Wednesday, April 22, 2015

Podcast: Configuration management with Red Hat's James Shubin

James Shubin is a configuration management architect at Red Hat. In this podcast, James talks about the state of configuration management and where it's going. He discusses some of the different approaches among major projects in the (broadly speaking) configuration management space including Puppet, Chef, and Ansible and offers his perspective on how they should be viewed relative to each other.

Links:

Listen to MP3 (0:15:35)
Listen to OGG (0:15:35)

[Transcript]

Gordon Haff:  Hi, everyone. This is Gordon Haff with Red Hat. Welcome to another edition of the "Cloudy Chat Podcast." Today, I'm joined by James Shubin, who writes "The Technical Blog of James, ttboj.wordpress.com." He goes by @purpleidea on Twitter and other places. He's a config management architect at Red Hat.
My guess, we're going to talk about configuration management today. Welcome, James.
James Shubin:  Hi, Gordon. Thanks for having me.
Gordon:  Great to have you here. Let me start off with ‑‑ to set the stage here, from your perspective, what is this config management thing?
James:  Config management. Let me run you through the quick, five‑second basics that I get everyone on the same page so at least we can use the same words. The three separations that I like to make is there is something that is provisioning, there's something that is config management, and then there is something that is orchestration.
Sometimes, we blur the lines. Just to set those straight from day one, provisioning is everything that happens that gets your machine up and running. Basically, a kickstart or a vagrant up or something like that. After that, everything that happens after that is config management. That configures the machine, it might install packages. Lots of things can go on.
After that, sometimes people like to manage things. We typically call that, "Orchestration." When there's some external force that goes and pokes things to do something, that's orchestration. The reason I like to make this clear is because there's lots of great tools out there, and some of them blur the lines of config management and orchestration.
A tool like Puppet is a pure config management tool. Same thing with Chef. There are things like Ansible which are crossing the line between config management and orchestration. It gets everyone a little confused, but those are the terms that we use.
Gordon:  I'm going to dig into a little more detail about the differences between some of the tools out there, and how they overlap and don't overlap. Before we get to that level of detail, from your perspective, what's changing about config management today? What's the interesting stuff that's happening?
James:  A lot of interesting stuff is happening. A lot of config management didn't used to happen in the past. People had a smaller number of machines, there weren't as many services. Now we have microservices, we have more and more machines. As things go on, automation isn't a question of if you do it well, you'll have to do it.
If you're not doing config management, if you're not doing automation, you simply won't be able to run your infrastructure. It's becoming more and more essential. Once upon a time, we would have little bash scripts that glued everything together. Some people still do. CFEngine was an early player that was somewhat popular, although I wasn't a huge CFEngine user, personally.
Newer technologies like Puppet and Chef are quite popular these days. In the future, I think that's going to change, too. The scales are getting bigger. There's a promise that ARM servers could happen and increase host counts by 10 or 100 or maybe more. A lot's changing there. The fact that containers are getting quite popular is definitely going to change the scene a little bit, too.
I don't have the magic glass ball that can tell me exactly what's going to happen. I'm definitely following it closely.
Gordon:  Let's talk about containers specifically. That brings a significant change in the way we operate systems. While you can look at containers as virtualization lite, as I've talked about in some of my previous podcasts, that's not the best way to use containers.
James:  Fair enough. The biggest message around containers that I would like to put out there...I think many of my colleagues and peers in the community agree, especially in the config management community.
While containers are a great technology, and there's definitely a lot of cool stuff happening there, this unfortunately, does not get rid of the need to still do config management. The role might change a little bit, how it glues together might be a little bit different, but it doesn't go away.
If anything, config management needs to be adapted and more flexible, and have new paradigms so that these things work that much better. The current generation of container stuff, containers have existed for ages, but quite recently they're getting quite hot and popular.
A lot of the current generation of config management tools, were written and designed before containers were very mainstream. That might have to change. They might have to make some big changes, and some of them are trying. We'll see how well they succeed. Maybe someone will come up with something entirely new that solves this problem much better.
Gordon:  Another change that's not directly related to containers, although they're part of the same general sea change, is this shift towards VMs or containers or services, that aren't long‑lived, that have a very short lifetime. The pets/cattle thing. One of these things goes bad, you just shoot it and start up a new one. How does that change config management?
James:  You've still got to configure how those things are built, and what goes in them and what settings get set for them. There are tools like Kubernetes that are particularly good at managing these cattle. That'll glue them together with containers.
I have a product, more of a project called "Oh‑My‑Vagrant," which I use to actually test and glue a lot of these technologies together. It's an environment on top of Vagrant, which is a great development tool. It will also glue in Puppet and even Ansible and Docker and Kubernetes and things like that.
If you want to spin up a test environment to test this out for yourself and to make it easy to develop your app, you can do that. I've got some new screencasts coming that I've been publishing on my blog. There's some stuff there now, and some more stuff is coming quite soon.
Gordon:  Let's dive a little deeper into some of the individual products and projects that are out there. What are they good for? Who are they interesting to? How are they evolving? Let's start with Puppet.
James:  Puppet is one of the most popular tools. It has a tricky learning curve for some people. I think it's a useful tool. I've done quite a lot of very complex and advanced Puppet things. I'm grateful that it existed, because it's where I did a lot of my learning about config management from.
The language, the Puppet DSL, the domain‑specific languages, is mostly declarative, which is uncomfortable for some people who are used to more imperative programming tools. Somehow, Puppet folks have made some really, really brilliant features. One of my favorite is exported resources.
It's a feature in Puppet that lets you declare things that seem to be part of one host but get exported, and those definitions get exported and used on a different host. When you're doing multi‑machine things, which I think are interesting. Puppet has some nice patterns that make it more natural.
It's not perfect. There're definitely some problems and missing pieces there, but it definitely has been very inspiring.
Gordon:  Now, the developer, as opposed to the operator or the ops community, Chef is probably the most popular tool today. Right?
James:  I'd like to see more of less developer versus ops and really more ops devs or dev ops, as we sometimes hear on the Internet. Getting those two to converge might be an impossible fight. A lot of places that I've seen people converge the two, and have much more truer DevOps environment, they see a lot of gains.
You can move a lot quicker. You can ship code that you own and are responsible from. You can make changes very quickly. There's a lot of organizations that are literally making changes every minute and every 10 minutes and moving that quickly.
When you can use DevOps techniques to move that quickly, you can innovate a lot quicker than some of the slower‑moving competition out there.
Gordon:  What is it, specifically, about Chef that makes it interesting for developers and the dev ops folks?
James:  I'm not a heavy Chef user. I've used Chef a bit, but not as extensively as Puppet. That's just a personal choice. I know that the Chef community is awesome. Nathan Harvey, one of their community guys, is a really nice guy. Every time you meet him at conferences, it's great. I wouldn't be surprised if one of the main answers is their community.
They have a lot of other great community people. Another thing about Chef is that it's not declarative. Not fully declarative, anyway. There's a lot more imperative Ruby being used. It might be a lot more comfortable for people who aren't familiar with the declarative paradigm. Personally, I think we should get used to the declarative paradigm.
For a lot of people that don't want to dive down that rabbit hole, Chef can be a great step into config management.
Gordon:  You've talked about this declarative versus imperative. Maybe for our listeners, spend a couple of minutes talking about what the differences are?
James:  This has changed a little bit over the lifetime of Puppet, because there are pure declarative languages and stuff. Puppet's maybe not purest. Imperative is what we're typically used to in a programming language. You have for loops and the typical structures you see in Python and Ruby.
The types, which are the things that we declare or state in config management exist. In Puppet, the constructs that set up those types can also be declarative. In pure Puppet, there didn't used to be for loops and things like that. There were classes and things that are called defines. You would set those up. You would have a relatively logical but well‑organized code base.
All of the more complex redefining of variables and little things that can cause programming errors in imperative space don't really exist in this language. It actually makes your code safer. I realize I'm probably explaining this extremely poorly. That's probably because I've got my head stuck in the sand, so deep into code that I forget how to define things.
Hopefully that gives you a good example, and can convince people to look into the specifics and see. If you're writing code that runs on thousands of machines, you could make a mistake. You could make a mistake.
Something like an off‑by‑one error, that could perhaps be prevented quite easily with a declarative language, could blow away a whole bunch of machines or a whole bunch of data. There is a use for safer declarative languages.
Gordon:  One of the ways I think about, and it's probably a bit simplistic, is imperative you're telling something...what to do, and declarative is you're telling it where you want it to end up.
James:  The concept of states and defining what state you want to converge towards exists in all the languages. How you define those state elements can be a bit more dynamic and be programmed a bit more classically in Chef than in Puppet.
Gordon:  One other project we're starting to hear quite a bit more about these days is Ansible.
James:  Ansible is an interesting project, because people lump it into the what's config management space. I don't see it in that space. The easy way you could decide if it's in the same space is is there some cluster, architecture, or infrastructure where you would have more than one of these tools?
If you would use Ansible with Puppet or Chef, would it still make sense? It might not be necessary. What Ansible really does is it's idempotent. What that means is that if you write some code or have some sort of line that says I'd like to do something and you run that over and over again, if it's idempotent it should converge towards that state.
Some operations, like appending to a file, they would not converge. They would diverge, and you would never end up at one, single state. Puppet, and Chef, and Ansible are all idempotent, which means you can run the code as many times, and you should converge towards that one state.
The difference is that Ansible, like I said, isn't purely config management. It's more of an orchestrator. You actually run it on your laptop or on, arbitrarily, one server. It goes out over SSH and applies things, in an idempotent way, to a number of servers. In contrast, Puppet and Chef run on that machine and configure directly.
Gordon:  Any other projects that have particularly interested you in this general space?
James:  Which ones have particularly interested me? There's a lot of little, random things that I look at and poke at. I don't think a lot of them are worth mentioning. I think we'll see some exciting changes over the next three months. I would wait and see what comes up. Nothing I wanted to call out today specifically.
Gordon:  Anything we haven't covered that you think our listeners might be interested in?
James:  There's a lot of stuff going on. I've been doing a lot of config management and containers work lately. The big thing that I'm hacking on at the moment, this week, is Oh‑My‑Vagrant. I definitely encourage you to have a look at that. It's a simple way to make a Vagrant environment without writing a thousand lines of Ruby.
There's some great stuff happening in the systemd‑nspawn world, which I think is quite interesting and I think is a serious technology win. That's quite interesting, if you're interested more in the container space. For config management, I don't know what's going to happen in the future. I have some ideas.

Hopefully, they'll be interesting things that we can talk about in a few months, or in a year down the road.

No comments: