DevOps Fundamentals, Part 5: Consistency in the Pipeline


So far in our series previewing the DevOps Fundamentals: Implementing Continuous Delivery (LFS261) course from The Linux Foundation, we have already looked at:

In this article, we’ll do a quick review of some of the tools and then discuss how to achieve consistency in the pipeline.

To start, we have source control, and Git is one of the more popular tools to use. But in the Microsoft world, you have Team Foundation Server, and then there’s Perforce and SVN, which is a bit older. Then, there is the really old CVS. There are also some SaaS-based source control systems like GitHub, Bitbucket, and GitLab.

In terms of what we call the build console or the Continuous Integration server, we have: Jenkins, Bamboo, Team City, as well as Travis CI, Circle CI, and Shippable.

For repository managers, we are going to use Nexus and Artifactory in this course, but we also have: Docker Trusted Registry, Docker Hub, and Google Container Registry.

In terms of operations consoles, there’s Rundeck and Marathon. Also, Asgard is interesting; it is part of Netflix’s open source tooling and particularly works with Amazon. There’s also Spinnaker and WeaveScope.

For automation, we have: Cfengine, Chef, Puppet, Ansible, Docker Compose, Cloud Formation, and Terraform.  This is in no way the entire list. There are new products every day. These are just some of the most commonly used. For more information, watch the video below:

Back to consistency in the pipeline. Here is the thing. When you get into containers, this is even more important, because you are running hundreds, maybe thousands of containers, running containers in a cluster.

So, the goal is to create consistency. In the early days, it was all checklist-based or somebody’s shell script, and that was somewhat inconsistent. Then, Chef and Puppet came in and created a high level of consistency, where all the environments would get built at every level through some domain specific language (DSL).

At the end of the day, all elements of the pipeline should be disposable and reproducible. All environments should look like production, from laptop, to integration, to any type of testing. You want to decrease variability between elements in the pipeline. Repeatability increases speed in rebuilding environments. Improved consistency also results in reduced errors and increased security.

You also want to version control everything. It keeps a history of all your changes, so you can check the differences. You can restore and rebuild elements. All changes are visible and audited by everybody, and changes can be automated. The magic really starts when everything is in version control and you can track where everything came from. Basically everything should be in version control.

More details on creating consistency in the pipeline are provided in the video:

Want to learn more? Access all the free sample chapter videos now!

This course is written and presented by John Willis, who has worked in the IT management industry for more than 35 years.