Understanding the Difference Between CI and CD

By

-

September 6, 2018

There is a lot of information out there regarding Continuous Integration (CI) and Continuous Delivery (CD). Multiple blog posts attempt to explain in technical terms what these methodologies do and how they can help your organization. Unfortunately, in several cases, both methodologies are usually associated with specific tools or even vendors. A very common conversation in the company cafeteria may be:

Do you use Continuous Integration in your team?
Yes, of course, we use X tool

Let me tell you a little secret. Continuous Integration and Delivery are both development approaches. They are not linked to a specific tool or vendor. Even though there are tools and solutions out there that DO help you with both (like Codefresh), in reality, a company could practice CI/CD using just bash scripts and Perl one-liners (not really practical, but certainly possible).

Therefore, rather than falling into the common trap of explaining CI/CD using tools and technical terms, we will explain CI/CD using what matters most: people!

MySQL High Availability at GitHub

By

GitHub Engineering

-

September 6, 2018

GitHub uses MySQL as its main datastore for all things non-git, and its availability is critical to GitHub’s operation. The site itself, GitHub’s API, authentication and more, all require database access. We run multiple MySQL clusters serving our different services and tasks. Our clusters use classic master-replicas setup, where a single node in a cluster (the master) is able to accept writes. The rest of the cluster nodes (the replicas) asynchronously replay changes from the master and serve our read traffic.

The availability of master nodes is particularly critical. With no master, a cluster cannot accept writes: any writes that need to be persisted cannot be persisted. Any incoming changes such as commits, issues, user creation, reviews, new repositories, etc., would fail.

To support writes we clearly need to have an available writer node, a master of a cluster. But just as important, we need to be able to identify, or discover, that node.

On a failure, say a master box crash scenario, we must ensure the existence of a new master, as well as be able to quickly advertise its identity. The time it takes to detect a failure, run the failover and advertise the new master’s identity makes up the total outage time.

This post illustrates GitHub’s MySQL high availability and master service discovery solution, which allows us to reliably run a cross-data-center operation, be tolerant of data center isolation, and achieve short outage times on a failure.

Helm, The Package Manager for Kubernetes

By

Cloud Native Computing Foundation

-

September 6, 2018

A few weeks ago, the CNCF family was extended with a new project – Helm, the package manager for Kubernetes.

Kubernetes was developed as a solution to manage and orchestrate containerized workloads. At the same time, managing the pure containers is not always enough. At the final end, Kubernetes is being used to run applications, and having a solution that will simplify the ability to run and deploy applications with Kubernetes, was a high demand. Helm was this solution.

Originally developed by Deis, Helm shortly became a de-facto open source standard for running and managing applications with Kubernetes.

Imagine Kubernetes as an Operating System (OS), Helm is the apt or yum for it. Any operating system is a great foundation, but the real value is in the applications. Package managers like apt, yum, or similar, simplify the operations so instead of building the application from the source files, you can easily install it with the package manager in a few clicks.

Linux Weather Forecast

By

Jonathan Corbet

-

September 5, 2018

Welcome to the Linux Weather Forecast

This page is an attempt to track ongoing developments in the Linux development community that have a good chance of appearing in a mainline kernel and/or major distributions sometime in the near future. Your “chief meteorologist” is Jonathan Corbet, Executive Editor at LWN.net. If you have suggestions on improving the forecast (and particularly if you have a project or patchset that you think should be tracked), please add your comments below.

Forecast Summaries

Current conditions: the 4.18 kernel was released on August 12. Some of the more important changes in this release are:

The power domains subsystem has seen a number of enhancements that will lead to improved whole-system power management.
It is now possible for unprivileged processes to mount filesystems when running in a user namespace. This will make it possible to mount filesystems within containers without the need for additional privileges.
Zero-copy TCP data reception is now supported.
The AF_XDP subsystem will eventually lead to highly accelerated networking in a number of settings. This work is part of a larger effort to win back users of user-space networking stacks by providing better facilities in the kernel.
Bpfilter is a new packet-filtering mechanism based on the BPF virtual machine. The 4.18 version of bpfilter won’t do much, but it is expected to be the base on which the next generation of kernel firewalling systems is built.
Restartable sequences, a new API for performing highly optimized lockless operations in user space, have finally made it into the mainline kernel.

It’s also worth noting that the new AIO-based polling mechanism, originally merged for 4.18, was later reverted pending further work.

The 4.18 development cycle saw the merging of 13,283 unique changesets from nearly 1,700 developers, 226 of whom where first-time contributors. See this article for more information on the changes merged for 4.18.

Short-term forecast: The 4.19 kernel can be expected in mid-October. Some of the key features merged for this release are:

The AIO-based polling mechanism that didn’t quite make 4.18 will be there in 4.19.
Load tracking in the scheduler has been augmented with a better understanding of the resources used by realtime and deadline processes; this information will support better power-management decisions.
Intel’s cache pseudo-locking feature is now supported.
An extensive set of mitigations for the L1TF vulnerability has been provided.
The block I/O latency controller allows an administrator to provide block (disk) I/O response-time guarantees to specific processes.
Time-based packet transmission is now supported in the networking subsystem.
The CAKE queueing discipline offers significantly better networking performance, especially in home or small-business settings.
The minimum version of GCC needed to build the kernel is now 4.6.
As usual, thousands of bug fixes, clean-ups, and small improvements have been merged as well.

The 4.19 kernel is in the stabilization phase now, with only bug fixes accepted. There is a small chance that this kernel will be called “5.0” when it is released, though your weatherman would predict that 5.0 won’t happen for one more release cycle.

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

Thoughts on the New State of DevOps Report

By

dbsmasher

-

September 5, 2018

This week, the latest State of DevOps report by the team at DORA was released. There is a lot of conversations in various social media platforms on what makes an engineering team ‘successful’ and how to increase feature velocity in tech companies and the DORA report does not just rely on anecdotes but surveys the field using rigorous scientific methods and statistical constructs to back or debunk common practices with real data. Because of the strong scientific backbone of their findings, I have been following the report for the last few years. If you have not, the book Accelerate is a great way to learn the findings of the previous four years.

Database management as a part of team performance

One of the new things in this year’s report is expanding behaviors by high performing teams to more than just ‘development’ practices. This year expands the view on high performing engineering organizations to how these organizations handle database management.

We discovered that good communication and comprehensive configuration management that includes the database matter. Teams that do well at continuous delivery store database changes as scripts in version control and manage these changes in the same way as production application changes.

How to See What’s Going On With Your Linux System Right Now

By

HPE Blog

-

September 5, 2018

Is that service still running? What application is using that TCP port? These questions and more can be answered easily by sysadmins who know simple Linux commands.

If you’re a system administrator responsible for Linux servers, it’s essential that you master the Bash shell, especially the commands that show what software is running. That’s necessary for troubleshooting, obviously, but it is always a good idea to know what’s happening on a server for other reasons—not the least of which is security awareness.

Previously, I summarized 16 Linux server monitoring commands you really need to know, what to do when a Linux server keels over, and Linux commands you should never use. Here are some more commands to further your system administration skills—and help you identify the current status of your Linux server. I consider these commands to be fundamentals, whether your servers run on bare iron, a virtual machine, or a container with a lifespan of only a few hours.

To put these in some kind of context, let’s follow sysadmin Sammie as she goes through a typical day.

Raspberry Pi-based ZeroPhone Promises ‘an Open-Source, Linux-Powered’ Handset

By

ZDNet

-

September 5, 2018

A project to build the $50 phone has been launched on Crowd Supply, promising to deliver “an open-source, Linux-powered” handset with “no carrier locks, bloated apps, or data mining” and that “doesn’t depend on big companies”.

Android smartphones have hit the headlines recently, first for Google tracking users who had switched location tracking off, and second for sending data to Google’s servers 50 times more often than an iPhone.

The ZeroPhone will be based around the tiny Raspberry Pi Zero, an Arduino microcontroller, and the ESP8266 Wi-Fi module.

To get the most of the phone, users will need to be technically proficient. While the splash page for the project says the likes of IR receivers and the Pi’s 5 and eight-megapixel cameras can be wired up the phone’s electronic interfaces, this will require users to hook up the electronics themselves.

Survey: Open Source Programs Are a Best Practice Among Large Companies

By

The New Stack

-

September 4, 2018

Open source software programs play an important role in how DevOps and open source best practices are adopted by organizations, according to a survey conducted by The New Stack and The Linux Foundation (via the TODO Group). By implementing open source best practices, organizations are helping developers become both more productive and more structured in how they manage the often abundant open source software their businesses rely on.

Thousands of open source projects have participants from organizations with open source offices that consume — and in turn contribute back to — any number of projects. The interest is showing enough results to merit companies considering deeper and more focused approaches to the development of the open source offices they manage.

Open source programs generally have three core characteristics: they 1) execute and communicate the organization’s open source software strategy, 2) maintain license compliance, and, most of all, 3) foster open source culture.

Linus Torvalds Talks Frankly About Intel Security Bugs

By

ZDNet

-

September 4, 2018

At The Linux Foundation‘s Open Source Summit North America in Vancouver, Linus Torvalds, Linux’s creator, and Dirk Hohndel, VMware VP and chief open source officer, had a wide-ranging conversation about Linux security, open-source developer, and quantum computing.

Torvalds would really like his work to get back to being boring. It hasn’t been lately because of Intel’s CPU Meltdown and Spectre security bugs. The root cause behind these security holes was speculative execution.

In speculative execution, when a program does a calculation, which might go several ways, the processor assumes several results and works on them. If it’s wrong, it goes back to the beginning and restarts with the correct data. Because CPUs are so fast these days, it’s much quicker to do this than to have the hardware sit idle waiting for data.

Torvalds “loves speculative execution. CPUs must do this.” But, Torvalds is annoyed that “people didn’t think about the problems of taking shortcuts with speculative execution. We knew speculative work that wasn’t used had to be thrown away.” It wasn’t. That problem is now baked in most modern processors. The long-term fix is a new generation of Intel CPUs.

Git Quick Start Guide

By

Linux Journal

-

September 4, 2018

If you have any experience with programming or just altering config files, I’m sure you’ve been dumbstruck by how one change you’ve made along the line affects the whole project. Identifying and isolating the problem without a version control system is often time- and energy-intensive, involving retracing your steps and checking all changes made before the unwanted behavior first occurred. A version control system is designed explicitly to make that process easier and provide readable comparisons between versions of text.

Another great feature that distributed version control systems such as git provide is the power of lateral movement. Traditionally, a team of programmers would implement features linearly. This meant pulling the code from the trusted source (server) and developing a section before pushing the altered version back upstream to the server. With distributed systems, every computer maintains a full repository, which means each programmer has a full history of additions, deletions and contributors as well as the ability to roll back to a previous version or break away from the trusted repository and fork the development tree (which I discuss later).

Quick Start Guide

The great thing about git is there’s so little you need to know! Without further ado, let’s begin with the most important commands.