SUSE Enterprise Linux Live Kernel Patching with Open Source kGraft


live patching talk

Imagine, if you will, applying a kernel patch to your production servers in the middle of the day, during peak transaction periods, and not … missing … a beat. Imagine that same update not requiring you to suffer downtime as the server reboots to apply the patch. These are ideas from the future, right? Wrong. The idea of live kernel patching is real and now ─ thanks to SUSE Enterprise Linux (SLE).

The kGraft project was developed in the SUSE Labs and has been submitted to be merged in the upstream Linux kernel. That means, once accepted, anyone that uses Linux can enjoy this same, remarkable feature.

But how did kGraft come about? And what exactly does it do? I attended a SUSECon session with Vojtech Pavlik (Directory of SUSE Labs) and Udo Seidel (Head of the Linux Strategy and Server Automation at Amadeus) to get the specifics of what is certainly the future of Linux kernel updates.

There is one thing of note ─ both SUSE and Red Hat Linux have their own takes on this live kernel patching. Although both are very different, the endgame is to wind up with a universal version in the upstream kernel. Both versions are open source and both work with existing upstream infrastructures. Both also have the same goal. How each reaches that goal is where the variation begins. Red Hat’s offering (kpatch), places a heavy focus on pre-patch analysis. With this analysis, kpatch determines minor inconsistencies vs. issues that could have major post-patch ramifications. Both technologies are massively complex and should be considered game changers. The biggest difference between kGraft and kpatch (other than kGraft is available now), is that kGraft works without kernel interruption. Period.

Vojtech Pavlik is the director of SUSE Labs and has been working with Linux since 1994 (Slackware 1.1.6 kernel 0.97). Early on he did a bit of kernel development (work on input driver layers, USB, and more), but kGraft should be considered his masterpiece.

The roots of kGraft (sort of) begin with the ksplice project. I remember using ksplice and thinking how amazing it was that I could effectively upgrade my computer’s kernel and not have to reboot the system. There was serious magic going on with that process. But then Oracle acquired ksplice in 2011 ─ and we all know what happens when Oracle acquires an open source project… The kGraft project has nothing to do with ksplice ─ other than the ultimate goal of patching the kernel without interruption.

How does it work?

The gist of kGraft is this:

  1. kGraft locates differences between the running kernel and the patch.

  2. It creates replacement functions based on those differences.

  3. It loads and links the patched functions.

  4. It redirects code execution to patched functions.

What you must understand about kGraft is simple ─ it is the piece that makes SUSE Linux Live Kernel Patching possible. What you may not know is that SUSE Linux Live Kernel Patching is a service associated with SLES and their highest level of support. If you happen to be a subscriber to the Live Patching service (and use SLES 12), Live Kernel Patching will be ready and waiting. In fact, all of the necessary pieces for kGraft to work will be there, by default. Any security patches to the Linux kernel will happen, through the magic of kGraft, using the same old SUSE GUI management tools you already know and love.

If, however, you are not a subscriber or you haven’t purchased SLES 12 ─ fret not ─ kGraft is open source. In fact, during the presentation, Udo Seidel ran a demo using kGraft on a Fedora machine. Get the kGraft source code from Git.

Why is this technology so important and who does it apply to? If you really dig into the muck and mire of kGraft, you’ll find those that most benefit from this level of technology are the exact target audience for SUSE ─ enterprise level clients. Businesses with massive server deployments that demand 24/7/365 uptime are ripe for Live Kernel Patching. As well, this technology is perfectly suited for big data. Why? When you’re looking at terabytes of in-memory data that will take hours to reload on reboot ─ you need Live Kernel Patching to ensure those security patches (patches that can range from a mere two lines of code to thousands) can be loaded without having to give the dreadful command to “shut it down”. This can be a real game-changer when the bureaucratic red tape of rebooting can delay the process days, weeks, and even months (or send the CEO, COO, and CFO into fits of apoplectic shock). To that end, SUSE has you covered.

As I’ve mentioned before, SUSE is bringing to the table serious technological advances ─ advances that should readily awaken the world to a company that has been crafting incredible tech mojo (mostly under the radar) for a very long time. So, if you’re ready for some of the highest availability you have ever experienced, it’s time you get on over to SUSE and get ready to completely change the way you view updating the kernel.

For even more information on kGraft, read Libby Clark’s interview with Vojtech Pavlik.