It was on this day in 1993 that Patrick Volkerding announced the Slackware 1.00 release that was inspired by the Softlanding Linux System…
Read more at Phoronix
Linux System Administration Skills are Changing
When was the last time you compiled a kernel? For many of the latest generation of Linux admins, the answer is really simple: never. I am one of those, provided we don’t count a few times I tried it just for fun, then couldn’t see why I would need a custom kernel and went back to my out-of-the-box kernel.
For many of the longer-time Linux admins and engineers this may seem laughable, but it is a reality: As Linux adoption grows in the enterprise, a new generation of Linux admins is created that has extremely good technical skills, but lacks these ‘simple’ low level skills seen by many as fundamental to being a good Linux admin. We can build a high performance, highly available web infrastructure that uses the latest of the latest techniques, but don’t ask us to fix a non-booting Linux machine: our advice will be to ditch it and set up a new vm.
Over the past decade or so, we have seen some interesting trends. Linux became a commodity in the enterprise, and as that happened the various distributions became powerful yet flexible enough to remove the need for the average admin to ever have to do low level things like compiling a kernel.
Next, we welcomed virtual machine technology as a commodity, which added another layer of abstraction. Users of clouds like amazon or VPS providers will possibly never have to deal with deploying Linux on bare metal. As hybrid and private clouds are becoming common as well, many enterprise admins will also not have to deal with this kind of thing anymore, they will just log into a web interface and spin up 5 more apache vm’s.
The newest two trends add even more abstraction: configuration management and the seemingly brand new (yet not new at all) containerization with tools like docker. Whenever a client asks us at OlinData to configure a Linux machine, our first action will be to set up Puppet. With our trusted library of well-functioning Puppet modules, that is very easy and will cost me less time then doing this manually.
For example with Puppet, I can install Apache on a new machine as simple as this:
node ‘web01.olindata.com‘ {
include apache
apache::vhost{ ‘www.olindata.com‘:
docroot => ‘/var/www/olindata’
}
}
Depending on the environment, I don’t even have to log into the machine anymore. Deploying this code through Continuous Deployment tools like Jenkins will allow me to deploy my infrastructure code automatically as it passes the tests I set up.
SysAdmin skills move up the stack
Even as we move toward higher levels of abstraction, ongoing Linux training is still highly valuable and desirable for admins today and will be well into the future. Knowing the fundamentals is key but as abstraction removes some of the old tasks, this requires sysadmins to move up further in the stack and enhance their skills in the higher level tools and practices. It is critical for a sysadmin to become familiar with the tools that enable these higher levels of abstraction. It pushes them to become more skilled in things like coding so that they can do more with these “new” tools.
Will the need for low(er) level linux skills ever go away completely? Of course not. We still have many other uses for Linux then just the commodity server deployments. Also, people will still benefit hugely from knowing how to do lower level operations in their everyday work. On top of that, with demonstrable Linux skills on your resume, I (and many other employers with me) will always prefer you over candidates that don’t have them. You never know when you need those low-level skills!
Walter Heck is CEO and Founder of OlinData, an authorized Linux Foundation training partner. Here’s a list of scheduled official Linux Foundation courses by OlinData.
Linux Foundation SysAdmin Konstantin Ryabitsev, an SELinux Expert
This is the fourth profile in a series on Linux Foundation system administrators leading up to SysAdmin Day. Do you have a super-hero sysadmin you’d like to recognize? Send your nomination to This e-mail address is being protected from spambots. You need JavaScript enabled to view it
by July 25 and enter them to win a free ticket to LinuxCon and CloudOpen North America taking place in Chicago August 20-22, 2014. (See the full contest announcement for more details.)
Konstantin Ryabitsev is a system administrator on the Linux Foundation’s Collaborative Projects IT team. Here he discusses why he became interested in IT security, his approach to working with developers, and his love of human languages, among other things. (Also, see our May 2013 article on what it’s like to be a Linux Foundation sysadmin, starring Konstantin.)
Linux.com: How long have you been a sys admin?
I set up my very first Linux system in 1998. It was a combination web server and mail server running on an old beige tower and served the non-profit where I worked at the time extremely well. I didn’t know enough about systems administration at the time to keep sendmail patched, so six months later the system was also serving “warez.” Needless to say, that catapulted IT security to the forefront of my interests.
When did you start at the Linux Foundation and how did you get the job?
I started in November 2011. I was working at McGill University InfoSec at the time, and was also active with Fedora Project — which is how my name showed up on the list of candidates. The Linux Foundation was looking for a systems administrator with a strong background in IT security — who would also be a good fit for a decentralized team of passionate open-source advocates. I’m extremely glad I was a good fit for the position, as I can’t imagine receiving as much satisfaction from any other job.
What do you do for the Linux Foundation? What’s your specialty?
I’m part of the Collaborative Projects IT team, which is responsible for providing IT hosting for projects like kernel.org, codeaurora.org, opendaylight.org, allseenalliance.org, and a few others. The requirements are different for each project, so our specialty is working with the developers involved in each one of them and finding solutions that would satisfy their needs. We provide a full development stack, from git repository hosting to continuous integration and release distribution. We try not to specialize in any one thing, but we
certainly have areas where each member of the team has higher expertise than others. I am usually the go-to resource for SELinux policies and other process confinement questions.
Will you describe a typical day at work for you?
We are very client-oriented, so we try to give priority to incoming developer requests. We try to automate as much as we can, but a lot of things still require admin involvement, especially with regards to continuous integration (CI) systems. Besides that, there’s always documentation to be written or updated, or some system refactoring to do. We’ve added a number of very large projects in a very short period of time throughout the last 18 months, so some of the decisions we’ve made along the way require rethinking in order to allow us to scale better. Unfortunately, once you have production systems in place, refactoring requires very careful planning — otherwise you risk downtime that would interfere with project timelines.
What’s your favorite part of the job/ thing to do and why?
Every sysadmin’s guilty pleasure is writing a tool that would automate away a menial task. I call it “guilty pleasure” because sometimes the total amount of effort spent on writing such tools goes vastly beyond what it would take to continue doing things manually (obligatory xkcd link: http://xkcd.com/1205/). However, sysadmins hate being distracted by small requests (we have “the zone,” too, just like programmers: http://heeris.id.au/2013/this-is-why-you-shouldnt-interrupt-a-programmer/), which is how we justify our efforts. Thankfully, we don’t have to justify them to management — only to fellow members of the sysadmin team.
What is your nightmare scenario? How have you prepared for it?
Well, if we’re prepared for it, then it’s no longer the nightmare scenario. 🙂 We try to introduce lots of redundancy into the infrastructure to assure that there are as few single points of failure as possible, but at some point you have to do risk-benefit analysis and draw the line past which such redundancy becomes prohibitively expensive either in terms of money or effort. For example, if there’s a major natural calamity on the North American West coast, we’ll definitely be impacted, since most of our infrastructure is in Portland, Oregon.
However, we will be able to recover without any data loss due to off-site backups in Montreal. A more realistic nightmare scenario would be a systems failure while the sysadmins responsible for that particular part of the infrastructure are not reachable, which is why we try to keep ample documentation. Oh, and the rule is that we don’t all fly to Linux Foundation events on the same days, so as not to all be in the air at the same time. 🙂
What is your favorite sysadmin tool and how do you use it?
I don’t really have a favorite tool — not any more than a plumber would have a favorite wrench (yes, I think plumbers are a great analogy for sysadmins). I do have a preferred set of tools that I use every day, but I expect it’s similar to what most other sysadmins keep in their proverbial tool belt.
What’s your favorite story about working at the Linux Foundation?
The thought that we are in a position to tell Linus what he can and cannot do on the Linux systems we manage makes me giggle every time – because it’s so wonderfully absurd. However, I would say my favorite bit is how many companies are so eager to help you out if they find out that you are working for the Linux Foundation. We rely on the goodness of a great number of donors to run our infrastructure — most of whom volunteered their services without any prompting on our part. My thanks go out to Google, Rackspace, ISC, Intel, Vexxhost, Yubico — among many others — for helping us with hardware and hosting. And, of course, to all the generous members of the Linux Foundation. You make it possible for us to run infrastructure where Linux development itself happens.
What do you do for fun, in your spare time?
I’m a (human) language nerd, so my favorite hobby is learning how to read books over and over again in different languages. It’s like crossword puzzles on a nightmare level. I can read fluently in four languages (English, Russian, French, and Spanish.) When I don’t have spare brain cells left, though, I enjoy walking, biking, and being generally silly with my family.
Read more about Linux Foundation System Administrators:
Linux Foundation SysAdmin Michael Halstead’s IT Career Started at Age 15
Linux Foundation SysAdmin Andy Grimberg Loves New Tech and Snowboarding
To Linux Foundation SysAdmin Ryan Day, Elegance is the Best Tool