Unsung heroes: Linux Kernel Janitors

Managing a free software project of any size is like wrestling an octopus.
No matter how strong or smart you are, there’s always something else waiting to
hit you in the head. Besides coding, you must organize volunteers, delegate
tasks, and motivate people to move in new directions. It’s no wonder little
maintenance issues fall through the cracks. Source files that go untouched for
long periods of time succumb to bitrot, and a
thousand small changes here and there evolve parts of the code faster than
others.

The Linux kernel is no exception. A huge project involving a wide range of
hardware and a rapid development pace, the kernel has several little jobs that can fall under the radar. Finding and fixing these issues takes time and knowledge — a rare
combination. Merging patches from dozens to hundreds of far-flung programmers
winnows out plenty of errors, and maintainers occasionally post task lists
before a major release. That doesn’t mean things automatically get done. Like
writing and running tests, updating documentation or old code after a small
change can be tedious and thankless.

As usual, the world of free software turns this on its head. Here, a list
of tasks is well-worth the effort to create it. Not only does a list serve as
a status report and a set of reminders, but it can also be a map of places for
a new developer to explore the code. He can learn the codebase, make very real
contributions, and receive feedback from the gurus. Running patches past
the maintainer responsible for the section of code being improved will hone his
skills. It’s dirty but manageable work.

Maintenance man

That’s what motivated Arnaldo
Carvalho de Melo to start the
Linux Kernel Janitors
Project. Interested in working on the kernel, he wanted to improve his
knowledge before tackling the big jobs. de Melo created a personal TODO list and
started working. Realizing the scope of the project exceeded his free time, he
published the list to attract other potential hackers. It gained attention,
indeed.

While the Janitors list grew in popularity, kernel programmer Ted T’so published a list of things to finish
before
the official 2.4.0 kernel release. It’s written in a terse shorthand,
accessible mostly to existing hackers. These things usually get done,
but keeping the list current is a big job. It rarely attracts fresh
coders, either. Dave Jones, himself a
kernel hacker, suggested that the Janitors list could solve both of
these problems with stronger administration, so he and de Melo started an
official project on SourceForge. This gave the group dedicated mailing
lists and an easy way to track task updates.

The chosen name, Janitors, indicates the relative glory of code cleanup. Still, the prestige of contributing to a major piece of free software is hard to
ignore. Half of becoming a free software developer is knowing where to start.
Finding a list of accomplishable tasks that don’t require voluminous experience
or esoteric knowledge can be just enough motivation to fire up a text editor and
start coding.

That’s not to say that kernel development enforces formal stages of
apprenticeship on new contributors. Proprietary companies may require new hires
to pay their dues in the customer support or quality assurance pits before
graduating to junior developer status. An informal meritocracy tends to play a
larger role in the free software world. Kernel Janitors take up that mantle
voluntarily. It’s not an exile or rite of initiation, but rather a
self-selected starting point.

Cleanup on files *.c

If one bug or flaw or bad assumption shows up in a program, it will have
friends lurking around the corner. (If that’s not an axiom yet, it should be.)
That knowledge fuels the security auditing of the OpenBSD team, and it’s given rise to a set of
tools for analyzing source code. It takes human experience and intuition to
discover a new error pattern, but a properly programmed computer can root out
occurrences of that pattern in megabytes of source in moments.

One such tool is the Stanford Checker, a modified version of the GNU C
compiler. It takes descriptions of error patterns and analyzes source code for
actual occurrences. Running it on the kernel has yielded several weak spots
now on the Janitors list of things to do. That’s no substitute for experienced coders, many of whom have also recommended areas and things to fix. Any error brought up on
the Linux Kernel mailing list is a signal for further exploration. It’s
possible to find these weak spots by hand, but as de Melo puts it, “[Tools] like
this can help a _lot_ in identifying possible bugs.”

A taste of success

Somewhere in the hidden alcoves of the lkml
archives is an ossuary of half-finished proposals to revolutionize kernel
development. Anyone can have an idea, but the coin of the realm is working
code. The Janitors have already proven their worth in this area. de Melo
recently found himself the maintainer of the IPX protocol after updating it to
take advantage of newer kernel features like multithreading. That helps the
Janitor project’s mindshare, as its successes lead to more responsibility and more
suggestions for other tasks.

As the scope of the project grows, it will need to attract more
contributors. So far, this hasn’t been a problem. Some contributors only
produce a few patches and then leave, while others stick around. To give a
better sense of the mission, the head Janitors have considered writing a series
of articles and documents describing larger tasks and good style guidelines.
For now, they recommend all interested parties participate in the Janitors
Discussion mailing list.

For the future, de Melo believes the available cleanup tasks will continue to
increase in complexity. There will still be plenty of introductory jobs,
including updating code comments to be converted into Postscript and DocBook
automatically. The job will never go away as long as the kernel continues to
evolve. For now, the Janitors have demonstrated that maintenance is a vital
part of kernel development. Further, they’re ready, willing, and able to
perform these tasks as needed. They may not receive glory or recognition
corresponding to their contributions, but they deserve a tremendous amount of
respect and encouragement.