2.4 kernel: How are decisions made on what goes into the kernel?

– By Dylan Griffiths –

The Linux kernel is the center of every Linux distribution. It is the foundation upon which the userland (the programs and environment you use to interact with the system) is built. The source code which goes into it is very special — any bugs it has will affect every part of the system, perhaps even destroying the work of the user. How do changes happen inside of it? How are decisions made about the direction that the kernel will take?The Linux kernel is split into many parts. The largest division is that of the core kernel code and the device drivers. The core kernel code is divided further into subsystems. Some subsystems contain architecture-specific code, whereas most device drivers are not architecture specific. By the kernel being divided in this way, many people can share the work of maintaining the kernel by watching over specific parts. Each device driver has a person who is responsible for it; each subsystem of the kernel core has a maintainer which monitors its development.

The various programmers working to maintain the kernel are in a loose hierarchy. More experienced programmers, who have contributed more and have volunteered for this responsibility, are assigned more important jobs. The ultimate authority over all of these programmers is the official tree maintainer. As of this writing, they are: Linus Torvalds for 2.4.x, Alan Cox for 2.2.x, and David Weinehall for 2.0.x. The official tree maintainer is like the director of a play — they assign tasks to the other programmers, sometimes giving examples of the direction they wish the code to go. They work to make the kernel match their vision of what it should be.

You may be wondering how a tree maintainer decides on a direction for the kernel; where their vision comes from. It comes from the users. The users ask for features, report bugs that interfere with their work, and generally provide feedback about what the kernel is doing for them. Based on this, and personal motivation, the tree maintainer decides on a set of concrete goals for the kernel. This is their vision.

The direction of kernel development is not static throughout the life of a specific branch of the code. As the needs of the users change, the goals for the tree change. The discussion of these changes takes place on the Linux-kernel mailing list. The list receives almost all of the public discussion between the various developers, as well as general bug reports, feature requests, and patches. Averaging 200 messages a day, it’s not for the faint of heart.

The vision for the 2.4.x tree involved fixes and improvements to the VM subsystem, expanded scalability in enterprise environments, increased portability, and better efficiency. These changes were motivated by the humiliating mindcraft benchmarks of 1999, as well as a desire to give enterprise Linux users the ability to use all their big iron with Linux. Over the course of the tree’s development (the 2.3.x series of kernels), changes not directly related to Linus’ goals for the tree were accepted because they cleaned up code, addressed misbehaviour of code, or paved the way for future code cleanups.

Over the course of the development series, some conflicts arose as well. Andrea Arcangeli and Rik van Riel both had ideas about how the virtual memory subsystem should be designed. Alan wanted to see whose approach to improving the VM was more effective in his AC branch of the 2.4.0-test series. The solution was to have both men submit their patches, and release two versions of the next AC test kernel. The most stable patch was then selected and included in the next release. This process was repeated a few times to give a fair chance for competition between the programmers. The eventual winner was Rik’s algorithm. Linus merged in the patch from Alan’s AC tree, and it became the official VM patch going into 2.4.x’s stable release.

The Linux kernel is one of the larger Open Source projects around. The 2.4.1 kernel is a 24 megabyte tarball; expanded, the source is over 100 megabytes in size. This is quite a growth over the original source released onto usenet in 1991 by Linus — that kernel was half a megabyte in size uncompressed!

By harnessing the ideas of the developers and users, the development process of the Linux kernel has allowed it to eclipse many traditional operating systems in performance and reliability. This process, which emphasizes reliable code and correctness over the feelings and politics of individuals, is an essential part of the success of Linux.