Author: Bruce Byfield
Reiser4 is the successor to ReiserFS (a.k.a. Reiser3), a journaled filesystem that has been in the kernel since version 2.4.1, and that also faced a rocky road to inclusion. Both are developed by Namesys, a company fronted by Hans Reiser, with sponsorship from Linspire and the Defense Advance Research Projects Agency. According to Namesys, Reiser4 offers twice the speed of other filesystems, as well as greater efficiency in file allocation and military-grade security. It has attracted a loyal group of users, and, while not in the official kernel, is included in Andrew Morton’s experimental -mm kernel sources.
Discussions about Reiser4’s status have flared periodically for months. The current discussion started when Diego Calleja posted a document entitled “Why Reiser4 is not in the Linux Kernel” on the Kernel Newbies wiki. Since the document was not issued by Linus Torvalds or Andrew Morton, it is not an official statement. Calleja describes it as “a sort of Linux kernel mailing list opinion” — that is, as a summary of prevailing views, written by him and edited by several others.
Reiser responded with a rebuttal on the kernel mailing list. Besides the main question, the resulting thread raised side issues including the reception of ext4 by kernel developers, the desirability of testing new features in distributions before the kernel, whether small patches are integrated faster than larger ones, and whether Reiser ignores bugs in ReiserFS in order to focus on Reiser4. Throughout these exchanges, both sides strained to keep the conversation polite, but neither completely resisted snide remarks or the temptation to lecture the other, and disagreements were as likely to be over half-truths or preferences as technical points.
Standard procedures vs. convention and favoritism
According to Calleja, the main explanation for Reiser4’s exclusion is that it is not yet sufficiently tested. Reiser complains that this criticism is not only untrue, but a double-bind. More than one feature is marked as experimental in the kernel, and one of the points of inclusion is to ensure thorough testing. To Calleja’s suggestion that he try the tactic of introducing Reiser4 through a distribution, Reiser replies that kernel developers are far more suitable than the average user of a distribution.
On the technical side, the debate over Reiser4 centers on the fact that that it uses its own structure for plugins, rather than those that already exist in the kernel’s Virtual File System (VFS) layer. Calleja speaks for many kernel developers — probably the majority — when he says, “The code that needs to be merged must adapt to Linux’s point of view, no matter how much submitters disagree. The Linux kernel is community-oriented, and it’s the community that must take the decisions. You can’t impose your point of view on the rest of the kernel developers.”
To complicate matters, Reiser4’s approach lands the filesystem in the middle of a longstanding convention of avoiding plugins in the kernel, mainly to avoid architectural complications, but also to discourage proprietary drivers that circumvent the kernel’s release under the GNU General Public License. According to Torvalds, “if a filesystem wants to do something fancy, it needs to do so with the Virtual File System (VFS) Layer, not as some plugin architecture of its own. We already have exactly the plugin interface we need, and it literally is the VFS interfaces.”
Similarly, when Reiser formally requested that Reiser4 be accepted into the kernel in September 2005, Christoph Hellwig, one of the lead maintainers for filesystems in the kernel, noted that its source code did not follow the kernel coding style. Unless it did, several developers suggested, other programmers would have a hard time contributing to the code or maintaining it, should Reiser and Namesys withdraw from active development in the future.
Often, Reiser’s initial response to these technical criticisms has been to make sweeping statements about the superiority of his team’s methods and results, and to denounce reviewers of his code as unqualified. For example, in response to criticisms of his coding style, Reiser remarked, “Most of my customers remark that Namesys code is head and shoulders above the rest of the kernel code,” while he claims that the problem with having Reiser4 code reviewed by most other kernel developers is that “You can’t really have code reviewed by persons less experienced and proven than those being reviewed.” As several developers suggest, these reactions may slow down Reiser4’s inclusion by making developers reluctant to become involved in the process.
If these were Reiser’s only reactions, the controversy would remain minor. As Calleja says, other developers in the past have strained against the kernel’s established policies and procedures, and “all of them failed or were just ignored.” However, Reiser claims that his own experiences are symptomatic of more general problems. Linux kernel development, he suggests, would benefit from an influx of people who did things differently. “Researchers who stay in clumps are not the ones who become enriched from the ideas of many others,” he says. Not only that, but Reiser and his supporters suggest that an insistence on conformity has caused leading filesystem programmers to withdraw from kernel development. The names raised include Donald Becker, Andre Hedrick, David Mazières, Jim Mostek, and Steve Lord. (Others have questioned the motivations for some of these withdrawals, or whether they have occurred at all. Jim Lord, for one, publicly denies ever leaving, and hopes to step up his activity again one day.)
Another problem, according to Reiser, is that standards are not being applied consistently. He compares what he perceives as the resistance to Reiser4 to the apparent fast-tracking of ext4, the recently announced successor to the widely used ext2 and ext3 filesystems. “The [ext4] code isn’t even written, benchmarked, or tested yet,” Reiser says, “and it is going into the kernel already so that its developers don’t have to deal with maintaining patches separate from the tree.” Some respond by pointing out that ext4 is a successor to technologies already in the kernel, and that its developers have a proven track record of following established procedures. But to Reiser, who is eager for the increased testing that would come with inclusion, the situation smacks of favoritism.
Reiser is also irked that conformity, not performance, is the gauge of whether Reiser4 is ready for the kernel. “If you offer twice the performance of others,” he says, “you should be perceived as useful, and getting in should be easy for your filesystem. I am concerned here that what we have is a culture in which, if you benchmark, it is harder for you to get in because feelings get hurt. We are developing an anti-scientific culture, in which people think that they know what is right code without measuring it.”
Reiser may score some philosophical points, but, in the end, he has little choice except to conform in order to realize his goal. While engaging in these policy debates, Reiser has also made considerable efforts to answer the criticisms about Reiser4’s architectural and coding styles, until now, even his critics say that it is only a matter of time before Reiser4 is part of the kernel.
Personality and pride
For now, the arrival of that moment seems delayed by personality conflicts. Calleja specifically denies this assertion in his FAQ, calling it “shocking,” and claiming the delay is due entirely to technical merits. Unfortunately, his inability to resist remarks such as “Hans Reiser has not helped a lot with his attitude” and “Hans is not the right person to deal with the rest of the kernel community” do more to support it than refute it.
The fact is, no one who has observed kernel development should be surprised that personality plays a role. As long ago as six years, ago, Jeff Garzik’s guide to kernel development listed “You are being annoying” as a reason why a patch might be rejected. That remark may have been partly a joke, but much of the archives suggest that it has a strong foundation in the truth.
Reiser is hardly blameless. As some posters remark, Reiser is often undiplomatic and insulting. He describes Al Viro and Christoph Hellwig, for example, as “90% of the problem.” Linus Torvalds remarks that Viro and Hellwig “aren’t always easy to work with,” so the conflicts are not wholly Reiser’s fault, but he does not shy away from them, either.
Kernel development is a large responsibility, and pride undoubtedly plays a large role in everybody’s reactions. Reiser, for instance, has spent years developing this filesystem, often through trial and error. Unsurprisingly, he is immensely proud of the result. “Reiser4 represents a huge pile of luck in design,” he says. “I will most likely never get so lucky again in one of my design experiments.” At the same time, he is also aware that the present technical superiority of his work will not last forever, and pleads, “Let me have my 15 minutes of technical relevance.” Such considerations make impatience and bad manners understandable, if not excusable.
If anything, in the current exchange, both sides seem to be straining to stay polite. Perhaps they are tired of the issue. The trouble is, the efforts are not only inconsistent, but probably coming too late.
Summing up
Clearly, any answer to questions about Reiser4’s status is complicated. But briefly — and as neutrally as possible — the reasons Reiser4 is not in the Linux kernel today can be summarized as claims that further testing are required and apparently genuine difficulties with integrating Reiser4 into the existing kernel structure, compounded by philosophical and personal differences. These difficulties may be unique to this situation, or may point to genuine problems in the conventions of kernel development. Either way, they cannot be ignored.
Whatever the case, the controversies surrounding Reiser4’s inclusion are far from kernel developers’ finest hour. Outsiders coming across the discussion could be forgiven for wondering sometimes whether they stumbled upon the Linux kernel mailing list or a group of script-kiddies who have just discovered Slashdot.
Bruce Byfield is a course designer and instructor, and a computer journalist who writes regularly for NewsForge, Linux.com and IT Manager’s Journal.
Category:
- Linux
