While Fedora is historically dominated by Red Hat and remains dependent on its resources, the project is evolving toward serving the needs of its community as well, Spevack says. Achieving the goal will not be not easy. Fedora has only just emerged from a chaotic beginning, and faces several ongoing challenges, including establishing the rules by which the community governs itself, continuing to balance Red Hat's and Fedora's interests, and improving software development policies -- all while encouraging greater community involvement. Yet while acknowledging the difficulties, Spevack remains firm in his faith that the necessary leaders can emerge from the community and that rational people can find solutions to disagreements, if only they talk long enough and honestly enough.
The history of Fedora
The Fedora Project started in 2003 as the community-based source for Red Hat Enterprise Linux (RHEL). Almost immediately, the project became the center of controversy. Many observers saw the replacement of Red Hat Linux with RHEL as proof that Red Hat was abandoning its community roots in favor of purely commercial interests. In addition, Red Hat's trademarking of the name Fedora was challenged by a similarly named FOSS project at Cornell University.
Even more importantly, although releases were getting done and sub-projects were emerging, the organization of the project was confused. Talking at Linuxfest Northwest last month, Greg DeKoenigsberg, Community Builder at Red Hat, said that the Fedora Project still seemed disorganized when he began his job in October 2004 -- a year after the project was founded.
Spevack was not employed by Red Hat when the Fedora Project began, but he acknowledges, "It's pretty obvious that we didn't handle [creating the Fedora Project] very well at all from the community perspective. There was a stretch where we were trying to build a community and we weren't sure about the right way to do it. We were waiting for community to happen, as opposed to taking the leadership needed to grow it."
Today, Spevack says the situation is vastly improved. "In the past year," he says, "tremendous strides have been made." He gives much of the credit for the building of the Fedora Project to the initiative of community members like Thomas Chung, the creator of the FedoraNEWS.org community news site.
Looking back, Spevack sees the gradual emergence from chaos as part of an inevitable evolution. He is now considering the next steps in that evolution.
Adding democracy via the Fedora Board
One of the most recent steps in the development of the Fedora community is the creation of the Fedora Board. In June 2005, Red Hat announced its intention to create the Fedora Foundation to help organize volunteers and to serve as a repository for copyrights and patents.
Spevack and DeKoenigsberg argued bluntly and passionately against the Foundation in a series of discussions in forums and within Red Hat. They suggested that the Open Invention Network would provide the repository, and that a foundation could do nothing that an elected community board could not. As a result of their efforts, the Fedora Board was announced last month and the Foundation was folded.
The board consists of five members appointed from within Red Hat, four from the community, and Spevack as chairman. Two facts stand out about the community members of the board: They are drawn from aspects of the project that tend to have been overlooked, such as documentation and the KDE packages, and several are involved with CentOS, a distribution based on RHEL.
According to Spevack, selection of board members with diverse backgrounds is intended to bring a wide a range of voices to the administration of the process. "We're interested in working with everybody to make Fedora better," he says. "We're not trying to pick and choose whom we work with." In addition, Spevack hopes that the diversity will "make sure that we do a good job of remembering that we need to meet the needs of folks out there who are using different types of software than we are."
Spevack is especially interested in encouraging distributions like CentOS that are based upon Fedora or RHEL. "I would hope that these people would think of Fedora Project as their upstream, and that they would want to help us," he says. "Because that ends up being a direct benefit to them."
Discussions are underway on the fedora-advisory-board mailing list now about how the initial board will be replaced by an elected one. Although Spevack foresees that, at least in the short term, the chair of the board will likely remain "whoever happens to have my job at Red Hat," no other positions will be reserved for Red Hat employees. Instead, the rest of the board will be elected through a process that is still being discussed. Spevack suspects that members will serve for two Fedora Core releases -- in other words, about a year -- and that half will be replaced at each election -- "so that [we] keep institutional memory," he explains.
Another issue still being discussed is who will be eligible to vote for board members. Unlike Debian, which has a formal process for accepting new developers, Fedora seems to be leaning toward a looser set of criteria. Programmers and debuggers should be obvious because of their contributions, but other eligible voters may be determined by who is registered with the project wiki. If problems arise, Spevack says, "We'll just have to make it clear well in advance how that's being decided. Then we'll deal with the questions or special cases as we go along."
Balancing between company and community
One of Spevack's major concerns in the development of the Fedora Project is how to balance the interests of Red Hat and community members. On one hand, Red Hat has a direct commercial interest in the Fedora Project because of its substantial investment in it, and the fact RHEL is a snapshot of Fedora Core. On the other, some members of the community may have resentments due to some of the confusion in the past, or perhaps because of FOSS culture's distrust of business.
However, Spevack says he feels that a combination of common interest, openness, and rationality can help to overcome any distrust on either side. "My own personal view on how to deal with conflict is by simply being honest and having long conversations," Spevack says.
According to Spevack, one of the main tools for avoiding conflict is the new board. At board meetings, he says, community members "get to hear firsthand about a lot of the things going on inside Red Hat." He also suggests that the direction for each release can be settled by comparing the lists of the company and community priorities and discussing which to pursue based on resources and time. "I'm not going to sit here and say there's some sort of foolproof way in which all those decisions are made," Spevack says, "but bringing some more clarity to that [process] and giving the community transparency into why these decisions are made -- that's the challenge. Dialog, I think, is an important thing to have,and bringing some of that to life is part of the community challenge here."
And if a conflict ever arose between Red Hat's and the Fedora Project's interest? "It would get resolved at the level of the Fedora Board," Spevack replies. "We would have a very frank discussion. If there's a big enough conflict, we just keep talking about it, all the way up to [Red Hat CEO] Matthew Szulik. We'd let him understand what the different conflicting viewpoints are and see what he thinks." If necessary, he would defend the community interest in much the same way that he did with the creation of the board, by making his position clear and encouraging public discussion.
But Spevack thinks a major conflict unlikely. Everyone, he says, should be able to see that "Fedora is progressing with each release" and it is in everybody's interest that the improvement continues. At any rate, he adds, "If there's anyone in the Fedora community, whether they work for Red Hat or anyone else, who can make an intelligent and well-reasoned argument about why something is a good decision, I'd like to hope that that decision gets made [and that] other people will see that it's right."
Revising development methods
Having been a quality assurance engineer and the QA team lead for Red Hat Network, Spevack is especially concerned with improving testing for the Fedora Project. In fact, Red Hat has recently hired someone to spearhead this effort. "I don't envy him," Spevack says, "because the position has two very demanding and separate goals."
The first goal, Spevack explains, is "to figure out on a technical level what's going on with Fedora QA. The second -- and the absolutely critical part of the job -- is to lead other Fedora folk in the building of a strong Fedora QA community. We need community members who are helping with bug triage, and we need community members who are helping us figure out hardware compatibility. We are [also] going to need some code written around that." Some of that code, in Spevack's vision, might include a tool to report hardware configuration to the project's QA team.
Even more importantly, Spevack wants to encourage a closer coordination of efforts between Fedora Core, the main package repository for the project, and Fedora Extras, the community-based repository that complements Fedora Core. Although he downplays the divisions that others see between the two repositories, Spevack considers Fedora Extras a main source of fresh ideas for the project, praising what he calls "an explosion of leadership out of Fedora Extras." In particular, he cites the fact that the Fedora Extras package guidelines are now being used within Red Hat. "A lot of people say that Red hat does everything and the community doesn't get to be involved," he notes. "Yet here we have an example of the complete opposite."
In addition, Spevack would like to work toward a common build system for the repositories, and to establish policies that would allow non-Red Hat employees to make commits to Fedora Core. "I don't like the idea that there's a giant repo of packages that the community can't touch," he says, "because in some cases the community can do a much better job. It's silly that we can't take advantage of this."
Within the next year, Spevack would also like to see a major change in how the Fedora distribution is built. "What if you had a big pile of packages that were all Fedora," he asks,"And you could go in and say, I want to build myself a Linux distribution, and I want KDE and not GNOME by default, and I want, say, a Web server and a DNS server -- and that could be your version of Fedora? And someone else who wanted GNOME could have their own version?"
He is referring to rPath's rBuilder and Conary, or their equivalent, which combine version control with package management. A switch to Conary might cause some resentment within Red Hat, since many of rPath's employees were once leading figures at Red Hat. If Fedora does go down that path, it would make it the first major distro to incorporate the technology.
Lessons from other communities
Looking beyond the Fedora community, Spevack says he would like to see closer ties with other FOSS communities. He talks, a bit wistfully, of getting representatives of different distributions together to talk about issues such as common package management. After all, he says, "it's the open source mentality that we are all champions of. Different distributions are just different interpretations of that mentality."
Most of all, in looking at how the members of other communities interact, Spevack says that he has learned important lessons about the need to balance community involvement with leadership. "You need to know that somewhere in the community there's someone who's ultimately given the ability to make a decision and will also be help accountable for that decision. Waiting for consensus isn't the best way to drive forward progress. You have to listen to different ideas and then you have to make a decision."
Bruce Byfield is a course designer and instructor, and a computer journalist who writes regularly for NewsForge, Linux.com, and IT Manager's Journal.