Author: Bruce Byfield
Woods hopes to prevent such dismissals by introducing Red Hat’s internal testing tools to Fedora and by organizing formal testing inside Fedora. By accomplishing these goals, Woods hopes to encourage more large-scale and thorough testing throughout the free software community at large.
Woods never set out to become a quality assurance expert. First introduced to testing at Compaq and IBM, Woods gained enough experience that he was hired two years ago when Red Hat needed someone to develop test automation tools. “It just turned out to be something I was good at,” he says. “I’ve sort of fallen into this role, and it worked out really well for me.”
During his time at Red Hat, Woods has seen a rapid evolution in quality assurance at the company. “Everybody recognized that there has been a problem with it,” he says, “but we have seen an upsurge in the quality of the stuff we’ve been releasing for RHEL.”
Woods believes that this improvement is due largely to increased automation. He notes that each version of RHEL is supported for seven years with quarterly updates, and available for seven different architectures. “The amount of stuff we have to test has multiplied at an incredible rate,” he says, “but we’re still staying on top of it. I attribute that to the automation work that we’ve done.”
However, little of the improvement has reached Fedora yet. Currently, “testing for Fedora is more ad hoc,” he admits. “We have a list of things on the Fedora wiki that should be tested, people work through them and everything gets tested, but it isn’t official.”
In his new position, Woods is tasked with changing that situation. “Red Hat signs the pay check,” he says, “but basically I work on the Fedora project exclusively.”
Integrating the Red Hat internal testing tools
As announced at the Red Hat Summit 2006 in May, Red Hat is releasing its internal testing tools to the free software community under the 108 project. One of Woods’ first priorities in his new position is to introduce these tools into Fedora development. Improving testing upstream from RHEL makes logistical sense for Red Hat, Woods explains, but is also a way of introducing the tools to the community. “Once something’s in the Fedora project,” he says, “it’s fully a part of the open source community, and is available for any other distribution or any other open source project.”
One of the tools Woods plans to introduce into Fedora is dogtail, a GUI test tool and automation framework written in Python. The basic software was already freely available before the summit, but Red Hat is now releasing some of the scripts it uses in-house with dogtail.
Even more importantly, Red Hat is releasing the soon-to-be-renamed Red Hat Test System. “It’s the automated test system that we’re using for all the enterprise Linux products and pretty much everything we can get our hands on,” Woods says. “There’s regression tests and functionality tests that we run on every tree that gets built. The first task I’ve been given is shepherding along this automated test suite, and finding the tests we can use” — some of the internal tests, he explains, being the property of Red Hat’s partners. When this initial work is done, Wood is charged with setting up within Red Hat a test lab dedicated entirely to Fedora.
For Woods, making these testing tools generally available is an essential first step in improving testing in both Fedora and the greater free software community. Compared to all the free development tools available,” he says, “the tools just aren’t there for software testing.” Testing in the free software community, he says, is currently comparable to what development would be like if even basic tools like diff didn’t exist.
Those that are available, he adds, “don’t gain the same exposure. If we can help to provide better tools, then things will change. That’s really what we want to address.”
Organizing Fedora testing
So far, Woods has found a handful of people who are interested in participating in the Fedora Testing project. He welcomes the involvement of these dedicated testers, foreseeing that they will be useful in developing test plans and improving the automation tools. However, considering the amount of testing required for a release of a distribution, he doubts that the dedicated testers will be sufficient for the required tasks.
At the same time, Woods recognizes that getting more people involved is difficult. “Software testing is not glamorous at all,” he says frankly. “There’s no Linus Torvalds of testing, so there isn’t an awareness of it.”
Woods is placing his hopes for recruitment in the fact that the barriers for getting involved in testing are much lower than those for development. “Any time you’re using software, you’re testing it,” he notes. His goal is to encourage many members of the Fedora community to contribute small bits of time to release test plans.
“What I’m hoping,” he says, “is to reduce the barrier to entry for formalized software testing. What we want to do is set up things so we have, for example, a Web site with a whole lot of test plans, so that users can test individual apps. If we can get everyone to contribute five minutes of their time, that’s going to all add up. We’ll have way more testing going on than we could ever hope to get from one guy who is a rock star tester.”
Woods acknowledges that this type of testing may not lend itself to the sort of peer review found in free software development. However, he anticipates few problems with the fact. “I don’t see the incentive for someone to test badly,” he says. “That’s not a problem, because, even if something goes wrong, the effect on the whole of the project would be small.”
When he has time, Woods would also like to educate members of the Fedora community about the importance of testing, but has no specific plans to do so yet. “That will come in time,” he says. “Once we start applying the principles we’re talking about here, I think we’ll get the chance to talk to people more.”
Setting the roadmap
When interviewed, Woods was drafting the roadmap for Fedora Testing, so that it could be recognized by the Fedora Board as an official project. He expects the first of Red Hat’s internal tests to be available shortly, and anticipates integrating a fuller set of them into the Fedora Core 6 test releases later in the summer. “We’re going to be running them basically through the same tests we run RHEL through,” he says. After the release of Fedora Core 6, he plans to move the testing lab outside of Red Hat to make it generally accessible to everyone in the Fedora community.
As soon as possible, Woods hopes to become fully integrated into the Fedora release cycle. “The Fedora release team has a set of processes for a release,” he says, “and I anticipate that I will end up being at least informed, if not involved in that process. If something is going horribly wrong with testing, I’m going to tell them. It’s in the interests of the community to change schedules and adapt to that sort of thing.”
Contrary to those who dismiss Fedora on Slashdot, Woods expects no pressure from Red Hat if testing delays a Fedora release. “Obviously, there’s obligations to RHEL,” he says, “but because there’s so much sharing of code, any problem that Fedora ends up having will be a problem for RHEL. So, any time that we find a problem in Fedora that’s big enough that we have to rearrange the schedule, we’re going to have the full weight of Red Hat’s testing and development organizing working on it. So I don’t anticipate there ever being a huge problem.
“I really want people to think of testing as part of development,” Woods says. “That’s a common theme in proper software engineering. Once something matures to a certain point, you have to start doing rigorous regression testing and things of that nature. I think the open source community needs some of that. It’s not that I really need to tell people — people know that testing is important. My real goal is to make testing easier.” By integrating Red Hat’s internal testing software into Fedora and bringing rigor to Fedora’s testing, Woods hopes to bring some of that ease not just to Fedora, but to free software in general — and, incidentally, to silence Fedora’s detractors once and for all.
Bruce Byfield is a course designer and instructor, and a computer journalist who writes regularly for NewsForge, Linux.com and IT Manager’s Journal.
Category:
- News
