Author: Joe 'Zonker' Brockmeier
Michlmayr’s work was sponsored by Google and carried out at the University of Cambridge as part of his research into quality management in free software projects.
He says he spent about two and half weeks building the entire Debian archive with GCC 4.1 on a quad-core MIPS machine donated by Broadcom, and on an EM64T machine from Intel. The results show that GCC 4.1 is “very stable” on MIPS and AMD64, but there are “a large number of packages using code (especially C++) [that] GCC 4.1 treats as errors” in the Debian archives.
In 6,192 packages, the tests uncovered more than 500 new bugs, of which 280 can be attributed to GCC’s increased strictness. Ben Hutchings created a page describing common C++ syntax errors that wouldn’t have been flagged as errors prior to GCC 4.1.
Though the tests uncovered a lot of syntax errors in Debian packages, they revealed few errors in GCC 4.1 itself. Michlmayr uncovered six bugs in GCC 4.1, two of which have already been fixed.
Why MIPS?
It might seem a little odd to spend time worrying about MIPS-specific problems, since most users run Linux on x86 and AMD64 hardware. However, Michlmayr says that it’s actually more important to look at an architecture like MIPS, because “non-mainstream architectures generally receive much less testing.”
So why MIPS and not another non-standard architecture such as ARM? According to Michlmayr, MIPS was chosen because it was possible to get fast hardware to use for testing, and because MIPS is fully supported by Debian.
“It would be nice to perform similar tests on other non-mainstream architectures, but for many of them it’s quite hard to get access to fast hardware,” says Michlmayr. “The ARM platform is an example for which 1.2GHz dual-core development boards are available, but they are very expensive. Broadcom understands the value in supporting Debian’s MIPS port by donating hardware, but we have yet to establish a similar working relationship with ARM licensees, such as Intel.”
Severity of bugs
Though Michlmayr’s work has uncovered quite a few bugs, he says that most of the problems he found were “completely harmless and trivial to fix. Out of the 280 problems I’ve found only a few have the potential to cause real problems. And now that they’ve been identified and reported, we can fix them.”
Primarily, bugs are being found in Debian’s archive because GCC 4.1 is “stricter in terms of conformance to standards.”
For example, Michlmayr said that GCC 4.1 would warn for “extra qualification,” which is when “a declaration within a class explicitly puts the class name before the declaration. Given that the class name is already known, this is an ‘extra qualification’ — however, the C++ standard requires unqualified name.” Extra qualification was far and away the most common error, with 187 bugs found identified as extra qualification.
GCC 4.1 for Etch?
Whether Debian will use GCC 4.1 for Etch is still up in the air. Michlmayr says that the number of bugs specifically related to GCC 4.1 has given Steve Langasek, one of Debian’s release managers, pause.
“He made it quite clear that moving to 4.1 as the default compiler won’t be an option at this stage of the release unless these bugs are fixed very quickly. Many of the bugs I have reported have already been closed, but I’ll see whether we can have a concentrated effort to get the numbers lower.”
This probably won’t be the last quality report from Michlmayr, either. He says that he’s established a good contact with the GCC project’s QA person, Andrew Pinksi, who’s “very keen that I continue my work.”
To that end, Michlmayr wants to do more work with GCC 4.1 on non-mainstream architectures if he can get fast hardware, and provide more feedback to the GCC team. “Obviously, Debian is a great test case because of its sheer size.”
Category:
- Linux
