Author: Jay Lyman
The Open Source Development Labs announced this week its major upgrade to version 3.0 of the Scalable Test Platform (STP) testing suite, promising a broader range of features for developers to simulate real-world datacenter environments on the Linux kernel and to check the workload impacts of databases and more. So what’s new in version 3.0?
OSDL Acting Director of Engineering Craig Thomas said the main area of focus in the upgraded STP was allowing the test framework to keep track of patches applied to any open source application as well as the kernel.
“Before STP 3.0, the framework only managed patches applied to the Linux kernel,” Thomas said. “This new feature allows other open source developers to try a patch against a known source tree for their application, compile the code through a variety of cross-compilers, and run tests against the patched application on a variety of hardware in OSDL’s lab.”
Thomas explained the advantage of STP is that it allows a developer to create an experimental build, and then obtain results from a variety of tests run against it.
“We’ve also changed the interface to make it easier to use,” Thomas said. “We have organized the selection of our tests into categories so a tester can focus on some element of the software package. Right now, we have the tests categorized for elements of kernel testing: general testing, memory, database, performance, file system, and scheduler.”
Thomas said other improvements are smaller, but help the more advanced tester. For instance, in addition to specifying a LILO string for booting, STP users can now specify custom environment variables and sysctl settings.
“We’ve also done a fair amount of bug cleanup and feature requests that were submitted on the STP SourceForge project page. In the spirit of open source, we have been updating and releasing improvements since the 1.0 release. We have also been adding new test cases and we are working to add more.”
Thomas, who said the number of STP users has not really changed since STP 3.0 was rolled out two weeks ago, indicated it will take time for others to join in and use the framework.
Right now, the tests running in the STP framework deal mostly with performance, according to Thomas. He said these tests range from file system I/O, to the scheduler, to database performance.
“We also run the functional tests such as LTP (Linux Test Project) with each Linus [Torvalds] and Andrew [Morton] kernel release,” Thomas said.
As for what’s still brewing in the STP framework, Thomas indicated there is much.
“After all, this is an open source project and we have a huge list of things we would like to accomplish, but they will take time,” Thomas said. “One of the items we are now working on is the ability for tests in the STP framework to control multiple systems. This would open up the realm for us to add client/server tests, networking tests, and multi-tier database tests in an automated fashion.”
Among smaller STP endeavors is work on the reports of some of the tests themselves to simplify the output and make it easier for a tester to analyze the data, according to Thomas.
“Of course, we are always looking for and working on the creation of new tests,” he added.
Thomas, who said the upgraded STP would open Linux testing to a range of new developers, also solicited other open source projects to be tested with the framework.
“The new capability added to the STP 3.0 release allows other open source project developers to try out new patches to their code and test it on systems ranging from a single processor to eight processors,” Thomas said. “Right now, we are testing the Linux kernel, a utility called systat, and PostgreSQL. We would like to offer other projects the opportunity to be added to this list.”
