GStreamer framework eases development of media applications


Author: Stephen Feller

The team behind the GStreamer media application framework, a library for developers, recently launched its stable series with version 0.10.0, and announced plugins to support files protected by digital rights management (DRM) software and for playback of the widely used MP3 format.

Developers of the framework toiled for 18 months leading up to the new release, which they tout as having improved playback and video seeking functions, support for real-time transport protocol, and fraction-based frame rates, among others.

Fluendo, the company that supports and develops significant portions of the project, announced in a press release last month that a plugin to support MP3 playback would be immediately available for download. Although MP3 plugins are already available for GStreamer, the company said that its new one would work “out of the box” for many open source audio applications built on the framework, which include amaroK, Totem, Sound Juicer, Annodex, and PiTiVi.

The eventual addition of DRM support will be as a “separate helper library using special plugins,” according to a December 7 post from GStreamer developer Christian Schaller to the Planet GStreamer blog, as well as to his own blog. The intention, he wrote, will be to enable playback of both DRMed and non-DRMed content, and will be in clear view of both programmers and end users, unlike the Sony-distributed software that caused an uproar late last year.

The more than five-year-old gStreamer project is a library of plugins for a variety of audio and video formats, devices, and hardware. The library allows multimedia software developers to work on applications by creating “media pipelines” that connect files and resources to the hardware required to play them, said GStreamer developer Andy Wingo.

Wingo is part of a small team of about half a dozen developers at Fluendo. GStreamer also benefits from 30 or so other developers, he said.

According to Wingo, much of the attention GStreamer has been receiving, as well as its development in the last few years, has been helped by automated builds, test suites, full bindings for Python, and API documentation worked on and distributed by Fluendo. He said, however, that the real growth of GStreamer comes as much from the applications developers have built on it as it does from Fluendo itself.

“The abstraction provided by GStreamer also allows developers to cooperate more effectively,” Wingo said. “Once someone implements a GStreamer plugin for a format, every application has access to that format, transparently.”

Although GStreamer is used for applications on a variety of Linux distributions, the project is most closely associated with GNOME and several pieces of multimedia software distributed with it, GNOME Foundation board member Murray Cumming said. A plugin-based framework is the first step along a path to solving the multimedia problem in Linux, said Cumming. Cumming, who also is a maintainer of gtkmm, the C++ interface for the GTK+ library, said the problem with audio and video players in Linux distributions comes down to licensing for the software and for use of the formats.

According to Cumming, proprietary licensed decoders can be difficult for open source and Linux developers because of restrictions on redistribution and the release of source code, which can complicate developers’ desires to provide such applications for users. But, he said, libraries like GStreamer that can eliminate the need for two players, one for open formats and one for proprietary formats, make it easier for both developers and users.

“The solution is to allow use of closed-source decoders [and] encoders by open source software,” Cumming said. “That needs a well-thought-out plugin system in a multimedia framework that’s good enough to be used by most developers. GStreamer is almost there, so proprietary companies can install proprietary GStreamer plugins, allowing free software music and video players to support those formats.”

Mark Kretschmann, a developer on the KDE-based music player amaroK, said the project had switched from using the aging and largely dormant aRts framework because GStreamer is updated and maintained more often, as well being more versatile.

In addition to being surprised by the framework’s flexibility, and that the pipeline system for designing applications can be constructed to do just about any multimedia task, Kretschmann said the developer community had been helpful to him and others on the amaroK project — which he pointed to as being especially beneficial for any open source project.

Although Wingo cautioned that it may be premature to say GStreamer has become a convergence point for Linux media player developers, he said the framework is the only tool of its kind out there right now. The roughly 60,000 lines of code developed for GStreamer itself, and the code from all of its plugins which he said as much as quadruples that estimate, is part of the reason he said the well-developed framework wouldn’t be surpassed any time soon.

“I don’t see anything replacing GStreamer in the next two years at least,” Wingo said. “With the range of formats we support [and] our growing set of language bindings, someone writing a media-handling application in 2006 would be a fool not to use GStreamer.”

Cumming said the reason GStreamer is being picked up by developers and garnering attention, aside from the recent release, is because it has helped to make video and audio playback “just work,” without requiring strange extra steps to gain those functions.

With support for both free standards, as well as proprietary ones, and by making content creation tools easier to develop and use, GStreamer is turning Linux into a “creative platform” that can be used to create new types of media. And more importantly, Cumming said, it lets Linux users take advantage of the formats and media that are already out there.

“We can’t help that people have some MP3 files instead of Ogg files, and we can’t help that you must pay money to play MP3 files,” Cumming said. “But we can make it easy to pay that money and play those files.”