March 4, 2010

The Mobile View: Linux Kernel 2.6.33

Last week brought the arrival of a new Linux kernel, version 2.6.33. With it, came quite a few changes likely to interest device developers. So, here is a breakdown of the most significant mobile/embedded updates to come in the latest kernel.

For instance, Android patches were dropped from the staging tree, due to lack of maintenance. Many were surprised that Google has apparently opted to maintain its kernel patches "out-of-tree." However, the ensuing discussions certainly stand to raise the general awareness level around embedded Linux best practices, which generally start with "work your changes upstream."

Some of the advantages of "mainlining" code include:

  • Code improvements from peer review
  • Code contributions and support from the kernel community
  • Seamlessly integrating new Linux improvements, such as new device drivers
  • Simpler merging of any other needed out-of-tree patches
  • Easier bring-up on new hardware
  • A much bigger potential user base, because development is so much simpler

At the same time, there's a long, if not especially proud, tradition of device companies maintaining their own kernel forks. Google certainly has adequate developer resources to take on this extra work load. It apparently plans to "rebase" its kernel approximately every other kernel release, an approach much better than letting "bit-rot" run its course for years at a time.

For his part, Linus doesn't seem fazed by the situation, reportedly commenting (in a Nexus404 story) "I don't worry about out-of-tree development for odd devices too much."

Lots more information about the technical issues in Google's Android patches can be found here, in Greg Kroah-Hartman's analysis, and here, in a follow-up discussion on

New Testing Tools

Device developers almost always modify the kernel in some way, whether writing new device drivers/modules, integrating unique bootloaders, or streamlining for specific targets and/or target functions. Meanwhile, devices are held to a higher standard of reliability and stability than other computing equipment (think "appliance-like" stability). Thus, device developers are always interested in new debugging and profiling tools.

Kernel debuggers range from expensive Linux-aware hardware debuggers (probes and in-circuit emulators) to the Linux kernel's own growing collection of available instrumentation, which can be used with utilities such as ftrace, SystemTap, and so on. In the latter category, Linux' "perf" tool gained quite a few new capabilities in the 2.6.33 release. New perf "subcommands" aim to enable dynamic probing, benchmarking, memory tracing, and more.

Much of the development appears to have been driven by enterprise Linux users, like Red Hat and Fujitsu. However, embedded architectures ARM and SH also gained patches to support some of the new perf tools. The ARM patches appear to have been driven by Intel and Picochip, a company that makes system-on-chip processors for femtocells (small cell phone repeaters). SH patches came from the Renesas-sponsored project.

Additionally, Linux gained considerable ftrace, stack trace, and oprofile hooks for use with Microblaze, a "soft processor" core distributed with Xilinx FPGAs.

New Thin Client RAM-Saver

Devices typically have less memory than servers or PCs, so one new memory swapping technology debuting in the staging tree of 2.6.33 looks promising, especially for thin clients. The "Ramzswap" framework, formerly known as "compcache," lets developers create RAM disks (/dev/ramzswapX) that compress swap cache pages stashed there using zlib compression.

Ramzswap could help Linux compete in low-margin, commoditized verticals where saving a few pennies on bill-of-materials can make all the difference. It could also help thin-client Linux projects like the LTSP support more existing devices and perform better on lower-spec equipment.

The idea is not new. However, ramzswap takes a simpler, less intrusive path than previous approaches, and thus appears to be gaining more acceptance. Still, it requires a new memory allocator (xvmalloc), because compressed pages can be of various sizes. Furthermore, as block devices, compressed ram disks could need a better way to retire stale pages, and even to defrag in certain situations. A good overview (and interesting discussion) can be found on, here.

Another nifty space-saver: LZO compression is now supported on initrd images for several architectures, including ARM, MIPS, and X86. LZO is reportedly much faster on ARM than the typical zlib compression, and that's welcome news to all those device developers looking to shave milliseconds off of boot time.

The Fun Stuff

All the above items represent pretty big items already extensively reported on in the mainstream press. But what about the small features, incrementally bringing Linux to new platforms, new hobbyist/hacker groups, new markets, and the chips destined for use in the devices of the future? Sometimes, the most fascinating news with kernel releases lies "below-the-radar," in the minutiae of architecture tree commits and driver lists. Here is a selection of choice tidbits.

Happy Hacking

Other nifty additions: support for new QNAP and Lacie NAS devices based on Marvell "Kirkwood" (ARMv7) processors, and new camera support for Motorola A780 and A910 EZX phones.

New Chip/EVM Support

  • Patches were merged for the OMAP3630, TI's latest/greatest mobile applications processor, as well as the Zoom3 board based on it.
  • Chipset support arrived for the Marvell Armada 510 (AKA Dove 88AP510), a highly superscalar ARM SoC with 1080p video and crypto decoders, and targeting smartbooks, mids, and tablets.
  • The China-made Loongson 2F processor for low-power computing applications gained basic support, and even CPU frequency scaling.
  • Logic's Zoom AM3517 evaluation board gained support. It's a development board for TI's fairly new Sitara AM3517 SoC, based on Cortex-A8 and targeting industrial computing.
  • Memory controller support was added for TI's OMAP-L127, a TI Davinci SoC with ARM926EJ-S and C674x cores.
  • SMP support arrived for the Sparc-Leon architecture, just in time for Leon4, which gained an AMBA AHB interface. These soft-cores are typically integrated onto logic devices also equipped with various fault-tolerant engines, and used in aeronautics and space exploration applications. Neat stuff!

New Boards and Reference Designs


And then, there are all the various driver additions, which make up fully half of the 2.6.33 change log. It's great to see more and more peripheral vendors working with to ensure the best viability of their products in embedded markets where Linux has become so important.

There are far too many drivers to delve into specifics. But, I can't resist calling out one: the wmc3200 driver adds support for an Intel chip with no fewer than four radio capabilities: GPS, bluetooth, wifi, and wimax. Now that's integration! Of course, the driver is just a shell that loads a blob, probably due to the legal sensitivity of altering parameters in software-defined radios like this.

The above is but a sample of the many patches rolled into 2.6.33 and its staging tree. Other resources for exploration include:

Of course, the best resource of all is to just download the code from, and read through the help screens in menuconfig. Enjoy!!!

Click Here!