November 14, 2011

Tuning Embedded Linux: When Less is More

There's a saying that you can never be too rich, or too thin. While that's a bit of hyperbole, thin is definitely in when it comes to embedded Linux. Luckily, trimming the fat off Linux for embedded use is a lot easier than getting rich or losing that spare tire. Intel's Darren Hart explained how he slimmed Linux down at the Embedded Linux Conference in October.

Hart says that the objectives were to reduce image size, memory use, and minimize boot time. Why? If you can save 2 pennies on a chip, for devices that ship in the millions it's worth it. He also notes that smaller images use less power — which is, of course, deeply important to users. (And vendors definitely like to tout better battery life.) This also means that there's better performance all-around.

Darren HartSome of the real-world examples that Hart cited, taken in part from a poll on the Yocto mailing list, were things like digital cameras and medical devices with very minimal resources. A digital camera might only have 10MB of memory – who wants to wait a long time for a digital camera to boot? You want to be ready with the camera as quickly as possible, so the image size is deeply important.

Other devices, said Hart, are booting over the network and require the smallest possible image.

Goal Weight

How slim is slim enough? Hart says that the goal was to get down to a Kernel and RootFS that are less than 4MB and that can boot with less than 8MB. They should boot to the shell in less than 2 seconds. "From power on to login," said Hart. That's right, 2 seconds.

Hart also said that "it's cheating" to turn off networking. In 2011, nearly every device should be able to have networking capabilities.

Where We Are Now

Minimal image in Yocto, said Hart, is 4MB kernel image. It has a 11MB RootFS, for a total of 15MB. Boot time was 9.5 seconds to a login. The RootFS includes files necessary to boot, libraries, applications, and the Linux kernel.

The "guiding principles" said Hart are to go for the "low-hanging fruit" first, the 90% that is easier to get rid of and not the "hairy" stuff that can take forever to optimize. The budget that Hart came up with includes 1MB for the Linux kernel and 3MB for the RootFS.

Interestingly, Hart says that he focused in his talk only on configuration to slim everything down rather than source modifications. According to Hart, there's enough to do in the source configuration.

Hart approached the storage by blocks, rather than bytes. Why? Hard said it doesn't matter how many bytes are used, it's how many blocks "that's how the disk is measured."

Hart then spent a fair amount of the talk covering the tools used and his methods for slimming down the image, and some of the larger pieces that he removed that weren't necessary. For example, Hart said that he got rid of Udev and v86d, since he didn't need things like dynamic device support or support for VGA. He was also able to get rid of 1MB by getting rid of the ext journal. (Especially since most devices won't use Ext in the end, anyway.)Kernel Size by Stage

Those things dropped the size by 7MB, and dropped 2.3 seconds from the boot time with a two-line change in the configuration.

Next he looked at built-in drivers. According to Hart, 66.53% of the kernel image was composed of drivers, networking, filesystems, sound and the core kernel. Naturally, you don't need a lot of that for an embedded device, so he went hunting for things above 500KB to get rid of.

Networking, for example, loses 131K for wireless. If you don't need wireless in a device, that's a ripe target to get rid of. Likewise, NFS consumes 230K, which may not be necessary for many devices. Quota support, that's 22K you may not need. Sound? That's nearly 700K. If you don't need sound, that's also ripe to be ditched.

Hart continued by stages, talking about the various things that were removed with each pass. By stage 5, he had the kernel "stupid small" with a total of 1.6MB. Ultimately, Hart was able to get below two seconds to booting the kernel and getting to login. The functionality, though, was correspondingly minimal – and perhaps less than one might want in a production device, even an embedded device.

Stage 4, though, is "what's relevant." Here, the kernel is less than 4MB and boot time right at 2 seconds.

Next Steps

What did this do? Hart says the outcome is "Meta-Tiny," an experimental layer for tweaking configs without mucking with original sources. Next, he says that they should restore "netbase" to the image, and split up udev packaging. Hart also says that Busybox needs a better way of exposing its configuration options, since he had to write his own tools to slim it down further.

He also says that one tiny-config option probably won't be adequate for the wide range of devices that are in use. Because there are so many options, Hart said that there need to be several tiny- configurations that match different hardware types.

For more information, check out the Yocto Project and the git repository on the Yocto repo. Naturally, you'll also want to watch Hart's talk as well if you're deeply interested in tuning embedded Linux. Many thanks to Thomas Petazzoni for posting the talk!