August 8, 2006

Processing RAW image files on Linux

Author: Nathan Willis

The easiest way to sound like a professional when you talk about photography these days is to grumble about the deficiencies of RAW file converters. The ability to save pictures in RAW format rather than JPEG or TIFF is what distinguishes a "pro level" camera from a consumer device. But rather than mentioning your camera's specs outright, the shrewd move is the heavy sigh followed by lamentations about your tireless search for the perfect software to properly manipulate your beautiful work. Let's take a look at how to do it using Linux and open source.

For starters, it is important to understand just what is meant by the term "RAW image file." Despite how the term is generally used (in the singular), RAW does not designate a file format. Rather, it is an umbrella term describing multiple file formats controlled by multiple hardware manufacturers. It is generally seen in capital letters, a convention probably meant to distinguish the term from the generic adjective raw, meaning "not yet edited."

That said, RAW does refer to image file formats not yet processed. The RAW formats are supposed to contain unaltered data from the camera's CMOS sensor. In reality, there is always a small bit of processing done in-camera, to remove dead pixels and so on. But the RAW formats do contain high bit-depth readings, uncompressed, and not yet remapped for white point compensation or other alterations eventually performed to tweak the image for display.

The idea is that interactive software can help users perform all these tweaks, taking advantage of adjustable algorithms to reduce noise and recover data from washed-out or underexposed areas (turns out we humans are better at that sort of stuff than our machines). In addition to adjustments themselves, there are multiple, competing algorithms for converting RAW to standard RGB, and there is no consensus on which is best. Capturing in RAW lets you experiment and try them all.

Every perfectionist knows you should always preserve the original and makes changes on a copy. Since the RAW file represents the original as seen by the camera, any camera that saves photos to its flash memory only in JPEG or TIFF format is discarding the original and handing the user an inferior, degraded copy.

File under fire

The RAW formats themselves are specific to digital camera manufacturers. Canon's current format uses the extension .cr2, a successor to the preceding generation's .crw. Nikon uses .nef, Minolta .mrw, Fuji .raf, Pentax .pef, Olympus .orf, Hasselblad .3fr. Although most are based internally on a variation of the hyper-extensible TIFF, all are proprietary and subject to change with each new camera model, and most are officially undocumented.

There are two RAW formats not produced by camera manufacturers. Foveon's .x3f was created by chip maker Foveon for its special image sensors, found in cameras manufactured by Sigma and Polaroid. In 2004, software giant Adobe announced it was launching a new RAW format called .dng that would serve as an umbrella format for all of the others. The company produced a free DNG Converter application, but as of today DNG has not taken over. Many see the DNG move as a blatant attempt to take over the RAW format market by a jealous software house not currently a player -- and for most digital SLR owners, since DNG is not the format their cameras actually capture in, it is nothing more than another (unnecessary) intermediate format.


Many photographers find the glut of undocumented, proprietary, and incompatible RAW formats a royal inconvenience. The OpenRAW project is an initiative to persuade camera manufacturers to publicly release full specifications for their formats, easing the jobs of photographer and image editor application programmer alike.

Freedom road

Fortunately, no special procedure is necessary to import RAW files from your digital camera in Linux. As long as your camera model is supported (newer models will experience some delay), you can import images through software such as the gPhoto suite.

Gtkam is the interactive front-end for gPhoto's background library workhorse libgphoto2, and is simple enough to use. You connect your camera with a USB cable, launch Gtkam, and the application will bring up a browseable list of the images on board. An alternative to Gtkam is command-line tool named gphoto2. KDE supports libgphoto2 as a kio slave, and there is even a FUSE implementation of libgphoto2 to let you mount your camera directly into your filesystem.

However you get your images off the camera and onto the Linux PC, the real work is converting them to a file format usable in image editors like the GIMP, CinePaint, and Photoshop. Here the major player is a free software project called dcraw, which is a small and fast RAW file converter that supports most major camera manufacturers' formats. Developer Dave Coffin puts his own time into supporting new camera models as they come out, and implements changes to the formats as the manufacturers update their own tools and firmware.

Dcraw can be used from the command line, but for a friendlier user experience, check out UFRaw, a GUI tool that uses dcraw for image import, but offers the user a wide array of adjustment tools. UFRaw can be launched as a standalone program, a GIMP plugin, or run non-interactively as a batch processor. It is GPL-licensed and probably available via your Linux distribution's package management system. If not, you can download sources and compile it for yourself; dependencies include the GIMP and libgimp for basic functionality, and LittleCMS and libEXIF for enhanced features.

Using UFRaw to extract the perfect image from your RAW files is an often subjective process that depends on a multitude of factors. UFRaw contains essential RAW workflow features such as color management, exposure control, white balance correction, multiple interpolation algorithms, and fine control over individual channels.

When called from within the GIMP, the completed import is opened as a GIMP file -- meaning that it is limited to the 8-bit color depth of the GIMP itself. As a standalone app, UFRaw can save to 8-bit or 16-bit PPM, 8-bit or 16-bit TIFF, or JPEG.

But don't let its complexity discourage you from getting started! UFRaw can import and convert your RAW files even if you don't know what white balance means or what RGB stands for. The histograms and adjustment curves are intuitive and provide live feedback in the preview window. You can play with the settings and observe the results -- in fact, that is the only way to gain experience.

But wait, there's more!

Although I have more experience with UFRaw, I should also mention a similar open source project, Rawstudio, which also builds on the functionality of dcraw. Rawstudio is a newer project (launched earlier this year), but it offers many of the same features as UFRaw. As yet Rawstudio is still young, and new releases are available only as source code tarballs.

In addition to free software tools, there are proprietary options available for Linux as well. Bibble Labs produces Bibble Lite and Bibble Pro tools for Linux, Mac OS X, and Windows PCs. Bibble is a highly regarded RAW converter on the other two platforms; it is routinely compared to the high-priced Aperture and Photoshop tools. It is shareware, with the two versions priced at $70 and $130 respectively. A no-cost but closed source alternative is LightZone, a Java application produced by Light Crafts. Finally, Google's Picasa supports RAW files, but of course its image-editing features are slim.

The big question in choosing a tool is which RAW converter produces the best images. Well, there is no end to that argument. The software houses are (naturally) biased, and the proprietary software photographers fill page upon page with discussions after poring over the details of every update to the major commercial RAW importers, without reaching a consensus. Each application is better than its competition in some area, but there is no be-all and end-all, hence the constant hemming and hawing of the professional photographers.

Dcraw and its derivatives produce results as good as what you will see on Macs and Windows boxes. On Linux, we have fewer native RAW choices, but we do have choices. Aren't you glad you, too, can hang your head and bemoan the neverending search for good RAW file support?