March 22, 2006

Resynthesizer changes your pictures

Author: Nathan Willis

Paul Harrison's Resynthesizer is a texture-synthesis plugin for the GIMP. Sure, you can use it for simple things like creating seamless, tileable textures, but you can also have it remove any object from a picture and automatically replace it with a synthetic background, with impressively professional-looking results.

To get started, visit Harrison's Resynthesizer page and download the package. The latest release is 0.13, currently available as source code for Linux and a compiled binary for Windows. The 0.12 release is available as a binary for Mac OS X, and can also be found pre-packaged with Gimpshop.

To compile the plugin for Linux, make sure you have the libgimp-dev package installed. It should be available on virtually every distro. The entire Resynthesizer tarball is smaller than 18KB, and should compile in a matter of seconds. Once installed, you will find Resynthesizer under the GIMP's Filters -> Map menu.

Up with texture

Basic usage involves creating a tileable texture from an image. Begin by selecting the portion of the image you want to work with (or select nothing if you want to tile the entire image). Open Resynthesizer from the Filters menu, check or uncheck your horizontal and vertical preferences under Output, and click the OK button.

Moments later, your tile will be ready. The simplest way to take a look at it is to choose the Tile filter from Filters -> Map and have the GIMP automatically paste together a few tiles.

There are, of course, other ways to make a tileable texture out of an image. The Texturizer plugin, for example, does just this, though it offers fewer options. The Texturizer site features a large gallery of before-and-after images, however, so you can compare the results it provides with those of Resynthesizer.

For my money, Resynthesizer does a better job of recognizing large features and preserving them in its output -- you can tweak its sensitivity to "outliers" (as it calls them) in the dialog box, while Texturizer offers no such control. Resynthesizer also creates its textures from the entire image area, whereas Texturizer only seems to create tileable edges.

While Texturizer always creates the same tile from a given input, Resynthesizer randomizes the contents of the image somewhat. This can noticeably rearrange image features, but it also means you can run the filter multiple times to get results you may be happier with.

But chances are, unless you are a game designer, you probably won't spend a significant portion of your life making textures. On the other hand, everybody and his brother takes photographs, and Resynthesizer has one big trick up its sleeve that you will want to see to believe.

How to get kicked out of the Associated Press

Click for Before
Click for After

Now, you can't get away with it if you are a photojournalist, but sometimes the rest of us need to remove something -- object, person, or obstruction -- from a picture. What if you finally get the whole family together for a portrait, but Cousin Emmett won't sign the release? You can take him out.

The old-fashioned way of doing this relied on the clone or rubber stamp tool, but the process is arduous and the results are inevitably sloppy, because removing an object demands painting over it with realistic background texture, and that is almost impossible to do by hand.

But creating texture is right up Resynthesizer's alley. Along with the basic Resynthesize menu item I mentioned in the previous section, Harrison has included a filter called smart remove selection that you will find under Script-Fu -> Enhance.

It is alarmingly simple to use. Draw a selection around the object you wish to remove from the picture, and run smart remove selection. Resynthesizer will fill the selection area with intelligently generated texture drawn from the surrounding image data. There is one option to set: the radius from within which Resynthesizer will sample as it creates texture. Trial and error can determine the best value for a given scene.

In this example, I decided for aesthetic reasons to remove my friend Matt from this otherwise appealing shot of the Appalachian Trail. You can see the original shot in the first picture, and the enhanced shot in the second. No other alterations were made.

Sure, it's not perfect. If you know where to look, you can detect some visual artifacts. But the quality is superior to the results of the clone tool alternative, and it is far faster.

I had intended to end this article with a comparison between Resynthesizer and closed-source alternatives, but there are no commercial equivalents to speak of. Texture synthesis is well-researched academically, but far from solved.

The plus side for those of us using open source software is that much of the leading research is available as free software first. Resynthesizer is but one example; the further reading on Harrison's personal site links to many more.

Click Here!