September 14, 2004

How to migrate from Ximian Evolution to Mozilla Thunderbird

Author: Nathan Willis

I've always liked Mozilla Mail -- when I have to use Windows, it's my preferred email client. So naturally I've kept an eye on the Mozilla Thunderbird standalone email application and given it a test drive or two. When the latest release hit the Internet a few weeks ago I decided that perhaps, as with the Firefox browser, it was finally time for me to make the switch.

Switching Web browsers is a comparatively easy task; essentially the only data you have to worry about moving is your bookmarks. An email client, on the other hand, may archive thousands of old messages you can't just leave behind, and you need to move your server settings and address book to the new program as well.

Here's how I migrated away from Ximian Evolution to Mozilla Thunderbird. I'll point out some of the pitfalls you'll want to avoid when making such a move yourself.

The first thing to do of course is to download Thunderbird. Mozilla.org provides tar.gz packages, and you can find RPMs for most distributions through third-party Web sites; .deb files for Debian-based distros are also available. I'm not going to cover the installation procedure or how to correctly configure Thunderbird to send and retrieve mail, as these are well-covered elsewhere, but you must create a profile in Thunderbird and go through the usual setup steps before you can migrate your existing data.

I decided to start my own migration with my archived messages rather than my address book, reasoning that if that transition didn't go smoothly there would be little point to attempting the other. Thunderbird includes an Import tool under the Tools menu, but the Linux build appears to be capable of importing mail only from Netscape Communicator 4.x. This isn't the end of the world, however, because both Evolution and Thunderbird store email in the mbox format, so with a little trickery you can copy your messages from your old Evolution folder to your new Thunderbird profile and Thunderbird will recognize them automatically.

You might call the mbox format folder-centric: each folder you see in your email client exists as a separate mbox file on your hard disk, which contains every email message in the folder, all stuck together. You need to create a duplicate set of empty mail folders in Thunderbird, which will give you one empty mbox file for each folder of mail you want to migrate, and then copy the mbox file from the corresponding Evolution folder to your Thunderbird profile, overwriting the empty files.

Specifically, for an Evolution folder called, say, Santa, create a folder in Thunderbird also called Santa and then from the command line type something like cp ~/evolution/local/Santa/mbox ~/.thunderbird/default.xyz/Mail/Local Folders/Santa. Note that in the Evolution hierarchy, each mbox file is actually named mbox and lives in a directory named for the email folder. In Thunderbird, all the folders reside in the same directory and each mbox file bears the name of the folder it represents.

It wouldn't hurt to browse through the two applications' directory hierarchies to get this distinction clear in your head before beginning -- it's not tricky, but if you have a lot of folders to migrate, it might get tedious and confusion could trip you up.

I didn't lose any email in this process, but I did decide to reexamine my love of creating folders and nested subfolders just for the fun of it.

When you've finished, your Collected Letters will be safely tucked away in their new home and Thunderbird should be able to see them all. They're probably all going to be marked as Unread, but you can fix that as you check the folders to make sure everything made it intact.

Migrating the address book

Of course, if you want to send any new email, you will have to migrate your address book data as well. Once again, Thunderbird and Evolution don't provide a direct way to move this data. The Thunderbird import wizard offers to import LDIF data and three varieties of text file: tab-delimited, comma-separated, and plain text. Evolution stores its address book internally and will export it only to a vCard.

vCard is a format designed for person-to-person contact information swapping and is supported by a lot of email clients (including the Outlook family and most of the popular Webmail providers), several IM programs, most cell phones, and the iPod. LDIF, on the other hand, is the LDAP interchange format that is supported almost solely by LDAP-specific software.

You have no choice but to select your entire address book in Evolution and export it as one large VCF file. I looked with Google for some way to automatically convert vCard data to LDIF format, but the results were spotty. There were a few homebrew applications (both compiled and scripted) written by others who had been down this path before, but none of them worked reliably.

Ultimately, the only method I found that afforded any degree of success was importing the VCF file into KDE's KAddressbook. For users of GNOME or other desktop environments, this may mean installing a lot of KDE-specific software that you don't need, due to dependencies. KAddressbook imported 57 of 58 entries in my VCF file, but not without problems. Two entries came through with blank name fields, and one contained just the letter "b." KAddressbook did not import any dates, job title/organization fields, or "notes," and only grabbed home addresses, not work.

When performing the import, KAddressbook opened a separate "edit Contact" window for every entry in the VCF file, which I then had to close manually. (The more popular you are, the more grueling this will be.) During this process, KAddressbook generated four error messages, but they popped up in generic windows while importing continued, making it impossible to associate them with the contacts that spawned them.

Once you have performed the vCard import, you can make corrections to the data before you export it again. I had read that KAddressbook could export data to LDIF format -- which was why I tried it -- but there was no such option in the Export menu. The only choices were "Export vCard 3.0..." and "Export List..." I chose the list option, and the result was a comma-separated text file.

Be wary of comma-separated-value files in this context. Some addresses have commas in them, as do some names and some organizations, not to mention any surviving "notes" fields. In general, putting structured data into plain-text files is red kryptonite (i.e., anything can happen and usually it's bad), regardless of the delimiter, but the comma is a particularly poor choice for contact information.

Thunderbird will open this CSV file, but you must match the comma-separated fields it finds to the address book fields it expects. This can be confusing, for a number of reasons.

First, the import wizard presents you with a two-column window -- one column labeled "Address Book fields" and the other "Record data to import." KAddressbook's CSV file, however, begins with a line that labels the fields in the records that follow, so you have two columns of field labels. Click the next-record button in the upper right corner to see the first address book entry, which will be much easier to align correctly.

The only method provided to align the fields is a pair of buttons labeled "Move Up" and "Move Down" out to the left of both columns. But highlighting an entry selects the entire row, leaving you to guess which of the columns will move when you hit the button. Spoiler alert: the Move buttons affect the left-hand column.

Also, you will find that selecting a row also unchecks that row, so you have to go back and check it again when you have the fields aligned. Moreover, moving a field displaces every field that it passes, so you have to start at one end of the list and adjust every single row in the window, in order.

Last but not least, KAddressbook populates its CSV file with a number of fields unused by any address book entry, which you then have to align with "custom" fields in the Thunderbird import wizard to keep the useful data straight.

Your patience will be rewarded, though, and Thunderbird will do its level best to import your data from the CSV file. Of course, given all the mutations the data has been through, you will want to go through the resulting Thunderbird address book and make corrections by hand. This would be wise to do no matter how you migrated the data, but I can tell you from experience that you may have to copy some data manually.

My last piece of parting advice is to keep Evolution installed on your system, for safety's sake, since there is no guarantee that the next Thunderbird update won't corrupt your data (or that this one won't: remember, it is still only a "preview" release).