August 6, 2010

Weekend Project: Secure Instant Messaging with Off The Record

Instant messaging, just like email or VoIP traffic, needs to be secure from eavesdroppers, man-in-the-middle attackers, and other security threats. Many IM clients can tunnel messages over transport layer security (TLS) to provide encryption, including XMPP (a.k.a. Jabber), IRC, and the OSCAR protocol used by AIM. TLS provides authentication and encryption at a low level, but a considerably secure solution for IM is a protocol called Off The Record (OTR). Pull up a chair and secure your instant messaging today.

What OTR does and why you care

Although there is not anything wrong per se with TLS encryption, it works best for a connection-oriented, ongoing stream of data. OTR provides better security by fitting into the asynchronous, single-message-at-a-time communication model used by IM. Unlike PGP, it does not require the users to already have a public/private key pair to be retrieved and verified in an outside channel.

With the OTR protocol, when both participants in a conversation agree to start an OTR session, the clients set up an encrypted channel with Diffie-Hellman key exchange, then perform a mutual authentication routine inside that channel to verify each other's identity. After the setup, a new key exchange is performed on every message sent, based on incrementing the previously acknowledged key. The participants can independently verify each other's identity using the "Socialist Millionaires' Protocol" (SMP) which allows mutual verification without exchanging private data.

The multiple key exchanges provide "perfect forward secrecy" — meaning that compromising one key does not let an attacker decrypt your previous conversations. This is one of OTR's big advantages over TLS-like encryption alone. The other advantage is "deniable authentication." Because a shared secret is sent with each message in addition to the keys, an attacker that steals a key and decrypts a message could use the information contained within to fake the entire message.

That might not seem like a security feature at first glance, but consider how it would work in practice: some attacker announces that he or she has intercepted and decrypted a message between you and your secret contact. But because the decrypted message itself contains enough information to forge the message, there is no way for the attacker to prove it is authentic and not a complete fabrication from the ground up. You may be lying when you deny its authenticity, of course, but the point is that the alleged decrypted message cannot ever be proven to be authentic -- unlike, say, an email that you signed with your PGP key.

OTR support under Linux

The best thing about OTR, though, is that the vast majority of the cryptographic setup is done automatically, without the need for the user to take a careful sequence of steps. The OTR project releases a library, libotr, that IM clients can use to encrypt messages sent over any protocol: AIM, XMPP, ICQ; you name it. Furthermore, the signal that one OTR client uses to wake up another is a sequence of whitespace characters that will go unnoticed by chat clients that do not support OTR at all, but can start an OTR conversation seamlessly between clients that do.

There are two desktop Linux IM clients that feature built-in support for OTR: the multi-protocol KDE application Kopete, and the console XMPP client mcabber. Both are common packages shipped with desktop Linux distributions. Kopete's OTR support is provided by a plugin which is included with the default package, but must be turned on in the Settings -> Configure dialog. You can choose between requiring OTR, initiating OTR whenever the other party supports it, manually initiating OTR only, or never using OTR.

The mcabber client runs in text mode, but is a full-fledged XMPP client in every respect. You initiate an OTR connection with the command /otr start contactname, and set your OTR policy with /otrpolicy default followed by either plain, manual, opportunistic, or always. You can set a specific policy for a particular contact by substituting that contact's name in place of "default."

In addition to these desktop Linux clients, Android users can take a look at OtRChat, a special XMPP client built specifically for OTR usage.

OTR for Pidgin

In addition to libotr, the OTR project itself maintains the OTR-enabling plugin for the most popular Linux desktop client, Pidgin. It may or may not be available through official Apt or RPM package repositories, but if it is not, you can always grab the latest release from the OTR site, which hosts builds for Pidgin's Windows port in addition to Linux. The Pidgin OTR plugin integrates a bit more deeply than some of the others, so it is worth a closer look.

You enable the OTR plugin from the Tools -> Plugins dialog box. When activated, choose "Configure Plugin" at the bottom of the window to bring up the OTR settings. As with Kopete, you can select OTR policy with simple checkboxes; the Pidgin plugin's labels are a bit clearer with regard to what the settings do, e.g., "Automatically initiate private messaging" rather than "Opportunistic." The config screen allows you to generate private keys for each Pidgin account, but doing so is not required, because the plugin will generate keys on an as-needed basis. You can also examine a list of keys you have seen from OTR conversations with your buddies, verify them with SMP, or forget them if you fear they may have been compromised.

When activated, the plugin adds an OTR menu to Pidgin's conversation window. The "Start private conversation" option, obviously, initiates an OTR session request with the other party. During an IM session, the menu displays one of four possible OTR states: "Not private" — meaning OTR is not in progress, "Private" — meaning an OTR session is in full swing, "Unverified" — meaning you have started an OTR session with the other party but have not performed an SMP authentication, and "Finished" — meaning the other party has closed the OTR session, and you must do so as well to return to normal chat mode. The last state is important; by blocking you from sending messages until you deactivate OTR on your end, the system prevents you from accidentally spilling the beans on a channel that you mistakenly think is secured.

The Pidgin plugin also adds a right-click context menu item to the Buddy List, allowing you to set per-buddy OTR preferences. Be careful, though, because the selection you make is not listed anywhere else in the interface, so it is possible to forget per-buddy policies if you customize too many of them. Both the plugin configuration dialog and the per-buddy OTR settings window allow you to specify that OTR conversations not be logged. If selected, this will override all other Pidgin logging settings that might apply to the conversation.


When the public at large thinks about IM, they probably think about services run by big corporate players, such as AIM or Google Talk. In the geek community, however, IRC is still a communications standby — in fact, you can hardly call yourself an open source project with your head held high if you don't have an IRC channel.

So it should come as no surprise that there are open source OTR tools built for use with IRC. Because the OTR protocol only handles one-to-one communication, it can only be used to secure a private chat session, not an entire channel, but those private messages can be reliably encrypted just like anything sent over XMPP or another system.

IRC OTR is made possible by the developers of the Bitlbee open source IRC-to-IM gateway application, but it does not require running Bitlbee to work. At you will find OTR plugins for both Irssi and for XChat. The XChat plugin is loaded at startup automatically; in Irssi you must type /load otr to load it instead.

In both applications, the plugin does not add menus or other GUI elements to the client, just additional IRC commands. You initiate an OTR conversation by typing ?OTR? to your buddy. The first time you so do, the plugin will generate a key for you, which could take some time, but on subsequent runs that is not a concern. The IRC plugin is semi-manual in its OTR policy; the ?OTR? command sends the special whitespace sequence to the other party, so you must execute it to initiate a conversation. On the other hand, when you receive the whitespace OTR trigger, the IRC plugin responds automatically.

While chatting, you can type /otr auth to trigger an SMP authentication sequence, and /otr finish to terminate the session. You can also set per-contact OTR policy with a list of "nick@server policy" pairs using the otr_policy variable.

The only real caveat to using OTR over IRC is that some IRC servers have the nasty habit of stripping out whitespace from messages, which undermines the automatic OTR setup "handshake." The IRC plugin developers assure users that this is only a problem when you use the "opportunistic" policy setting, and that manual OTR triggers are handled correctly.

Extra credit: other clients!

Listed above are just five Linux instant messaging clients, and that simply isn't enough. Unfortunately, it looks as if the developers of the Empathy client, which is slated to replace Pidgin as the default IM application in future Ubuntu releases, have long been unwilling to add support for OTR to the code because they believe encryption should be built in to the IM protocol itself, and because Empathy has no plugin system, a third-party add-on is out of the question. Luckily, work on this may have finally started progressing, so there is hope.

In the mean time, there are still several other IM and chat applications that could use an OTR plugin of their own. That includes XChat-GNOME, which as of now cannot run the existing XChat OTR plugin, many of the single protocol IM clients (Gajim, Emesene, KMess, etc.), and most if not all of the voice-chat programs that include SIP chat functionality as a second communications option.

Why not pick up your favorite IM app, and see about integrating off-the-record security through libotr? Remember, despite what the name suggests, OTR offers security for all instant messaging users, not just reporters breaking the next Earth-shattering story.

Click Here!