October 27, 2004

SysAdmin to SysAdmin: A Procmail recipe primer

Author: Preston St. Pierre

Procmail is a local delivery agent and processor for electronic mail.
What this means to you is there is a tool available that enables you to
filter mail into different folders, create automatic replies, and
forward or copy mail to other email accounts. This document aims to
explain the basics of how to get started using procmail. It will also
provide a few working examples and tips.

How does all this work?

First, I'm forced to make some assumptions, lest this become a 40-page
diatribe on architecting a corporate email solution (a diatribe I'm not
qualified to write.) I'm assuming that you are either using an IMAP mail
server, or that whatever other mail system you're using is configured to
use procmail as a local delivery agent. If you're doing this on a home
machine, your mail server and workstation may be the same, which makes
things a bit easier. If you have a file server that exports home
directories to you and to the mail server, this can make things easier,
too, since procmail will need to find a config file in your home
directory.

When email arrives for someone@yourdomain.com, the mail is accepted for
delivery by a Mail Transfer Agent (MTA), which in many cases is
sendmail. However, this doesn't mean you have to use sendmail as a
Local Delivery Agent (LDA). Instead, all mail accepted for delivery by
sendmail can be handed off to Procmail, which is a LDA. Procmail will
then check ~someone for a file called '.procmailrc', which, if it
exists, will contain the rules for delivering mail to this user (we'll
go over how to create this in a moment). If there is no .procmailrc
file, Procmail will put the mail in the user's mail spool (or Inbox).

Why Bother?

First, if you can honestly say that you will only ever check mail from
one machine for the remainder of your days, and you only need very
simple filtering, if any at all, then Procmail very well might not be
worth it. You can probably get the little bit of filtering you need from
your mail client (Pine, Evolution, Kmail, Sylpheed, etc). However, most
people check mail from more than one place, on more than one machine,
etc. If you check mail from more than one place and you don't use
Procmail, you have to create all of your filtering rules and folders on
every machine you use, for every mail client you use, so that mail that
shows up in the 'MyList' folder at work also appears in a folder called
'MyList' at home. If you do use a Procmail recipe, this is all taken
care of on the server, and you just point your email client to the
parent directory for all of your mail on the server, and voila! The
folder list you see in all of your mail clients will be consistent, and
the messages in those folders will also be consistent no matter where
you check your mail from.

Though the user who wants to exploit every possible feature of Procmail
will have a good deal of reading ahead of them, most users find that
they can create their first simple Procmail recipe in just a few
minutes. Compared to the useful things that Procmail can do, the
learning curve is really relatively small.

Does this work with
Netscape/Outlook/Pine/Mutt/Elm/Exmh/Evolution/KMail/enter-random-client-here?

Procmail isn't a plugin for a mail client, it handles mail
delivery. When you launch a mail client like KMail, all it's
doing is reading the mail from files and directories on the mail server.
So in short, procmail works with any mail client.

How do I set all this up?

Procmail can function nicely with nothing but a .procmailrc file in your
home directory (~/.procmailrc). Any logs or other directories you want
Procmail to use can be defined in ~/.procmailrc. While a simple recipe
can be created in (probably) seconds, this recipe would probably not be
extremely useful, and would also not provide niceties like logging which
folder each incoming mail was sent to and delivering to a default
location of your choice if none of the rules match. Here's a very simple
recipe file, which I'll explain below.


PMDIR=$HOME/.procmail
MAILDIR=$HOME/mail
LOGFILE=$PMDIR/procmail.log
VERBOSE=1
EOL="
"
LOG="Filtering Mail$EOL"

:0 :
* ^To:.*webmaster@somesite.com
My\ Website

:0 :
^TO_softball-list@softball-nerds.com
Softball\ List

:0
* ^From:.(friend@hisjob.com|her@herplace.net|some@one.org)
{

:0 c
! myacct@tepidmail.com

:0 :
Personal

}

:0 :
$DEFAULT

This is a very small, yet very powerful file example, and will
hopefully answer a lot of questions about Procmail, its capabilities and
its syntax. You can probably see that the general layout of the file
consists of the definition of a few variables I'd like Procmail to use,
followed by the list of rules that make up the Procmail recipe. Let's go
through the sections in order:

I've defined a few variables that I should explain, the first of
which is PMDIR. This is a directory I've created under my home
directory. If I tell procmail to do logging or to include other recipe
files or anything like that, I'll tell it to look for what it needs in
PMDIR.

Next is MAILDIR. Later on when I make rules that tell Procmail to
dump mail in a specific folder, Procmail will put it in a folder under
this directory (and create it if it doesn't exist). This is also the
directory where you'll point your mail client.

LOGFILE is probably pretty obvious. It's the file where I want
Procmail to log its activity. The next variable, VERBOSE, goes with
it, and defines how much detail will be included in the log for each
piece of mail Procmail touches. For testing a recipe, setting this to
'1' will generally give you all the information you need to debug.

You can also place things into the LOGFILE yourself for debugging.
The "LOG=" line above sends a user-defined message into the LOGFILE, and
in the message I've sent in the example, you'll notice I tag the $EOL
variable onto the end, which will just make sure the log is readable by
placing a newline at the end of my log entry.

How do I create the actual filtering recipes?

Let's walk through the rules from the example above. There are just a
couple of important things to remember when creating your recipes. One
is that every new rule begins with a ':0'. You should also leave a blank
line between rules. Now on to the example rules:


:0 :
* ^To:.*webmaster@somesite.com
My\ Website

The first line starts the recipe using ':0'. The additional ':' tells
Procmail to use a lockfile when performing an action for this recipe (if
you're not sure you need this extra colon, use it -- it won't hurt). The
second line is called the 'condition line', and always starts with an
asterisk (*). However, the actual condition to check for starts with a
caret (^). Though Procmail can filter on other parts of the email, the
default action is to filter based on the email headers. So '^To:.' is
going to filter by looking at the 'To:' header and seeing if it matches
'webmaster@somesite.com'. Adding the asterisk (^To:.*) will make the
matching process case insensitive. If there is a match, then the third
line of the recipe is processed, which just specifies a folder (under
$MAILDIR) to put the email. Notice that if you want folders that contain
spaces in their names, the spaces need to be escaped using a '\' escape
character. Let's try the second recipe.


:0 :
* ^TO_softball-list@softball-nerds.com
Softball\ List

Again, the ':0' signifies the beginning of a new recipe, and I'm also
using a lockfile here again, denoted by the trailing colon (:) on the
first line. The second line is slightly more interesting. At first
glance, it looks almost the same as the first example, but a closer
inspection shows the word 'TO' is now in all caps, and is followed by an
underscore (_) instead of a colon and dot. This is a shortcut that's
especially useful for mailing lists. 'TO_' actually matches against any
headers in the email that represent a recipient (To:, Cc:, Original-To:,
Apparently-To:, Bcc: and many more.) Once again, the third line
specifies a folder under $MAILDIR to put mail that matches this rule.


:0
* ^From:.(friend@hisjob.com | her@herplace.net | some@one.org)
{

:0 c
! myacct@tepidmail.com

:0 :
Personal

}

This recipe is a little more complicated, but not hard to understand
if you've followed this far. The first thing you'll notice is that the
first line does not tell Procmail to use a lockfile. This will be taken
care of later. The second line (the condition line) basically says 'if
the mail is coming from any of these addresses'. It checks the 'From:'
header of the email, and if the email is from one of the specified
addresses, two things happen to it. Whenever you're doing more than one
thing with a piece of mail that matches a filter, the separate events go
inside curly braces ({}).

The first thing that happens (on line 4 of the example) is the email
gets copied (':0 c' means 'copy'.) The address to send the copy to is on
the following line (the exclamation point (!) tells Procmail that what
follows is an address, not a folder.) In addition to sending it to that
email address, we also want to keep a local copy in a folder called
'Personal' under $MAILDIR. Notice that we use a lockfile here when we
copy an email to a local file! Finally, the last rule looks like this:


:0 :
$DEFAULT

The very last rule in every .procmailrc file should be some kind of
catch-all rule so that your mail has somewhere to go if it doesn't match
any of the other rules in the file. You'll notice here that the first
line is the normal ':0', and we use a lockfile. The variable $DEFAULT,
if not defined elsewhere in the .procmailrc file, will tell Procmail to
place the mail in its globally defined default mail location, which in
the case of CS is the user's spool or Inbox. NOTE: If this rule is NOT
the last rule in your .procmailrc file, it IS POSSIBLE that a bug
elsewhere in your recipes will cause mail to be sent to oblivion, never
to be returned.

Finally, understand that these examples, while useful, constitute only a
small fraction of Procmail's capabilities. If you want to know more
about how to create recipes and see some more complicated examples and
explanations, try using Google to browse the thousands of sites online
where loads of people have posted useful procmail tips and recipes for
your perusal. Also, man procmail and man
procmailrc
can be very helpful if you have them handy.
Finally, if you really want your entire mail world to revolve around
procmail, I highly recommend "The Procmail Companion", by Martin
McCarthy.

Click Here!