Fetchmail howto
How to use TMDA with fetchmail.
This document explains how to install TMDA (tested with ver 1.0) to work with fetchmail. Only things that should be made differently from normal TMDA configuration are explained; please read the TMDA documentation first!
My MTA is postfix, but I quess this should be easily converted to work with other MTA's also. You need c-compiler to do this (unless someone will translate the xtmdaheader.c to python...). My TMDA version is 1.0
1. Install fetchmail.
2. Install TMDA
3. Compile the xtmdaheader-1.1.c (new version! finds also To: and Cc: - headers) and copy the result to your preferred directory. I copied it to ~/.tmda.
cc xtmdaheader.c -o xtmdaheader
4. when feeding the mail to TMDA, feed it first throught xtmdaheader. With other MTA's, this might be different. This is how to do it in postfix: In .forward - file, instead of using
"| /path/to/bin/tmda-filter"
use
"| ~/.tmda/xtmdaheader youremailaddress@domain.xxx | /path/to/bin/tmda-filter"
5 Add the following lines (with your own data) to your ~/.tmda/config. Use the hostname of your email address, not your local hostname.
Recipient header is needed, because the recipient information is lost with fetchmail. xtmdaheader extracts this from the mail headers. Email addresses are needed, because your local email address will propably not be the same than the address fetchmail gets the mail to.
RECIPIENT_HEADER = "X-RecFor-Recipient" CONFIRM_ADDRESS = "youremailaddress@domain.xxx" USERNAME = "username" HOSTNAME = "domain.xxx" FULLNAME = "Surname Lastname"
6. Optionally you can patch the tmda source; With the original source, the emails that are released from the pending queue will go to another round on the outside word. This does not stop it from working, but it adds some (minimal) overhead to email servers. Also mail is delayed a bit; you will not get the released mail until the next fetchmail check. By patching the TMDA, the mail won't get the extra round and you get the mail immediately when it is released from the pending queue. And here is the patch:
In the TMDA/Pending.Py, class Message: Change the line
self.recipient = self.msgobj.get('x-tmda-recipient')
to
self.recipient = self.msgobj.get('delivered-to')
In the bin/tmda-rfilter, in function def release_pending, change the line
recipient = msg.get('x-tmda-recipient')
to
recipient = msg.get('delivered-to')
Remember to compile the source by running compileall... --- That should be it. I have been happily using this for over a year!
A new version of xtmdaheader searches also To: and Cc: headers for your own email address. It works better with emails sent to several recipients on the same domain.
Comments, questions: please mail me (Hannu Kotipalo) by adding @iki.fi after my names separated with a dot. (And see the drawback of using fetchmail with tmda; you might need to wait until I turn my computer on before you get the confirmation message)
If you would like to convert the c-file to python, you can do it, but please share the result with the others. Basicly xtmdaheader.c replaces the RECIPIENT environment variable with added header, "x-recfor-recipient". The recipient is searched from 'received for' headers in the mail. Please note; I'm not sure, if this is 100% accurate in all cases. Anyway, it has been working for over a year for me.
Perl version of xtmda_header
If my reading/interpretation of the above code is correct, it will: 1. check To: for an address that "matches" yours. Use this if found. 2. check CC: for an address that "matches" yours. Use this if found. 3. use the last email from the Received: headers.
This will work if you are in the To, CC, or BCC header of the mail.
For mailing list mail, it will say that the Recipient was the list. This could be a problem if you subscribed to the list using a keyword address.
For mail that is forward in transit (say by a .forward file from old_user@old_domain to username+keyword_address@domain), it will say that the recipient was old_user@old_domain.
I *think* that an MTA would report the recipient the keyword address in the above two examples. The perl code below does just that.
#!/usr/bin/perl
# xtmda_header.pl - extract received for info from headers
# by Corey Halpin <chalpin@cs.wisc.edu>
# This is public domain software, use it however you want.
########## Need to set this:
$my_name = "chalpin";
$my_domain = "cs\.wisc\.edu";
##########
use Email::Filter;
my $email = Email::Filter->new();
my $fallback = "";
my $recfor = "";
my @to_cc;
for $header ( $email->to(), $email->cc()){
my @addrs = split(/,/,$header);
for $addr ( @addrs ){ push(@to_cc, $addr); }
}
for $header ( @to_cc, $email->received() ){
if( $header =~ /([A-Za-z0-9+.-]+\@[A-Za-z0-9+.-]+)/ ){
my $addr = $1;
if( length($fallback)==0 ){ $fallback=$addr; }
if( $addr =~ /$my_name(\+[^@]+)?\@$my_domain/ &&
length($addr) > length($recfor) ){
$recfor = $addr;
}
}
}
$email->simple->header_set("X-RecFor-Recipient",
length($recfor)>0 ? $recfor : $fallback );
print($email->simple->as_string);
Using Fetchmail and TMDA with a catch-all e-mail address
Many hosting companies allow users to specify a catch-all address for domains hosted on the company's shared hosting servers. This catch-all address generally receives all e-mail sent to the hosted domain that isn't explicitly delivered to a mailbox or forwarding address.
This is very convenient for those using TMDA as all the dated, keyword, and whatnot addresses will all dump into the catch-all address. However, not all addresses and hosting companies are created equally. As such, using the approach suggested above will only get you so far.
For instance, while e-mail sent to my primary e-mail address worked just fine with TMDA, e-mail sent to other virtual addresses that were configured to pass through TMDA for delivery would always trigger confirmation.
The solution I found after much experimentation and rooting around online was to add the following to my .procmailrc file:
RECIPIENT=`formail -UDelivered-to: -xDelivered-To: | perl -e '$_=<STDIN>;s#\sdbogen-example:com-##;print "$_";'` :0fw | formail -i "X-Procmail-Recipient: $RECIPIENT" :0fw | /usr/local/bin/xtmdaheader $RECIPIENT # Run the message through tmda-filter.
This uses formail to isolate the last Delivered-To: header in the message. From this I can derive the address to which the message was originally delivered. perl is used to strip out the part of the Delivered-To: header I don't want, which leaves me with just the e-mail address to which the message was originally delivered. Obviously, you'll need to remove/replace the dbogen-example:com from the perl code before it will work for you. In addition, if your hosting company uses a different scheme for their Delivered-To: headers, you may need to tweak the perl regexp to get something that works for you. You may need to examine the headers of some messages, and experiment with the commands above before implementing anything in a live environment.
Then, I add this information to the messages headers for later debugging, if necessary.
Finally, I call xtmdaheader as described above with the value we derived from header parsing.
I won't guarantee that this is the Right Way to solve this problem, but it works for me. You can contact me at djb at bogen org if you have questions about this.