Wuytens.net

Step 6: Configure Dovecot

Let us configure Dovecot which provides both a POP3 and an IMAP service. The configuration files for Dovecot is found under /etc/dovecot. Start with the...
/etc/dovecot/dovecot.conf

Set the line protocols to:

protocols = imap imaps pop3 pop3s

so that Dovecot starts the IMAP and POP3 services and also its equivalents that work over an encrypted SSL (secure socket layer) connection.

If users start to complain that they cannot fetch their emails consider setting:

disable_plaintext_auth = no

This will allow plaintext passwords over an unsecured (non-SSL) connection. By default it is set to 'yes' for security reasons. Setting it to 'no' will mean less security but may help the "less fortunate".

A more important setting is:

mail_location = maildir:/home/vmail/%d/%n

which will tell that the users' mailboxes are always found at /home/vmail/DOMAIN/USER and that it should be in maildir format.

If you already have virtual mailboxes on your system because you followed the previous tutorials for Sarge or Woody you may want to define the IMAP namespace explicitly so that the users find their folder where they have always been:

namespace private {
    separator = .
    prefix = INBOX.
    inbox = yes
}

Next look for a section called "auth default". First define the allowed authentication mechanisms:

mechanisms = plain login

Then inside that same section you need to change:

passdb sql {
    args = /etc/dovecot/dovecot-sql.conf
}

which tells Dovecot that the passwords are stored in an SQL database and:

userdb static {
    args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes
}

to tell Dovecot where the mailboxes are located. This is similar to the mail_location setting.

You will want to comment out the section called passdb pam that deals with system users. Otherwise Dovecot will also look for system users when someone fetches emails which leads to warnings in your log file.

Now look for another section called socket listen. Here you define socket files that are used to interact with Dovecot's authentication mechanism. Make the section read:

socket listen {
    master {
        path = /var/run/dovecot/auth-master
        mode = 0600
        user = vmail
    }

    client {
        path = /var/spool/postfix/private/auth
        mode = 0660
        user = postfix
        group = postfix
    }
}

The master section is needed to give Dovecot's delivery agent (the program that saves a new mail to the user's mailbox) access to the userdb information. The client section creates a socket inside the "chroot" directory of Postfix. chroot means that parts of Postfix are jailed into /var/spool/postfix and can only access files beneath that directory. It is a good security measure so that even if Postfix had bugs and were attacked the attacker would not be able to access /etc/passwd for example.

And finally the protocol lda section needs to be customized. The LDA (local delivery agent) is more capable than Postfix' built-in virtual delivery agent. It allows for quotas and Sieve (ships with the dovecot-common package) filtering. Let the section be:

protocol lda {
    log_path = /home/vmail/dovecot-deliver.log
    auth_socket_path = /var/run/dovecot/auth-master
    postmaster_address = postmaster@example.com
    mail_plugins = cmusieve
    global_script_path = /home/vmail/globalsieverc
}

Please change the above postmaster email address to a valid address where the administrator can be reached.

Edit /etc/dovecot/dovecot-sql.conf and change these settings:

driver = mysql
connect = host=127.0.0.1 dbname=mailserver user=mailuser password=mailuser2007
default_pass_scheme = PLAIN-MD5
password_query = SELECT email as user, password FROM view_users WHERE email='%u';

Restart Dovecot:

$> /etc/init.d/dovecot restart

Now look at your /var/log/mail.log logfile. You should see:

dovecot: Dovecot v1.0.rc15 starting up
dovecot: auth-worker(default): mysql: Connected to 127.0.0.1 (mymailserver)

Upon the first restart of Dovecot it will also generate Diffie-Hellman parameters and fix persmissions of /var/run/dovecot and /var/run/dovecot/login. That is perfectly normal.

Before you send a first test email you will need to fix file system permissions for the /etc/dovecot/dovecot.conf file so that the vmail user can access the Dovecot configuration. The reason is that Postfix starts the delivery agent with vmail permissions:

$> chgrp vmail /etc/dovecot/dovecot.conf
$> chmod g+r /etc/dovecot/dovecot.conf

Step 7: Send a test mail through TELNET
Emulating an SMTP session with telnet

Let us try to send an email to the user now. As the "example.com" is not really existing and your DNS settings will likely not point to the right server we are simulating an SMTP session with the telnet command:

$> telnet localhost smtp

The server should reply:

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 mailtest ESMTP Postfix (Debian/GNU)

Great. Postfix is listening and wants us to speak SMTP. First we need to be friendly:

ehlo example.com

Postfix appreciates our friendliness and tells us which features it provides:

250-my-new-mailserver
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

Hey, Postfix, we have a mail from steve@example.com:

mail from:

Looks like Postfix is happy with that because return codes that start with a '2' are good news:

250 2.1.0 Ok

Tell Postfix who the recipient of the mail is:

rcpt to:

Postfix accepts that:

250 2.1.5 Ok

Then we are ready to send the actual mail:

data

Postfix agrees and tells us we can send the actual mail now and end our input with a dot on an empty line:

354 End data with .

Okay, type in the mail:

Hi John,

just wanted to drop you a note.
.

Postfix tells us it has received the mail and queued under a queue ID:

250 2.0.0 Ok: queued as A9D64379C4

Thanks, Postfix, we are done:

quit

Checking the logs

Take a look at the /var/log/mail.log file now. You should see something similar to:

postfix/smtpd[...]: connect from localhost[127.0.0.1]
postfix/smtpd[...]: 5FF712A6: client=localhost[127.0.0.1]
postfix/cleanup[...]: 5FF712A6: message-id=<...>
postfix/qmgr[...]: 5FF712A6: from=, size=364, nrcpt=1 (queue active)
postfix/pipe[...]: 5FF712A6: to=, relay=dovecot, ..., status=sent (delivered via dovecot service)
postfix/qmgr[...]: 5FF712A6: removed
postfix/smtpd[...]: disconnect from localhost[127.0.0.1]

The delivery has worked. Postfix has correctly determined that the destination domain is a virtual domain and forwarded the email to the 'dovecot' service.
Checking the user's mailbox

The email should now be somewhere under /home/vmail/example.com/john. Let us take a look:

$> cd /home/vmail/example.com/john
$> find
.
./cur
./new
./new/1179521979.V801I2bbf7M15352.mailtest
./tmp

There sits the email. Try to read the mail with the "mutt" program:

$> mutt -f .

The new email is shown:

q:Quit  d:Del  u:Undel  s:Save  m:Mail  r:Reply  g:Group  ?:Help
   1 N   May 18 steve@example.c (0.1K)

Press ENTER to read the email:

From: steve@example.com
To: undisclosed-recipients: ;

Hi John,

just wanted to drop you a note.

So the email arrived at John's account. Perfect. Press 'q' twice to exit mutt again.
Step 8: Test fetching emails with IMAP and POP3

John will surely prefer to read his mail in a comfortable mail program. So he needs a way to get access to his mailbox. Two protocols come to play here:

    * POP (Post Office Protocol) is a simple protocol that lets you fetch email from a single mailbox. It is usually used to get all emails and then delete them on the server.
    * IMAP (Internet Messaging Application Protocol) is also used to fetch email but you can maintain different mailboxes. The inbox is where your incoming emails are stored. But users can move emails to different directories. IMAP is useful when you want to access your email from different locations without losing mail because you fetched it from another location.

Testing POP3

Let us try to create a POP3 connection and retrieve John's email:

$> telnet localhost pop3

The server replies:

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
+OK Dovecot ready.

Login as John:

user john@example.com

The server should accept that:

+OK

Send the password:

pass summersun

The server should recognize the correct password:

+OK Logged in.

Get a list of John's emails:

list

Dovecot will tell you that one email is in the mailbox:

+OK 1 messages:
1 474
.

Fetch that email number 1:

retr 1

Dovecot sends you the email:

+OK 474 octets
Return-Path: 
X-Original-To: john@example.com
Delivered-To: john@example.com
Received: from example.com (localhost [127.0.0.1])
    by ... (Postfix) with ESMTP id 692DF379C7
    for ; Fri, 18 May 2007 22:59:31 +0200 (CEST)
Message-Id: <...>
Date: Fri, 18 May 2007 22:59:31 +0200 (CEST)
From: steve@example.com
To: undisclosed-recipients:;

Hi John,

just wanted to drop you a note.
.

Close the connection to the POP3 server:

quit

The server will disconnect you:

+OK Logging out.
Connection closed by foreign host.

Test IMAP

Instead of going through the following procedure (IMAP is rather complicated) you may as well just use mutt to create an IMAP connection:

$> mutt -f imaps://john@example.com@localhost

Alternatively you can open up a raw IMAP connection to the server and enter the IMAP commands yourself:

$> telnet localhost imap2

You should get a connection:

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK Dovecot ready.

IMAP commands always start with a number and reply to that command with the same number. So the following commands must be entered with the number at the beginning of each line. Login with username and password:

1 login john@example.com summersun

Dovecot logs you in:

1 OK Logged in.

Ask Dovecot for a list of John's mail folders:

2 list "" "*"

Here comes the list:

* LIST (\HasNoChildren) "." "INBOX"
2 OK List completed.

Select your inbox:

3 select "INBOX"

Dovecot gives you all kinds of information about that folder:

* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1180039205] UIDs valid
* OK [UIDNEXT 3] Predicted next UID
3 OK [READ-WRITE] Select completed.

You see that one email exists. Fetch it:

4 fetch 1 all

IMAP will just give you basic information on the email:

* 1 FETCH (FLAGS (\Seen) INTERNALDATE .........
4 OK Fetch completed.

To read the actual mail body you need to fetch it explicitly:

5 fetch 1 body[]

Here it comes:

* 1 FETCH (BODY[] {474}
Return-Path: 
X-Original-To: john@example.com
Delivered-To: john@example.com
Received: from example.com (localhost [127.0.0.1])
        by ... (Postfix) with ESMTP id 692DF379C7
        for ; Fri, 18 May 2007 22:59:31 +0200 (CEST)
Message-Id: <...>
Date: Fri, 18 May 2007 22:59:31 +0200 (CEST)
From: steve@example.com
To: undisclosed-recipients:;

Hi John,

just wanted to drop you a note.
)
5 OK Fetch completed.

Disconnect from the server:

6 logout

Dovecot logs you out:

* BYE Logging out
6 OK Logout completed.
Connection closed by foreign host.

POP3 and IMAP appear to work. You could now use any email program like kmail, evolution or thunderbird/icedove and set up a POP3 or IMAP email account. The quickest way to check encrypted connections is using mutt again:

$> mutt -f imaps://john@example.com@localhost

If you use other mail programs note that the username will be the email address 'john@example.com' and the password is 'summersun'. You can try these kinds of connections:

    * POP3
    * IMAP
    * POP3 with TLS/SSL enabled
    * IMAP with TLS/SSL enabled

When using TLS/SSL you will get a warning that the certificate of the server cannot be trusted. Dovecot ships with a sample certificate so that you can test your setup and use TLS/SSL to fetch emails securely. Unfortunately the so called "postinst" script (that is called after the package 'dovecot-common' is installed) does not seem to create the certificate correctly. The common name lacks the domain part. (I have reported this issue under bug number #425917 bugs.debian.org but this will probably not be fixed in Etch.) So it is advised that you create your own certificate with the proper server name:

$> openssl req -new -x509 -days 3650 -nodes -out /etc/ssl/certs/dovecot.pem \
    -keyout /etc/ssl/private/dovecot.pem

The certificate and key will be created while you get asked a few questions:

Generating a 1024 bit RSA private key
.........++++++
............................++++++
writing new private key to '/etc/ssl/certs/dovecot.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:DE
State or Province Name (full name) [Some-State]:Hamburg
Locality Name (eg, city) []:Hamburg
Organization Name (eg, company) [Internet Widgits Pty Ltd]:workaround.org
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []:mailtest.workaround.org
Email Address []:postmaster@workaround.org

Of course you should fill in your own information here. The most important setting is the Common Name which must contain the fully-qualified name of your mail server. Oh, and this certificate will be valid for 10 years (3650 days) - adjust that period as you want.

Do not forget to set the permissions on the private key so that no unauthorized people can read it:

$> chmod o= /etc/ssl/private/dovecot.pem

Step 9: Authenticated SMTP

It is important that you understand the problem of open relays first. For security reasons Postfix allows either all mails from IP addresses listed in mynetworks or to local or virtual domains to send emails. All other cases like someone sending mail to another person on the internet is called relaying and is limited for security reasons. If you would relay mail for everybody then spammers would abuse your mail server to send out emails on their behalf. You would be a so called open relay. You would add to the spam problem and probably get blacklisted quickly. That will make it hard to ever send out emails again. Setting the smtpd_recipient_restrictions right (as described below) is very important.

Generally you can define the networks that are allowed to relay through your mail server by setting the mynetworks parameter in your main.cf. Usually you set it to your local network so local user do not need to authenticate:

$> postconf -e mynetworks=192.168.50.0/24

One case of relaying from outside of your network is but important in real-life. Imagine you have a user who is currently not connected to your local network but wants to send out an email to the internet through your mail server. According to the above rules the user would not be able to do that because neither are they on your network nor are they sending an email to one of your domains. You need to find a way to make your remote user be trusted by your mail server. The users will need to show you their username and password so you know relaying should be permitted. This is what authenticated SMTP is about. And most email clients have that feature built in.

Authenticated SMTP with Postfix has always been a pain. It was done through the SASL (Simple Authentication and Security Layer) library that was part of the Cyrus mail server. It was nearly impossible to debug and threw error messages that were gibberish and misleading. Fortunately starting with Postfix 2.3 we can make Postfix ask the Dovecot server to verify the username and password. And since you already configured Dovecot this is really easy now. Postfix just needs some extra configuration:

$> postconf -e smtpd_sasl_type=dovecot
$> postconf -e smtpd_sasl_path=private/auth
$> postconf -e smtpd_sasl_auth_enable=yes
$> postconf -e smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject_unauth_destination

smtpd_sasl_auth_enable enables SMTP authentication altogether. And smtpd_recipient_restrictions defines rules that are checked after the remote user sends the RCPT TO: line during the SMTP dialog. In this case relaying is allowed if:

    * permit_mynetworks: the user is in the local network (mynetworks) or
    * permit_sasl_authenticated: if the user is authenticated or
    * reject_unauth_destination: the mail is destined to a user of a domain that is a local or virtual domain on this system (mydestination, virtual_alias_domains or virtual_mailbox_domains).

There are further restrictions (smtpd_client_restrictions, smtpd_helo_restrictions, smtpd_sender_restrictions) but unless you really want to customize them it is okay to put all restrictions into the smtpd_recipient_restrictions.

Try to authenticate during an SMTP session:

$> telnet localhost smtp

The server will let you in:

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 mailtest ESMTP Postfix (Debian/GNU)

Say hello:

ehlo example.com

Postfix will present a list of features that are available during the SMTP dialog:

250-mailtest
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

Send the authentication string with a Base64-encoded password:

auth plain am9obkBleGFtcGxlLmNvbQBqb2huQGV4YW1wbGUuY29tAHN1bW1lcnN1bg==

The server should accept that authentication:

235 2.0.0 Authentication successful

Disconnect from Postfix:

quit

Good bye:

221 2.0.0 Bye

Authentication works. Great.

Note

If you have set John's password to somthing else than 'summersun' you need Base64-encode it yourself. Use:

perl -MMIME::Base64 -e \
    'print encode_base64("john\@example.com\0john\@example.com\0password")';

Now you can test sending email with authentication enabled. To make even your local network untrusted temporarily you can set:

$> postconf -e mynetworks=

temporarily. Fire up your mail program and watch your mail.log while you send an email to a domain on the internet. I recommend you send a test email to devnull@workaround.org which is an email address that will just discard your email. If everything worked well your logfile will show:

postfix/smtpd[4032]: 1234567890: client=..., sasl_method=PLAIN, sasl_username=john@example.com
postfix/cleanup[4040]: 2EAE8379CB: message-id=<...>
postfix/qmgr[3963]: 1234567890: from=, size=830, nrcpt=1 (queue active)
postfix/smtpd[4032]: disconnect from ...
postfix/smtp[4041]: 1234567890: to=,
    relay=torf.workaround.org[212.12.58.129]:25, delay=6,
    delays=0.09/0.08/5.6/0.23, dsn=2.0.0, status=sent
    (250 OK id=1HsPC3-0008UJ-O5)
postfix/qmgr[3963]: 2EAE8379CB: removed

Otherwise in case of an error your logfile would look like:

postfix/smtpd[4032]: connect from ...[10.20.30.40]
postfix/smtpd[4032]: warning: ...[10.20.30.40]: SASL PLAIN authentication failed:
postfix/smtpd[4032]: lost connection after AUTH from ...[10.20.30.40]
postfix/smtpd[4032]: disconnect from ...[10.20.30.40]

Your email program may have warned you that the mail server uses an untrusted SSL certificate. The default certificate that comes with Postfix is made out to the funny nonexisting "Office for Complication of Otherwise Simple Affairs" organisation. This is sufficient for testing but just like you did for Dovecot it is advised you create a proper SSL certificate that at least has the correct host name included. The default certificate is stored at /etc/ssl/certs/ssl-cert-snakeoil.pem and the default private key is stored at /etc/ssl/private/ssl-cert-snakeoil.key. Consider creating a new certificate/key pair and store it as postfix.pem:

$> openssl req -new -x509 -days 3650 -nodes -out /etc/ssl/certs/postfix.pem \
    -keyout /etc/ssl/private/postfix.pem

Same procedure as above when you created a certificate for Dovecot. Just remember to set the "Common Name" to the fully-qualified hostname. You could as well use the same certificate you created for Dovecot if the server name is the same. In that case just use the files /etc/ssl/certs/dovecot.pem and /etc/ssl/private/dovecot.pem below.

Do not forget to set the permissions on the private key so that no unauthorized people can read it:

$> chmod o= /etc/ssl/private/postfix.pem

You will just have to tell Postfix where to find your certificate and private key:

$> postconf -e smtpd_tls_cert_file=/etc/ssl/certs/postfix.pem
$> postconf -e smtpd_tls_key_file=/etc/ssl/private/postfix.pem

When you relay through Postfix again you should not get that certificate warning any longer.

By default Postfix will allow that the login data for SMTP authentication is sent in plain text. You better only allow encrypted transmission of the credentials by setting these parameters:

$> postconf -e smtpd_use_tls=yes
$> postconf -e smtpd_tls_auth_only=yes

What these parameters do is require encryption for SASL authentication to be used so that if you're using the PLAIN or LOGIN SASL methods your passwords aren't transmitted in the clear. When the client initially connects to the server, the AUTH command isn't offered by the server. If the client issues the STARTTLS command and successfully negotiates the TLS connection, the client sends the EHLO command a second time and this time the server offers the AUTH command. This won't require any kind of encryption from clients who don't need to authenticate (i.e. servers that connect to send mail to the domains our server is a final destination for, as opposed to our end users attempting to relay mail through it to external domains).

If you truly do want to forbid unencrypted SMTP connections (I do this on ports 587 and 465), you'd want to use either "smtpd_tls_security_level = encrypt" (for STARTTLS, generally on port 587) or "smtpd_tls_wrappermode = yes" (for SSL encryption from the initial connection on, generally on port 465).
Step 10: AMaViS: Filtering spam and viruses

Two annoying issues when using email are spam and viruses. Fortunately you can fight both using AMaViS (A Mail Virus Scanner). AMaViS is an interface between Postfix, spamassassin (famous for its Bayesian spam filtering capabilities) and optionally a virus scanner. AMaViS contains the spam filtering part but has no virus scanner built in. You should install ClamAV for that purpose. It is a free virus scanner that gets updated frequently.

In Debian Etch the AMaViS configuration has been moved from the /etc/amavis/amavisd.conf to different files in the /etc/amavis/conf.d directory. Fortunately the virus scanner "ClamAV" is already configured by default so you do not need to touch that. However I suggest you take a closer look at /etc/amavis/conf.d/20-debian_defaults:

    * $sa_spam_subject_tag: if this line stay uncommented then every email that AMaViS suspects to be spam get this string inserted into the email's subject. If you do not want to alter the subject you should comment this line out. Users can still filter out spam by checking the X-Spam-Status header.
    * $sa_tag_level_deflt: spam with a score that is greater or equal to this level will get spam headers added. Debugging is easier if you always add the headers so you should consider setting it to some very low score (e.g. "undef" - which is the lowest possible level - so that headers are always added.
    * $sa_tag2_level_deflt: emails with a spam score greater or equal to this level will be marked as spam. Usually this default is reasonable. Lower it if too much spam gets through. Raise it if you get too many regular mails caught as spam.
    * $sa_kill_level_deflt: should be set to the same value as $sa_tag2_level_deflt
    * $final_spam_destiny: the default D_BOUNCE setting is plain stupid here. The usual approach is to tag email as spam. But if you bounced them you would hit some innocent person because spam mails never contain the correct sender address. Just let the user decide what to do with spam. So it is strongly recommended that you set this to D_PASS.
    * $banned_filename_re: carefully check this list because these patterns tell AMaViS when to bounce an email because it contains data that you do not like to receive in an email

In the file /etc/amavis/conf.d/15-content_filter_mode you also need to remove the '#' from the @bypass_... lines so that spam and virus filtering gets enabled.

In the /etc/spamassassin/local.cf file you should add a line to disable automatic expiration of learned spam emails because that feature conflicts with AMaViS:

bayes_auto_expire 0

Restart AMaViS if you have made changes to the config files:

$> /etc/init.d/amavis restart

Make sure that AMaViS is listening on TCP port 10024:

$> netstat -nap | grep 10024

You should get this output:

tcp  0   0 127.0.0.1:10024     0.0.0.0:*    LISTEN   12345/amavisd

If you get such a line then AMaViS is running and waiting for incoming SMTP sessions. Otherwise check your /var/log/mail.log file - perhaps you have made a mistake in the configuration files.

A few words on how AMaViS will be plugged into Postfix. If some person sends you an email from the internet it will be received by Postfix (the smtpd daemon) first on TCP port 25 (SMTP). If Postfix accepts the email it will be forwarded to AMaViS on TCP port 10024 (SMTP). And if AMaViS is happy with the email's contents it will be sent back to Postfix on TCP port 10025 (SMTP). Postfix finally delivers the email to the actual recipient.

Making Postfix forward emails to AMaViS is done by setting the content_filter setting. Also set another option that will be explained later:

$> postconf -e content_filter=smtp-amavis:[127.0.0.1]:10024
$> postconf -e receive_override_options=no_address_mappings

But we need to define the smtp-amavis service first in the /etc/postfix/master.cf. And also we need Postfix to listen on TCP port 10025 for emails that get sent back from AMaViS:

smtp-amavis unix -      -       n     -       2  smtp
    -o smtp_data_done_timeout=1200
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20

127.0.0.1:10025 inet n  -       -     -       -  smtpd
    -o content_filter=
    -o local_recipient_maps=
    -o relay_recipient_maps=
    -o smtpd_restriction_classes=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_data_restrictions=reject_unauth_pipelining
    -o smtpd_end_of_data_restrictions=
    -o mynetworks=127.0.0.0/8
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
    -o local_header_rewrite_clients=

Restart Postfix first:

$> postfix reload

Now two settings here need some explanation. First the receive_override_options are set to no_address_mappings. This disables all address mappings. Your virtual aliases for examples are not considered at first. Then the email is sent to smtp-amavis and in the end returned to the 127.0.0.1:10025 service that sets a lot of options. One of those options is the receive_override_options again. But this time the no_address_mappings setting is left out. So at this stage Postfix finally checks your virtual aliases. Sounds complicated? Well, it has to be done like this or otherwise your aliases would be evaluated twice which leads to mails sent twice. The other options are used to disable checks that Postfix has already done during the first stage.

Note

Setting receive_override_options=no_address_mappings makes Postfix not consider aliases any more. So if you wish to disable AMaViS as a content filter then you must not set this parameter either.

Another tiny caveat is that the user "clamav" must be a member of the system group "amavis" so that the two services are allowed to talk to each other:

$> adduser clamav amavis
$> /etc/init.d/clamav-daemon restart

And another issue to take care of: AMaViS tries to find out whether a certain email is incoming (sent from the internet to your domains) or outgoing (sent from your system to the internet) by looking at the @acl_local_domains setting. You need to tell AMaVis where to check if a certain domain is one of your destination domains. Edit the /etc/amavis/conf.d/50-user file and before the "1;" enter these lines:

@lookup_sql_dsn = (
    ['DBI:mysql:database=mailserver;host=127.0.0.1;port=3306',
     'mailuser',
     'mailuser2007']);

$sql_select_policy = 'SELECT name FROM virtual_domains WHERE CONCAT("@",name) IN (%k)';

The @lookup_sql_dsn defines how AMaVis can access your database. And the $sql_select_policy sets the SQL query that is run when AMaVis wants to determine if the destination domain of the currently scanned email is one of your virtual domains. The %k is a list of strings that AMaVis expects to find. The actual query will look like this:

SELECT name
FROM virtual_domains
WHERE CONCAT("@",name)
IN (
    'john@example.com',
    'john',
    '@example.com',
    '@.example.com',
    '@.com',
    '@.')

This may look a bit weird. But in the end the string '@example.com' is searched for. Restart AMaVis:

$> /etc/init.d/amavis restart

Now try sending an email to john@example.com. If you examine the email headers of that mail you should find lines that got added by AMaVis:

X-Virus-Scanned: Debian amavisd-new at mymailserver
X-Spam-Score: 0
X-Spam-Level:
X-Spam-Status: No, score=0 tagged_above=-9999 required=6.31 tests=[none]

Your users will be able to filter out spam depending on this information. The X-Spam-Status line will be set to "Yes" if the spam score is above the required score ($sa_tag2_level_deflt). The X-Spam-Level line contains a number of "*" that tell the spam score. If your email program looked for a X-Spam-Level: ***** header then it would catch spam with at least a score of 5. A real-life example of caught spam:

X-Spam-Status: Yes, hits=16.0 tagged_above=-9999.0 required=6.31
    tests=BAYES_99, FORGED_MUA_OUTLOOK, MSGID_FROM_MTA_ID,
    RCVD_IN_BL_SPAMCOP_NET, UNDISC_RECIPS, URIBL_OB_SURBL, WORK_AT_HOME
    X-Spam-Level: ***************
    X-Spam-Flag: YES

All incoming emails should now be tested for viruses and spam. Try that. Spamassassin comes with a spam sample containing the GTUBE (Generic Test for Unsolicited Bulk Email) that contains a signature that is supposed to trigger spam filters - just like EICAR.COM for virus scanners. Send John that spam email:

$> sendmail john@example.com < /usr/share/doc/spamassassin/examples/sample-spam.txt

Among other information your /var/log/mail.log will now contain a line being written by AMaViS:

amavis[13001]: (13001-02) Passed SPAM, <...> -> , ...

So the email was detected as spam and passed through to John. Very well. Finally fix the permissions of the AMaViS configuration files so that no unauthorized user can read the database password:

$> chmod o= /etc/amavis/conf.d/50-user

Step 11: Learning spam and ham

SpamAssassin is the software that is supposed to tell spam from regular emails ("ham"). It comes with a large number of tests that are applied to emails. It also queries real-time blacklists on the internet. But in addition to the fixed tests there is the Bayesian filter. It uses conditional probabilities to analyse the words used in the email mathematically. So it can derive if an email is likely to be spam. This filter can be trained by telling it what you think is ham or spam. The more ham and spam emails you show it the better the spam detection will become. You will need to have trained at least 200 ham mails and 200 spam mails or the Bayes filter will not be taken into account. The configuration described here will not distinguish ham and spam per-user but rather keep a global Bayes database. Results may be better if every user had their own database and that's actually possible but currently not used here.

As the 'amavis' user is the owner of the /var/lib/amavis/.spamassassin directory that is used to store the Bayes database and the 'vmail' user is used to store the users' mailboxes we have a permissions problem here. Unfortunately it cannot be easily solved by adding the 'amavis' user to the 'vmail' group. The permissions on /home/vmail do not allow group write access. So currently the spam learning is done by the 'root' user which may be a security problem.

A script that will teach SpamAssassin ham and spam can be run once a day by the 'root' user. Example for a script that you can run through crontab:

#!/bin/bash -e

SADIR=/var/lib/amavis/.spamassassin
DBPATH=/var/lib/amavis/.spamassassin/bayes
SPAMFOLDERS="\
    /home/vmail/well-known-customer.com/fred/.spam/cur \
    /home/vmail/spamtrap.org/jeanne/.spam/cur \
    "
HAMFOLDERS="\
    /home/vmail/spamtrap.org/jeanne/.trash/cur \
    "

for spamfolder in $SPAMFOLDERS ; do \
    echo Learning spam from $spamfolder ; \
    nice sa-learn --spam --showdots --dbpath $DBPATH $spamfolder
done

for hamfolder in $HAMFOLDERS ; do \
    echo Learning ham from $hamfolder ; \
    nice sa-learn --ham --showdots --dbpath $DBPATH $hamfolder
done

chown -R amavis:amavis $SADIR

You can see what AMaViS has recorded into its database by running:

$> sa-learn --dbpath /var/lib/amavis/.spamassassin/bayes --dump magic
0.000          0        977          0  non-token data: nspam
0.000          0      27866          0  non-token data: nham
[...]

The nspam value is the number of spam emails SpamAssassin has seen. And nham is the number of ham emails being learned.

Note: this way you are training AMaViS' global Bayesian database. So it applies to all emails that enter your system. Make sure you don't learn spam mails from unreliable or untrusted users. They can ruin your spam detection ratio by putting messages in their 'spam' folder that are not really spam. Just because a user never wants to get email from his ex-girlfriend again does not make it spam for everybody. So be careful about the mails you feed to the global bayes database. Also do not make SpamAssassin learn emails that users just 'forwarded' to you. The header information would be gone that contains much of the information that helps SpamAssassin recognize spam. Make sure they use the 'bounce' feature or send the spam email as an attachment. Your best choice is to take the email messages directly from the user's maildir directory.
Step 12: Populate and administer the users in the database

Now that the setup is working with your test entries it is now time to fill the database with real data.
Converting your database from the Sarge tutorial

If you followed the previous tutorial for Sarge then you can simply use the conversion script:

$> wget http://workaround.org/articles/ispmail-etch/dbconvert.py
$> chmod +x dbconvert.py
... edit the db_sarge and db_etch variables in the script ...
$> ./dbconvert.py

That should convert your data from a Sarge-based database layout to the database layout suggested here.
Ronny Tiebel's PHP administration frontend

Of course you need a convenient user interface to administer the data in the database. Many readers offered to write a web interface to simplify the administration of domains, accounts and aliases. Robert Klikics published a PHP script written by his apprentice Ronny Tiebel.
Peter Gutwein's PHP administration frontend

Another PHP based web interface to administer user accounts was written by Peter Gutwein and is available at http://www.grs-service.ch/pub/grs_mminstallation.html
Optional features
Offering webmail access

This step is optional but most email service providers offer their users a way to read emails in a browser. Luckily this is easy. The package you need is called "squirrelmail" - a web mail system that can use any IMAP server - just like the Dovecot IMAP server we use here. To set it up you first need to add its configuration to your Apache configuration:

$> ln -s /etc/squirrelmail/apache.conf /etc/apache2/conf.d/squirrelmail.conf
$> apache2ctl restart

You need to change one configuration option at least so please run the configuration utility:

$> squirrelmail-configure

select option 3 (Folder Defaults) and set option 1 (Default Folder Prefix) to 'none'. You may also want to browse through the other menu options to e.g. set your organisation's name right.

Point your browser at http://localhost/squirrelmail and you should see the webmail interface. Login as 'john@example.com' with password 'summersun' and you should be able to read John's email. It couldn't be simpler.

In case you are unhappy with the http://my.server/squirrelmail URL and prefer to use http://my.server instead you should read the users will prefer a simple URL section in the /etc/squirrelmail/apache.conf file.
Sieve: Filtering out spam

You already have a fully functioning mail server that even tags spam. But so far you left the actual task of sorting out the tagged spam to the user. We can do better by setting up server-side filters. The task ahead is to move all spam emails to a "Spam" folder of each user. We are using the raw power of the Sieve_ plugin which is a mail filter like procmail (which is not useful for virtual mailboxes). Sieve has a simple scripting language that allows us to forward all emails that are tagged by Spamassassin to a "spam" folder automatically. Some way above we defined a protocol lda and there set global_script_path = /home/vmail/globalsieverc. So that file will be the filter that is executed whenever an email is delivered to a virtual user. A basic recipe will do that. Create a /home/vmail/globalsieverc file containing these lines:

require ["fileinto"];
# Move spam to spam folder
if header :contains "X-Spam-Flag" ["YES"] {
  fileinto "spam";
  stop;
}

Send John another spam email:

$> sendmail john@example.com < /usr/share/doc/spamassassin/examples/sample-spam.txt

Then watch the /home/vmail/dovecot-deliver.log file. The last line should look like:

deliver(john@example.com): "2007-06-18 22:17:46 "Info: msgid=: saved mail to spam

This tells that the email was saved into the "spam" folder. If you fetch email you should see the email show up in that folder automatically.
Mailing lists with mailman

A frequently asked question is how to run mailing lists with virtual domains using the famous 'mailman' software. Actually this is not very hard. There are different ways to accomplish that. The simplest but least flexible way uses local aliases:
Using local aliases

This first approach will use the aliases that you usually insert into /etc/aliases. You just have to understand that virtual users cannot have piped aliases. Piped... what? Well, once you have created a mailing list with the newlist command you are requested to add certain aliases to your /etc/aliases file like chitchat-admin, chitchat-bounces, chitchat-join and so on that point to destinations that look like "|/var/lib/mailman/mail/mailman post powerdns-debian". This means that the email is piped into the mailman program. Mailman is called and gets the email as input.

This works well for local domains but will fail if you put a piped alias like this into a virtual alias. That is because you do not have a system user whose user-id you could use to run this pipe. So you will have to make a circuit and forward your virtual email address to a local email address where you can use piped aliases.

I recommend that you use 'localhost' as your local domain (mydestination = localhost) and forward each of the mailman aliases to the @localhost equivalent. So if you have a chitchat-subscribe@virtual.domain address you just forward it to chitchat-subscribe@localhost and use the /etc/aliases as suggested by 'mailman'. Perhaps someone comes up with a nifty solution to put the /etc/aliases into the MySQL database, too. Let me know if you have invented something. :)
Using transport maps (contributed by Michael Dürgner)

If you want to allow your customers/clients to have their own mailing lists then the above solution is not flexible enough. Consider using Postfix' powerful transport maps. They allow you to tell postfix which transport (service) to use for certain email addresses or even whole domains.

Using a regular expression lookup table you can forward emails where the recipient email address matches a certain pattern to mailman. The following is a static example of a transport mapping that forwards mylist@example.com and anything looking like mylist-*@example.com to the mailman service:

/^(mylist|mylist-.*)@example\.com$/      mailman:

If you want to maintain your mailing list addresses in a MySQL table then use this /etc/postfix/mysql-transport-mailman.cf configuration file:

user = mailuser
password = mailuser2007
hosts = 127.0.0.1
dbname = mailserver
query = SELECT 'mailman:' FROM transports WHERE email LIKE '%s'

The transport table would consist at least of a column email that contains mailing list addresses. Use '%' in the address as a placeholder.

Use this config file by setting:

$> postconf -e transport_maps=mysql:/etc/postfix/mysql-transport-mailman.cf

Greylisting with Postgrey

Some administrators are enthusiatic about a mechanism called greylisting. How does it work? Usually MTAs have a queue so that they keep mails queued in case of temporary delivery problems on the recipient's side. SMTP distinguishes temporary errors that have a numeric starting with "4" and permanent errors starting with "5". If a 5xx error is returned then the sender is notified that the delivery failed. In case of a 4xx error though the delivery is tried again for a few days. Some spammers do not use MTAs with a queue because they want to annoy thousands of innocents and do not want to care about queues. If your system rejected their first attempt to deliver an email they might not try again. This is what greylisting is about. Postfix rejects the first delivery attempt from an unknown system with a 4xx error. After five minutes the server Postfix will accept the next delivery attempt.

Greylisting has pros and cons. You will block off spammers that do not keep a queue. But you will delay legitimate email for at least five minutes what might annoy your users who may impatiently wait for an urgent email. Greylisting seems to become less useful with the time because most of the time spammers use compromised (cracked) systems to relay their emails. Those systems usually have a queue. It is your decision. Some administrators love it - some say it is useless to fight spam.

Fortunately using greylisting with Postfix is rather trivial. You would need to install the postgrey package first:

$> aptitude install postgrey

Right after its installation the postgrey daemon will start running in the background and listen for requests on TCP port 60000. To make Postfix use that policy daemon you have to add it to the smtpd_client_restrictions setting in your /etc/postfix/main.cf. Example:

smtpd_client_restrictions =
    permit_mynetworks
    permit_sasl_authenticated
    reject_unauth_pipelining
    reject_rbl_client bl.spamcop.net
    reject_rbl_client dynablock.njabl.org
    reject_rbl_client zen.spamhaus.org
    reject_rbl_client list.dsbl.org
    check_policy_service inet:127.0.0.1:60000

In this example a couple of RBLs (real-time blacklists) are checked to see if the sending system is a known spammer. If none of these criteria match then the postgrey policy daemon is asked about the sender's system and will reject the delivery for five minutes. Such delays will show up in your mail.log like:

postfix/smtpd[1234]: NOQUEUE: reject: RCPT from mail.example.com[10.20.30.40]:
    450 4.7.1 : Client host rejected:
    Greylisted, see http://isg.ee.ethz.ch/tools/postgrey/help/yourdomain.html.html;
    from= to= proto=ESMTP helo=<[10.20.30.40]>

Vacation / Autoresponder

Remo Fritzsche has kindly published his PHP-based goldfish autoresponder that can easily be used in conjunction with this setup.
Using Horde/Imp as a webmail interface

Oliver Ladner kindly provides instructions on how to use Horde/Imp with this tutorial's SQL schema.
Removing old deleted mails

With IMAP you can mark emails as deleted and some email clients will not even show them any more. But the emails are still there and occupy space. Usually there is an option to purge all marked emails but many users do not care. So Michael Weisgerber suggests to run this command frequently via crontab to remove such emails:

find /home/vmail/ -name '*,ST' -ctime +7 | xargs rm -f

Dovecot renames all deleted emails so that they get a ,ST added at the end of the filename. Adjust the parameter to -ctime as you like. In this example deleted mails older than 7 days are purged.
Troubleshooting

If you are having trouble receiving or sending email you may try these general steps:

   1. Check your /var/log/mail.log. 93.7% of all problems cause a more or less clear error message there. :)
   2. Run postfix check. No output means everything is well.
   3. Ask on the #postfix IRC channel on irc.freenode.net. I usually attend there as Signum but please just ask and wait for people to help you. Especially read the hints in the channel's topic.
   4. Ask on the workaround chitchat mailing list (english please)
   5. This tutorial is meant for Debian "Etch". If you are trying this setup on any other distribution you are completely on your own. No, Ubuntu does not count as Debian because the package versions differ.
   6. Sending an email directly to the author asking for help is the worst choice. Sending an email directly to the author giving suggestions or feedback is a good choice. :)

Thanks

This tutorial is maintained since the age of Debian Woody. And a lot of people translated it, contributed suggestions and helped finding bugs. Thank you:

    * Alexander Stielau
    * Axel Zöllich
    * Cameron Jones from vpslink for sponsoring a virtual root server for testing this setup
    * Christian Garling
    * Christian Kurz
    * Christof Gruber
    * Colleen Hatfield
    * Daniel Hackenberg
    * Daniel Wladow
    * Devdas Bhagat
    * Eicke Kemm
    * Florian Fritsch
    * Jasper Slits
    * Jesper Krogh
    * Jon Cox
    * Kay-Michael Voit
    * Lucas Baltes
    * Mario Duve
    * Matthias Quade
    * Michael Dürgner
    * Michael Weisgerber
    * Mikael Tokarev
    * Oliver Ladner
    * Patrick Blitz
    * Patrick Jäger
    * Pete Boyd
    * Ricardo Arguello
    * Simon Kammerer
    * Thomas "Balu" Walter
    * Tim Weippert
    * Tobias Scheeper

You can find additional contributions that did not (yet) make it into the tutorial at the appropriate contributions wiki page. Also thanks to all readers who sent feedback on the tutorial - both technical and emotional. Perhaps you are curious to read what others think. If you like to support the author please consider donating:

tenslotte kan je hier een script afhalen dat de users mails als ze meer als een bepaalde grootte in het mail box hebben zitten.

---

Laatst gewijzigd op: 2008-02-08 13:44:35