Abstract

You surely know the big web hosters that allow you to rent a domain and use it to receive emails. If you have a Debian computer connected permanently to the internet you can offer the same service. You do not even need to have a fixed IP address thanks to dynamic DNS services like dyndns.org. All you need is this document, a pot of delicious tea and a little time. When you are done with the setup your server will be able to...

• receive and store emails for your users
• let your users retrieve the email through IMAP and POP3 - even with SSL to encrypt to connection
• receive and forward (relay) email for your users if they are authenticated
• offer a webmail interface to read emails in a web browser
• detect most spam emails and filter them out or tag them

This document is not a simple copy-and-paste tutorial. The intention is rather to make you understand the different components that you are using. There is deliberately no script that will do all the setup automatically. But in the end you will be skilled enough to debug problems yourself. If you feel you need help with your setup then read the hints on Troubleshooting. The setup in this tutorial has been tested very thoroughly by several readers. Unlike many other Postfix tutorials on the internet this is already the third edition. Writing this tutorial took a month of work so these are not just quick draft notes thrown together but a consistent document guiding you.

The configuration described here is not very complicated but still needs to be done carefully. You are expected to have at least basic knowledge of:

• MySQL (creating a database, granting access to users, basic SQL queries)
• SMTP (what it is and what a basic SMTP dialog looks like)
• POP3, IMAP (what they do and what the differences are)
• basic Postfix configuration (understand the default settings in your main.cf, have read through the basic configuration document and know that your mail log file is at /var/log/mail.log)
• Debian/Linux (general system administration, using a text editor, reading log files)

Migrating from the Sarge Tutorial

The "ispmail" tutorial is maintained since the ancient times of Debian "Woody". You may have followed former tutorials and now want to know how to upgrade to Etch properly. It is hard to provide exact instructions on what steps to go through. And you will definitely have a downtime unless you have multiple mail servers so you can migrate them one-by-one. Some basic information:

• The mailbox format and path stays the same. (Phew.)
• Courier POP/IMAP is replaced by Dovecot.
• SASL is no longer done through auxprop. It uses the Dovecot authentication setup thus saving you time.
• Passwords are now encrypted in the database.
• The virtual delivery agent is replaced by Dovecot's LDA.
• The database schema has changed completely. But there is a conversion script available to help you migrate your database.

Also you should definitely read Etch's release notes before attempting to upgrade your system from Sarge to Etch.

Please note that when switching from Courier to Dove your users will have to subscribe to their IMAP folders again. The subscription information is not migrated.

The Components

The configuration described here uses these components:

• Postfix for receiving incoming emails from the internet and storing them to the users' mailboxes on the harddisk. And for receiving emails from your users that are sent out to the internet (relaying)
• Dovecot to allow your users to get their emails into their email client through POP3 and IMAP
• Squirrelmail as a webmail interface (although any IMAP capable webmail interface will do)
• MySQL as the database system that stores information about your domains, the user accounts and email forwardings
• AMaViS for scanning incoming emails for viruses, spam and unwanted attachments

Virtual Domains in a Database

Have you ever wondered how internet service providers (ISPs) maintain their thousands of domains they receive mail for? There is surely nobody entering all these domains and aliases into a main.cf text configuration file manually. Postfix offers a better way to handle such domains and accounts by means of virtual domains and virtual users.

In addition to local users (those being listed in your /etc/passwd) Postfix can handle any number of virtual users on virtual domains. Virtual users cannot log into your computer and they have neither a user ID nor a home directory. They just exist in the database. But if you connect Postfix to the database those users can suddenly receive emails. Postfix can work on a list of those virtual users and deliver emails to any directory you like. This would look like:

You also need to tell Postfix which virtual domains you want to use. If a domain is not used on your system then Postfix will reject emails. This would be a list of domains:

So far the information could have been written into a text file. But you can imagine that this will become confusing quickly. Fortunately Postfix can also get this information from other sources like LDAP or SQL databases. So I am using SQL database tables here.

You have now seen that a mapping assigns one value to another. If you query a database you need to tell Postfix which two columns you mean. This is done through 'cf' files as documented at http://www.postfix.org/MYSQL_README.html or through "man 5 mysql_table".

Example file:

# Information on how to connect to your MySQL server
user = someone
password = some_password
hosts = 127.0.0.1

# The database name on the servers.
dbname = mailserver

# The SQL query template.
query = SELECT destination FROM virtual_aliases WHERE source='%s'

This file defines the way that Postfix can access data from your database. It would be suitable for a virtual_alias_maps mapping. Imagine you saved the above lines into a file /etc/postfix/mysql-virtual-alias-maps.cf. Then the following line in your main.cf would make Postfix query the database:

virtual_alias_maps = mysql:/etc/postfix/mysql-virtual_alias_maps.cf

How does this work? Imagine that Postfix is about to send an email to john@doe.net and wants to check the virtual alias map. Postfix then opens up a connection to the MySQL server at the IP address 127.0.0.1 and authenticates to the MySQL server with the username someone and the password some_password. It selects the database mailserver and finally runs a query:

SELECT destination FROM virtual_aliases WHERE source='john@doe.net'

Let us assume this query returns several results:

• jack@example.com
• jeff@example.com
• kerstin@example.com

That would be equal if you used a text file with aliases like this:

john@doe.net jack@example.com, jeff@example.com, kerstin@example.com

So much as a quick introduction on how mappings are used with databases.

Virtual domains versus local domains

It is important to understand the three different kinds of domains that Postfix knows. Most of the "it does not work" emails result from people mixing virtual and local domains. A domain is either a...

mydestination = localhost

Step 1: Installing the needed Debian packages

Now that you hopefully understood the basic concepts of virtual domains, mappings and database accesses it is time to get your hands dirty. Become root on your server and make sure that your /etc/hostname contains the host name without the domain part. The file /etc/mailname is supposed to contain the fully-qualified host name with the domain part.

Your /etc/hosts likely also needs to be fixed. Run hostname --fqdn and see if you get the fully-qualified hostname. If you just get the hostname without the domain please check that your /etc/hosts file has the fully-qualified hostname first in the list.

Wrong:

20.30.40.50   mailserver42 mailserver42.example.com

Right:

20.30.40.50   mailserver42.example.com mailserver42

Then begin by installing Postfix with MySQL map support:

$> aptitude install postfix-mysql

This will also install the postfix package as a dependency. When asked for the general type of configuration choose "Internet Site". Answer the question for the "Mail name" by entering the fully qualified hostname (that means including the domain part) of your system.

If you intend to run the MySQL server on the same server:

$> aptitude install mysql-server-5.0

As you probably want to offer POP3 and IMAP services to your users you need to install:

$> aptitude install dovecot-pop3d
$> aptitude install dovecot-imapd

Your users may like it if you filter out spam and (windows) viruses for them. Amavis is doing a good job here:

$> aptitude install amavisd-new libclass-dbi-mysql-perl

The Perl DBI module is needed so that AMaViS can read domain information from the database later. AMaViS is nearly useless unless you install its suggested packages to scan attachments for viruses and detect spam. You want to install these packages:

$> aptitude install spamassassin clamav-daemon cpio arj zoo nomarch lzop cabextract pax

Some of the suggested packages are not free enough to be included in Debian's main section. If you want to install them you first need to add the contrib and non-free sections to your Debian mirror in the /etc/apt/sources.list file. You usually just need to add the words contrib and non-free to your existing mirror line:

deb http://ftp.debian.org/debian/ etch main contrib non-free

Then you are ready to install the non-free packages:

$> aptitude update
$> aptitude install lha unrar

Next you will want to install the OpenSSL package so that you can later create a proper SSL certificate letting your users use your mail server securely:

$> aptitude install openssl

If you intend to offer a webmail service I can recommend the Squirrelmail package. It will automatically install an Apache server if you do not yet have one installed. Type:

$> aptitude install squirrelmail

As your control information for Postfix will be stored in a MySQL database you may want to install the PhpMyAdmin software that allows you to administer the database and its data in your web browser:

$> aptitude install phpmyadmin libapache2-mod-php5 php5-mysql

You may want to test the mail server with the simple "telnet" client later:

$> aptitude install telnet

The console-based mutt email client lets you read mail from mailboxes directly from the hard disk. It will be helpful for testing the configuration. And it's even a very powerful IMAP email client that many people user as their main mail program. Maybe you start to like it, too. You should install it:

$> aptitude install mutt

Step 2: Create the database and user

Now it is time to create the MySQL database and its tables. If you are experienced in using MySQL you can enter SQL statements on the 'mysql' command line. Alternatively you may use phpmyadmin by pointing your browser at http://yourmailserver/phpmyadmin

If you just installed your MySQL server you will be able to login as user 'root' with an empty password. Set a new password for that account now. In the shell you need to run:

$> mysqladmin password root2007

Then create the database. Call it 'mailserver' (you will be asked for the above password):

$> mysqladmin -p create mailserver

For security reasons you will want to create another less privileged MySQL user account that your mail server will use. Connect to your database:

$> mysql -p

When you see the mysql> prompt enter the following SQL statement to grant the appropriate privileges:

mysql>
GRANT SELECT ON mailserver.*
TO mailuser@localhost
IDENTIFIED BY 'mailuser2007';
exit

This will create a user called 'mailuser' that has only the privilege to select/read data from the database but not to alter it. If you want to add or alter data in the database either use the 'root' account or create another account for that purpose. The password 'mailuser2007' is just an example. Please replace it by a more decent password.

Step 3: Create the database tables

Inside the newly created database you will have to create tables that store information about domains, forwardings and the users' mailboxes. Connect to the MySQL again and choose the 'mailserver' database:

$> mysql -p mailserver

You will see the mysql> prompt again. First create a table for the list of virtual domains that you want to host:

mysql>
CREATE TABLE `virtual_domains` (
id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(50) NOT NULL
) ENGINE = InnoDB;

The next table contains information on the actual user accounts. Every user has a username and password. It is used for accessing the mailbox by POP3 or IMAP, logging into the webmail service or to send mail if they are not in your local network. As users tend to easily forget things the user's email address is also used as the login username. Create the users table:

mysql>
CREATE TABLE `virtual_users` (
id int(11) NOT NULL AUTO_INCREMENT PRIMARY KEY,
domain_id INT(11) NOT NULL,
user VARCHAR(40) NOT NULL,
password VARCHAR(32) NOT NULL,
CONSTRAINT UNIQUE_EMAIL UNIQUE (domain_id,user),
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE = InnoDB;

And finally a table is needed for aliases (email forwardings) from one account to another:

mysql>
CREATE TABLE `virtual_aliases` (
id int(11) NOT NULL AUTO_INCREMENT PRIMARY KEY,
domain_id INT(11) NOT NULL,
source VARCHAR(40) NOT NULL,
destination VARCHAR(80) NOT NULL,
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE = InnoDB;

You wonder about the foreign keys? They express that entries in the virtual_aliases and virtual_users tables are connected to entries in the virtual_domains table. This will keep the data in your database consistent because you cannot create virtual aliases or virtual users that are not connected to a virtual domain. And you avoid redundancy because you just store the domain name once - in the virtual_domains table - and nowhere else. The suffix 'ON DELETE CASCADE' means that if you delete a row from the referenced table that the deletion will also be done on the current table automatically. So you do not leave orphaned entries accidentally. Imagine that you do not host a certain domain any longer. You can remove the domain entry from the virtual_domains table and all dependent/referenced entries in the other tables will also be removed. This approach is also called a normalized database.

An example of the data in the tables:

The email addresses of these three users in the database would be:

• john@example.com
• steve@example.com
• kerstin@foobar.org

Let us add a simple alias:

This will make the mail for steve@example.com be forwarded to steve.miller@gmail.com. And the mail for kerstin@foobar.org is forwarded to both kerstin42@yahoo.com and kerstin@mycompany.com

Do not be scared if this way looks incredibly complicated. Such a database is not supposed to be maintained by manual SQL commands. But it is still important that you understand the schema of the database tables. There are already web-based administration tools that are supposed to make managing accounts easier.

Step 4: Create the database mapping files

user = mailuser
password = mailuser2007
hosts = 127.0.0.1
dbname = mailserver
query = SELECT 1 FROM virtual_domains WHERE name='%s'

$> postconf -e virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf

$> mysql -p mailserver

mysql>
INSERT INTO virtual_domains (id, name) VALUES (1, 'example.com');
exit

$> postmap -q example.com mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf

$> groupadd -g 5000 vmail
$> useradd -g vmail -u 5000 vmail -d /home/vmail -m

$> postconf -e virtual_uid_maps=static:5000
$> postconf -e virtual_gid_maps=static:5000

mysql>
INSERT INTO virtual_users (id, domain_id, user, password)
VALUES (1, 1, 'john', MD5('summersun'));

mysql>
SELECT CONCAT(virtual_users.user, '@', virtual_domains.name) AS email
FROM virtual_users
LEFT JOIN virtual_domains ON virtual_users.domain_id=virtual_domains.id;

+------------------+
| email            |
+------------------+
| john@example.com |
+------------------+

mysql>
SELECT CONCAT(virtual_users.user, '@', virtual_domains.name) AS email,
       virtual_users.password
FROM virtual_users
LEFT JOIN virtual_domains ON virtual_users.domain_id=virtual_domains.id;

+------------------+----------------------------------+
| email            | password                         |
+------------------+----------------------------------+
| john@example.com | 14cbfb845af1f030e372b1cb9275e6dd |
+------------------+----------------------------------+

mysql>
CREATE VIEW view_users AS
SELECT CONCAT(virtual_users.user, '@', virtual_domains.name) AS email,
    virtual_users.password
FROM virtual_users
LEFT JOIN virtual_domains ON virtual_users.domain_id=virtual_domains.id;

mysql>
SELECT * FROM view_users;

+------------------+----------------------------------+
| email            | password                         |
+------------------+----------------------------------+
| john@example.com | 14cbfb845af1f030e372b1cb9275e6dd |
+------------------+----------------------------------+

user = mailuser
password = mailuser2007
hosts = 127.0.0.1
dbname = mailserver
query = SELECT 1 FROM view_users WHERE email='%s'

$> postconf -e virtual_mailbox_maps=mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

$> postmap -q john@example.com mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

mysql>
INSERT INTO virtual_aliases (id, domain_id, source, destination)
VALUES (1, 1, 'john', 'john@example.com'),
       (2, 1, 'john', 'jmiller@gmail.com');

mysql>
CREATE VIEW view_aliases AS
SELECT CONCAT(virtual_aliases.source, '@', virtual_domains.name) AS email,
       destination
FROM virtual_aliases
LEFT JOIN virtual_domains ON virtual_aliases.domain_id=virtual_domains.id;

mysql>
SELECT * FROM view_aliases;

+------------------+-------------------+
| email            | destination       |
+------------------+-------------------+
| john@example.com | john@example.com  |
| john@example.com | jmiller@gmail.com |
+------------------+-------------------+

user = mailuser
password = mailuser2007
hosts = 127.0.0.1
dbname = mailserver
query = SELECT destination FROM view_aliases WHERE email='%s'

$> postmap -q john@example.com mysql:/etc/postfix/mysql-virtual-alias-maps.cf

john@example.com,jmiller@gmail.com

user = mailuser
password = mailuser2007
hosts = 127.0.0.1
dbname = mailserver
query = SELECT email FROM view_users WHERE email='%s'

$> postmap -q john@example.com mysql:/etc/postfix/mysql-email2email.cf

john@example.com

$> postconf -e virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf,mysql:/etc/postfix/mysql-email2email.cf

chgrp postfix /etc/postfix/mysql-*.cf
chmod u=rw,g=r,o= /etc/postfix/mysql-*.cf

Step 5: Deliver emails through the Dovecot LDA

Postfix comes with a mail delivery agent (MDA) called "virtual" that will usually deliver emails to virtual mailboxes. But Dovecot comes with an own (local) delivery agent also called "Dovecot LDA" that offers more functionality. To make Postfix use that agent you will have to add a service to your /etc/postfix/master.cf:

dovecot   unix  -       n       n       -       -       pipe
    flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient}

Also make Postfix use that service for virtual delivery lines to your /etc/postfix/main.cf:

$> postconf -e virtual_transport=dovecot
$> postconf -e dovecot_destination_recipient_limit=1

So far this will make Postfix pass on incoming emails to virtual users to the /usr/lib/dovecot/deliver program. Now it is time to configure Dovecot.