When you tell your MUA to send a message, it simply hands off the message to a mail server running a mail transfer agent (MTA). Figure 1-1 shows the components involved in a simple email transmission from sender to recipient. MTAs (like Postfix) do the bulk of the work in getting a message delivered from one system to another. When it receives a request to accept an email message, the MTA determines if it should take the message or not. An MTA generally accepts messages for its own local users; for other systems it knows how to forward to; or for messages from users, systems, or networks that are allowed to relay mail to other destinations. Once the MTA accepts a message, it has to decide what to do with it next. It might deliver the message to a user on its system, or it might have to pass the message along to another MTA. Messages bound for other networks will likely pass through many systems. If the MTA cannot deliver the message or pass it along, it bounces the message back to the original sender or notifies a system administrator. MTA servers are usually managed by Internet Service Providers (ISPs) for individuals or by corporate Information Systems departments for company employees.

Figure 1-1. Simple Internet message flow

Ultimately, a message arrives at the MTA that is the final destination. If the message is destined for a user on the system, the MTA passes it to a message delivery agent (MDA) for the final delivery. The MDA might store the message as a plain file or pass it along to a specialized database for email. The term message store applies to persistent message storage regardless of how or where it is kept.

Once the message has been placed in the message store, it stays there until the intended recipient is ready to pick it up. The recipient uses an MUA to retrieve the message and read it. The MUA contacts the server that provides access to the message store. This server is separate from the MTA that delivered the message and is designed specifically to provide access for retrieving messages. After the server successfully authenticates the requester, it can transfer that user’s messages to her MUA.

Because Internet email standards are open, there are many different software packages available to handle Internet email. Different packages that implement the same protocols can interoperate regardless of who wrote them or the type of system they are running on. If you are putting together a complete email system, most likely the software that handles SMTP will be a different package than the software that handles POP/IMAP, and there are many different software choices for each aspect of your complete email system.

The communications that occur between each of these email system components are defined by standards and protocols. The standards documents are maintained by the Internet Engineering Task Force (IETF) and are published as Request For Comments (RFC) documents, which are numbered documents that explain a particular technology or protocol.

The Simple Mail Transport Protocol (SMTP) is used for sending messages, and either the Post Office Protocol ( POP) or Internet Mail Application Protocol ( IMAP) is used for retrieving messages. SMTP, defined in RFC 2821, describes the conversation that takes place between two hosts across a network to exchange email messages. The IMAP (RFC 2060) and POP (RFC 1939) protocols describe how to retrieve messages from a message store. The IMAP protocol was developed after POP and offers additional features. In both protocols, email messages are kept on a central server for message recipients who generally retrieve them across a network.

Note that the MUA does not necessarily use the same system for POP/IMAP as it does for SMTP, which is why email clients have to be configured separately for POP/IMAP and SMTP. An ISP might provide separate servers for each function to their customers, and corporate users who are away from the office often retrieve their messages from the company POP/IMAP server, but use the SMTP server of a dial-up ISP to send messages. MTA software running on SMTP servers constantly listens for requests to accept messages for delivery. Requests might come from MUAs or other MTA servers.

The modular architecture of Postfix forms the basis for much of its security. Each Postfix process runs with the least amount of privilege necessary to get its particular job done. Many of Sendmail’s security problems were exacerbated because Sendmail ran as a privileged process most of the time. Postfix operates with the minimum privilege necessary to accomplish a particular task. Postfix processes that are not needed on a system can be turned off, making it impossible to exploit them. For example, a network firewall system that only relays mail and does not need local delivery can have all the Postfix components for local delivery turned off. Postfix processes are insulated from each other and depend very little on any interprocess communication. Each process determines for itself what it needs to know.

In most cases, the delivery of mail does not require a Unix shell process, but when a configuration does make use of one, Postfix sanitizes information before placing it into environment variables. Postfix tries to eliminate any harmful characters that might have special meaning to a shell before making any data available to the shell.

Most Postfix processes are executed by a trusted master daemon. They do not run as user child processes, so they are immune to any of the security problems that rely on parent-child inheritance and communications. These attacks that use signals, shared memory, open files, and other types of interprocess communication are essentially useless against Postfix.

A buffer overflow is another common type of attack against applications. In this type of attack, crackers cause a program to write to memory where it is not supposed to. Doing so might allow them to change the path of execution in order to take control of the process. I’ve already mentioned that Postfix processes run with as little privilege as possible, so such an attack would not get very far; moreover, Postfix avoids using fixed-size buffers for dynamic data, making a successful buffer overflow attack highly unlikely.

An important security protection available on Unix systems is the ability to chroot applications. A chroot establishes a new root directory for a running application such as /var/spool/postfix. When that program runs, its view of the filesystem is limited to the subtree below /var/spool/postfix, and it cannot see anything else above that point. Your critical system directories and any other programs that might be exploited during an attack are not accessible. Postfix makes it very simple to cause its processes to run within a chroot (see more about chrooting in Chapter 4). By doing so, you cause Postfix to run in its own separate compartment. Even if Postfix is somehow subverted, it will not provide access to many of the methods an attacker typically employs to compromise a system.

Because Postfix is designed to run even under stressful conditions, denial-of-service (DOS) attacks are much less effective. If a system runs out of disk space or memory due to a DOS attack or another type of problem, Postfix is careful not to make the situation worse. It backs off from what it is trying to do to allow the system to recover. Postfix processes are configured to use a limited amount of memory, so they do not grow uncontrollably from an onslaught of messages.

The difficulty in planning for security is that you don’t know what the next attack will be or how it will be carried out. Postfix is designed to deal with adverse conditions no matter what their cause. Its built-in robustness is a major factor in the degree of security that Postfix provides. Indeed, Dr. Venema has said that he is not so much interested in security as he is in creating software that works as intended, regardless of the circumstances. Security is just a beneficial side effect.

The list of recognized users on a system is stored in the /etc/passwd file. Every user should have a unique login name and user ID number (commonly written as uid or UID). The UID, not the user’s login name, is the important attribute for identity and ownership checks. The login name is a convenience for humans, and the system uses it primarily to determine what the UID is. Some Postfix configuration parameters require UIDs rather than login names when referring to user accounts. Postfix sometimes takes on the identify of different users. A process is said to be using the rights or privileges of that account when assuming its identity.

A pseudo-account is a normal Unix system account except that it does not permit logins. These accounts are used to perform administrative functions or to run programs under specific user privileges. Your system most likely came installed with several pseudo-accounts. Account names such as bin and daemon are common ones. Generally, these accounts prevent logins by using an invalid password and nonexistent home directories and login shells. For Postfix administration, you need at least one pseudo-account for Postfix processes to run under. You may need additional ones for other functions, such as mailing-list programs and filters.

Nearly all processes on a Unix system have a standard input stream and a standard output stream when they start. They read data on their standard input and write data on their standard output. Normally, standard input is the keyboard and standard output is the monitor, which is how users interact with running programs. Standard input and output can be redirected so that programs can get input from, and send output to, a file or another program. This is often how batch mode programs operate. For the purpose of email, you should be aware of standard input and output because your mail system may have to interact with other programs over their standard inputs and outputs. A mail filter program, for example, might accept the contents of an email message on its standard input and send the revised contents to its standard output. Programs usually also have a standard error stream that, like standard output, is normally a user’s monitor, but it can also be redirected. Standard input/output/error are often written as stdin, stdout, and stderr. For more information, consult an introductory book on Unix.

The administrative login on Unix systems is the root account. It is also referred to as the superuser account, and you should treat it carefully. You should log in as the root user only when its privileges are required to accomplish a particular task. Administering Postfix sometimes requires root privileges. If you do not have superuser access on your system, you cannot administer Postfix.

When working with an interactive shell, you are normally greeted with a command prompt that indicates the system is ready for you to enter a command. By convention, user command prompts are shown as either the $ character or the % character, while the root prompt is presented as the # character. You should use the root account only when it is necessary. In examples in this book, a normal user prompt is shown as $, and that for root is shown as #. If the example shows the prompt as #, you know that you must execute the command as root.

It is common usage in Unix to break long commands into multiple lines with a backslash (\) at the end of the line, which indicates that two or more lines continue as if they were a single line. The continuation backslash can be used at a command prompt and in shell scripts, and it is commonly used in configuration files (but not in Postfix configuration files—see Chapter 4). In this book, lines that don’t fit on the page are continued with backslashes. If you follow the examples, you can type lines exactly as shown with the backslashes, or simply combine the continued lines into a single one.

Documentation for Unix systems is kept in an online manual known as manpages. You can read the documentation for a particular item by issuing the man command with the item as its argument. For example, to read about the mailq command, simply type:

$ man mailq

A description of the command is presented on your screen, one page at a time. Press the spacebar to continue scrolling through the information.

Manpages have a standard organization showing the syntax of the command, all options, and descriptions of behavior and other context. Some users find manpages daunting, but you’ll do yourself a great service by getting comfortable with manpages. All Unix and Postfix commands as well as many other features are documented in manpages. See an introductory Unix text or your system documentation to learn more about manpages.

RFCs, or Request for Comments documents, define the standards for the Internet. There are several RFCs relating to Internet email, all of which are relevant to you if you are administering an email system on the Internet. The two most commonly referenced RFCs for email are RFC 821 and RFC 822, which deal with how email messages are transferred between systems, and how email messages should appear. These documents were put into effect more than 20 years ago. They were updated in April 2001 with the proposed standards RFC 2821 and RFC 2822, although you will still see many references to the original documents. RFC documents are maintained by the Internet Engineering Task Force, whose site is available at http://www.ietf.org/.

Chapter 1 introduced several of the email agents involved in message composition to final delivery. For convenience, Table 2-1 contains a summary of these agents.

An email administrator is commonly referred to as a postmaster . An individual with postmaster responsibilities makes sure that the mail system is working correctly, makes configuration changes, and adds/removes email accounts, among other things. You must have a postmaster alias at all domains for which you handle email that directs messages to the correct person or persons. RFC 2142 specifies that a postmaster address is required.

If a receiving MTA determines during the SMTP conversation (see Section 2.2.8 later in the chapter) that it will not accept the message, it rejects the message. At that point the sending system should generate an error report to deliver to the original sender. Sometimes the MTA accepts a message and later discovers that it cannot be delivered—perhaps the intended recipient doesn’t exist or there is a problem in the final delivery. In this case, the MTA that has accepted the message bounces it back to the original sender by sending an error report, usually including the reason the original message could not be delivered.

The MTA that accepts a message takes responsibility for the message until it is delivered or handed off to another MTA. When a system is responsible for a message and cannot deliver or relay it, the responsible system informs the sender that the mail is undeliverable.

A common source of confusion for email users is the fact that the To: address in email message headers has nothing to do with where a message is actually delivered. The envelope address controls message delivery. In practice, when you compose a message and provide your MUA with a To: address, your MUA uses that same address as the envelope destination address, but this is not required nor is it always the case. From the MTA’s point of view, message headers are part of the content of an email message. The delivery of a message is determined by the addresses specified during the SMTP conversation. These addresses are the envelope addresses , and they are the only thing that determine where messages go. See Section 2.2.8 later in the chapter for an explanation of the SMTP protocol.

Mailing lists and spam are common examples of when the envelope destination address differs from the To: address of the message headers. For more information, see RFC 2821 and RFC 2822. Also see Section 2.2.7 later in the chapter for more information about the format of email messages. If you follow the SMTP session in Example 2-2, try substituting any address you want in the To: field of the message contents to see that it has no effect on where the message is delivered.

RFC 2822 describes the format of email addresses in great detail. It specifies how things such as quoting and comments should work in email addresses. If we ignore the more obscure details, a simple email address is generally composed of three parts: the local part (which is usually a username), the @ separator, and the domain name . The local part might also be an alias to another address or to a mailing list. The local part is sometimes referred to as the lefthand side (LHS), and the domain is sometimes called the righthand side (RHS). For more information, see RFC 2822.

Since RFC 822 was the document that originally described how Internet email messages should be formatted, messages are commonly referred to as “in the RFC 822 format” or as an “RFC 822 message.” You should understand the basics of the format since it is referred to in this book and you will likely see it elsewhere. I’ll use the newer proposed standard and refer to “RFC 2822 messages.”

Return-Path: <info@oreilly.com>
Delivered-To: kdent@mail.example.com
Received: from mail.oreilly.com (mail.oreilly.com [192.168.145.34])
        by mail.example.com (Postfix) with SMTP id 5FA26B3DFE
        for <kdent@example.com>; 
        Mon, 8 Apr 2003 16:40:29 -0400 (EDT)
Date: Mon, 8 Apr 2003 15:38:21 -0500
From: Customer Service <info@oreilly.com>
To: <kdent@example.com>
Reply-To: <info@oreilly.com>
Message-ID: <01a4e2238200842@mail.oreilly.com>
Subject: Have you read RFC 2822?

This is the start of the body of the message. It could continue
for many lines, but it doesn't.

The SMTP protocol is defined in RFC 2821. The protocol is actually quite simple to follow, and was designed to be easily comprehensible both to humans and computers. A client connects to an SMTP server, whereupon the server begins the SMTP conversation, which consists of a series of simple commands and replies, including the transmission of the email message. The best way to understand the protocol is to see it in action. You can easily try it yourself once you have your mail server set up. Using a Telnet client, you can pose as a delivering MTA. Example 2-2 shows the steps and the basic commands to deliver a message.

$ telnet mail.example.com 25
Trying 10.232.45.151
Connected to mail.example.com.
Escape character is '^]'.
220 mail.example.com ESMTP Postfix
HELO mail.oreilly.com
250 mail.oreilly.com
MAIL FROM:<info@oreilly.com>
250 Ok
RCPT TO:<kdent@example.com>
250 Ok
DATA
354 End data with <CR><LF>.<CR><LF>

Date: Mon, 8 Apr 2003 15:38:21 -0500
                  From: Customer Service <info@oreilly.com>
                  To: <kdent@example.com>
                  Reply-To: <service@oreilly.com>
                  Message-ID: <01a4e2238200842@mail.oreilly.com>
                  Subject: Have you read RFC 2822?

                  This is the start of the body of the message. It could continue
                  for many lines, but it doesn't.
                  .

250 Ok: queued as 5FA26B3DFE
quit
221 Bye
Connection closed by foreign host.
$

The SMTP session depicted in Example 2-2 is actually the delivery that produced the sample message in Example 2-1. To follow the example yourself, start by using a Telnet client to connect to the mail server on port 25 at mail.example.com. You should connect to your own Postfix server and type in your own email addresses for the envelope addresses. Port 25 is the well-known port for SMTP servers. After the Telnet messages:

Trying 10.232.45.151
Connected to localhost.
Escape character is '^]'.

the server greets you with its banner:

220 mail.example.com ESMTP Postfix

SMTP server replies, such as the greeting message, always start with a three-digit response code, usually followed by a short message for human consumption. Table 2-2 provides the reply code levels and their meanings. The first digit of the response code is enough to know the status of the requested command. In documentation the response codes are often written as 2xx to indicate a level 200 reply.

After receiving the welcome banner, introduce yourself with the HELO command. The hostname after the HELO command should be the name of the system you’re connecting from:

HELO mail.oreilly.com

The server replies with a success. So you may continue:

250 mail.oreilly.com

Indicate who the message is from with the MAIL FROM command:

MAIL FROM:<info@oreilly.com>

The server accepts the sending address:

250 Ok

Indicate who the message is to with the RCPT TO command:

RCPT TO:<kdent@example.com>

The server accepts the recipient address:

250 Ok

Now you are ready to send the content of the message. The DATA command tells the server that you have an RFC 2822 message ready to transfer:

DATA

The server replies that it accepts the command and is expecting you to begin sending data:

354 End data with <CR><LF>.<CR><LF>

At this point, you can transfer the entire contents of your message. The contents of messages start with the message headers. When the message itself is finished, indicate the end by sending a single period on a line by itself.

The server acknowledges the end of your message and replies that the transfer was successfully completed:

250 Ok: queued as 5FA26B3DFE

At this point the server has taken responsibility for the message. If you wanted to continue with more commands, you could do so now. Since you have no other messages to deliver to this server, you can start to disconnect with the quit command:

quit

The server replies with a success and disconnects:

221 Bye

Finally, the Telnet client tells you that the connection has ended returns to the command prompt:

Connection closed by foreign host.
$

This was, of course, the simplest example of an SMTP transaction. The basic protocol provides additional commands and has been extended to allow for many enhancements. RFC 1869 provides a framework for adding additional features to the basic SMTP protocol. The enhanced protocol is referred to as ESMTP. A client indicates its willingness to use the enhanced protocol by beginning with the EHLO command instead of HELO. If the server also supports enhancements, it replies with a list of the features it provides.

Many enhancements have been specified in various RFCs. You can learn about them by searching for SMTP information on the IETF web site (http://www.ietf.org/). There are many other resources available on the Web regarding the SMTP and ESMTP protocols .

The various Postfix components work together by writing messages to and reading messages from the queue. The queue manager has the responsibility of managing messages in the queue and alerting the correct component when it has a job to do.

Figure 3-2 illustrates the flow when a local email message enters the Postfix system. Local messages are deposited into the maildrop directory of the Postfix queue by the postdrop command, usually through the sendmail compatibility program. The pickup daemon reads the message from the queue and feeds it to the cleanup daemon. Some messages arrive without all of the required information for a valid email message. So in addition to sanity checks on the message, the cleanup daemon, in conjunction with the trivial-rewrite daemon inserts missing message headers, converts addresses to the user@domain.tld format expected by other Postfix programs, and possibly translates addresses based on the canonical or virtual lookup tables (see Chapter 4 for more information on lookup tables).

The cleanup daemon processes all inbound mail and notifies the queue manager after it has placed the cleaned-up message into the incoming queue. The queue manager then invokes the appropriate delivery agent to send the message to its next hop or ultimate destination.

Figure 3-2. Local email submission

Figure 3-3 illustrates the flow when a network email message enters the Postfix system. Messages received over the network are accepted by the Postfix smtpd daemon. This daemon performs sanity checking and can be configured to allow clients to relay mail on the system or deny them from doing so. The smtpd daemon passes the message to the cleanup daemon, which performs its own checks then deposits the message into the incoming queue. The queue manager then invokes the appropriate delivery agent to send the message to its next hop or ultimate destination.

Figure 3-3. Email from the network

When a user message is deferred or can’t be delivered, Postfix uses the defer or bounce daemons to create a new error message. The error message is handed off to the cleanup daemon. It performs its normal checks before depositing the error message into the incoming queue, where it is picked up by the queue manager.

Sometimes, after processing an email message, Postfix determines that the destination address actually points to another address on another system. It could, at that point, simply hand off the message to the SMTP client for immediate delivery, but to make sure that every recipient is processed and logged correctly, Postfix resubmits it as a new message where it is handled like any other locally submitted message.

The local delivery agent handles mail for users with a shell account on the system where Postfix is running. Domain names for local delivery are listed in the mydestination parameter. Messages sent to a user at any of the mydestination domains are delivered to the individual shell account for the user. In the simple case, the local delivery agent deposits an email message into the local message store. It also checks aliases and users’ .forward files to see if local messages should be delivered elsewhere. See Chapter 7 for more information on local delivery.

When a message is to be forwarded elsewhere, it is resubmitted to Postfix for delivery to the new address. If there are temporary problems delivering the message, the delivery agent notifies the queue manager to mark the message for a future delivery attempt and store it in the deferred queue. Permanent problems cause the queue manager to bounce the message back to the original sender.

Virtual alias addresses are all forwarded to other addresses. Domain names for virtual aliasing are listed in the virtual_alias_domains parameter. Every domain has its own set of users that do not have to be unique across domains. Users and their real addresses are listed in lookup tables specified in the virtual_alias_maps parameter. Messages received for virtual alias addresses are resubmitted for delivery to the real address. See Chapter 8 for more information on virtual aliases.

The virtual delivery agent handles mail for virtual mailbox addresses. These mailboxes are not associated with particular shell accounts on the system. Domain names for virtual mailboxes are listed in the virtual_mailbox_domains parameter. Every domain has its own set of users that do not have to be unique across domains. Users and their mailbox files are listed in lookup tables specified in the virtual_mailbox_maps parameter. See Chapter 8 for more information on virtual mailboxes.

The smtp delivery agent handles mail for relay domains. Email addresses in relay domains are hosted on other systems, but Postfix accepts messages for the domains and relays them to the correct system. Relay configurations are common when Postfix accepts mail over the Internet and passes it to systems on an internal network. Domain names for relay domains are listed in the relay_domains parameter. See Chapter 9 for more information on relaying.

Messages that do not fit into one of the address classes are generally destined for other domains hosted elsewhere on the network. Postfix accepts such messages only from authorized clients, such as systems that run on the same local network. When a message has to be delivered across the network, the queue manager calls the smtp delivery agent. The smtp agent determines which host or hosts can receive the message and makes a connection to each in turn until it finds one to accept it. If there are temporary problems delivering the message, the smtp delivery agent notifies the queue manager to mark the message for a future delivery attempt and store it in the deferred queue. Permanent problems cause the queue manager to bounce the message back to the original sender.

When a destination system that has been unavailable comes back online, Postfix is careful not to overwhelm it with all its pending messages. Whether delivering previously deferred messages or new messages, Postfix, at first, makes only a limited (configurable) number of connections to a receiving system. After Postfix has detected successful deliveries to a particular site, it slowly increases (up to a configurable limit) simultaneous connections to it. If Postfix detects any trouble from the receiving site, it starts to back off deliveries immediately.

There are other Postfix delivery agents that can be configured to handle messages for a particular class or destination. Other delivery agents must be configured in the master.cf file. They are invoked either through the class _transport parameter or through an entry in a transport table, listed in the transport_maps parameter. Two common alternate delivery agents are the lmtp and pipe agents.

The main.cf file is the core of your Postfix configuration. Nearly all configuration changes occur in this file. The default main.cf file lists only a portion of the nearly 300 Postfix parameters. Most Postfix parameters do not need to be changed, but the flexibility is there when it’s required. All Postfix parameters are listed and described in the various sample configuration files. The sample files are located in the directory specified by the sample_directory parameter, which is usually the same directory as your main.cf file. Both the main.cf file and the sample files that come with the Postfix distribution contain comments that explain each of the parameters.

You can edit main.cf with the postconf command, as you saw earlier in the chapter, or you can change the file directly with any text editor^[2] (such as vi or emacs). The file contains blank lines, comment lines, and lines that assign values to parameters. Comment lines start with the # character and continue to the end of the line. Blank and comment lines are ignored by Postfix. Parameters can appear in any order within the file, and are written as you would expect:

               parameter = value

A parameter definition must start in the first column of the line. The spaces around the equals sign are optional.

Here is an example parameter assignment with a comment:

# The myhostname value must be a fully qualified hostname.
myhostname = mail.example.com

# The rest of the file continues below...

You cannot have a comment on the same line as a parameter. The following example is incorrect and, with some parameters, could cause unexpected behavior that might be difficult to track down:

#
# This is a bad parameter assignment. Never do this.
#
myhostname = mail.example.com   # must be fully qualified hostname

Do not use quotation marks around values. They have no significance in the Postfix configuration, so they would be considered part of the value, which is probably not what you want.

mydestination = example.com oreilly.com ora.com postfix.org

mydestination = example.com
  oreilly.com
  ora.com
  postfix.org

mydomain = example.com
myorigin = $mydomain

myorigin = $mydomain
mydomain = example.com

mydestination = $mydomain, example.com, oreilly.com
mydestination = $mydomain example.com oreilly.com
mydestination = $mydomain
     example.com
     oreilly.com

mydestination = /etc/postfix/destinations

# postfix reload

Rather than using complicated rewriting or pattern transformation rules as Sendmail does, Postfix makes use of simple, yet flexible, lookup tables. Many parameters point to lookup tables to obtain important configuration information. One such parameter is canonical_maps . It’s used to rewrite email addresses in messages. Consider a site that uses account names internally for email addresses, but wants any publicly visible addresses to have the form firstname.lastname@example.com. For example, the address kdent@example.com should appear as kyle.dent@example.com. A canonical_maps lookup table provides the mapping from a key (kdent@example.com) to a value (kyle.dent@example.com).

There are many parameters that use lookup maps, but they all work on the same principle. An email message (or client) provides some kind of key used to look up a value. Based on the value, Postfix takes some action or makes some change.

#
# canonical mappings
#
kdent@example.com     kyle.dent@example.com

# postmap /etc/postfix/canonical

# postmap -q kdent@example.com /etc/postfix/canonical
kyle.dent@example.com

$ postconf -m
static
pcre
nis
regexp
environ
proxy
btree
unix
hash

$ postconf default_database_type
default_database_type = hash

                  parameter = type:name

canonical_maps = hash:/etc/postfix/canonical

parent_domain_matches_subdomains =
   debug_peer_list
   fast_flush_domains
   mynetworks
   permit_mx_backup_networks
   qmqpd_authorized_clients
   relay_domains
   smtpd_access_maps
   transport_maps
transport_maps = hash:/etc/postfix/transport

body_checks = regexp:/etc/postfix/re_body_checks

/pattern/   value

Postfix can make use of other backend systems for its lookup tables. (Later chapters discuss using MySQL and LDAP lookup tables.) When you make use of these external sources for lookup values, you should start with one of the simple database formats, such as dbm or hash. Make sure your configuration works as expected. After setting up your external data source, verify that it returns the same results as your simple tables.

The postmap with the -q option is an important tool for testing any kind of lookup table. For example, the following two commands should return the same values when you test your MySQL database:

$ postmap -q hash:/etc/postfix/transport

$ postmap -q mysql:/etc/postfix/transport.cf

See Chapter 15 for more information on using Postfix with external data sources.

Alias files are a special case of Postfix lookup tables because they use a Sendmail-compatible format. The file has traditionally been called aliases, and its location depends on your platform, but it is normally within the /etc directory or a subdirectory below it. By default, Postfix is configured to point to your original aliases file, so if you are migrating from Sendmail, your existing aliases continue to work.

alias_maps = hash:/etc/aliases, nis:mail.aliases

alias_maps = hash:/etc/aliases

alias_maps = hash:/etc/postfix/aliases

alias_database = hash:/etc/postfix/aliases

# postalias /etc/aliases

                  alias: target1, target2, ...

allow_mail_to_commands =
allow_mail_to_files =

allow_mail_to_commands = alias, forward, include
allow_mail_to_files = alias, forward, include

There are four parameters dealing with your system’s hostname and domain that you want to consider, no matter how you use Postfix: myhostname, mydomain, myorigin, and mydestination.

myorigin = $mydomain

mydestination = $myhostname, localhost.$mydomain, $mydomain

In addition to accepting mail and delivering messages to your local users, Postfix also relays messages to other systems. It’s very important to restrict who is allowed to relay messages through your system. Systems on your own network may require the ability to send messages anywhere, but you do not want to provide the rest of the world with the same service. Relay control is an important topic in email administration because of the prevalence of Unsolicited Bulk Email (UBE), or spam. (See Chapter 11for more information on UBE.) A common practice among spammers is to find a well-connected system that allows them to relay their mail. You want to prevent anyone who is not authorized from using your system to relay mail. If you leave yourself configured as an open relay, not only will you be contributing to the spam problem, but your own machine may become unusable as it is abused by spammers. Furthermore, you may find that other systems start refusing mail from you as they discover that your system is the source of spam. They’ll refuse the spam as well as any legitimate messages your own systems send. Mail servers that permit anyone to relay mail are called open relays .

Since Postfix is a long-running program, you should regularly check your system’s log file for warnings or messages. Things can change on your system that might impact Postfix. Almost all Postfix activity, successful or not, is logged. Whenever you start or reload Postfix, it is a good idea to check your log file for messages.

Postfix logging is accomplished by using your system’s syslog daemon. System log files are an aspect of system administration that vary across versions of Unix, so you may have to consult your own system documentation to fully understand Postfix logging.

In general, the syslog daemon (syslogd) receives messages from various system processes and writes them to their final destination (often a file). syslogd organizes messages according to their importance and the application or facility that generated the message. The file /etc/syslog.conf tells syslogd where to write each type of message. The logging facility used by Postfix is mail . If you don’t know where to find messages logged by Postfix, the file /etc/syslog.conf should point you in the right direction. Some operating systems, by convention, log nearly everything to a single file, such as /var/log/syslog, while others prefer to separate messages by applications or services, so that Postfix messages go to a file like /var/log/maillog. For the latter type of systems, you might find an entry like the following in /etc/syslog.conf:

mail.*         -/var/log/maillog

Once you locate your mail log file, check it regularly. You’ll probably want to check it at least daily, but decide for yourself, depending on the volume of mail your server handles and your existing log rotation scheme. You can use the following command to find Postfix messages that might be of interest:

$ egrep '(reject|warning|error|fatal|panic):' /var/log/maillog

assuming that your log file is /var/log/maillog. If not, substitute the name of your own mail log file.

You saw earlier in the chapter how to use the postfix command to start Postfix:

# postfix start

Once Postfix is running, if you make any changes to main.cf or master.cf, have Postfix reread its configuration by executing postfix with the reload argument:

# postfix reload

Postfix gracefully terminates running processes after they have finished any tasks they are working on, rereads its configuration files, and continues to receive mail without interruption.

The most important thing when starting or reloading Postfix is to check your system log to see if Postfix reports any errors or warnings.

You can stop Postfix with the stop argument. Running processes will still finish any tasks they’re working on and then terminate:

# postfix stop

You should not stop and start Postfix when a reload will suffice. Also, do not stop, restart, or reload frequently, since any of these actions can impact performance.

Most systems automatically start Postfix when they boot up because of Postfix’s built-in Sendmail compatibility. Sendmail is typically launched at startup with a command like:

sendmail -bd -q15m

The Postfix sendmail command understands nearly all of the same options as Sendmail, so if your server already has scripts that start Sendmail, those same scripts will start Postfix. One common Sendmail option ignored by Postfix is -q , which is used by Sendmail to specify the time between queue scans. The time between queue scans for Postfix is set in the main.cf file with the queue_run_delay parameter, which defaults to 1000 seconds.

Your system may have a configuration option to turn on automatic startup of Sendmail. After you install Postfix, turning on this option should be sufficient to cause Postfix to start at system initialization. Different versions of Unix have different idioms for configuring a server to start a process at system initialization. If your system’s Sendmail start script doesn’t work, or you prefer to use a Postfix-specific script, you can easily create a start script.

#!/sbin/sh
#
# Set the path to your own logger and postfix commands.
#
LOGGER="/usr/bin/logger"
POSTFIX="/usr/sbin/postfix"
rc=0

if [ ! -f $POSTFIX ] ; then
    $LOGGER -t $0 -s -p mail.err "Unable to locate Postfix"
    exit(1)
fi
if [ ! -f /etc/postfix/main.cf ] ; then
    $LOGGER -t $0 -s -p mail.err "Unable to locate Postfix configuration"
    exit(1)
fi

case "$1" in
    start)
        echo -n "Starting Postfix"
        $POSTFIX start
        rc=$?
        echo "."
        ;;

    stop)
        echo -n "Stopping Postfix"
        $POSTFIX stop
        rc=$?
        echo "."
        ;;

    restart)
        echo -n "Restarting Postfix"
        $POSTFIX reload
        rc=$?
        echo "."
        ;;

    *)
        echo "Usage: $0 {start|stop|restart}"
        rc=1

esac
exit $rc

# ln -s /etc/init.d/postfix /etc/init.d/rc2.d/S95postfix

The Postfix queue is also an important part of email administration. See Chapter 5 for information on the Postfix queue manager.

Postfix provides another type of address rewriting that lets you map disparate addresses into a standard format for your entire site. The canonical_maps parameter points to a lookup table of address mappings. (While the word canonical has many meanings, among computer professionals it means “the usual, standard, or normal.”) If different mail systems on your network create addresses in different ways, you can relay them all through your Postfix gateway and have it fix up the addresses into your standard format. Canonical maps are often used to change addresses from an internal format to a public one. Include entries like the following in your canonical table:

#
# /etc/postfix/canonical
#
pabelard@example.com   peter.abelard@example.com
hfulbert@example.com   heloise.fulbert@example.com

They can also rewrite addresses completely.

#
# /etc/postfix/canonical
#
pabelard@example.com   abelard@oreilly.com
hfulbert@example.com   heloise@oreilly.com

In main.cf, point the canonical_maps parameter to the canonical file:

canonical_maps = hash:/etc/postfix/canonical

Be sure to execute postmap against your canonical file and reload Postfix so that it recognizes your changes to main.cf:

# postmap /etc/postfix/canonical
# postfix reload

The canonical_maps parameter affects all of the addresses, including envelope and message headers. If Postfix finds a match, it makes the change. If you want your changes to affect only sender or recipient addresses, Postfix provides the additional parameters sender_canonical_maps and recipient_canonical_maps. They both work the same as canonical_maps, but only on their respective classes of addresses. If you use either of these two parameters in addition to canonical_maps, Postfix first fixes the addresses according to sender_canonical_maps and recipient_canonical_maps, and then canonical_maps.

Address masquerading refers to the idea that you can hide the names of internal hosts, and make all addresses appear as if they originated from the gateway system itself. You may have internal systems that use your Postfix server as a gateway. When mail is sent from these systems and the sender addresses include the fully qualified hostname, you may want addresses to appear with the domain name only. The masquerade_domains parameter strips hostnames down to their simpler domain names.

The parameter takes a list of domains. Any address whose fully qualified hostname matches the domain portion is stripped down to just the domain name:

masquerade_domains = example.com

Addresses that look like heloise@server1.example.com and frank@server2.example.com are converted to heloise@example.com and frank@example.com.

You can list multiple domains and subdomains. Postfix processes addresses against masquerade domain names in the order you list them. Consider a network that includes the two subdomains, acct.example.com and hr.example.com. You want addresses from these domains to show the subdomain, but you want addresses from any other domain or host in the network to show the parent domain. Set masquerade_domains as follows:

masquerade_domains = acct.example.com hr.example.com example.com

With this setting, the address heloise@sys3.acct.example.com matches acct.example.com, so that it becomes heloise@acct.example.com. The address frank@db.hr.example.com matches hr.example.com, and becomes frank@hr.example.com. Finally, helene@server1.example.com matches the last value, example.com, to become helene@example.com.

If you want to preserve a domain name that would otherwise be stripped down, you can preface the domain with an exclamation point:

masquerade_domains = !it.example.com, example.com

In this case, the domain it.example.com will not be rewritten, so the address kdent@it.example.com stays as it is.

You can exclude specific account names from masquerading. For example, if you want an address like root@db.example.com to stay intact, add the account to the masquerade_exceptions parameter:

masquerade_exceptions = admin, root

When you use masquerading, it is normally applied to all envelope and header addresses but not envelope recipient addresses. This allows mail addressed to a specific host to be delivered from the mail gateway to that particular system, while still rewriting addresses for messages sent from the host. If you prefer to have all addresses masqueraded, set the masquerade_classes parameter to include the complete list of address classes recognized by Postfix:

masquerade_classes = envelope_recipient, envelope_sender,
        header_sender, header_recipient

Be aware that if you set masquerade_classes this way, a gateway mail system may no longer know where to deliver a message that was originally addressed to kdent@server1.example.com once it has been rewritten as kdent@example.com.

The relocated_maps parameter points to a lookup table where you can store a list of addresses or domains that have moved to another location:

relocated_maps = hash:/etc/postfix/relocated

The lookup table uses the old address as the key and its new location as the value. When a message is delivered to a relocated address, Postfix rejects the delivery attempt with a message that includes the user’s new address as specified in the lookup table. You can also list just a domain name to have all recipients at that domain rejected with your specified message.

The file /etc/postfix/relocated contains entries like:

kdent@ora.com      kdent@oreilly.com
heloise@ora.com    hfulbert@oreilly.com
@example.com       oreilly.com

Messages sent to either kdent@ora.com or heloise@ora.com are rejected with an error message that gives their respective new addresses. Any messages sent to example.com are rejected regardless of what the local part is. The message reports that the address has moved to oreilly.com.

A local address that is not listed in relocated or other maps, and is not an account on the system is an unknown user. Normally, when Postfix receives mail for an unknown user, it rejects it. If you prefer to capture all of the messages sent to nonexistent accounts, you can use the luser_relay parameter. Set it to any email address to have messages destined for unknown users sent to the address you provide. You must also set local_recipient_maps to blank to prevent Postfix from rejecting mail for unknown users:

luser_relay = catchall
local_recipient_maps =

Assuming catchall is a legitimate address (alias or user account) on your system, it will receive all messages sent to nonexistent users. Be careful when using luser_relay, since spammers often launch dictionary attacks, where they try enormous lists of addresses hoping to find a legitimate one at your site. If luser_relay is configured, it will catch all of the spam.

Messages in the deferred queue stay there until they are either delivered successfully or expire and are bounced back to the sender. The bounce_size_limit parameter determines how much of a message that could not be delivered is bounced back to the sender in the error report. The default is 50,000 bytes.

Once a message has failed delivery, Postfix marks it with a timestamp to indicate when the next delivery attempt should occur. Postfix keeps a short-term list of systems that are down to avoid unnecessary delivery attempts. If there are deferred messages scheduled for a redelivery attempt, and there is space available in the active queue, the queue manager alternates between taking messages from the deferred and incoming queues, so that new messages are not forced to wait behind a large backlog of deferred ones.

Postfix periodically scans the queue to see if there are deferred messages whose timestamps indicate they are ready for another delivery attempt. Subsequent failed attempts at delivery cause scheduled delays to double, so Postfix waits longer each time before it attempts to deliver a message. You can configure the maximum delay with the maximal_queue_lifetime parameter. When the time has expired, Postfix gives up trying to deliver the message and bounces it back to the sender. By default the period is five days (5d). You can set it to any length of time, or to 0 to have undeliverable mail returned immediately.

Queue scans occur at an interval specified by the queue_run_delay parameter. By default the parameter is set to 1,000 seconds (1000s). With this setting, every 1,000 seconds, Postfix checks the deferred queue to see if there are any messages due for another delivery attempt.

The parameters minimal_backoff_time and maximal_backoff_time set minimum and maximum time limits on how often Postfix attempts to redeliver deferred messages. Each time a message is deferred, the queue manager increases the amount of time it waits to attempt to deliver that message again. The calculated increase of time is never allowed to exceed maximal_backoff_time (default 4,000 seconds) and is never less than minimal_backoff_time (default 1,000 seconds). If you find that you have a large backlog of deferred messages, you may want to increase the maximal_backoff_time so that Postfix doesn’t expend system resources in trying to deliver messages to unavailable servers.

The queue manager arranges for the delivery of messages by invoking the appropriate delivery agent. Postfix is careful not to overwhelm destination systems and provides several parameters to control resources for outgoing messages. For most situations the default settings are correct, but if you are experiencing resource problems or you are trying to optimize deliveries, you may want to experiment with the queue manager configuration.

Outgoing messages might be delivered over any of the transports available in the master.cf file. Each transport can have a limit on its total number of processes, specified in the maxproc column (see Section 4.5). If a value is not specified there, Postfix uses default_process_limit for its limit.

The initial_destination_concurrency parameter limits the number of messages initially sent (default is five). You can increase the value, but it can’t go higher than the maxproc value or default_process_limit for the transport used. After the initial delivery of messages, if there are more messages in the queue for a particular destination, Postfix increases the number of concurrent delivery attempts, as long as it doesn’t detect any problem from the destination system at the current load. Postfix continues to increase the number of simultaneous deliveries up to the number specified in the default_destination_concurrency_limit parameter, which is 20 by default. In general, you don’t want to increase the concurrency limit, or you risk overwhelming the receiving system.

You can override the default_destination_concurrency_limit value for any transport by setting a parameter of the form transport _destination_concurrency_limit. For example, you can limit concurrent connections to external systems with the parameter smtp_destination_concurrency_limit , or limit local deliveries with local_destination_concurrency_limit .

There are also parameters of the form transport _destination_recipient_limit that control how many recipients Postfix specifies for a single copy of an email message. If a transport-specific parameter is not configured, it takes its default value from default_destination_recipient_limit . If the number of recipients for a message exceeds the limit, Postfix breaks up the list of recipients into smaller groups of addresses and sends separate copies of the message to each group of addresses.

The corrupt queue is simply used to store damaged or otherwise unreadable messages. If a message is too damaged to do anything with it, Postfix places it here. If you want to investigate an issue, the problem message is available in this queue where you can view it manually, if necessary. Corrupt messages are very rare. If you have them, they may be a symptom of an underlying operating system or hardware problem.

Postfix can report certain errors by sending error messages to an administrator. Postfix classifies errors for notification, as shown in Table 5-1. The notify_classes parameter in main.cf contains the list of error classes that should generate error notices. By default the parameter includes " resource” and “software” errors.

Each class of error can be configured to send the notification to a particular email address, using parameters of the form class _notice_recipient. By default they all go to postmaster . Table 5-1 provides a list of possible error classes, along with the parameters that indicate who should receive the error notices.

If you would like to receive all problem notices, set the parameter as follows:

notify_classes = bounce, 2bounce, delay, policy, protocol,
   resource, software

The queue display contains an entry for each message that shows the message ID, size, arrival time, sender, and recipient addresses. Deferred messages also include the reason they could not be delivered. Messages in the active queue are marked with an asterisk after the Queue ID. Messages in the hold queue are marked with an exclamation point. Deferred messages have no mark.

You can list all the messages in your queue with the postqueue -p command. Postfix also provides the mailq command for compatibility with Sendmail. The Postfix replacement for mailq produces the same output as postqueue -p.

A typical queue entry looks like the following:

$ postqueue -p
-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------
DBA3F1A9        553 Mon May  5 14:42:15  kdent@example.com
        (connect to mail.ora.com[192.168.155.63]: Connection refused)
                                         kdent@ora.com

Since this entry is not marked with either an asterisk or an exclamation point, it is in the deferred queue.

The postsuper command allows you to remove messages from the queue. To remove the message in the sample entry displayed above, execute postsuper with the -d option:

# postsuper -d DBA3F1A9
postsuper: DBA3F1A9: removed
postsuper: Deleted: 1 message

If you have a lot of messages to remove, you can clear out your entire queue with the ALL argument:

# postsuper -d ALL
postsuper: Deleted: 23 messages

The ALL argument must be capitalized. Be very careful when using the command, since it will delete all queued messages without asking any questions.

Rather than deleting all of the queued messages or just one at a time, frequently you want to delete messages with a specific email address. Example 5-1 is a Perl script that provides a convenient way to specify an email address to delete particular messages from the queue.

#!/usr/bin/perl -w
#
# pfdel - deletes message containing specified address from
# Postfix queue. Matches either sender or recipient address.
#
# Usage: pfdel <email_address>
#

use strict;

# Change these paths if necessary.
my $LISTQ = "/usr/sbin/postqueue -p";
my $POSTSUPER = "/usr/sbin/postsuper";

my $email_addr = "";
my $qid = "";
my $euid = $>;

if ( @ARGV !=  1 ) {
        die "Usage: pfdel <email_address>\n";
} else {
        $email_addr = $ARGV[0];
}

if ( $euid != 0 ) {
        die "You must be root to delete queue files.\n";
}


open(QUEUE, "$LISTQ |") || 
  die "Can't get pipe to $LISTQ: $!\n";

my $entry = <QUEUE>;    # skip single header line
$/ = "";                # Rest of queue entries print on
                        # multiple lines.
while ( $entry = <QUEUE> ) {
        if ( $entry =~ / $email_addr$/m ) {
                ($qid) = split(/\s+/, $entry, 2);
                $qid =~ s/[\*\!]//;
                next unless ($qid);

                #
                # Execute postsuper -d with the queue id.
                # postsuper provides feedback when it deletes
                # messages. Let its output go through.
                #
                if ( system($POSTSUPER, "-d", $qid) != 0 ) {
                        # If postsuper has a problem, bail.
                        die "Error executing $POSTSUPER: error " .
                           "code " .  ($?/256) . "\n";
                }
        }
}
close(QUEUE);

if (! $qid ) {
        die "No messages with the address <$email_addr> " .
          "found in queue.\n";
}

exit 0;

The hold queue is available for messages you would like to keep in your queue indefinitely. Figure 5-2 shows the hold queue and how you can move messages into the hold queue where they will not be delivered until you specifically remove them or move them back for normal queue processing. To place the example message into the hold queue, use the postsuper command with the -h option:

# postsuper -h DBA3F1A9

The queue entry now contains an exclamation point to show that the message is on hold:

-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------
DBA3F1A9  !     553 Mon May  5 14:42:15  kdent@example.com
        (connect to mail.ora.com[192.168.155.63]: Connection refused)
                                         kdent@ora.com

Figure 5-2. Putting messages on hold

To move the message back into the normal queue for regular processing, execute the command with a capital -H option instead:

# postsuper -H DBA3F1A9

After the message is moved back, the queue manager marks it for redelivery according to its normal scheduling, or you can flush the message to have it sent out immediately (see Section 5.2.6).

If you have messages that were deferred because of a configuration problem that has been corrected, you may have to requeue the messages to have them delivered successfully. If the misconfiguration caused Postfix to store incorrect information about the next hop or transport method, or to rewrite the address incorrectly, requeuing causes Postfix to update the incorrect information based on your new configuration. The postsuper command uses the -r option to requeue messages. You can specify a queue ID for a single message, or the word ALL in capital letters to requeue everything:

# postsuper -r ALL

Requeued messages get a new queue ID and an additional Received: header.

The postcat command displays the contents of a queue file:

# postcat -q DBA3F1A9

Earlier versions of postcat did not support the -q option but required the full path to the queue file. Since a message can be in any of the queue compartments (maildrop, incoming, active, deferred, hold), and each of these has multiple subdirectories, the path to a particular queue file is not immediately apparent. If you are using an earlier version of postcat, which doesn’t support the -q option, you can create a shell script like the one in Example 5-2 as a convenient way to view a queue file by specifying only the queue ID. The script accepts one queue ID as an argument, checks all of the queue directories to locate the queue file, and executes postcat with the full path as its argument. The contents are then displayed. This simple script displays only one queue file at a time.

#!/bin/sh

PATH=/usr/bin:/usr/sbin
QS="deferred active incoming maildrop hold"
QPATH=`postconf -h queue_directory`

if [ $# -ne 1 ]; then
        echo "Usage: pfcat <queue id>"
        exit 1
fi

if [ `whoami` != "root" ]; then
        echo "You must be root to view queue files."
        exit 1
fi

if [ ! -d $QPATH ]; then
        echo "Cannot locate queue directory $QPATH."
        exit 1
fi

for q in $QS
do
        FILE=`find $QPATH/$q -type f -name $1`
        if [ -n "$FILE" ]; then
                postcat $FILE
                exit 0
        fi
done

if [ -z $FILE ]; then
        echo "No such queue file $1"
        exit 1
fi

Flushing the queue causes Postfix to attempt to deliver messages in the queue immediately. You can flush queue messages with the postqueue -f command. However, unless you have a reason to expect successful deliveries, it’s best to leave redelivery attempts to the Postfix queue manager. Repeated attempts to flush the queue can have a severe performance impact on your mail server.

You can flush messages destined for a particular site with the -s option. The site must be eligible for fast flush in order for this to work. To be eligible, the site must be listed in the fast_flush_domains parameter. By default, fast_flush_domains includes all of the hosts listed in relay_domains , but you can add additional sites if you want to flush them before the normally scheduled redelivery attempt.

fast_flush_domains = $relay_domains example.com

If you know that a previously unavailable, eligible site is ready to accept mail, execute postqueue with the -s option and name the site:

# postqueue -s example.com

See Chapter 9 for more information about fast flush and the SMTP command ETRN.

The Postfix SMTP delivery agent must be able to obtain IP address and MX records for mail-routing information. Postfix must make at least two DNS lookups: one to get the MX hostname and one to get the IP address for that hostname. Since Postfix uses the normal operating system resolver libraries for its DNS queries, the system that runs Postfix must have access to a DNS server. The DNS server does not have to be on the same system, although for most circumstances it should be.

If your system does not seem to be resolving domain names correctly, there are three common command-line tools that you can use to troubleshoot the problem: nslookup , dig, and host. You should check your system documentation to see which of these tools is available on your server and how to use them. You can use these tools to query all types of resource records for a domain, including the MX record that Postfix needs in order to successfully deliver mail to a domain.

DNS problems might stem from your own system’s configuration or a problem with the DNS server configuration for the domain Postfix is trying to send mail to. When you are troubleshooting a problem, it is very important to remember that Postfix first looks for MX records and not A records. Even if you can resolve a domain to an IP address, Postfix may not be able to deliver mail for that domain if there is a problem in retrieving MX information.

smtp_skip_4xx_greeting = no
smtp_skip_5xx_greeting = no

smtp_randomize_addresses = no

For Postfix to accept email for a particular domain, the system must be specified as an MX host in the domain’s DNS setup, and Postfix must be configured to accept mail for the domain. Postfix accepts mail for domains that are either local to the system, relay domains, or virtual domains. Virtual domains might use virtual aliases or virtual mailboxes (see Chapter 8). Each type of domain must be listed in a different Postfix parameter, as shown in Table 6-1.

Do not list a domain in more than one of the parameters. Postfix issues a warning if it detects a domain listed in two of the parameters. The error message “mail for example.com loops back to myself” occurs when the DNS configuration points to your mail server, but Postfix has not been configured to accept mail for the domain.

If your Postfix server accepts mail for the two local domains example.com and porcupine.org, then the mydestination parameter should look like the following in your main.cf file:

mydestination = example.com, porcupine.org

Chapter 9 explains configuration of relay domains. Chapter 8 covers virtual mailbox and virtual alias domains.

Historically, Unix systems have used a single file to store each user’s email messages. This type of message store format is commonly referred to as mbox. Each message within the file starts with a line that begins with the word From. It is important that the string start on the first character of the line, and that there is a space after the end of the word. The From line is commonly referred to as From_ with an underscore character to indicate the space following the word. Don’t confuse the From_ line used for separating messages within an mbox file with the From: line included in email message headers. The last line of a message is always a blank line.

A complete From_ line looks like the following:

From jmbrown@example.com Sun Feb  3 16:54:01 2002

As described, the line starts with the word From followed by a space. Following the space is an email address that is usually the envelope address of the message. Following the envelope address is the date of delivery in the common Unix date format occupying 24 characters. The mbox format allows for an optional comment string following the date, but it is generally not used.

When Postfix delivers a message to an mbox file, it first creates the From_ line using the envelope sender and the current date. Postfix then copies the contents of the delivered message into the mbox file. If Postfix encounters any lines that begin with From followed by a space, it has to quote them by adding a > to the beginning of the line, so that they won’t be confused with the start of the next message.

When a POP/IMAP server reads messages from the mbox file, it scans the file, looking for From_ lines, which mark the beginning of each message. It can read to the next From_ line (or the end of the file) to know when a message is finished. The POP/IMAP server may unquote any of the “>From” quoted lines, or they may remain in the quoted form.

Since both Postfix and the POP/IMAP servers access the mailbox file, they must use file locking. Postfix must obtain an exclusive lock on the file when it is delivering a message, so that it can write the message to the file. Postfix offers a variety of locking mechanisms, depending on the platform. You can use the postconf -l command to see which mechanisms Postfix can use on your system:

$ postconf -l
flock
fcntl
dotlock

If you want more information about the locking types listed by Postfix on your system, check your system’s man pages for the specific lock name:

$ man flock

The dotlock type, which should be available on all systems, is probably not documented on your system, because it is not a function of the operating system or supporting libraries as flock and fcntl are. The dotlock is simply a file. The lock file name is made up of the name of the file to be locked with a .lock extension appended to it. If such a lock file exists, then Postfix knows that another process is using the mail file. If the file does not exist, Postfix creates it to signal other processes that it is using the file. When Postfix is finished, it removes the lock file, making the mail file available again. The drawback of dotlock locking is that it is susceptible to stale locks, and it is not very efficient.

For the most part, you do not need to worry about locking, and the lock types available, because Postfix does a good job of figuring out the best option.

The maildir mailbox format differs from mbox in that it uses a structure of directories to store email messages. It was designed to solve some of the reliability and locking problems of the mbox format. For example, if a system crashes at the instant an email message is being delivered to an mbox file, it is possible that the message will be truncated at the point where the delivery was interrupted. When the system comes back online, the mail transport agent will attempt to deliver the message again. The partially written message at the bottom of the mbox file may cause problems when the next message is appended to the file.

Other problems can occur if a POP/IMAP server tries to access the mbox file at the same time as the SMTP server. If the programs do not use the same locking mechanism, the mail file will most likely be corrupted. There are several possible mail file locking mechanisms (see above), which are not necessarily used by all mail programs. With the maildir format, no locks are necessary because each message gets its own file. Different mail processes do not need access to the same files at the same time.

A maildir-style directory has three subdirectories, which must all be on the same filesystem: tmp, new, and cur. These subdirectories are usually below a mail directory in a user’s home directory:

$ ll /home/kdent/maildir
total 12
drwxr-x---    2 kdent    kdent        4096 Mar 13 12:24 cur
drwxr-x---    2 kdent    kdent        4096 Mar 13 12:24 new
drwxr-x---    2 kdent    kdent        4096 Mar 13 12:24 tmp

Files in the new directory are messages that have been delivered but have not yet been read. The modification time of the file is the delivery date of the message. The file usually contains the message in RFC 2822 format, and no From_ line is needed.

Once a message has been viewed, it is moved to the cur directory. The tmp directory is used during message delivery to store the contents of a file before it can be confirmed to have been written to the new directory.

There is no simple answer to help you decide which type of mailbox format is best for you. The mbox format has the advantage of being almost universally supported, but has the file-locking problems that prompted the development of the maildir format. On the other hand, there are concerns about the ability of the maildir format to scale to handle large numbers of messages on some filesystems. There are performance arguments to support both formats: locating and accessing or deleting a particular message is probably quicker with maildir, but delivery by simply appending the text of a message to the end of a single file is probably quicker in the mbox format. Your choice will most likely be driven by your selection of a POP/IMAP server. If you settle on a POP/IMAP server that requires the maildir format, the choice is made for you. Postfix easily supports either format, so you can safely allow other considerations to drive your decision. If you think it will be significant in your environment, you should run tests of both formats, simulating your own mail tasks as closely as possible.

.forward files allow local users to set up their own aliases. The contents of the .forward file are the same as the righthand side of an alias entry. When an alias entry has multiple values on the righthand side, they are separated by commas; while .forward files use the same convention, they also allow multiple entries to be entered on multiple lines.

.forward files must be owned by the recipient, and are normally found in users’ home directories. You can specify different locations with the forward_path parameter. When specifying a path for the parameter, there are eight variables whose values are expanded at delivery time:

If you want to add support for a nonstandard .forward file, you could configure forward_path as follows:

forward_path = /home/$user/.forward /home/$user/other_forward

See the Postfix local manpage for more information on specifying paths with variable expansion.

When Postfix delivers to a command or file specified in alias files, it makes the delivery or executes the command as the user who owns the alias file. The exception is when the file is owned by root, in which case Postfix uses the account specified in the default_privs parameter. By default it is set to the account nobody . Aliases are discussed in Chapter 4.

When Postfix delivers a message to a local user, it writes the message to the system’s message store. By default Postfix uses the mbox format for deliveries. When you install Postfix, it can normally figure out the default location of the mail spool directory depending on the type of Unix system you have. The mail_spool_directory parameter can be used to specify a directory other than the default. To change the directory to something other than the default for your system, edit the main.cf file, and add or modify the mail_spool_directory parameter:

mail_spool_directory = /var/spool/mail

To cause Postfix to use the maildir format for delivery, append the directory with a trailing slash:

mail_spool_directory = /var/spool/mail/

Postfix can also be configured to deliver messages to mailboxes within users’ home directories. Assign a relative path to the home_mailbox parameter to indicate which file should be used for mailboxes:

home_mailbox = mbox

Append the path with a trailing slash to indicate that Postfix should use the maildir-style delivery:

home_mailbox = maildir/

This causes Postfix to deliver messages into a directory called maildir, below users’ home directories.

The POP protocol works best when you have limited, or less than full-time, network access because it allows you to connect to your mail server, fetch all of your messages, and disconnect from the network. You now have local copies that you can read offline. Most POP clients have a configuration option to delete your messages from the server when you retrieve them, since you then have the local copies. If you don’t delete them at some time, the messages accumulate, taking up more and more space on your mail server. POP was designed to be easy to implement, but the major problem with the POP protocol is that if you ever work from more than one computer, your messages may not be where you need them. It also does not handle multiple mailboxes very well, and it forces you to download complete messages. There is no option to retrieve just the subject, for example, to decide if you want the complete message.

The IMAP protocol was designed to overcome some of POP’s shortcomings. It keeps all messages on the server. You have to be connected while working with your email messages, but you can manage them as if they were local. Since everything happens on the server, it doesn’t matter if you work from your desktop computer at home, another machine at work, and even on a laptop while traveling. IMAP still allows for saving messages locally, if necessary, and it also provides much more flexibility than POP. You can download just the headers from your messages and then decide to retrieve the rest of a message if you want to read it. You don’t have to be stuck downloading a huge message or attachment that you might not be interested in. You can maintain multiple mailboxes and folders on the IMAP server.

The cooperation between Postfix and POP/IMAP servers is simple. When Postfix accepts delivery of an email message, it places it in the message store. The POP/IMAP server simply retrieves messages from the same store when a user requests them. Figure 7-1 shows how simple the cooperation is between Postfix and POP/IMAP servers. Postfix and the POP/IMAP server must agree on the type of mailbox format and the style of locking. Postfix should work with any standards-compliant POP/IMAP server that uses one of the traditional message stores. You may have to adjust the mail_spool_directory parameter, as described earlier in the chapter, but for most POP/IMAP servers, you can simply follow the standard installation instructions and start the server. For POP/IMAP servers that don’t use a traditional message store, Postfix can still deliver messages using the Local Mail Transfer Protocol, which is discussed in the next section.

Figure 7-1. Postfix and POP/IMAP servers

Cyrus IMAP is intended to run on servers that provide POP/IMAP access only, where users do not need a shell account. If you are creating a mail server for existing users on a system, you will probably want to use another simpler POP/IMAP solution, such as Qualcomm’s Qpopper (POP access only) or the University of Washington’s IMAP Toolkit, which doesn’t require any special configuration to work with Postfix. This section deals with configuration issues for getting Postfix to work together with Cyrus IMAP.

Figure 7-2. Postfix and Cyrus IMAP

Cyrus IMAP can listen for LMTP deliveries using either Unix-domain sockets or TCP sockets. You must know which method you are using so that you can configure Postfix appropriately. If you want to use Unix-domain sockets, both Postfix and the Cyrus IMAP server must be on the same machine. If you use TCP sockets, the Cyrus IMAP server could be on the same system or any other system on your network. LMTP delivery is configured in your main.cf file or in a transport map.

For Postfix to accept messages to be delivered locally to Cyrus IMAP, the destination domain name of the email address must be listed in the mydestination parameter. (You may also want to configure Cyrus deliveries via the virtual transport. See Chapter 8.) Then you must configure Postfix to pass messages to Cyrus IMAP. Use the mailbox_transport, local_transport, or fallback_transport parameter to tell Postfix how much local processing to do before handing off messages to Cyrus. If you are using local_transport or fallback_transport, make sure that Postfix knows about all of the Cyrus users, by including the usernames in a lookup table listed with the local_recipient_maps parameter.

Specify your chosen transport type using the following format:

               xxx_transport = service:socket_type[:/path/to/socket]

For LMTP delivery, service must be lmtp, which refers to the lmtp service in the /etc/postfix/master.cf file. The socket_type is either unix or inet for Unix domain, or TCP sockets, respectively. The default is inet, which means that if your LMTP server uses an inet socket, you can simply specify the service as:

local_transport = lmtp

A typical LMTP transport configuration in /etc/postfix/main.cf using local_transport and a Unix domain socket looks like the following:

local_transport = lmtp:unix:/var/imap/socket/lmtp

To build Cyrus IMAP, you need the Cyrus SASL library, which is used to authenticate users for the IMAP server.^[1] You must first build and install the Cyrus SASL library, and then you can build the Cyrus IMAP server. The Cyrus software requires at least Version 3 of Berkeley DB. If you were using a version of Berkeley DB prior to Version 3, you may need to update your entire system. Having different versions of Berkeley DB intermixed on your system will likely lead to problems that can be difficult to track down. If you have to upgrade your libraries, consider rebuilding other packages that use Berkeley DB (such as Perl and Postfix), so that everything on your system uses the same version of the library.

Follow the instructions in the Cyrus SASL and IMAP distributions to compile and install them correctly on your system. There might be binary distributions available for your platform. Check your normal software sources to see if you can save yourself the trouble of building the Cyrus software.

For this example, assume that you have a Postfix server receiving mail for the domain example.com. All of the email accounts are set up within the Cyrus IMAP server running on the same system, so there are very few actual login accounts on the system. However, you want mail destined for the root account or postmaster alias to be sent to the correct person, which means that you need to expand local aliases before handing off messages to the Cyrus IMAP server. To achieve this, set the mailbox_transport parameter to point to the lmtp delivery agent, which will be configured to deliver mail to the Cyrus IMAP server:

The virtual mailbox files must be owned by a user account and associated with a group on your system. How your users retrieve their messages determines what the ownership of mailbox files should be. Often, your POP/IMAP server executes under its own account and expects all of the mailbox files to be owned by this user, but if necessary, Postfix lets you configure ownership for mailbox files in any way you need. Each can be owned by a separate user, or one user can own all of the mailboxes for one domain, while a different user owns the mailboxes of another.

The virtual_uid_maps and virtual_gid_maps parameters determine the owner and group Postfix uses when making deliveries to virtual mailbox files. You can specify that all of the virtual mailboxes should be owned by the same user account with the static map type. Assume, for this example, that you have created an account called vmail that has a UID of 1003, and a group called vmail that has a GID of 1005. You want all of the virtual mailbox files to be owned by this user and group.

Set the virtual_uid_maps and virtual_gid_maps parameters in main.cf:

virtual_uid_maps = static:1003
virtual_gid_maps = static:1005

If you want to use different UIDs for different mailbox files, you must create a lookup file that maps the addresses to the UIDs. Then point the mapping parameter to your lookup file:

virtual_uid_maps = hash:/etc/postfix/virtual_uids

If most of your virtual mailboxes should have the same fixed ownership but some require different UIDs, you can combine static and table lookups:

virtual_uid_maps = hash:/etc/postfix/virtual_uids static:1003

If you also need separate group mappings, they work exactly the same way.

The file /etc/postfix/virtual_uids contains entries like the following, with each address mapped to the correct UID. In this case, the mailboxes for http://ora.com use one ID and those for http://oreilly.com use another:

#
# /etc/postfix/virtual_uids
#
info@ora.com         1004
kdent@ora.com        1004
info@oreilly.com     1007
service@oreilly.com  1007

It is possible for a virtually hosted domain to have some addresses that are delivered to the local message store and some that are forwarded. Since all recipient addresses are checked for virtual aliasing regardless of their class, simply place the forwarded addresses in the virtual_alias_maps file instead of the virtual_mailbox_maps file. Make sure the virtual_alias_maps parameter points to a virtual alias lookup table:

virtual_alias_maps = hash:/etc/postfix/virtual_alias

The /etc/postfix/virtual_alias file contains entries for addresses that should be forwarded elsewhere:

kdent@oreilly.com   kyle.dent@onlamp.com

Do not list a domain in both virtual_mailbox_domains and virtual_alias_domains . Use virtual_mailbox_domains for domains that have a mix of aliases and mailboxes and virtual_alias_domains only when all of the addresses are aliases.

For either virtual mailboxes or virtual aliases, your lookup table can have a key value of the domain without a local part to catch any message destined for the domain addressed to a nonexistent address. Catchall addresses should be used advisedly, since they tend to receive a lot of spam. Spammers often send messages to nonexistent accounts at a domain, which are received by catchall addresses.

@ora.com        ora.com/service

@ora.com        customer.service@onlamp.com

info@ora.com         ora.com/info
info@oreilly.com     oreilly.com/info

@ora.com             customer.service@onlamp.com
kdent@oreilly.com    kyle.dent@onlamp.com
info@ora.com         info@ora.com
info@oreilly.com     info@oreilly.com

To configure an auto-responder to work with virtual domains, you must create a special transport type in master.cf for delivery to the specific command. In order to have messages delivered to your new component, you have to map an address to the transport you created using transport maps.

Many auto-responders can handle only a single message at a time with only one recipient. You can limit the number of recipients to any transport type by setting a parameter of the form transport _destination_recipient_limit, where the string transport is the name of the transport type. If a transport called inforeply should be limited to only one recipient at a time, set the following parameter:

inforeply_destination_recipient_limit = 1

The following steps walk through setting up the email address info@ora.com to use inforeply.pl. The domain http://ora.com is configured as a virtual domain. The local domain on the host is http://example.com:

In the next example, you’ll set up a mailing list for a virtual domain. Mailing-list managers are discussed in Chapter 10. You may want to review that chapter before setting up your virtual mailing lists. This example creates a mailing list for Majordomo. You should first install and configure Majordomo according to the directions in Chapter 10.

Virtual mailing lists work by creating a parallel version of the list under a local domain. The local version is only used internally on your system. External users can use the virtual addresses and never know that the local version exists. When naming the local version, you may want to include the virtual domain name in some way to distinguish the list from lists for other virtual domains hosted on your system. The following procedure creates a mailing list at the virtual address astronomy@ora.com that is handled by the local version astronomy-ora@example.com:

You should now be able to send messages to astronomy@ora.com for distribution to all of the addresses in your list file.

It is highly recommended that you maintain a list of valid recipients for domains you provide backup MX services to. You should develop a regular process for obtaining an updated user list from your primary MX servers. If your system does not know all of the available mailboxes on the primary mail server, it must accept all messages. It’s only when your backup MX server tries to deliver them to the primary server that it discovers that a message cannot be delivered. At that point, your server must bounce the message back to the original sender.

Since spammers often send messages to made-up addresses, if your server does not know all the valid email addresses on the primary server, your server will unnecessarily accept a lot of mail that must be bounced. The bounce problem is exacerbated by the spammer tactic of forging sender addresses by using the real email addresses of innocent bystanders. The forged addresses receive all of the error notices for messages they never sent (see Chapter 11). The relay_recipient_maps parameter specifies lookup tables that should contain all of the addresses for domains listed in your relay_domains parameter:

relay_recipient_maps = hash:/etc/postfix/relay_recipients

The relay_recipients file should contain entries with the recipient address on the lefthand side. The righthand side is not used by Postfix, but you must specify a value:

#
# relay_recipients
#
user1@example.com         any_value
user2@example.com         any_value
user3@example.com         any_value

If your system is on the same network as the primary, and the user accounts are stored in some kind of database, you may be able to perform real-time lookups using MySQL or LDAP (see Chapter 15).

A potential problem is that once you set relay_recipient_maps, you must include email addresses for all domains you provide backup service to. If not, Postfix will reject messages that don’t appear in the lookup table. If you don’t know the valid addresses for some domains, you can specify a wildcard entry for that domain:

#
# relay_recipients
#
user1@example.com       any_value
user2@example.com       any_value
user3@example.com       any_value
@oreillynet.com         any_value

The final entry is a wildcard entry that allows messages for any address at the domain. Obviously, it’s better to obtain the list of valid addresses for the reasons mentioned earlier.

Networks that receive mail for many sites, such as ISP networks, typically have some customers whose systems aren’t always connected to the network. When the customer network is offline, the ISP queues its messages. When the site comes online, it can request immediate delivery of all its queued mail with the ETRN SMTP command:

220 mail.ora.com ESMTP Postfix
EHLO mail.example.com
250-auger.seaglass.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250 8BITMIME
ETRN example.com
250 Queuing started

If there are a lot of messages queued when a domain is ready to accept mail, searching every queue file would be time-consuming. Postfix provides a capability called fast flush to speed up queue processing for a particular domain. Fast flush is handled by the flush daemon, which maintains lists of messages that are queued for specific domains so that Postfix knows which messages to deliver when it receives an ETRN command.

By default, all of the sites listed in relay_domains are eligible for the fast flush service. You can include domains in addition to your relay domains by adding them to the fast_flush_domains parameter. Add a domain name as follows:

fast_flush_domains = $relay_domains, example.com

In this case http://example.com is a domain not already listed in relay_domains.

You can manually notify Postfix that a fast flush domain is ready to accept messages by issuing the postqueue -s command (or its equivalent, sendmail -qR) with the site name:

$ postqueue -s example.com

Under some circumstances you want Postfix to postpone delivery of messages until it has received an explicit command to deliver them. Deferred messages are delivered when you issue the postqueue -f domain command or Postfix receives an ETRN SMTP command from a fastflush-eligible domain.

A common scenario for deferring messages is when an ISP receives mail for a customer network that is not always online. The ISP must queue messages until the network is online and can receive them. Similarly, users on the customer network should send messages through a local gateway that queues them until they can be delivered once the network is online. This section presents configurations for both situations.

$ postqueue -f example.com

If any messages cannot be delivered to one of the addresses listed, the original sender of the message receives an error message explaining that there was a delivery problem. For small or internal lists this may be perfectly acceptable; however, if you are creating a large list, or the members of the list do not necessarily know each other, it is probably more appropriate to have error messages sent to the administrator of the list. The convention is to create an additional alias for lists using a format like owner-<list_alias>@example.com, where owner- is prepended to the name of the list alias. For the previous example, we would create the alias owner-needlepoint.^[1] This owner- alias should point to an administrator, who is generally in a better position than the original sender to deal with bounced messages:

owner-needlepoint: kdent@example.com

Sending error notifications to the owner- alias is achieved by setting the envelope sender to the owner-needlepoint@example.com alias instead of the original sender’s email address. Example 10-1 shows typical headers from a mailing-list message.

Return-Path: <owner-needlepoint@example.com>
Delivered-To: rgrier@oreilly.com
Received: from cowrie.example.com (cowrie.example.com[192.168.100.7])
     by mail.oreilly.com (Postfix) with ESMTP id B712120DD5B
     for <needlepoint@example.com> Mon, 13 May 2002 11:55:40 -0400 (EDT)
Date: Mon, 13 May 2002 12:00:43 -0400 (EDT)
From: G.M. Hopper <gmhopper@onlamp.com>
X-Sender: gmhopper@cowrie
To: needlepoint@example.com
Subject: Just finished latest project
Message-ID: <Pine.GSO.4.10.10205131200230.692-100000@cowrie>

When the owner- alias exists, Postfix automatically uses it as the envelope sender address when sending out messages to list members. If, for some reason, you don’t want Postfix to use the owner- alias but rather to keep the originator’s address, you can set the parameter owner_request_special to no:

owner_request_special = no

You can also cause Postfix to use the actual administrator’s email address instead of the owner- alias by setting expand_owner_alias to yes:

expand_owner_alias = yes

If this parameter is set, the address kdent@example.com is used instead of owner-needlepoint@example.com.

Although users do not generally need to send mail directly to the list owner, you should create owner aliases even for simple lists so other postmasters can contact the correct person in case they run into any problems with your list.

Another list convention is to provide a request alias for your lists. Request aliases use the format <list_alias>-request@example.com. The request alias for the needlepoint alias looks like needlepoint-request@example.com. Request aliases are used for requests to subscribe and unsubscribe from lists or to get nontechnical information about a list.

If you have more than just a few names on a list, it is more convenient to create a text file that lists all of the email addresses for the list. The format of the alias entry that points to a file is as follows:

               email_alias: :include:/path/to/file.

Let’s take the needlepoint alias from earlier in the chapter and move the list addresses into a separate file. Your alias entry should be revised to point to the text file that contains the list of addresses:

needlepoint:        :include:/etc/postfix/needlepoint

The file /etc/postfix/needlepoint contains the email address of each member of the group. Put one address on each line. When you need to make changes to the list, simply edit the file:

rgrier@oreilly.com
gmhopper@onlamp.com
grayburn@oreilly.com
bogus@example.com

I’m adding an invalid address, bogus@example.com, for testing later in the chapter.

Recall from Chapter 4 that the alias_maps parameter allows you to specify any number of alias files to use with Postfix. For example, you might want to use a separate alias file to store your mailing lists. Simply include the separate alias filename along with the system alias to set the alias_maps parameter. You should also set the alias_database parameter, so that you can run the command newaliases to update all of your alias-mapping files:

alias_maps = hash:/etc/postfix/aliases, hash:/etc/postfix/mail_lists
alias_database = hash:/etc/postfix/aliases, hash:/etc/postfix/mail_lists

It may be more convenient to assign all of your alias files to alias_database and then assign alias_database to alias_maps. If you use other map types for aliases, simply assign them to alias_maps as well:

alias_database = hash:/etc/postfix/aliases, hash:/etc/postfix/mail_lists
alias_maps = $alias_database, nis:mail.aliases

Remember to reload Postfix when you make changes to main.cf.

Let’s review everything discussed so far and consider all the pieces of our needlepoint mailing list. The alias file contains the following lines:

needlepoint:          :include:/etc/postfix/needlepoint
owner-needlepoint:    kdent@example.com
needlepoint-request:  kdent@example.com

The first line in the example causes messages sent to needlepoint@example.com to be delivered to every address listed in the /etc/postfix/needlepoint file. This file should contain a list of email addresses of all members of the list. Bounce messages and requests are forwarded to the real address kdent@example.com. If necessary, users or other postmasters can send messages to the list owner, and users can send messages to the request alias for subscription or other information.

When a message is sent to the list, the To: header contains just the address of the mail list alias and not an expansion of all the names on the list (which could be hundreds or even thousands of names). Each member of the list receives a copy of the message with headers that resemble those shown in Example 10-1. In this example, gmhopper@onlamp.com has sent a message to the list. Notice that the Return-Path: contains the owner alias rather than the actual originator of the message (gmhopper@onlamp.com).

You can test your list by sending a message to the alias you created for it. In this example, we’ll use the list alias needlepoint@example.com. Example 10-2 shows the log entries for a sample test message. Imagine that the address bogus@example.com is invalid.

postfix/local[7411]: 6C2CE20DD5B: to=<needlepoint@example.com>, 
   relay=local, delay=1, status=sent (forwarded as ACDC120DD70)
postfix/qmgr[8163]: ACDC120DD70: from=<owner-needlepoint@example.com>, 
   size=1121, nrcpt=8 (queue active)
postfix/local[0835]: ACDC120DD70: to=<bogus@example.com> relay=local,
   delay=1, status=bounced (unknown user: "bogus")
postfix/smtp[6556]: ACDC120DD70: to=<grayburn@oreilly.com>
   relay=mail.oreilly.com[10.82.6.11], delay=1, 
   status=sent (250 Mail accepted)
postfix/smtp[6556]: ACDC120DD70: to=<rgrier@oreilly.com>
   relay=mail.oreilly.com[10.82.6.11], delay=1, 
   status=sent (250 Mail accepted)
postfix/smtp[5954]: ACDC120DD70: to=<gmhopper@onlamp.com>
   relay=mail.onlamp.com[10.171.8.111], delay=1, 
   status=sent (250 Message received: GZCLUC00.E8F)

Some of the information, such as the timestamp and hostname, has been removed for clarity. Notice that at the end of the first line there is a comment saying (forwarded as ACDC120DD70) and the rest of the log entries use the new queue ID. Also notice in the first line of the example that the message enters the system addressed to needlepoint@example.com. The second line shows that Postfix uses the owner alias as the envelop sender address (from=<owner-needlepoint@example.com>) while delivering the message to all members of the list. The bogus address shows a status of “bounced.” The address kdent@example.com pointed to by the owner alias receives the bounce notification, which looks like Example 10-3. Notice in the example that the bounce notification message is delivered to owner-needlepoint@example.com. The sender of the message does not receive a notification.

From MAILER-DAEMON@mail.example.com Tue Jul 16 12:03:49 2002
Date: Tue, 16 Jul 2002 11:25:27 -0400 (EDT)
From: Mail Delivery System <MAILER-DAEMON@mail.example.com>
To: owner-needlepoint@example.com
Subject: Undelivered Mail Returned to Sender

...

<bogus@example.com>: unknown user: "bogus"

...

Majordomo is one of the more popular MLMs and has been available since the early 1990’s. It offers a complete set of MLM features, and nearly all administration takes place by sending commands through email messages. Little to no intervention is required by a postmaster once a list has been created. There are also web-based administration packages available to work with Majordomo, allowing much of the list administration to take place from a web site.

Majordomo is available at the Majordomo home page (http://www.greatcircle.com/majordomo/.) It requires Perl and works with Perl4 Version 4.036 or Perl5 Version 5.002 or better. Future releases will probably require Perl5. Majordomo also makes use of a small wrapper program written in C. If you are planning to build the package from scratch, you must have an ANSI C compiler.

If you configure Majordomo for moderated lists, where a list administrator approves posts using the Majordomo-supplied approve, you have to make an adjustment for Postfix and Majordomo to work together correctly. Postfix prepends a Delivered-To: header to messages it handles. It then uses the header to detect mailer loops. When a Majordomo message is delivered to a moderator for approval who then pipes the message through the approve command, it is sent back to the list with all of its original headers intact. When Postfix receives the message again, it recognizes that it has already seen the message and reports a mail delivery loop.

The easiest way to fix this issue is to make a small change to the Majordomo approve script (which is written in Perl). You’ll have to edit the file, normally located in the /bin directory located below the main Majordomo installation directory. If you follow the steps in the procedure below, your file will be located at /usr/local/majordomo/bin/approve. Edit the file and find the subroutine called process_bounce. Within that routine, there is a while loop, as shown below. Insert the emphasized line as shown, save the file, and you’re done:

while (<$FILE>) {
        if (/^>?From / && ! defined($from_skipped)) {
                # Skip any initial "From " or ">From " line
                $from_skipped = 1;
                next;
        }
        next if ( /^delivered-to:/i );  # Added for Postfix
        s/^~/~~/;
        print MAIL $_;
}

$ echo 'lists' | mail majordomo

Date: Tue, 16 Jul 2002 18:14:59 -0400 (EDT)
From: Majordomo@example.com
To: kdent@example.com
Subject: Majordomo results

--

>>>> lists
Majordomo@example.com serves the following lists:

  astronomy               

Use the 'info <list>' command to get more information
about a specific list.
>>>> 
>>>>

To: majordomo@example.com
From: tbrahe@porcupine.org
Subject:

subscribe astronomy

...
                        The Postfix program

<astronomy@example.com>: cannot open include file
    /usr/local/majordomo/lists/astronomy: Permission denied

...

# chmod 4755 /usr/local/majordomo/wrapper

Mailman is another full-featured MLM. It is available at the Mailman home page at http://www.gnu.org/software/mailman/. It includes web-based administration and creates a home page for each list where list administrators and members can perform administrative functions. It also accepts administrative commands via email much like Majordomo does.

Mailman requires at least Version 1.5.2 of Python. It includes some security wrapper programs that are written in C, so you must have an ANSI C compiler if you are planning to build the package from scratch.

There is one slightly tricky aspect to get Postfix and Mailman working together correctly. Mailman expects to be invoked by a process running with a particular group ID (GID). The GID it expects is specified at the time the Mailman package is built. If you are building the package yourself, make sure that you first create an account and a group called mailman . You should be able to use the normal administrative tools on your system to create both the account and the group. When you are finished, you should have an entry in /etc/passwd that resembles the following:

mailman:*:26413:60003:Mailman List Manager:/home/mailman:/bin/sh

and an entry in /etc/group like the following:

mailman:*:60003:

Make sure that the account mailman has the group mailman as its primary group. In the examples above, 60003 specifies the mailman group and the mailman account has that as its primary group.

When you run configure for Mailman, be sure that you include the option --with-mail-gid= xxx, where xxx is the actual GID for the mailman group that you created. According to the examples above, you should execute configure using 60003 for the GID option:

$ ./configure --with-mail-gid=60003

You may have additional options for configure according to your environment. Be sure to read the Mailman documentation for building the package. If you have already built your Mailman package and you did not specify the group, build it again. If you didn’t build your Mailman package, see the sidebar below.

Failure to exec script. WANTED gid 12 GOT gid 99 (Reconfigure
to take 99?)

# bin/newlist
Enter the name of the list: astronomy
Enter the email of the person running the list: kdent@example.com
Initial astronomy password:
Entry for aliases file:

## astronomy mailing list
                     ## created: 08-Mar-2002 root
                     astronomy:          "|/home/mailman/mail/wrapper post astronomy"
astronomy-admin:    "|/home/mailman/mail/wrapper mailowner astronomy"
astronomy-request:  "|/home/mailman/mail/wrapper mailcmd astronomy"
astronomy-owner:    astronomy-admin

Hit enter to continue with astronomy owner notification...

Client-blocking techniques use I P addresses, hostnames, or email addresses supplied by clients when they connect to deliver a message. Each piece of information supplied can be compared to lists of items from known spamming systems. Spamming systems might be owned by actual spammers, but they might also be unintentionally open relays managed by hapless, (almost) innocent mail administrators. In either case, if a system is regularly sending you spam, you will probably decide to block messages from it. One problem with identifying spam by IP address, hostname, or email address is that these items are easily forged. While the IP address of the connecting system requires some sophistication to spoof, envelope email addresses are trivial to fake.

In addition to identifying clients, you can often recognize spam by its contents. Certain strings within email messages mark them as likely to be spam (“Our Rates Have Never Been Lower!!”). But trying to distinguish spam by the contents of the message can be problematic. Imagine that you receive lots of spam offering new house mortgages. You figure you can eliminate most of it by blocking messages that contain words like “really low interest rate on a new mortgage.” This may indeed block many spam messages, but you might also block a message from your friend (or one of your user’s friends) who just got a great deal on a new house and wrote to tell you about it.

The problem with both client- and content-based techniques to identify spam is that spammers are constantly finding ways to get around them. There is a sort of arms race going on between legitimate users of email and spammers. You can compile lists of open relays, but spammers expend a great deal of effort seeking out new open relays or proxy servers to abuse (and there always seem to be more of them).

You may discover that you receive a lot of spam with the same return address. You can block messages that use that return address, but spammers use hit-and-run tactics. They obtain an email address from one of the free email sites and use that address to send thousands or millions of spam messages, and then discard it for another. Within a couple of days, you’ll never see the address you listed again.

Even content filters have to adjust for spammers escalating tactics. Some spammers embed HTML codes within the words of their messages to break up phrases you might filter against. Or they encode the entire message so that when Postfix scans it for recognized spam phrases, there are no intelligible phrases. Most email clients oblige users by automatically rendering such messages—decoding or ignoring extraneous HTML codes. Recipients often don’t even notice that the message had originally been encoded.

The SMTP conversation in Figure 11-1 should be familiar to you from Chapter 2. Example 11-1 shows the log entries for the transaction. First, an SMTP client connects to Postfix over a socket. Because of the way sockets function, Postfix learns the IP address of the client when it establishes the connection. You don’t see the client IP address in the figure, but it is logged by Postfix. You can accept or reject a message based on the client hostname or IP address, thus blocking specific hostnames or IP and network addresses.

1. postfix/smtpd[866062]: connect from mail.ora.com[10.143.23.45]
2. postfix/smtpd[866062]: D694B20DD5B: client=[10.143.23.45]
3. postfix/cleanup[864868]: D694B20DD5B: \
  message-id=<20030106185403.D694B20DD5B@smtp.example.com>
4. postfix/qmgr[861396]: D694B20DD5B: from=<info@ora.com>, \
  size=486, nrcpt=1 (queue active)
5. postfix/local[864857]: D694B20DD5B: to=<kdent@smtp.example.com>, \
  relay=local, delay=98, status=sent (mailbox)
6. postfix/smtpd[866062]: disconnect from mail.ora.com[10.143.23.45]

Once connected, the client sends a HELO command with an identifying hostname. The hostname provided can be used to accept or reject a message using smtpd_helo_restrictions .

In the next step, the client issues a MAIL FROM command to indicate the sender’s email address, followed by a RCPT TO command to indicate the recipient’s email address.

If everything is acceptable up to the point of the DATA command, the client is permitted to send the contents of the message, which consist of message headers followed by the message body. Postfix provides another opportunity to reject the message based on its contents (see Section 11.9 later in this chapter). If the final header and body checks are acceptable, the message is delivered.

Postfix indicates to the client that it has rejected a message by sending reply codes. Standard reply codes are described in Chapter 2. In this chapter, we consider codes in the 4xx and 5xx range. More information appears in a sidebar later in this chapter.

When you assign restrictions to Postfix UBE rules, it is not necessary to use all of the rules. You can define restrictions for the ones you need and leave out the others. The default setting if no rules are set in main.cf looks like the following:

smtpd_client_restrictions =
smtpd_helo_restrictions =
smtpd_sender_restrictions =
smtpd_recipient_restrictions =
     permit_mynetworks, reject_unauth_destination

This prevents your system from being an open relay by allowing any computer on your network to relay while rejecting all others unless they are sending messages destined for one of your users.

There are many restrictions available. Table 11-1 lists each one along with the client information it operates on. One important concept that confuses many people at first is that any of these restrictions can be used in any rule. While it may seem logical that check_helo_access should be assigned to smtpd_helo_restrictions, it could equally be assigned to smptd_sender_restrictions or any of the others. This gives you a lot of flexibility in ordering your restrictions when deciding what to accept and what to block.

You’ll notice from Table 11-1 that some rules take an argument of the form maptype:mapname. The mapname refers to a normal Postfix lookup table whose lefthand key is matched against the piece of client information, and the righthand value is the action to perform. Access maps are discussed in Restriction Definitions following.

smtpd_delay_reject = no

soft_bounce = yes

smtpd_recipient_restrictions =
        permit_mynetworks
        reject_unauth_destination
        warn_if_reject reject_invalid_hostname
        reject_unknown_recipient_domain
        reject_non_fqdn_recipient

smtpd_recipient_restrictions =
     permit_mynetworks
     reject_unauth_destination
     reject_invalid_hostname
     reject_unknown_sender_domain

There are six types of restrictions introduced below. Each of the restrictions are defined in the sections that follow.

smtpd_client_restrictions = 
     check_client_access hash:/etc/postfix/client_access
smtpd_sender_restrictions =
     check_sender_access hash:/etc/postfix/sender_access
smtpd_recipient_restrictions =
     permit_mynetworks
     reject_unauth_destination
     reject_invalid_hostname
     reject_unknown_sender

10.157                   REJECT
192.168.76.23            REJECT
currentmail.com          REJECT

hardsell@example.com               REJECT
marketing@                         REJECT
specials.digital-letter.com        REJECT

With what we know so far, let’s trace what happens with some simple HELO restrictions. Consider that smtpd_helo_restrictions is assigned the following rules:

smtpd_helo_restrictions = 
    check_helo_access hash:/etc/postfix/helo_access
    reject_invalid_hostname

and helo_access contains the following entries:

greatdeals.example.com     REJECT
oreillynet.com             OK

Let’s follow four different scenarios when clients connect with different HELO commands:

By default mime_header_checks and nested_header_checks use the same lookup tables as header_checks. If you want to distinguish checks for each one, you can configure them separately; otherwise, configuring header_checks causes mime_header_checks and nested_header_checks to use the same patterns as header_checks. When you assign the checking parameters, indicate both the lookup table and which type of regular expression you are using (see Chapter 4):

header_checks = regexp:/etc/postfix/header_checks
body_checks = regexp:/etc/postfix/body_checks

In a pattern-checking lookup table, the lefthand key is a regular expression enclosed by two delimiters (usually forward slashes):

/match string/              REJECT

A typical header_checks file contains lines like the following:

/free mortgage quote/       REJECT
/repair your credit/        REJECT
/take advantage now/        REJECT

If any of the strings shown appear in any of the headers of a message (these would most likely show up in the Subject: header), the message is rejected. Postfix logs the rejection along with the offending line, and if you specified a message, it is also logged and sent to the client.

The right hand action can be one of the following values. The values that allow an optional text message are indicated. The specified message is sent to the client and logged with the rejection. If you don’t supply a message, Postfix uses the default.

Actions cannot include specific error reply codes or customized restrictions as with access maps.

Header checks compare each header against every pattern in the listed lookup files. Multiline headers are combined into a single line before making comparisons. Each pattern is checked in the order you list them, and checking stops as soon as Postfix finds a match, at which point the message is handled according to the action you specified.

The patterns indicated by the body_checks parameter are checked against each line of the body of the message. Lines are compared one at a time, and each one is checked against every pattern in the order you list them. Checking stops as soon as Postfix finds a match, at which point the message is handled according to the action you specified.

Very long body lines are compared in chunks that are at most as long as the value of the parameter line_length_limit . The default is 2048. Also, by default, Postfix checks the contents of the body only up to the value of body_checks_size_limit . The default is 50 KB. Message headers are compared in chunks that are limited by header_size_limit. These limits are useful in preventing Postfix from scanning the entire file when messages contain large attachments.

Some administrators use header checks for simple virus scanning. You can reject all messages that include attachments with file extensions that might be dangerous to your users:

/name ?="?.*\.(bat|com|dll|exe|hta|pif|vbs)"?/        REJECT

You should include any other extensions that you know might pose a problem for your users. Be aware, however, that this pattern is not really sufficient for true virus scanning since you are certain to miss some extensions, and many PC clients may execute files regardless of their extension.

A typical body_checks file contains lines like the following:

/increase your sales by/             REJECT
/lowest rates.*\!/                   REJECT
/in compliance (with|of) strict/     REJECT
/[:alpha:]<!--.*-->[:alpha:]/        REJECT Suspicious embedded HTML comments

The second line matches any string that starts with “lowest rates” followed by any text leading to an exclamation point (“We have our lowest rates in 40 years!”). The fourth line checks for HTML comments that are embedded in the middle of words. Remember that this is a common spammer trick to defeat your content filters, but it’s also a dead giveaway that the message contains spam.

You can test your regular expressions with the postmap command. Place the contents of a message into a file, then redirect the file to postmap:

$ postmap -q - regexp:/etc/postfix/body_checks < msg.txt
opportunity. increase your sales by 500%. Consider      REJECT

postmap prints any lines that match any of the regular expressions along with the action specified.

Study the spam you receive to refine and add to your patterns. However, be aware of potential performance problems with poorly written regular expressions. Another potential issue with content checking is that there is no way to whitelist individual messages that you might want to receive despite their containing phrases that trigger a rejection. In particular, if a message is whitelisted during the restriction parameter checking (described earlier in this chapter), it might still be rejected by header and body checks.

As you create rules for detecting spam, keep in mind that your users may differ in what balance they’ll accept between some spam and the possibility of blocking some real messages. If you must create different rules for different users, it’s probably best not to try to accomplish this with an MTA. Instead consider a specialized delivery agent such as procmail, maildrop, or sieve to set up per-user UBE rules. You can use Postfix to set up broad per-user class restrictions, as you’ll see in the next section.

We’ll call the two classes “spamlover” and “spamhater.” You must list all of the restriction classes you plan to define in the smtpd_restriction_classes parameter:

smtpd_restriction_classes = spamlover, spamhater

We’ve invented the names of the classes, but once listed with smtpd_restriction_classes, they can be treated like any other restriction rule. You can assign a list of restrictions to be considered for the class. Once defined, the restriction class can be used as an action in an access table. When Postfix encounters the class, it steps through the assigned restrictions.

We’ll define “spamhater” with several restrictions:

spamhater =
        reject_invalid_hostname
        reject_non_fqdn_hostname
        reject_unknown_sender_domain
        reject_rbl_client nospam.example.com

and “spamlover” with a simple “permit”:

spamlover = permit

You could, of course, refine these with restrictions that make sense for your own configuration.

Now that the restriction classes have been declared and defined, you can put them to use by assigning the appropriate class to each of our recipients in a lookup table. We’ll call the table per_user_ube.

#
# per_user_ube
#
abelard@example.com   spamhater
heloise@example.com   spamlover

Next, tell Postfix that it should check your recipient lookup table when checking restrictions:

smtpd_recipient_restrictions =
    permit_mynetworks
    reject_unauth_destination
    check_recipient_access hash:/etc/postfix/per_user_ube

When a message comes in addressed to abelard@example.com, Postfix goes through the normal default restrictions and then encounters check_recipient_access pointing to the recipient lookup table. Postfix finds the recipient address in the file, reads the action spamhater, and then invokes the restrictions defined for spamhater. If any of the “spamhater” restrictions returns REJECT, Postfix rejects the message; otherwise, it is delivered. Messages for heloise@example.com go through the same process, but when Postfix checks the “spamlover” restrictions, it finds permit and immediately accepts the message.

The client and server must agree on the authentication mechanism they’ll use. (See the Cyrus documentation for currently supported mechanisms.) Some of the more common mechanisms are listed below:

When a client connects to a mail server, the server typically lists all of the password mechanisms it supports, in order of preference. The client tries the first one it supports. If that fails, it may be configured to try additional mechanisms until it can authenticate successfully. If the client and server cannot successfully negotiate over a common mechanism, authentication fails.

Once the server and client agree on a mechanism, they begin the authentication process, consisting of one or more challenges and responses that are governed by the agreed-upon mechanism. The protocol also specifies how these exchanges should be encoded.^[1]

The SASL authentication framework can use your existing Unix system passwords (for example, passwd, shadow, or PAM) or a separate password file just for authenticating SMTP users. Other options include Kerberos or even a new scheme of your own.

Ultimately, your choice comes down to where and how you want to store your authentication information. Consider your network and how your users currently authenticate to decide which framework works best for you. If your mail users already authenticate on your network through PAM, for example, then you probably want to configure SASL to use your existing system. If, on the other hand, most of your SMTP users are virtual accounts (without system logins), you should opt for a separate password database for SMTP users. Often your POP/IMAP server can share the same user database, making this a convenient option for virtual mail accounts.

The SASL library uses a separate configuration file for each application it works with. Postfix uses a file named smtpd.conf for SASL purposes. This file is usually located at /usr/local/lib/sasl2/smtpd.conf. At a minimum, smtpd.conf contains a line indicating the framework to use. We are going to look at specifying either Unix passwords or separate SASL passwords for Postfix authentication. See the Cyrus documentation to see other options you might include in smtpd.conf.

pwcheck_method: saslauthd

# saslauthd -a pam

pwcheck_method: auxprop

# chown postfix:sasl /etc/sasldb2
# chmod 440 /etc/sasldb2

# saslpasswd2 -c -u `postconf -h myhostname` kdent
Password: 
Again (for verification):

All of the relevant Postfix parameters for SASL password authentication start with smtpd_sasl* for the SMTP server or smtp_sasl* for the SMTP client. For server configuration you need at a minimum the smtpd_sasl_auth_enable parameter and the permit_sasl_authenticated restriction, which must be assigned to one of the smtpd restriction parameters. See Chapter 11 for more information on UBE restrictions.

smtpd_sasl_auth_enable = yes

broken_sasl_auth_clients = yes

kdent@example.com        kdent

smtpd_sender_login_maps = hash:/etc/postfix/sasl_senders

smtpd_recipient_restrictions = permit_mynetworks, 
        permit_sasl_authenticated, reject_unauth_destination

smtpd_sasl_security_options = noanonymous, noplaintext

Following are step-by-step instructions summarizing the configuration described in this chapter. This is a broad overview of what’s required to set up your Postfix system with SASL: