Table of Contents for

sendmail, 4th Edition

IDA and V8 sendmail allow comments in :include: files. Comment lines begin with a # character. If the # doesn’t begin the line, it is treated as the beginning of an address, thus allowing valid usernames that begin with a # (such as #1user) to appear first in a line by prefixing them with a space:

# Management               ← a comment
frida
george@wash.dc.gov
# Staff                    ← a comment
ben
steve
 #1user                    ← an address

Note that because comments and empty lines are ignored by sendmail, they can be used to create attractive, well-documented mailing lists.

Under older versions of sendmail, comments can be emulated through the use of RFC-style comments:

( comment )

By surrounding the comment in parentheses, you cause sendmail to view it (and the parentheses) as an RFC-style comment and, thus, to ignore it:

( Management )
frida
george@wash.dc.gov
( Staff )
ben
steve

This form of comment works with both the old and new sendmail programs.

As has been noted, the aliases file should be writable only by root for security reasons. Therefore, ordinary users, such as nonprivileged department heads, cannot use the aliases file to create and manage mailing lists. Fortunately, :include: files allow ordinary users (or groups of users) to maintain mailing lists. This offloads a great deal of work from the system administrator, who would otherwise have to manage these lists, and gives users a sense of participation in the system.

In some circumstances, reading :include: lists can be slower than reading entries from an aliases database. At busy sites or sites with numerous mail messages addressed to mailing lists, this difference in speed can become significant. Note that the -bv command-line switch (-bv on page 237) can be used with sendmail to time and contrast the two different forms of lists. On the other hand, rebuilding the aliases(5) database can sometimes be very slow. In such instances, the :include: file can be faster because it doesn’t require a rebuild each time it changes.

One possible common disadvantage to all types of mailing lists is that they are visible to the outside world. This means that anyone in the world can send mail to a local list that is intended for internal use. Many lists are intended for both internal and external use. One such list might be one for discussion of the O’Reilly Nutshell Handbooks, called, say, nuts@oreilly.com. Anyone inside oreilly.com and anyone in the outside world can send mail messages to this list, and those messages will be forwarded to everyone on the nuts mailing list.

It is possible to protect your internal lists from use by outsiders, but doing so requires writing custom rules. For possible rules you might be able to adapt for your site’s needs, see http://www.sendmail.org/~ca/email/examples/internal.aliases.html.

The eventual recipient of a mailing-list message should be able to reply to the message and have that reply go to either the original sender or the list as a whole. Which of these happens is an administrative decision. In general, replies go to the address listed in the From: or Reply-To: header. If the intention is to have replies go to the list as a whole, these headers need to be rewritten by a filter at the originating site:

list:   "|/etc/local/mailfilter list -oi -odq -flist-request list-real"

Here, the name of the filter has replaced sendmail in the aliases file entry. Writing such a filter is complex. The original addresses need to be preserved with appropriate headers before they are rewritten by the filter.

The converse problem is that not all mail-handling programs handle replies properly. Some programs, such as UUCP and certain versions of emacs-mail, insist on replying to the envelope sender as conveyed in the five-character "From" header. By setting up lists correctly (as we showed earlier), an administrator can at least guarantee that those replies are sent to the list maintainer, who can then forward them as required.

A more serious problem is the way other sites handle bounced mail. In an ideal world, all sites would correctly bounce mail to the envelope sender and (less desirably) to the Errors-To: address, which, beginning with V8.8, is supported only if the UseErrorsTo option (UseErrorsTo on page 1115) is set to true.^[212] Unfortunately, not all sites are so well behaved. If a mailing list is not carefully set up, there is a possibility that bounced mail will be re-sent to the list as a whole. To minimize such potential catastrophes, follow the guide in Table 13-1.

When gatewaying a mailing list to Usenet news, the inews(1) program bounces the message if it is for a moderated group and lacks an Approved: header, which can be added by a filter program (Write a Delivery Agent Script on page 470) or by a news gateway delivery agent.

If your site is running (or has access to) Usenet news, the recnews(1) program that is included therein can be used to gateway mail to newsgroups. It inserts the Approved: header that inews needs and generally handles its gateway role well. One minor pitfall to avoid with recnews is making separate postings when you intend cross-postings:

mail-news: "|/usr/local/recnews comp.mail comp.mail.d" ← separate postings
mail-news: "|/usr/local/recnews comp.mail,comp.mail.d" ←  cross-posted
                                         ↑
                                    note the comma

There are many ways to handle bounced mail in managing a mailing list. One of the best ways for large lists is to create a bounce alias for a list:

list-bounce: :include:/usr/local/lists/list-bounce

When an address in the main list begins to bounce, move it from the main list’s file to the corresponding list-bounce file. Then send a message to that list nightly (via cron(8)), advising the users in it that they will soon be dropped. To prevent the bad addresses from deluging you with bounced mail, set up the return address and the envelope to be an alias that delivers to /dev/null:

black-hole:   /dev/null

Finally, arrange to include the following header in the outgoing message:

Precedence: junk

This prevents most sites from returning the message if it cannot be delivered.

Programs are available that can help to manage large and numerous mailing lists. We will cover them later in this chapter.

It is impossible to cause all users to interact properly with a mailing list. For example, all submissions to a list should (strictly speaking) be mailed to list, whereas communications to the list maintainer should be mailed to list-request. As a list maintainer, you will find that users mistakenly reverse these roles surprisingly often.

One possible cure is to insert instructions in each mailing at the start of the message. In the header, for example, Comment: lines can be used like this:

Comment: "listname" INSTRUCTIONS
Comment: To be added to, removed from, or have your address changed
Comment: in this list, send mail to "listname-request".

Unfortunately, user inattention usually dooms such schemes to failure. You can put instructions everywhere, but some users will still send their requests to the wrong address.

A solution some sites use when the list is used only for official and rare mailings is to install the list name in the aliases file just before the mailing:

list:        :include: /usr/local/lists/official.list     ← before

Then run newaliases(1) and send mail to the list. After all the mail for the list has been queued, edit the aliases file, comment out that entry, and create a new one:

#list:        :include: /usr/local/lists/official.list     ← after
list:   owner-list

Run newaliases(1) again, and you will have disabled that list. That way, mail that is wrongly sent to list will be received only by the list’s owner (who can notify the sender of the error) instead of wrongly being broadcast to the list as a whole.

All mass mailings, such as mailing-list mailings, should have a header Precedence: line that gives a priority of bulk, junk, or list. On the local machine, these priorities cause the message to be processed from the queue after higher-priority mail. At other sites, these priorities will cause well-designed programs (such as the newer vacation(1)^[213] program) to skip automatically replying to such messages.

The X.400 telecommunications standard is finding some acceptance in Europe and by the U.S. government. Addresses under X.400 always begin with a leading slash, which can cause sendmail to think that the address is the name of a file when the local delivery agent is selected:

/PN=MS.USER/O=CORP/PRMD=CORP/ADMD=TELE/C=US/

To prevent this misunderstanding, all such addresses should be followed by an @domain part to route the message to an appropriate X.400 gateway:

/PN=MS.USER/O=CORP/PRMD=CORP/ADMD=TELE/C=US/@X.400.gateway.here

Each mailing to a list should contain clear information describing how a subscriber may be removed from the list and to whom to send questions or complaints.

As we illustrated earlier, some mailing list software inserts the information into custom X- headers on your behalf. For example:

X-Unsubscribe: remove@mailing.list.domain
X-Owner: list-request@mailing.list.domain

Others arrange for standard headers to work. For example:

From: list-request@mailing.list.domain

Here, merely replying to this message will get the subscriber’s comments delivered directly to the list administrator (see RFC2142 Common Mailbox Names on page 474 for a -request suffix discussion).

Other mailing list software appends a standard footer to the body of every message. For example:

This list is brought to you by the power of mailing.list.domain.
To unsubscribe visit http://www.mailing.list.domain/unsubscribe
or send email to unsubscribe@mailing.list.domain. To report
abuse or problems, send email to abuse@mailing.list.domain. In
the event email fails you may also telephone +1-800-555-1234 or send
surface mail to MailingList, Inc. P.O. Box 555, City, CA 12345

This footer solves most of a mailing list’s needs. It allows the recipient to unsubscribe either via a web site or by sending email to a clearly indicated email address. It also indicates to whom to send complaints and reports of problems. It is vital (and required) that if you send email from your site, you maintain an alias for the user named abuse, which causes mail for that name to be delivered to an actual person. Note that if email fails, there is a telephone number and surface mail address to fall back on.

We recommend that you adopt as many of these techniques as you can. A recipient should be able to communicate with the administrator of the list by simply sending a message to the name of the list suffixed with a -request. Also, information about unsubscribing should be placed in clear text in the body of every message.

Many businesses routinely reject messages that contain attachments, or accept them and silently strip attachments. Mailing list management should adopt a similar sort of strategy when accepting messages that will be broadcast to subscribers. To protect the recipients of the mailing list, either reject submissions that contain attachments or silently remove attachments (perhaps with an indication of that removal placed in the body of the message).

The method for rejecting or removing attachments varies depending on the type of mailing list software you use, and therefore, we must leave the discovery of that method up to you.

Some lists discuss matters that, by their very nature, require readers to view or hear examples. When administrating lists that discuss images or sounds, for example, try to encourage list members to send web references instead of embedding the images or sounds directly into each message. The following lines illustrate one appropriate technique:

I put my latest 3D images up for you to see at
http://www.my.domain/3d/bob/newimages. Let me know
if you like them.

Here, a half kilobyte message distributes images vastly more efficiently than would a potentially two or three megabyte message that embedded the images directly inside itself. Because of that efficiency, use of references is kinder to ISP machines, and reduces the risk that the images will be removed or email rejected because they have attachments.

Hands down, the most offensive way to email a message to a mailing list is by placing all the recipients into a To: or CC: header or both. Not only will this likely mark you as a spammer, but it also risks that your site will become listed at one or more blacklisting sites.

Never send mail to a mailing list like this:

To: list-owner@mailing.list.domain
Cc: bob@a.domain, ben@another.domain, bill@yet.another.domain,
       carrie@somewhere.gov, jose@there.domain, ...
        etc. for hundreds of addresses

There are two serious problems with this approach. First, it reveals all the members of the list to every recipient on the list. This violates the privacy of each recipient on the list. Most who join an organization, or mailing list, expect that their membership will be private and not advertised to others.

Second, messages with too many header recipients (typically more than 25 or so) are consider spam email by many sites. Mailing list messages are not spam email and therefore should never appear to be.

See Internal Mailing Lists on page 485 to learn the correct way to set up mailing lists using sendmail.

As mailing lists become large, or emailings to them become frequent, lists eventually need to be moved from self-serviced lists to fully automated lists. Three classes of software are available for this transition. Open source software for Unix is mature and well written. Commercial software for Unix and Windows is widely available. Commercial services (some free with advertising included in each message) are also available.

See Packages That Help on page 499 for a discussion of several available packages.

Each subscriber that joins your mailing list should be made aware of your list’s policies from the beginning. One common method of distributing policy to subscribers is to include it in the initial greeting sent to a new subscriber. Another common method is to post it on a web site. Naturally, we encourage you to do both.

As the administrator of a mailing list, it is your job to police that list. Anytime a subscriber sends an offensive or spam email message to the list, you should immediately contact that subscriber and take corrective action. Many administrators will immediately unsubscribe the offender. Some administrators must find other solutions, because members may have to pay to subscribe.

Find out what your rights are as an administrator before you accept the job or before you set up the mailing list. Make certain you can remove offending subscribers in a timely manner to protect the remainder of your subscriber base.

As a courtesy to your remaining subscribers, you should let them know that you handled a certain problem and that the offending subscriber won’t post to the list again.

The Majordomo mailing-list management software was originally written by Brent Chapman using the perl(1) language. Its chief features are that it allows users to subscribe to and remove themselves from lists without list manager intervention and that it allows list managers to manage lists remotely. In addition, users can obtain help and list descriptions with simple mail requests. Note that Majordomo aids in managing a list (the list addresses) but does not aid in list moderation (the contents of mail messages). But Majordomo does catch administrative mail erroneously sent to the list as whole. Majordomo is available from http://www.greatcircle.com/majordomo/.

The Mailman mailing-list management software was written and is distributed as part of the GNU project. It claims to be fully web-based for ease of management. Even list managers can manage lists entirely over the Web. Mailman is written in Python (with a little additional C code for better security). The Mailman package is available from http://www.gnu.org/software/mailman/.

The ListProcessor system was written by Tasos Kotsikonas. It is an automated system for managing mailing lists that replaces the aliases file for that use. According to the author, it includes support for “public and private hierarchical archives, moderated lists, peer lists, peer servers, private lists, address aliasing, news connections and gateways, mail queueing, list ownership, owner preferences, crash recovery, and batch processing.” The system also accepts Internet connections for “live” processing of requests at port 372 (as assigned by the IANA).^[214] The ListProcessor system is available from http://www.listserv.net/.

According to its author, Murray S. Kucherawy, the ListManager system “is written entirely in C, so it’s faster and more efficient than mailing list systems based on scripted languages. Rather than flat files, it uses fast B-tree databases courtesy of Sleepycat Software for faster performance.” This is the mailing list software used by sendmail.org. Among the features claimed on its web site are:

The Listmanager software is available at http://www.listmanager.org/.

The traditional use of the ~/.forward file, as its name implies, is to forward mail to another site. Unfortunately, as users move from machine to machine, they can leave behind a series of ~/.forward files, each of which points to the next machine in a chain. As machine names change and as old machines are retired, the links in this chain can be broken. One common consequence is a bounced mail message (“host unknown”) with a dozen or so Received: (Received: on page 1162) header lines.

As the mail administrator, you should beware of the ~/.forward files of users at your site. If any contain offsite addresses, you should periodically use the SMTP expn command^[217] to examine them. For example, consider a local user whose ~/.forward contains the following line:

user@remote.domain

This causes all local mail for the user to be forwarded to the host remote.domain for delivery there. The validity of that address can be checked with nslookup and telnet(1) at port 25^[218] and the SMTP expn command:

% ns -q=mx remote.domain
Address:  123.45.67.89

remote.domain preference = 0, mail exchanger = mail.remote.domain
remote.domain preference = 10, mail exchanger = mx.another.domain

% telnet mail.remote.domain 25
Trying 123.45.123.45 ...
Connected to mail.remote.domain.
Escape character is '^]'.
220 mail.remote.domain Sendmail 8.14.1/8.14.1 ready at Thu, 13 Dec 2007 09:48:09 −0600
(MDT)
220 ESMTP spoken here
expn user
250 <user@another.site>
quit
221 remote.domain closing connection
Connection closed by foreign host.
%

This shows that the user is known at remote.site but also shows that mail will be forwarded (yet again) from there to another.site. By repeating this process, you will eventually find the site at which the user’s mail will be delivered. Depending on your site’s policies, you can either correct the user’s ~/.forward file or have the user correct it. It should contain the address of the host where that user’s mail will ultimately be delivered.

But beware that the world of email is becoming less friendly for the well-intentioned administrator. Because EXPN can be used to harvest addresses for spam lists, it is more and more frequently turned off. If you connect to a site with EXPN turned off, you will see an error such as the following, instead of the forwarding address you need:

502 5.7.0 Sorry, we do not allow this operation

If EXPN fails, try finger(1) in its place, which also might fail (another illustration of the harm caused by spam email).

Because ~/.forward files are under user control, the administrator occasionally needs to break loops caused by improper use of those files. To illustrate, consider a user who wishes to have mail delivered on two different machines (call them machines A and B). On machine A, the user creates a ~/.forward file such as this:

\user, user@B

Then, on machine B, the user creates this ~/.forward file:

\user, user@A

The intention is that the backslashed name (\user) will cause local delivery and the second address in each will forward a copy of the message to the other machine. Unfortunately, this causes mail to go back and forth between the two machines (delivering and forwarding at each) until the mail is finally bounced with the error message “too many hops.”

On the machine that the administrator controls, a fix to this looping is to temporarily edit the aliases database and insert an alias for the offending user, such as this:

user:  \user

This causes mail for user to be delivered locally and that user’s ~/.forward file to be ignored. After the user has corrected the offending ~/.forward files, this alias can be removed.

The ~/.forward file can contain the names of files onto which mail is to be appended. Such filenames must begin with a slash character that cannot be quoted. For example, if a user wishes to keep a backup copy of incoming mail:

\user
/home/user/mail/in.backup

the first line (\user) tells sendmail to deliver directly to the user’s mail spool file using the local delivery agent. The second line tells sendmail to append a copy of the mail message to the file specified (in.backup).

Note that prior to V8, sendmail did no file locking, so writing files by way of the ~/.forward file was not recommended. Beginning with V8, however, sendmail locks those files during writing, so such use of the ~/.forward file is now OK.

If the SafeFileEnvironment option (SafeFileEnvironment on page 1084) is set, the user should be advised to specify the path of that safe directory:

\user
/arch/bob.backup          ←  here /arch was specified by the SafeFileEnvironment
option

When the SafeFileEnvironment option is used, the cooperation of the system administrator might be needed if users are to have the ability to save mail to files via the ~/.forward file.

The ~/.forward file can contain the names of programs to run. A program name is indicated by a leading pipe (|) character, which might or might not be quoted (Delivery Via Programs on page 468). For example, a user might be away on a trip and want mail to be handled by the vacation(1) program:

\user, "|/usr/ucb/vacation user"

Recall that prefixing a local address with a backslash tells sendmail to skip additional alias transformations. For \user, this causes sendmail to deliver the message (via the local delivery agent) directly to the user’s spool mailbox.

The quotes around the vacation program are necessary to prevent the program and its single argument (user) from being viewed as two separate addresses. The vacation program is run with the command-line argument user, and the mail message is given to it via its standard input.

Beginning with V8 sendmail, a user must have a valid shell to run programs from the ~/.forward file and to write files via the ~/.forward file. See The /etc/shells File on page 180 for a description of this process and for methods to circumvent it at the system level.

Because sendmail sorts all addresses and deletes duplicates before delivering to any of them, it is important that programs in ~/.forward files be unique. Consider a program that doesn’t take an argument and suppose that two users both specified that program in their ~/.forward files:

user 1 →  \user1, "|/bin/notify"
user 2 →  \user2, "|/bin/notify"

Prior to V8 sendmail, when mail was sent to both user1 and user2, the address /bin/notify appeared twice in the list of addresses. The sendmail program eliminated what seems to be a duplicate,^[219] and one of the two users did not have the program run.

If a program requires no arguments (as opposed to ignoring them), the ~/.forward program specifications can be made unique by including a shell comment:

user 1 →  \user1, "|/bin/notify #user1"
user 2 →  \user2, "|/bin/notify #user2"

Rather than expecting users to write home-grown programs for use in ~/.forward files, offer them any or all of the publicly available alternatives. The most common are listed in the following sections.

"|exec /usr/local/bin/procmail #user"

"| /usr/local/lib/mh/slocal -user user"

Normally, a program in the user’s ~/.forward file is executed with the Bourne shell:

Mprog, P=/bin/sh,   F=lsDFMeuP,  S=10, R=20, A=sh -c $u
                                               ↑
                                         the Bourne shell

One drawback to using the Bourne shell to run programs is that it exits with a value of 1 when the program cannot be executed. When sendmail sees the exit value 1, it bounces the mail message.

There will be times when bouncing a mail message because the program could not execute is not desirable. For example, consider the following ~/.forward file:

"| /usr/local/lib/slocal -user george"

If the directory /usr/local/lib is unavailable (perhaps because a file server is down or because an automounter failed), the mail message should be queued rather than bounced. To arrange for requeuing of the message on failure, users should be encouraged to construct their ~/.forward files like this:

"| /usr/local/lib/slocal -user george || exit 75"

Here, the || tells the Bourne shell to perform what follows (the exit 75) if the preceding program could not be executed or if the program exited because of an error. The exit value 75 is special, in that it tells sendmail to queue the message for later delivery rather than to bounce it.