The primary reason for this book, the fourth edition of sendmail, is the release of version 8.14 of the sendmail program. Since the release of the third edition, V8.13 and V8.14 sendmail have been released. Each sendmail release has shown marked improvements over earlier releases and, together, they call for a full update of this book.
In addition to folding the new V8.14 information into this book, we have fixed all the errata in the third edition to make this fourth edition much more accurate.
This edition of the sendmail book assumes you are using V8.14, the current version of the sendmail program. It follows the same general format as earlier editions, but we realize this might not be the most convenient arrangement for readers who are primarily interested in what has changed since the last edition. To help minimize this problem, we have added Appendix B, in which the many improvements of the intervening versions of sendmail are categorized by chapter, complete with references to the appropriate sections within this book.
King Gordius of Phrygia once created a knot so tangled that no one could undo it. The Gordian knot stayed tangled, or so the story goes, until Alexander the Great came along and took a different approach to untying the knot. With a sweep of his sword, he parted the great knot once and for all.
It would be nice if the knot that is sendmail could be undone with one quick stroke of fresh insight, but alas, it cannot. Instead, a more mundane approach must be taken, so in this book we untie the hard way, one strand at a time.
But, you might ask, “Why the effort? Doesn’t sendmail predate the dawn of computing time? Hasn’t the time come to replace sendmail with something new, something better, something modern?” Not so. Age has brought sendmail maturity and reliability. The sendmail program has withstood the test of time because it is more than just a program, it is a philosophy: a general-purpose, internetwork mail-routing facility with the flexibility and configurability to solve the mail-routing needs of all sites large or small, complex or simple.
These strengths of sendmail are also its weaknesses. Configurability has bred complexity. The sendmail program is difficult to configure and even more difficult to understand. Its configuration file, for example, can be positively frightening. But don’t despair. With this book in hand, you should be able to configure sendmail to meet any need and bring the days of the sendmail guru to an end.
The sendmail program was originally written by Eric Allman while he was a student and staff member at the University of California at Berkeley. At the time, one campus machine (Ingres) was connected to the ARPAnet and was home to the INGRES project where Eric was working. Another machine (Ernie CoVax) was home to the Berkeley Unix project and had recently started using the Unix to Unix Communication Protocol (UUCP). These machines (as well as several others on campus) were connected via a low-cost network built by Eric Schmidt, called BerkNet. Software existed to move mail within ARPAnet, within UUCP, and within BerkNet, but none yet existed to move mail between these three networks.
A sudden increase in protocol types, coupled with the anticipation of an explosion in the number of networks, motivated Eric Allman to write delivermail—the precursor to sendmail. The delivermail program was shipped in 1979 with 4.0 and 4.1 BSD Unix. Unfortunately, delivermail was not flexible enough to handle the changes in mail-routing requirements that actually occurred. Perhaps its greatest weakness was that its configuration was compiled in.
In 1980, ARPAnet began converting from Network Control Protocol (NCP) to Transmission Control Protocol (TCP). This change increased the number of possible hosts from 256 to more than 1 billion. Another change converted from a “flat” hostname space (such as MIT-XX) into a hierarchical namespace (such as XX.MIT.EDU). Prior to these changes, mail was transported using the File Transfer Protocol (FTP). Afterward, a new protocol was developed for transporting mail, called Simple Mail Transfer Protocol (SMTP). These developments were not instantaneous. Some networks continued to run NCP years after most others switched to TCP. And SMTP underwent many revisions before finally settling into its present form.
Responding to these and other changes, Eric evolved delivermail into sendmail. To ensure that messages transferred between networks would obey the conventions required by those networks, Eric took a “liberal” approach—modifying address information to conform rather than rejecting it. At the time, for example, UUCP mail often had no headers, so sendmail had to create them from scratch.
The first sendmail program was shipped with 4.1c BSD (the first version of Berkeley Unix to include TCP/IP). From that first release to the present,[1] Eric has continued to enhance sendmail, first at UC Berkeley, then at Britton Lee, then back at UC Berkeley, then with InReference Inc., and now with Sendmail, Inc. The current major version of sendmail is V8, a major rewrite that includes many bug fixes and significant enhancements.
But Eric wasn’t the only one working on sendmail. In 1987, Lennart Lovstrand of the University of Linköping, Sweden, developed the IDA enhancements to BSD sendmail Version 5. IDA (which stands for Institutionen för Datavetenskap) injected a number of improvements into sendmail (such as support for .dbm files and separate rewriting of headers and envelopes) and fixed a number of bugs. As the 1990s approached, two offspring of IDA appeared.
Neil Rickert (Northern Illinois University) and Paul Pomes (The University of Illinois) took over maintenance of IDA sendmail. With contributions from around the world, their version (UIUC IDA) represents a continuation of the work begun by Lennart Lovstrand. Neil focused on fixing and enhancing the configuration files into their current m4-based form. Paul maintained the code, continually adding enhancements and fixing bugs. In general, their version was large, ambitious, and highly portable. It succeeded in solving many complex mail-routing problems.
A variation on IDA sendmail was also developed by Paul Vixie (while at Digital Equipment Corporation). Called KJS (for King James sendmail), it was a more conservative outgrowth of Lennart Lovstrand’s last IDA release. The focus of KJS was on code improvement rather than changes to configuration files.
In addition to these major offshoots, many vendors modified
sendmail to suit their needs. Sun
Microsystems made many modifications and enhancements to
sendmail, including support for
nis and nisplus
maps. Hewlett-Packard also contributed many fine enhancements,
including 8BITMIME
support.
This explosion of sendmail versions led to a great deal of confusion. Solutions to problems that work for one version of sendmail failed miserably for another. Even worse, configuration files were not portable, and some features could not be shared.
In 1992, Eric started creating a new version of sendmail to merge all the earlier versions. V8 officially adopted most of the good features from IDA, KJS, Sun, and HP’s sendmail, and kept abreast of the latest standards from the Internet Engineering Task Force (IETF). In 1996, Eric began work on V8.8 sendmail. This release continued the trend begun with V8.7, adding many requested new features and options, and tightening security. In 1998, V8.9 was released, continuing the direction started by V8.8.
In 1999, Sendmail, Inc., was founded in Emeryville, California. Sendmail, Inc., took over maintenance and development of the open source version of sendmail, and began work on a commercial version. Sendmail, Inc., has the web site:
| http://www.sendmail.com |
and is also one of the sponsors of the open source sendmail’s web site:
| http://www.sendmail.org |
For more information on the open source community and the Open Source Initiative (OSI), go to:
| http://www.opensource.org |
The first major offering from Sendmail, Inc., was V8.10 sendmail, released in 2000. It was mentored by Eric Allman, but largely written by Greg Shapiro.
V8.10 and V8.11 were developed in parallel. Claus Aßmann added SMTP AUTH and STARTTLS to V8.10, as well as a number of security changes, bringing that version up to V8.11. V8.11 was released as a commercial version because of export restrictions. Shortly afterward, export restrictions were relaxed and V8.11 was released in open source form.
Claus Aßmann took sendmail in a somewhat new direction with V8.12, in which he added a suite of new features. V8.13 expanded the Milter interface and added several new ways to suppress mail abuse, such as email address harvesting and denial of service. V8.14 continued this trend by further expanding the Milter interface, adding more antispam features, and creating more configuration flexibility.
I have to admit that I’m surprised by how well sendmail has succeeded. It’s not because of a large marketing organization or a deep-pockets budget. I think there are three reasons.
First, sendmail took the approach that it should try to accept, clean up, and deliver even very “crufty” messages instead of rejecting them because they didn’t meet some protocol. I felt this was important because I was trying to gateway UUCP to the ARPAnet. At the time, the ARPAnet was small, UUCP was anarchy, and Unix mail programs generally didn’t even understand headers. It was harder to do, but after all, the goal was to communicate, not to be pedantic.
Second, I limited myself to the routing function—I wouldn’t write user agents or delivery backends. This was a departure from the dominant thought of the time, in which routing logic, local delivery, and often the network code were incorporated directly into the user agents. But it did let people incorporate their new networks quickly.
Third, the sendmail configuration file was flexible enough to adapt to a rapidly changing world: the 1980s saw the proliferation of new protocols, networks, and user agents.
And, of course, it didn’t hurt that it was free, available at the right time, and did what needed to be done.
Configuring sendmail is complex because the world is complex. It is dynamic because the world is dynamic. Someday sendmail, like X11, will die—but I’m not holding my breath. In the meantime, perhaps this book will help.
When I started reviewing Bryan’s first-edition manuscript, I had been avoiding any major work on sendmail. But then I started reading about various petty bugs and annoyances that all seemed easy to fix. So I started making small fixes, then larger ones; then I went through RFC1123 to bring the specs up-to-date, cleaned up a bunch of 8-bit problems, and added ESMTP. It would be fair to say that the first book and sendmail Version 8 fed on each other—each improving the other.
We’ve divided this book into an introduction and two parts, each part addressing a particular aspect of sendmail.
Chapter 1, will be of special help to the new user. It covers the basic concepts underlying mail delivery and the roles sendmail plays in that delivery.
Part I, covers all aspects of handling sendmail, from downloading and installing new releases to managing mailing lists and aliases.
Part II, contains a heavily cross-referenced guide for configuring and tuning sendmail.
Part III, contains topic not directly germane to any particular chapter.
This book is primarily intended for system administrators who also administer email. But not all Unix systems are managed by administrators. Many are managed by programmers, network engineers, and even inexperienced users. It is our hope that this book satisfies all of you, no matter what your level of experience.
The true beginner should begin with Chapter 1, skipping ahead as needed.
The beginning system administrator should probably start with Part I to learn how to build, install, and administer sendmail, then skip ahead to topics of interest.
The experienced system administrator who wants to install and manage V8 sendmail should read Part I first to gain the needed background. Then explore Part II to discover further topics of interest.
Unix gurus and sendmail specialists should find Part II to be of value (even Eric keeps a copy on his desk). In it, every arcane detail of sendmail is listed alphabetically. For example, in Part II you’ll find a single chapter dedicated to options, with every option listed and explained.
No matter what your level of expertise, the sheer size of this book forces us to assume that you are familiar with the day-to-day system workings of Unix. If you aren’t, you must learn Unix elsewhere.
For the most part, we illustrate sendmail under BSD Unix and its variants (such as FreeBSD). Where AT&T System V (SysV) differs (such as Sun’s Solaris 2.x and Linux) we illustrate those differences.
Our primary focus throughout this book is on V8.14 sendmail. For completeness, and where necessary, we also discuss V8.13 and earlier (such as BSD’s version 5,[2] IDA, early Sun, Ultrix, and NeXT) but do not cover them in detail in this edition.
The following typographic conventions are used in this book:
Used for names, including pathnames,
filenames, program and command names, usernames,
hostnames, machine names, and mailing-list names,
as well as for mail addresses. It also is used to
indicate that part of a program’s output is not
specific. For example, "error:
number or file" indicates
that the error will be shown either as a number or
as a filename. Italic is also used to emphasize
new terms and concepts when they are
introduced.
Constant
WidthUsed in examples to show the contents of files
or the output from commands. This includes
examples from the configuration file or other
files such as message files, shell scripts, or
C-language program source. Constant-width text is
quoted only when necessary to show enclosed space;
for example, the five-character "From" header.
Single characters, symbolic expressions, and
command-line switches are always shown in
constant-width font. For instance, the o option illustrates a
single character, the rule $- illustrates a
symbolic expression, and -d illustrates a command-line
switch.
Constant
BoldUsed in examples to show commands or some
other text that is to be typed literally by the
user. For example, the phrase cat
/var/run/sendmail.pid means the user
should type “cat /var/run/sendmail.pid” exactly as
it appears in the text or example.
Constant ItalicUsed in examples to show variables for which a
context-specific substitution should be or will be
made. In the string Snum, for
example, num will be a
user-assigned integer.
%Indicates a user shell.
#Indicates a root shell.
This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: "sendmail, by Bryan Costales et al. Copyright 2008 Bryan Costales et al., 978-0-596-51029-9.”
The source for the sendmail program comes with a document written by the sendmail program’s authors that is required reading. Sendmail Installation and Operations Guide (located in doc/op in the source distribution) provides installation instructions and a succinct description of the configuration file. Many vendors also provide online manuals which might reveal vendor-specific customizations not documented in this book. Also, if you have the source, see the RELEASE_NOTES file and all the */README files.
Two topics that are only touched upon in this book are the Domain Name System (DNS) and TCP/IP network communications. At a typical site, a significant number of mail-related problems turn out to be problems with one of these other areas rather than with sendmail.
The DNS is well documented in the book DNS and BIND, Fifth Edition by Paul Albitz and Cricket Liu (O’Reilly).
The protocols used to communicate over the Internet are well documented in the book TCP/IP Network Administration, Third Edition by Craig Hunt (O’Reilly).
Finally, many mail problems can be solved only by the system administrator. The sendmail program runs as root and can be installed and managed only by root. The art of functioning effectively as root is superbly covered in the UNIX System Administration Handbook, Third Edition by Evi Nemeth, Garth Snyder, Scott Seebass, and Trent R. Hein (Prentice Hall).
We have tested and verified the information in this book to the best of our ability, but you might find that features have changed (or even that we have made mistakes!). Please let us know about any errors you find, as well as your suggestions for future editions, by writing to:
| O’Reilly Media, Inc. |
| 1005 Gravenstein Highway North |
| Sebastopol, CA 95472 |
| 800-998-9938 (in the United States or Canada) |
| 707-829-0515 (international or local) |
| 707-829-0104 (fax) |
We have a web page for this book, where we list errata, examples, or any additional information. You can access this page at:
| http://www.oreilly.com/catalog/9780596510299 |
To comment on or ask technical questions about this book, send email to:
| bookquestions@oreilly.com |
For more information about our books, conferences, Resource Centers, and the O’Reilly Network, see our web site at:
| http://www.oreilly.com |
When you see a Safari® Books Online icon on the cover of your favorite technology book, that means the book is available online through the O’Reilly Network Safari Bookshelf.
Safari offers a solution that’s better than e-books. It’s a virtual library that lets you easily search thousands of top tech books, cut and paste code samples, download chapters, and find quick answers when you need the most accurate, current information. Try it for free at http://safari.oreilly.com.
First and foremost, I must thank Greg Shapiro for his careful review of the new chapter about Milters.
Bruce Mah and Sean Brennan were guinea pigs for the first and second editions, respectively. Chris Fedde was guinea pig for the third edition. Scott Palmer bravely functioned as guinea pig for the fourth edition. They set up and ran sendmail based on early drafts and thereby uncovered omissions and mistakes that required correction. Gavin Cameron bravely applied the checkcompat() examples to real-world situations, thereby helping to debug that code for me. Mark D. Roth kindly reviewed the ph database type and provided valuable clarification.
Needless to say, this book would not have been possible if Eric Allman had not written sendmail in the first place.
For the second and fourth editions, Cricket Liu kindly reviewed the DNS chapter and found several errors that slipped by everyone else.
George Jansen,[3] editor extraordinaire, has turned all my early drafts of new text into a form suitable for publication. He has stuck with me through all editions and has never tired.
Thanks and praise must go to Tim O’Reilly for agreeing to do this book in the first place. His experience has shaped this book into its current form. He was aware of the “big picture” throughout and kept his fingers on the pulse of the reader. Without his advice, a book this complex and massive would have been impossible.
Additional thanks must go to Edie Freedman for gracefully accepting my unhappiness with so many cover designs except the current one, which I consider perfect.
The production folks at O’Reilly did a yeoman’s job of achieving an outstanding finished book. For the previous editions a special thank you to Barbara Willette for copyediting, Nancy Kotary for help with final production, Kismet McDonough-Chan for her help in each phase of the production, Chris Reilley for the figures, Mary Anne Weeks Mayo for helping with quality control, Curt Degenhart, Madeleine Newell, and Ellie Fountain Maden for making the edits, Seth Maislin for doing the index, and Danny Marcus for proofreading.
For the third edition, a special thank you to Robert J. Denn for managing the project, Darren Kelly for help with final production, Rob Romano and Jessamyn Read for the figures, Mary Brady, Linley Dolby, Matt Hutchinson, and Claire Cloutier for helping with quality control, Reg Aubry, Julie Hawks, Genevieve d’Entremont, and Judy Hoer for providing production support, Brenda Miller for updating the index from the second edition, and Audrey Doyle for proofreading.
For the fourth edition, thanks to Tatiana Apandi, Audrey Doyle, Colleen Gorman, Mary Brady, John Bickelhaupt, and Marlowe Shaeffer for their work in editorial and production.
Finally, thanks to a list of folks, each of whom helped in small but notable ways: Paul Vixie; Neil Rickert; Keith Johnson; Paul Pomes; Frederick Avolio; John Halleck; John Beck; Brad Knowles; Andrew Chang; Shau-Ping Lo; and the many who sent interesting questions to the sendmail questions mailing list, and all the postings to the comp.mail.sendmail news group.
—Bryan Costales
[1] * With one long gap between 1982 and 1990.
[2] * The versions jump from 5 to 8 because the managers of the BSD 4.4 Unix distribution wanted all software to be released as version 8. Prior to that decision, the new BSD sendmail was designated Version 6. V6 survived only the alpha and beta releases before being bumped to V8.
[3] * Author of The Jesse James Scrapbook and The Fade-away (http://www.georgejansen.com).