You have a Red Hat Linux server and want to install or upgrade the Apache Web server on it using the packages that Red Hat prepares and maintains.

If you are a member of the Red Hat Network (RHN), Red Hat’s subscription service, you can use Red Hat’s up2date tool to maintain your Apache package:

# up2date -ui apache apache-devel apache-manual

If you’re running a more recent version:

# up2date -ui httpd httpd-devel httpd-manual

If you aren’t a member of RHN, you can still download the packages from one of Red Hat’s servers (either ftp://ftp.redhat.com or ftp://updates.redhat.com), and install it with the following command:

# rpm -Uvh apache

The -Uvh option to the rpm command tells it to:

If you use the packages Red Hat maintains for its own platform, you will benefit from a simple and relatively standard installation. However, you can only update versions for which Red Hat has put together an RPM package, which typically means that you may be lagging weeks to months behind the latest stable version.

There is also the issue of platform compatibility; for instance, at some point the version of Apache provided for Red Hat Linux changed from 1.3 to 2.0, and newer versions of the operating system will probably only have the 2.0 packages available. Similarly, if you run an older version of Red Hat Linux, the newer packages will probably not install properly on your system.

It’s a good idea to install the apache-devel package as well. It’s quite small, so it won’t have much impact on your disk usage; however, it includes files and features that a lot of third-party modules will need in order to install properly.

You have a computer running Debian, or one of the Debian-based distributions, such as Ubuntu, and wish to install Apache.

Using apt-get, install the apache2 package:

# apt-get install apache2-mpm-prefork

As with any package-based Linux distribution, it’s usually best to stick with the packages supplied by that distribution in order to have ease of updates, and maximum interoperability with other packages installed on the same system. On Debian, this means using apt-get.

It’s a good idea to install the apache2-dev package as well, as it provides utilities, such as apxs, which will be useful in installing third-party modules, should the need arise.

Debian has its own unique arragement of configuration files, which is unlike that of any other distribution. Both modules and sites (virtual hosts) are arranged in subdirectories so that they can be enabled or disabled at will using utilities that come with Debian’s version of Apache. For example, to enable a particular module, you will use the a2enmod command, which makes the appropriate changes to the server configuration file to cause that module to be loaded. For example:

# a2enmod rewrite

For a full description of where Debian places its files and directories, you should consult http://wiki.apache.org/httpd/DistrosDefaultLayout.

You want to install the Apache Web server software on a Windows platform.

Primarily, Windows is a graphically oriented environment, so the Apache install for Windows is correspondingly graphical in nature.

The simplest way to install Apache is to download and execute the Microsoft Software Installer (MSI) package from the Apache Web site at http://httpd.apache.org/download. The following screenshots come from an actual installation made using this method.

Each step of the installation procedure is distinct in the process and you can revise earlier decisions, until the files are installed. The first screen (Figure 1-1) simply confirms what you’re about to do and the version of the package you’re installing.

Figure 1-1. First screen of Apache MSI install

The second screen (Figure 1-2) presents the Apache license. Its basic tenets boil down to the following: do what you want with the software, don’t use the Apache marks (trademarks like the feather or the name Apache) without permission, and provide proper attribution for anything you build based on Apache software. (This only applies if you plan to distribute your package; if you use it strictly on an internal network, this isn’t required.) You can’t proceed past this screen until you agree to the license terms.

Figure 1-2. License agreement

Figure 1-3 shows the recommended reading for all new users of the Apache software. This describes special actions you should take, such as making configuration changes to close security exposures, so read it closely.

Figure 1-3. Recommended reading for new users

If you are installing Apache for the first time, the installation process asks for some information so that it can make an initial configuration for your server (Figure 1-4). If you already have a version of Apache installed, nothing you enter on this screen will override your existing configuration.

Figure 1-4. Initial server configuration information

The Server Name in the figure is filled with the same value as the Network Domain field; this is a nod to the growing tendency to omit the “www” prefix of Web sites and use the domain name (e.g., http://oreilly.com/ instead of http://www.oreilly.com/ ). What name you specify for the server is just advisory, allowing the installation process to configure some initial values; you can change them later by editing the configuration file. The important thing is that the Server Name value be resolvable into an IP address.

Next comes a screen asking what portions of the package you want to install, as shown in Figure 1-5. Just go with Typical unless you’re an advanced user. The Custom option allows you to choose whether to install the Apache documentation.

Figure 1-5. Installation type

Figure 1-6 asks where you want the software installed. The screenshot shows the default location, which will become the ServerRoot.

Figure 1-6. ServerRoot directory

Once you’ve answered all the questions, a screen similar to Figure 1-7 will come up. This is your last chance to go back and change anything; once you click the Install button on this screen, the installation puts the pieces of the package in place on your system.

Figure 1-7. The last chance to change your mind

Figures 1-8 and 1-9 show the last screens for the Windows MSI install; they show the progress of the installation. When they’re finished, Apache has been installed (and started, if you chose the “Run as a service” option shown in Figure 1-4).

Figure 1-8. The installation progress

Figure 1-9. MSI installation finished

A lot of effort has been put into making the Apache server run well on Windows and be managed like other Windows applications. As a consequence, the primary installation method (InstallShield or MSI) should be familiar to Windows users.

If you’ve never run Apache before, accept the defaults the first time you install it. This makes it easier for others to provide assistance if you need help, because the files will be in predictable locations.

If you chose to start the Apache server as a service (see Figure 1-4), then you can modify the conditions for it to start, such as the user it should run as or whether it should start automatically, just as you would any other service. Figure 1-10 shows one way to do this; bring up the window by right-clicking on the My Computer icon on the desktop and choose Manage from the pop-up menu.

Figure 1-10. Modifying the Apache service

You want to build the Apache Web server yourself from the sources directly (see Recipe 1.4), but don’t know how to obtain them.

There are a number of ways to obtain the sources. You can access the latest version in close to real-time by using Subversion (the tool used by the Apache developers for source control), you can download a release tarball, or you can install a source package prepared by a distributor, among others.

To install from a prepackaged tarball, download the tarball from http://httpd.apache.org/download.cgi, and then:

% tar xzvf httpd-2.0.59.tar.gz

If your version of tar doesn’t support the z option for processing zipped archives, use this command instead:

% gunzip -c < httpd-2.0.59.tar.gz | tar xvf -

From the very latest up-to-the-minute Apache 2.0 source repository (not guaranteed to be completely functional), use:

% svn checkout http://svn.apache.org/repos/asf/httpd/httpd/branches/2.0.x/ httpd-2.0

You can fetch a particular release version instead of the bleeding edge code if you know the name the developers gave it. For example, this will pull the sources of the 2.0.59 release, which is expected to be stable, unlike the up-to-the-minute version:

% svn checkout http://svn.apache.org/repos/asf/httpd/httpd/tags/2.0.59/ httpd-2.0.59

You can find the names of the tags used in the source tree by visiting either http://svn.apache.org/viewvc/httpd/httpd/tags/ or with:

% svn ls http://svn.apache.org/repos/asf/httpd/httpd/tags/

No matter how you install the source, the directory tree will be ready for configuration and building. Once the source is in place, you should be able to move directly to building the package (see Recipe 1.4).

If you chose to install the sources using the Subversion method, you can keep your sources up-to-date by simply executing the following command from the top level of the source directory:

% svn update

This will update or fetch any files that have been changed or added by the developers since the last time you downloaded or updated.

If you update to the latest version of the sources, you’re getting whatever the developers are currently working on, which may be only partially finished. If you want reliability, stick with the released versions, which have been extensively tested.

You want to build your Apache Web server from the sources directly rather than installing it from a prepackaged kit.

Assuming that you already have the Apache source tree—whether you installed it from a tarball, Subversion, or some distribution package, the following commands—executed in the top directory of the tree, builds the server package with most of the standard modules as DSOs:

% ./buildconf
% ./configure --prefix=/usr/local/apache
> --with-layout=Apache --enable-modules=most --enable-mods-shared=all \
> --with-mpm=prefork
% make
#make install

If you want more detailed information about the various options and their meanings, you can use the following command:

% ./configure --help

Building the server from the sources can be complex and time-consuming, but it’s essential if you intend to make any changes to the source code. It gives you much more control over things, such as the use of shareable object libraries and the database routines available to modules. Building from source is also de rigeur if you’re developing your own Apache modules.

If you want to build the modules statically into the server, replace any occurrences of --enable-mods-shared=list with --enable-mods=list.

The options to the configure script are many and varied; if you haven’t used it before to build Apache, locate some online tutorials (such as those at http://apache-server.com/tutorials/ or http://httpd.apache.org/docs-2.0/install.html) when you want to change the defaults. The default options generally produce a working server, although the filesystem locations and module choices may not be what you’d like; they may include modules you don’t want or omit some you do. (See Chapter 2 for some examples.)

You have a complicated collection of modules you want to install correctly.

Download ApacheToolbox from http://www.apachetoolbox.com/. (Note that the version numbers will probably be different than these, which were the latest available when this section was written.) Unpack the file:

% bunzip2 Apachetoolbox-1.5.65.tar.bz2
%tar xvf Apachetoolbox-1.5.65.tar

(Depending on your version of tar, you may be able to combine these operations into a single tar xjvf command.)

Then run the installation script:

# cd Apachetoolbox-1.5.65
#./install.sh

ApacheToolbox is developed and maintained by Bryan Andrews. It is a shell script that assists in the configuration and installation of Apache. It includes support for over 100 commonly used or standard modules.

When you run the script, you select modules from lists appearing on various screens. Once you have decided on your list of modules, ApacheToolbox downloads the third-party modules you have selected and the tools that you don’t have installed, and then runs the Apache configure script with any arguments needed to create the combination you have requested.

The main screen (see Figure 1-11) lists the most popular third-party modules that ApacheToolbox can install. Select or deselect a particular module by typing the number next to that module’s name.

Figure 1-11. Main screen of ApacheToolbox install

Typing apache moves you to the second screen (see Figure 1-12), which lists the standard Apache modules. Add or remove individual modules by typing the number next to their module names.

Figure 1-12. ApacheToolbox screen for standard Apache modules

You can choose options for configuring the modules on additional menus, and you can build an RPM on your installation configuration, which you can then install on multiple machines without requiring that ApacheToolbox be installed.

Once you have made all your module selections, type go to tell ApacheToolbox to start the configuration process.

Your preferences are saved to a file (etc/config.cache) so that if you want to reinstall Apache with the same configuration, you merely need to run ApacheToolbox again, and it will start up with the selections from the last run. To upgrade to a new version of Apache, get the latest version of ApacheToolbox, and ask it to run the installation script with your last selections (without going through the menu process), by typing the following commands:

# ./install.sh --update
#./install.sh --fast

Once ApacheToolbox has completed its work, you can edit the configuration script to insert or modify arguments. Once you are satisfied and ApacheToolbox has run the configuration script, go into the Apache source subdirectory and run make and make install to compile and install Apache:

# cd apache_1.3.27
# make
#make install

You want to be able to start and stop the server at need, using the appropriate tools.

On Unixish systems, use the apachectl script; on Windows, use the options in the Apache folder of the Start menu.

The basic Apache package includes tools to make it easy to control the server. For Unixish systems, this is usually a script called apachectl, but prepackaged distributions may replace or rename it. It can only perform one action at a time, and the action is specified by the argument on the command line. The options of interest are:

For Windows, the MSI installation of Apache includes menu items for controlling the server, as shown in Figure 1-13.

Figure 1-13. Using the Start menu to control Apache

Both of the solutions shown (for Unixish and Windows systems) illustrate the basic server control operations: start, stop, and restart. The purpose of the start and stop functions should be self-evident. Any time you modify the server-wide configuration files (such as httpd.conf), you must restart the server for the changes to take effect.

You have the Apache software installed on your system, and you want to remove it.

On Red Hat Linux, to remove an Apache version installed with the RPM tool, use:

# rpm -ev apache

Other packaging systems may provide some similar mechanism. If they don’t, however, chances are that cleaning out all the files will require a lot of manual work.

On Windows, Apache can typically be removed like any other MSI-installed software (see Figure 1-14).

Figure 1-14. Uninstalling the Apache software

Unfortunately, there is no generic works-for-all removal method for Apache installations on Unixish systems. Some packages, such as Red Hat’s RPM, do remember what they installed so they can remove all the pieces, as shown in the solution. However, if the software was installed by building from the sources (see Recipe 1.4), the burden of knowing where files were put rests with the person who did the build and install. The same applies if the software was installed from source on a Windows system; it’s only the MSI or InstallShield packages that make the appropriate connections to allow the use of the Add/Remove Software control panel.

For a Unixish system, if you have access to the directory in which the server was built, look for the --prefix option in the config.nice file. That will give you a starting point, at least. Here is a list of the directories an Apache 2.0 installation usually puts somewhere on your disks:

You want to know which version of Apache is the right one for you.

Although there is not necessarily one right answer for everyone, the Apache HTTP Server development team works very hard to ensure that every release of the software is the best, most stable, most secure product that they are able to put together, and each release of the product fixes problems that were found in earlier releases. So, it’s always our position that the latest version of the server is the one that you should be running.

As of this writing, that means the latest release of the 2.2.x branch, which, right now, is 2.2.4. When 2.4 is released, we will recommend that you upgrade to 2.4.

This question is not always quite as simple as we would like it to be. We want to give the One Right Answer, but there are sometimes very good reasons for sticking with an older version of the software. However, these reasons are less frequently valid than they were a few years ago.

The most common reason that people give for remaining on the 1.3 version of the server is that they are running mod_something and it’s not available for 2.2 yet. In the early days of Apache 2, this was a valid reason for many people that were sticking with Apache 1.3, and it entered the commonly accepted wisdom that most modules weren’t yet available for Apache 2.

However, as various major Linux distributions started including Apache 2 as the default Web server, more and more modules became available for Apache 2, or people developed alternative modules implementing the same functionality, and this became less and less true.

As of this writing, it seems to be that only a very few commercial modules still satisfy the “not available for Apache 2” category, and this reason is not nearly as believable as it once was.

Another common reason given is that a large installation, with many virtual hosts and complex configuration, is built on Apache 1.3, and it would be an enormous undertaking to migrate it to Apache 2. This is a much more compelling reason. However, it must also be factored in that Apache 1.3 is in maintenance-only mode, and will never get the new features that are being developed for the 2.x branch. Also, perhaps more importantly, the people who provide free online support for Apache are, for the most part, themselves using Apache 2, and their knowledge of Apache 1.3 is waning. So if you have a stable installation, and have no technical difficulties, and are content to slip gradually further and further behind in terms of new functionality, then perhaps staying with 1.3 is a valid solution in that case.

If, however, you are doing a new Web server installation, there is absolutely no good reason not to do with the latest version of the product. You’ll benefit from the experience gained in the 1.3 days, and you’ll get the new features that come with the 2.2 server, as well as the better-implemented old features.

You built your Apache Web server software from the source, and now you want to upgrade it while keeping all the same configuration options.

Unpack the source of the new version into a separate tree, and execute the config.nice script created by your build of the earlier version.

For example, suppose you built and installed version 2.0.17 long ago, and you now want to upgrade your system to 2.0.59:

# cd /usr/local/build
# tar xvf /tmp/httpd-2.0.59.tat.gz
# cd httpd-20.0.59
# ../httpd-2.0.17/config.nice
#make

When you execute the configure script to set up your compilation and installation preferences, it creates a file called config.script with all the options you chose. The file config.nice executes configure with all those options. This means you don’t need to remember or write down all the options you specified when you finally got it working.

In addition, config.nice allows you to specify additional options, which it adds to those with which it invokes configure. When configure runs, it will create config.nice again with the complete new set of options.

You want your Apache Web server to start automatically when your system boots up.

On a Windows system, if you installed Apache as a service, you can configure it to start automatically just as you would any other service. Go to the Services control panel and make the desired changes there.

On Unixish systems, how you set this up differs by platform. For Red Hat-based systems:

# cp path/to/apachectl /etc/rc.d/init.d/httpd
# vi /etc/rc.d/init.d/httpd   # add '# chkconfig 3 92 10'
# chkconfig --add httpd
#chkconfig --levels 35 httpd on

This will cause Apache to be started up (and shut down) as part of the normal sequence for runlevels 3 and 5.

The solution provided is specific to Red Hat-based platforms such as Fedora Core or RHEL. For other platforms or distributions you may instead need to edit /etc/rc.local, or copy the apachectl script into /etc/rc3.d, or something similar. Consult your operating system’s documentation for specifics.

The configure script, that is used to set up a build from source, has many options, and it’s not clear which ones are really important.

Here are some of the most important and useful options that you you might want to use:

Minimal documentation for all of the configure options is available from the script itself:

          % 
          ./configure --help

However, for more detailed understanding you need to consult the Apache documentation itself—or look at the source code.

You’ve installed the Apache Web server, whether from source or an installation kit, but you’re not sure where all the files have been put. (This is useful to know if you want to uninstall it later.)

If you installed the software from a source kit, look at the config.layout file in the top level of the source directory. Look for a <Layout> stanza that matches the --enable-layout option given to the configure script. (If none was supplied, the Apache layout will have been used.)

If you installed the software from an RPM package, use the -ql option to see where the files have been installed:

          % 
          rpm -ql httpd

If you installed from a kit prepared by a distributor, such as Ubuntu, check with the distribution documentation to find out where the files are stored.

One of the advantages—and disadvantages—of open software is that everyone can build an installation kit. And everyone pretty much chooses options different from everyone else.

The Apache source package includes a list of “common” layouts, and most installation kits use one or another of these.

You have downloaded a third-party module that isn’t listed in this chapter, and you want to install it.

Move to the directory where the module’s source file was unpacked, and then:

% /path/to/apache/bin/apxs -cia module.c

In the case of a third-party module that consists of a single .c file, there is a good chance that it can be built and installed using the Solution. Modules that involve multiple source files should provide their own installation instructions.

The -cia options mean to compile, install, and activate. The first is pretty straightforward; install means put the .so file in the place Apache expects to find it; and activate means to add the module to the httpd.conf file.

You want to add or enable WebDAV capabilities to your server. WebDAV permits specific documents to be reliably and securely manipulated by remote users without the need for FTP, to perform such tasks as adding, deleting, or updating files.

If you’re using Apache 2.0 or later, mod_dav is automatically available, although you may need to enable it at compile time with --enable-dav.

If you are using Apache 1.3, download and unpack the mod_dav source package from http://webdav.org/mod_dav/, and then:

% cd mod_dav-1.0.3-1.3.6
% ./configure --with-apxs=/usr/local/apache/bin/apxs
% make
#make install

Restart the server, and be sure to read Recipe 6.18.

The mod_dav source package is an encapsulated and well-behaved module that is easily built and added to an existing server. To test that it has been properly installed, you need to enable some location on the server for WebDAV management and verify access to that location with some WebDAV-capable tool. We recommend cadaver, which is an open source command-line WebDAV tool. (The URL for the cadaver tool is found at the end of this recipe.)

To enable your server for WebDAV operations, you need to add at least two directives to your httpd.conf file. The first identifies the location of the locking database used by mod_dav to keep WebDAV operations from interfering with each other; it needs to be in a directory that is writable by the server. For example:

# cd /usr/local/apache
# mkdir var
# chgrp nobody var
#chmod g+w var

Now add the following line to your httpd.conf file, outside any containers:

<IfModule mod_dav.c>
    DAVLockDB var/DAVlock
</IfModule>

Next, create a temporary directory for testing WebDAV functionality:

# cd /usr/local/apache
# mkdir htdocs/dav-test
# chgrp nobody htdocs/dav-test
#chmod g+w htdocs/dav-test

Add a stanza to your httpd.conf file that will enable this directory for WebDAV operations:

<Directory "/usr/local/apache/htdocs/dav-test">
    DAV On
</Directory>

Now restart your server. It should be ready to handle WebDAV operations directed to the /dav-test local URI. To test it with the cadaver tool, try the following commands; your output should look very similar to that shown:

% cd /tmp
% echo "Plain text" > dav-test.txt
% cadaver
dav:!> open http://localhost/dav-test
Looking up hostname... Connecting to server... connected.
dav:/dav-test/> put dav-test.txt
Uploading dav-test.txt to '/dav-test/dav-test.txt': (reconnecting...done)
Progress: [=  ==  ==  ==  ==  ==  ==  ==  ==  ==  ==  ==  ==  ==  ==>] 100.0% of 11 
    bytes succeeded.
dav:/dav-test/> propset dav-test.txt MyProp 1023
Setting property on 'dav-test.txt': (reconnecting...done) succeeded.
dav:/dav-test/> propget dav-test.txt MyProp
Fetching properties for 'dav-test.txt':
Value of MyProp is: 1023
dav:/dav-test/> propdel dav-test.txt MyProp
Deleting property on 'dav-test.txt': succeeded.
dav:/dav-test/> close
Connection to 'localhost' closed.
dav:!> exit
%rm dav-test.txt

Properties are attributes of a WebDAV resource. Some are managed by the system, such as the resource’s size, but others can be arbitrary and added, changed, and removed by the user.

Once you have verified that mod_dav is working correctly, remove the htdocs/dav-test directory, and the corresponding <Directory> stanza in your httpd.conf file, and follow the guidelines in Recipe 6.18.

You want to enable WebDAV capabilities on your existing Apache 1.3 server with mod_dav.

Apache 2.0 includes mod_dav as a standard module, so you do not need to download and build it.

Download and unpack the mod_dav Windows package from http://webdav.org/mod_dav/win32. Verify that your Apache installation already has the xmlparse.dll and xmltok.dll files in the ServerRoot directory; if they aren’t there, check through the Apache directories to locate and copy them to the ServerRoot. mod_dav that requires the Expat package, which is included with versions of the Apache Web server after 1.3.9; these files hook into Expat, which mod_dav will use.

Put the mod_dav DLL file into the directory where Apache keeps its modules:

C:\>cd mod_dav-1.0.3-dev
C:\mod_dav-1.0.3-dev>copy mod_dav.dll C:\Apache\modules
C:\mod_dav-1.0.3-dev>cd \Apache

Add the following line to your httpd.conf file:

LoadModule dav_module modules/mod_dav.dll

You may also need to add an AddModule line if your httpd.conf file includes a ClearModuleList directive and re-adds the other modules. Alternatively, you can insert the LoadModule for mod_dav after the ClearModuleList directive.

The mod_dav Package is an encapsulated and well-behaved module that is easily built and added to an existing server. To test that it has been properly installed, you need to enable some location on the server for WebDAV management and verify access to that location with some WebDAV-capable tool, or browse to it in Windows Explorer, which knows how to access WebDAV locations (as of Windows 2000), or access it from a different system where cadaver or another WebDAV tool is available.

To enable your server for WebDAV operations, you need to add at least two directives to your ServerRoot/conf/httpd.conf file. The first identifies the location of the locking database used by mod_dav to keep WebDAV operations from interfering with each other; it needs to be in a directory that is writable by the server. For example:

C:\Apache-1.3>mkdir var

Now add the following lines to your httpd.conf file to enable WebDAV:

<IfModule mod_dav.c>
    DAVLockDB "C:/Apache-1.3/var/dav-lock"
</IfModule>

Create a temporary directory for testing mod_dav’s ability to function:

C:\Apache-1.3>mkdir htdocs\dav-test

Modify the <IfModule> container to enable WebDAV operations for this test directory:

<IfModule mod_dav.c>
    DAVLockDB "C:/Apache-1.3/var/dav-lock"
    <Directory "C:/Apache-1.3/htdocs/dav-test">
        DAV On
    </Directory>
</IfModule>

Now restart your server and try accessing the /dav-test location with a WebDAV client. If you’re using cadaver from another system, see Recipe 2.2 for detailed instructions. If you want to use Windows Explorer to test mod_dav, read the following section.

You want to install the mod_perl scripting module to allow better Perl script performance and easy integration with the Web server.

For Apache 1.3, download and unpack the mod_perl 1.0 source package from http://perl.apache.org/. Then use the following command:

% perl Makefile.PL \
>     USE_APXS=1 \
>     WITH_APXS=/usr/local/apache/bin/apxs \
>     EVERYTHING=1 \
>     PERL_USELARGEFILES=0
% make
%make install

Restart your server.

For Apache 2.0 and later, the process is similar. Download and unpack the mod_perl 2.0 source package, then use the following command:

% perl Makefile.PL MP_APXS=/usr/local/apach2/bin/apxs 

The mod_perl source package is quite a complex module, and there are several different ways to add it to your server. This recipe is the fastest and lowest-impact one; if it doesn’t suit your needs, check the various README.* files in the package directory after unpacking. Because its primary language is Perl rather than C, the installation instructions are significantly different from those for most other modules.

Once you have restarted your server successfully, mod_perl should be available and configured as part of it. You can test it by making some changes to the httpd.conf file, adding a few scripts, and seeing whether the server processes them correctly. Here is a sample set of steps to test mod_perl’s operation:

If your configuration is valid, the response should be a page containing simply the words, “Hello, world! Love, mod_perl.”

You want to add the mod_php scripting module to your existing Apache Web server.

Download the mod_php package source from the Web site at http://php.net (follow the links for downloading) and unpack it. Then:

% cd php-5.2.3
% ./configure \
>     --with-apxs2=/usr/local/apache/bin/apxs
% make
#make install

Restart the server.

To test that your installation was successful, create a file named info.php in your server’s DocumentRoot; the file should contain the single line:

<?php phpinfo(); ?>

Add the following lines to your server’s httpd.conf file:

<IfModule mod_php4.c>
    AddHandler application/x-httpd-php .php
</IfModule>

After restarting your server, try fetching the document info.php using a browser. You should see a detailed description of the PHP options that are active. If you do, indicating a successful installation, remove the info.php file.

There are numerous additional options and extensions available for PHP; the recipe given here is only for the most basic installation.

You want to add the mod_php scripting module to your existing Apache server on Windows.

This recipe needs to be described largely in terms of actions rather than explicit commands to be issued.

The PHP module installation on Windows requires a lot of nitpicky manual steps. To test that your installation was successful, create a file named info.php in your server’s DocumentRoot; the file should contain the single line:

<?php phpinfo(); ?>

After restarting your server, try fetching the document info.php from it using a browser. You should see a detailed description of the PHP options that are active.

There are numerous additional options and extensions available for PHP; the recipe given here is only the most basic installation. See the install.txt file in the PHP4 directory and the documentation on the Web site for more details.

You want to add SSL support to your Apache server with the mod_ssl secure HTTP module.

The mod_ssl package requires source-level changes to the base Apache code, and so the version of the mod_ssl package you install must match the version of the Apache distribution you have. If your Apache installation doesn’t include the source, such as if you installed a binary-only RPM or other vendor distribution, you won’t be able to add mod_ssl to it.

In addition to the Apache source, mod_ssl requires that you have Perl and the OpenSSL libraries installed. The --with-ssl option on the build configuration statement indicates where this is located; if it is in a vendor-distributed directory, the special keyword SYSTEM tells the build to look for it, and you don’t have to find it yourself.

Unlike most other Apache modules, when adding mod_ssl you run the ./configure script that’s in mod_ssl’s directory, rather than the one in the Apache source directory; the module’s script makes changes to Apache’s and then invokes it directly.

This recipe is the bare basics; there are many optional components and features that mod_ssl allows you to specify at configuration time. For more information, consult the README and INSTALL files in the mod_ssl source directory, or the mod_ssl Web site at http://www.modssl.org.

You’re looking for Apache modules with a particular functionality, or by name, and you’ve heard about the Apache Module Registry.

Visit http://modules.apache.org and search for keywords related to the functionality you want, or portions of the module name.

The Apache modules registry is an unofficial site at which module authors can voluntarily register their work for easy location.

You want to install the mod_security module to take advantage of its simple and powerful filtering mechanisms.

The Makefile included with the mod_security package will do the building of the module and put it in the right place, but activating it in your server is your responsibility. Recent versions of the package include a set of core rules for handling things like blog spam and common attacks, and the rules are also available as a separate tarball, which may or may not be updated more frequently than the ones bundled with the software.

The current version of mod_security only supports version 2 of the Apache Web server. There is an older version that supports the 1.3 versions, but it is unlikely to be maintained for long.

You are trying to install a third-party module, but the Apache Web server refuses to recognize it.

Consult the sources for the module, or its documentation, or ask the author, in order to determine which version of Apache the package supports.

As significant changes are made to the Apache Web server, sometimes compatibility suffers as the API is changed. Although efforts are made to keep this sort of thing to a minimum, sometimes it is unavoidable.

To keep an incompatible module from being loaded and crashing the Web server when used, both modules and the server have a built-in “magic” number that is recorded when they’re built, and that relates to the version of the API. When the server tries to load a module DSO, it compares the module’s magic number with the server’s own, and if they aren’t compatible, the server refuses to load it.

The development team tries to keep the magic number compatibility within major version numbers, but not across them. That is, a module built for Apache 1.3 should work with almost any 1.3 version of the server built after the module was, but it definitely won’t work with a 2.0 server. Contrariwise, a 2.0 module won’t work with a 1.3 server under any circumstances.

You want to add a little more detail to your access log entries.

Use the combined log format, rather than the common log format:

CustomLog logs/access_log combined

The default Apache logfile enables logging with the common log format, but it also provides the combined log format as a predefined LogFormat directive.

The combined log format offers two additional pieces of information not included in the common log format: the Referer (where the client linked from) and the User-agent (what browser they are using).

Every major logfile parsing software package is able to handle the combined format as well as the common format, and many of them give additional statistics based on these added fields. So you lose nothing by using this format and potentially gain some additional information.

You want more information in the error log in order to debug a problem.

Change (or add) the LogLevel line in your httpd.conf file. There are several possible arguments, which are enumerated here.

For example:

LogLevel Debug

There are several hierarchical levels of error logging available, each identified by its own keyword. The default value of LogLevel is warn. Listed in descending order of importance, the possible values are:

emerg results in the least information being recorded and debug in the most. However, at debug level a lot of information will probably be recorded that is unrelated to the issue you’re investigating, so it’s a good idea to revert to the previous setting when the problem is solved.

Even though the various logging levels are hierarchical in nature, one oddity is that notice level messages are always logged regardless of the setting of the LogLevel directive.

The severity levels are rather loosely defined and even more loosely applied. In other words, the severity at which a particular error condition gets logged is decided at the discretion of the developer who wrote the code—your opinion may differ.

Here are some sample messages of various severities:

[Thu Apr 18 01:37:40 2002] [alert] [client 64.152.75.26] /home/smith/public_html/
     test/.htaccess: Invalid command 'Test', perhaps mis-spelled or defined by a
     module not included in the server configuration
[Thu Apr 25 22:21:58 2002] [error] PHP Fatal error:  Call to undefined function:
     decode_url(  ) in /usr/apache/htdocs/foo.php on line 8
[Mon Apr 15 09:31:37 2002] [warn] pid file /usr/apache/logs/httpd.pid overwritten --
     Unclean shutdown of previous Apache run?
[Mon Apr 15 09:31:38 2002] [info] Server built: Apr 12 2002 09:14:06
[Mon Apr 15 09:31:38 2002] [notice] Accept mutex: sysvsem (Default: sysvsem)

These are fairly normal messages that you might encounter on a production Web server. If you set the logging level to Debug, however, you might see many more messages of cryptic import, such as:

[Thu Mar 28 10:29:50 2002] [debug] proxy_cache.c(992): No CacheRoot, so no caching.
     Declining.
[Thu Mar 28 10:29:50 2002] [debug] proxy_http.c(540): Content-Type: text/html

These are exactly what they seem to be: debugging messages intended to help an Apache developer figure out what the proxy module is doing.

At the time of this writing, there is an effort underway to provide a dictionary of Apache error messages, what they mean, and what to do about the conditions they report, but it doesn’t have anything concrete to show at this point. When it does, it should be announced at the Apache server developer site:

It will be mentioned on this book’s companion Web site, as well:

In addition, see the detailed documentation of the LogLevel directive at the Apache site:

You want to record data submitted with the POST method, such as from a web form.

Ensure that mod_dumpio is installed and enabled, and put the following in your configuration file:

# DumpIOLogLevel notice - 2.3.x and later
LogLevel debug
DumpIOInput On

Or, with mod_security:

SecAuditLogType Concurrent
SecAuditLogStorageDir /var/www/audit_log/data/
SecAuditLog /var/www/audit_log/index
SecAuditLogParts ABCFHZ

mod_dumpio is a new module in Apache 2.0 (that is to say, it’s not available for Apache 1.3) that allows the complete input and output of each HTTP transaction to be logged. In the example above, we’re enabling input logging only, using the DumpIOInput directive.

On Apache 2.0 and 2.2, LogLevel needs to be set to debug in order for these records to be logged. In 2.3 and later, there’s a new directive DumpIOLogLevel that allows you to set the LogLevel at which the entries will be logged. For example, if you set DumpIOLogLevel to notice, then these entries will be logged when LogLevel is set to notice or higher.

Log entries for POST data will look like:

[Sun Feb 11 16:49:27 2007] [debug] mod_dumpio.c(51): mod_dumpio:dumpio_in (data-HEAP): 11 bytes
[Sun Feb 11 16:49:27 2007] [debug] mod_dumpio.c(67): mod_dumpio: dumpio_in (data-HEAP): foo=example

In the log entry shown here, the form value foo was set to example.

The output from mod_dumpio is very noisy. A typical request may generate somewhere between 30 and 50 lines of log entries. The entry shown here is just a tiny part of what was logged with the POST.

mod_security also permits the logging of request data. In the mod_security configuration shown in the recipe above, a logfile is created containing all available request headers, and the request body itself.

You want to log the IP address of the actual client requesting your pages, even if they’re being requested through a proxy.

None.

Unfortunately, the HTTP protocol itself prevents this from being possible. From the client side, proxies are intended to be completely transparent; from the side of the origin server, where the content actually resides, they are meant to be almost utterly opaque, concealing the identity of a request.

Your best option is to log the IP address from which the request came. If it came directly from a browser, it will be the client’s address; if it came through one or more proxy servers, it will be the address of the one that actually contacts your server.

Both the combined and common log formats include the %h format effector, which represents the (remote) client’s identity. However, this may be a hostname rather than an address, depending on the setting of your HostNameLookups directive, among other things. If you always want the client’s IP address to be included in your logfile, use the %a effector instead.

You want to record the MAC (hardware) address of clients that access your server.

This cannot be logged reliably in most network situations and not by Apache at all.

The MAC address is not meaningful except on local area networks (LANs) and is not available in wide area network transactions. When a network packet goes through a router, such as when leaving a LAN, the router will typically rewrite the MAC address field with the router’s hardware address.

You want to record all the cookies sent to your server by clients and all the cookies your server asks clients to set in their databases; this can be useful when debugging Web applications that use cookies.

To log cookies received from the client:

CustomLog logs/cookies_in.log "%{UNIQUE_ID}e %{Cookie}i"
CustomLog logs/cookies2_in.log "%{UNIQUE_ID}e %{Cookie2}i"

To log cookie values set and sent by the server to the client:

CustomLog logs/cookies_out.log "%{UNIQUE_ID}e %{Set-Cookie}o"
CustomLog logs/cookies2_out.log "%{UNIQUE_ID}e %{Set-Cookie2}o"

In versions before to 2.0.56, using the %{Set-Cookie}o format effector for debugging is not recommended if multiple cookies are (or may be) involved. Only the first one will be recorded in the logfile. See the Discussion text for an example.

Cookie fields tend to be very long and complex, so the previous statements will create separate files for logging them. The cookie log entries can be correlated against the client request access log using the server-set UNIQUE_ID environment variable (assuming that mod_unique_id is active in the server and that the activity log format includes the environment variable with a %{UNIQUE_ID}e format effector).

At the time of this writing, the Cookie and Set-Cookie header fields are most commonly used. The Cookie2 and corresponding Set-Cookie2 fields are newer and have been designed to correct some of the shortcomings in the original specifications, but they haven’t yet achieved much penetration.

Because of the manner in which the syntax of the cookie header fields has changed over time, these logging instructions may or may not capture the complete details of the cookies.

Bear in mind that these logging directives will record all cookies, and not just the ones in which you may be particularly interested. For example, here is the log entry for a client request that included two cookies, one named RFC2109-1 and one named RFC2109-2:

PNCSUsCoF2UAACI3CZs RFC2109-1="This is an old-style cookie, with space characters
     embedded"; RFC2109-2=This_is_a_normal_old-style_cookie

Even though there’s only one log entry, it contains information about two cookies.

On the cookie-setting side, here are the Set-Cookie header fields sent by the server in its response header:

Set-Cookie: RFC2109-1="This is an old-style cookie, with space characters embedded";
     Version=1; Path=/; Max-Age=60; Comment="RFC2109 demonstration cookie"
Set-Cookie: RFC2109-2=This_is_a_normal_old-style_cookie; Version=1; Path=/; Max-
     Age=60; Comment="RFC2109 demonstration cookie"

And here’s the corresponding log entry for the response (this was all one line in the logfile, so line wrapping was added to make it all fit on the page):

eCF1vsCoF2UAAHB1DMIAAAAA RFC2109-1=\"This is an old-style cookie, with space 
    characters embedded\"; Version=1; Path=/; Max-Age=60; Comment=\"RFC2109 
    demonstration cookie\", RFC2109-2=This_is_a_normal_old-style_cookie; 
    Version=1; Path=/; Max-Age=60; Comment=\"RFC2109 demonstration cookie\"

You want to log requests for images on your site, except when they’re requests from one of your own pages. You might want to do this to keep your logfile size down, or possibly to track down sites that are hijacking your artwork and using it to adorn their pages.

Use SetEnvIfNoCase to restrict logging to only those requests from outside of your site:

<FilesMatch \.(jpg|gif|png)$>
    SetEnvIfNoCase Referer "^http://www.example.com/" local_referrer=1
</FilesMatch>
CustomLog logs/access_log combined env=!local_referrer

In many cases, documents on a Web server include references to images also kept on the server, but the only item of real interest for log analysis is the referencing page itself. How can you keep the server from logging all the requests for the images that happen when such a local page is accessed?

The SetEnvIfNoCase will set an environment variable if the page that linked to the image is from the www.example.com site (obviously, you should replace that site name with your own) and the request is for a GIF, PNG, or JPEG image.

The CustomLog directive will log all requests that do not have that environment variable set, i.e., everything except requests for images that come from links on your own pages.

This recipe only works for clients that actually report the referring page. Some people regard the URL of the referring page to be no business of anyone but themselves, and some clients permit the user to select whether to include this information or not. There are also “anonymizing” sites on the Internet that act as proxies and conceal this information.

You want to automatically roll over the Apache logs at specific times without having to shut down and restart the server.

Use CustomLog and the rotatelogs program:

CustomLog "| /path/to/rotatelogs /path/to/logs/access_log.%Y-%m-%d 86400" combined

The rotatelogs script is designed to use an Apache feature called piped logging, which is just a fancy name for sending log output to another program rather than to a file. By inserting the rotatelogs script between the Web server and the actual logfiles on disk, you can avoid having to restart the server to create new files; the script automatically opens a new file at the designated time and starts writing to it.

The first argument to the rotatelogs script is the base name of the file to which records should be logged. If it contains one or more % characters, it will be treated as a strftime(3) format string; otherwise, the rollover time (in seconds since 1 January 1970), in the form of a 10-digit number, will be appended to the base name. For example, a base name of foo would result in logfile names like foo.1020297600, whereas a base name of foo.%Y-%m-%d would cause the logfiles to be named something like foo.2002-04-29.

The second argument is the interval (in seconds) between rollovers. Rollovers will occur whenever the system time is a multiple of this value. For instance, a 24-hour day contains 86,400 seconds; if you specify a rollover interval of 86400, a new logfile will be created every night at midnight—when the system time, which is based at representing midnight on 1 January 1970, is a multiple of 24 hours.

You want to close the previous month’s logs and open new ones on the first of each month.

The Apache distribution doesn’t come with a script that does this, but there is a free program that provides this and many other useful features. It is called Cronolog, and may be obtained from http://cronolog.org.

Obtain and install Cronolog, and then place the following in your configuration file:

CustomLog "|/usr/bin/cronolog /www/logs/access%Y%m.log" combined

Cronolog has been around for a long time, and provides many of the features that people wished were available in the standard rotatelogs utility. Over the years, rotatelogs has improved, but Cronolog has a number of other useful features that are of interest to sites with rapidly growing logfiles.

One of these is the ability to automatically rotate logfiles by day, week, month, or year, based on the format of the filename specified in the CustomLog directive.

In the example given, the logfile is rotated at the start of a new month, because the logfile name given contains only the year and month variables (%Y and %m, respectively).

You want to see hostnames in your activity log instead of IP addresses.

You can let the Web server resolve the hostname when it processes the request by enabling runtime lookups with the Apache directive:

HostnameLookups On

Or you can let Apache use the IP address during normal processing and let a piped logging process resolve them as part of recording the entry:

HostnameLookups Off
CustomLog "| /path/to/logresolve -c >> /path/to/logs/access_log.resolved" combined

Or you can let Apache use and log the IP addresses, and resolve them later when analyzing the logfile. Add this to http.conf:

CustomLog /path/to/logs/access_log.raw combined

And analyze the log with:

% /path/to/logresolve -c < access_log.raw > access_log.resolved

The Apache activity logging mechanism can record either the client’s IP address or its hostname (or both). Logging the hostname directly requires that the server spend some time to perform a DNS lookup to turn the IP address (which it already has) into a hostname. This can have some serious impact on the server’s performance, however, because it needs to consult the name service in order to turn the address into a name; and while a server child or thread is busy waiting for that, it isn’t handling client requests. One alternative is to have the server record only the client’s IP address and resolve the address to a name during logfile postprocessing and analysis. At the very least, defer it to a separate process that won’t directly tie up the Web server with the resolution overhead.

In theory, this is an excellent choice; in practice, however, there are some pitfalls. For one thing, the logresolve application included with Apache (usually installed in the bin/ subdirectory under the ServerRoot) will only resolve IP addresses that appear at the very beginning of the log entry, and so it’s not very flexible if you want to use a nonstandard format for your logfile. For another, if too much time passes between the collection and resolution of the IP addresses, the DNS may have changed sufficiently so that misleading or incorrect results may be obtained. This is especially a problem with dynamically allocated IP addresses such as those issued by ISPs. Although, for these dynamically allocated IP addresses, the hostnames tend not to be particularly informative anyway.

An additional shortcoming becomes apparent if you feed your log records directly to logresolve through a pipe: as of Apache 1.3.24 at least, logresolve doesn’t flush its output buffers immediately, so there’s the possibility of lost data if the logging process or the system should crash.

In practice, however, all log analysis software provides hostname resolution functionality, and it generally makes most sense to use that functionality than trying to resolve the IP addresses in the logfile before that stage.

You want to have separate activity logs for each of your virtual hosts, but you don’t want to have all the open files that multiple CustomLog directives would use.

Use the split-logfile program that comes with Apache. To split logfiles after they’ve been rolled over (replace /path/to/ServerRoot with the correct path):

# cd /path/to/ServerRoot
# mv logs/access_log logs/access_log.old
# bin/apachectl graceful
[wait for old logfile to be completely closed]
# cd logs
#../bin/split-logfile < access_log.old

To split records to the appropriate files as they’re written, add this line to your httpd.conf file:

CustomLog "| /path/to/split-logfile /usr/local/Apache/logs" combined

In order for split-logfile to work, the logging format you’re using must begin with “%v” (note the blank after the v). This inserts the name of the virtual host at the beginning of each log entry; split-logfile will use this to figure out to which file the entry should be written. The hostname will be removed from the record before it gets written.

There are two ways to split your access logfile: after it’s been written, closed, and rolled over, or as the entries are actually being recorded. To split a closed logfile, just feed it into the split-logfile script. To split the entries into separate files as they’re actually being written, modify your configuration to pipe the log messages directly to the script.

Each method has advantages and disadvantages. The rollover method requires twice as much disk space (for the unsplit log plus the split ones) and that you verify that the logfile is completely closed. (Unfortunately there is no guaranteed, simple way of doing this without actually shutting down the server or doing a graceless restart; it’s entirely possible that a slow connection may keep the old logfile open for a considerable amount of time after a graceful restart.) Splitting as the entries are recorded is sensitive to the logging process dying—although Apache will automatically restart it, log messages waiting for it can pile up and constipate the server.

You want to log requests that go through your proxy to a different file than the requests coming directly to your server.

Use the SetEnv directive to earmark those requests that came through the proxy server, in order to trigger conditional logging:

<Directory proxy:*>
    SetEnv is_proxied 1
</Directory>
CustomLog logs/proxy_log combined env=is_proxied

Or, for 2.x, use a <Proxy> block:

<Proxy *>
    SetEnv is_proxied 1
    </Proxy>
CustomLog logs/proxy_log combined env=is_proxied

Apache 1.3 has a special syntax for the <Directory> directive, which applies specifically to requests passing through the proxy module. Although the * makes it appear that wildcards can be used to match documents, it’s misleading; it isn’t really a wildcard. You may either match explicit paths, such as proxy:http://example.com/foo.html, or use * to match everything. You cannot do something like proxy:http://example.com/*.html.

If you want to apply different directives to different proxied paths, you need to take advantage of another module. Because you’re dealing with requests that are passing through your server rather than being handled by it directly (i.e., your server is a proxy rather than an origin server), you can’t use <Files> or <FilesMatch> containers to apply directives to particular proxied documents. Nor can you use <Location> or <LocationMatch> stanzas because they can’t appear inside a <Directory> container. You can, however, use mod_rewrite’s capabilities to make decisions based on the path of the requested document. For instance, you can log proxied requests for images in a separate file with something like this:

<Directory proxy:*>
    RewriteEngine On
    RewriteRule "\.(gif|png|jpg)$" "-" [ENV=proxied_image:1]
    RewriteCond "%{ENV:proxied_image}" "!1"
    RewriteRule "^" "-" [ENV=proxied_other:1]
</Directory>
CustomLog logs/proxy_image_log combined env=proxied_image
CustomLog logs/proxy_other_log combined env=proxied_other

Directives in the <Directory proxy:*> container will only apply to requests going through your server. The first RewriteRule directive sets an environment variable if the requested document ends in .gif, .png, or .jpg. The RewriteCond directive tests to see if that envariable isn’t set, and the following RewriteRule will set a different envariable if so. The two CustomLog directives send the different types of requests to different logfiles according to the environment variables.

Unlike access logs, Apache only logs errors to a single location. You want Apache to log errors that refer to a particular virtual host to the host’s error log, as well as to the global error log.

There are at least two possible ways of doing this:

Unlike activity logs, Apache will log error messages only to a single location. If the error is related to a particular virtual host and this host’s <VirtualHost> container includes an ErrorLog entry, the error will be logged only in this file, and it won’t appear in any global error log. If the <VirtualHost> does not specify an ErrorLog directive, the error will be logged only to the global error log. (The global error log is the last ErrorLog directive encountered that isn’t in a <VirtualHost> container.)

Currently, the only workaround to this is to have the necessary duplication performed by a separate process (i.e., by using piped logging to send the error messages to the process as they occur). Of the two solutions given earlier, the first, which involves a custom script you develop yourself, has the most flexibility. If all you want is simply duplication of entries, the second solution is simpler but requires that your platform have a tee program (Windows does not). It also may be subject to lagging messages if your tee program doesn’t flush its buffers after each record it receives. This could also lead to lost messages if the pipe breaks or the system crashes.

An alternate approach may be to send the error log to syslog, and then have your syslog server log entries to multiple places.

You want to log the IP address of the server that responds to a request, possibly because you have virtual hosts with multiple addresses each.

Use the %A format effector in a LogFormat or CustomLog directive:

CustomLog logs/served-by.log "%A"

The %A effector signals the activity logging system to insert the local IP address—that is, the address of the server—into the log record at the specified point. This can be useful when your server handles multiple IP addresses. For example, you might have a configuration that includes elements such as the following:

Listen 10.0.0.42
Listen 192.168.19.243
Listen 263.41.0.80
<VirtualHost 192.168.19.243>
    ServerName Private.Example.Com
</VirtualHost>
<VirtualHost 10.0.0.42 263.41.0.80>
    ServerName Foo.Example.Com
    ServerAlias Bar.Example.Com
</VirtualHost>

This might be meaningful if you want internal users to access Foo.Example.Com using the 10.0.0.42 address rather than the one published to the rest of the network (such as to segregate internal from external traffic over the network cards). The second virtual host is going to receive requests aimed at both addresses even though it has only one ServerName; using the %A effector in your log format can help you determine how many hits on the site are coming in over each network interface.

You want to record the URL of pages that refer clients to yours, perhaps to find out how people are reaching your site.

Add the following effector to your activity log format:

%{Referer}i

One of the fields that a request header may include is called the Referer. Referer is the URL of the page that linked to the current request. For example, if file a.html contains a link such as:

<a href="b.html">another page</a>

When the link is followed, the request header for b.html will contain a Referer field that has the URL of a.html as its value.

The Referer field is not required nor reliable; some users prefer software or anonymizing tools that ensure that you can’t tell where they’ve been. However, this is usually a fairly small number and may be disregarded for most Web sites.

You want to know the software visitors use to access your site, for example, so you can optimize its appearance for the browser that most of your audience uses.

Add the following effector to your activity log format:

%{User-Agent}i

Request headers often include a field called the User-agent. This is defined as the name and version of the client software being used to make the request. For instance, a User-agent field value might look like this:

User-Agent: Mozilla/4.77 [en] (X11; U; Linux 2.4.4-4GB i686)

This tells you that the client is claiming to be Netscape Navigator 4.77, run on a Linux system and using X-windows as its GUI.

The User-agent field is neither required nor reliable; many users prefer software or anonymizing tools that ensure that you can’t tell what they’re using. Some software even lies about itself so it can work around sites that cater specifically to one browser or another; users have this peculiar habit of thinking it’s none of the Webmaster’s business which browser they prefer. It’s a good idea to design your site to be as browser-agnostic as possible for this reason, among others. If you’re going to make decisions based on the value of the field, you might as well believe it hasn’t been faked—because there’s no way to tell if it has.

You want to record the values of arbitrary fields clients send to their request header, perhaps to tune the types of content you have available to the needs of your visitors.

Use the %{...}i log format variable in your access log format declaration. For example, to log the Host header, you might use:

%{Host}i

The HTTP request sent by a Web browser can be very complex, and if the client is a specialized application rather than a browser, it may insert additional metadata that’s meaningful to the server. For instance, one useful request header field is the Accept field, which tells the server what kinds of content the client is capable of and willing to receive. Given a CustomLog line such as this:

CustomLog logs/accept_log "\"%{Accept}i\""

a resulting log entry might look like this:

PNb6VsCoF2UAAH1dAUo "text/html, image/png, image/jpeg, image/gif, 
     image/x-xbitmap, */*"

This tells you that the client that made that request is explicitly ready to handle HTML pages and certain types of images, but, in a pinch, will take whatever the server gives it (indicated by the wildcard */* entry).

You want to record the values of arbitrary fields the server has included in a response header, probably to debug a script or application.

Use the %{...}o log format variable in your access log format declaration. For example, to log the Last-Modified header, you would do the following:

%{Last-Modified}o

The HTTP response sent by Apache when answering a request can be very complex, according to the server’s configuration. Advanced scripts or application servers may add custom fields to the server’s response, and knowing what values were set may be of great help when trying to track down an application problem.

Other than the fact that you’re recording fields the server is sending rather than receiving, this recipe is analogous to Recipe 3.17 in this chapter; refer to that recipe for more details. The only difference in the syntax of the logging format effectors is that response fields are logged using an o effector, and request fields are logged using i.

Rather than logging accesses to your server in flat text files, you want to log the information directly to a database for easier analysis.

Install the latest release of mod_log_sql from http://www.outoforder.cc/projects/apache/mod_log_sql/ according to the modules directions (see Recipe 2.1), and then issue the following commands:

          # 
          mysqladmin create apache_log
          # 
          mysql apache_log < access_log.sql
          # 
          mysql apache_log
          mysql> 
          grant insert,create on apache_log.* to webserver@localhost identified by 'wwwpw';

Add the following lines to your httpd.conf file:

<IfModule mod_log_sql.c>
    LogSQLLoginInfo mysql://webserver:wwwpw@dbmachine.example.com/apache_log
    LogSQLCreateTables on
</IfModule>

Then, in your VirtualHost container, add the following log directive:

LogSQLTransferLogTable access_log

Replace the values of webserver and wwwpw with a less guessable username and password when you run these commands.

Consult the documentation on the referenced Web site to ensure that the example here reflects the version of the module that you have installed, as the configuration syntax changed with the 2.0 release of the module.

You want to send your log entries to syslog.

To log your error log to syslog, simply tell Apache to log to syslog:

ErrorLog syslog:user

Logging your access log to syslog takes a little more work. Add the following to your configuration file:

CustomLog |/usr/local/apache/bin/apache_syslog combined

Where apache_syslog is a program that looks like the following:

#!/usr/bin/perl
use Sys::Syslog qw( :DEFAULT setlogsock );

setlogsock('unix');
openlog('apache', 'cons', 'pid', 'user');

while ($log = <STDIN>) {
    syslog('notice', $log);
}

closelog;

There are several compelling reasons for logging to syslog. The first of these is to have many servers log to a central logging facility. The second is that there are many existing tools for monitoring syslog and sending appropriate notifications on certain events. Allow Apache to take advantage of these tools, and your particular installation may benefit. Also, in the event that your server is either compromised, or has some kind of catastrophic failure, having logfiles on a dfferent physical machine can be of enormous benefit in finding out what happened.

Apache supports logging your error log to syslog by default. This is by far the more useful log to handle this way, since syslog is typically used to track error conditions, rather than merely informational messages.

The syntax of the ErrorLog directive allows you to specify syslog as an argument, or to specify a particular syslog facility. In this example, the user syslog facility was specified. In your /etc/syslog.conf file, you can specify where a particular log facility should be sent—whether to a file, or to a remote syslog server.

Because Apache does not support logging your access log to syslog by default, you need to accomplish this with a piped logfile directive. The program that we use to accomplish this is a simple Perl program using the Sys::Syslog module, which is a standard module with your Perl installation. Because the piped logfile handler is launched at server startup, and merely accepts input on STDIN for the life of the server, there is no performance penalty for using Perl.

If you have several Web servers, and want to have all of them log to one central logfile, this can be accomplished by having all of your servers log to syslog, and pointing that syslog facility to a central syslog server. Note that this may cause your log entries to be in non-sequential order, which should not really matter, but may appear strange at first. This effect can be reduced by ensuring that your clocks are synchronized via NTP.

Consult your syslogd manual for further detail on setting up a networked syslog server.

Finally, depending on what particular operating system you are using, you may be able to use the logger utility to accomplish the same thing:

AccessLog "|/usr/bin/logger" combined

You want each user directory Web site (i.e., those that are accessed via http://server/~username) to have its own logfile.

In httpd.conf, add the directive:

CustomLog "|/usr/local/apache/bin/userdir_log" combined

Then, in the file /usr/local/apache/bin/userdir_log, place the following code:

#!/usr/bin/perl

my $L = '/usr/local/apache/logs'; # Log directory

my %is_open = (); # File handle cache
$|=1;
open(F, ">>$L/access_log"); # Default error log

while (my $log = <STDIN>) {
    if ($log =~ m!\s/~(.*?)/!) {
        my $u = $1;
        unless ($is_open{$u}) {
            my $fh;
            open $fh, '>>' . $L . '/'. $u;
            $is_open{$u} = $fh;
        }
        select ($is_open{$u});
        $|=1;
        print $log;
    }
    else {
        select F;
        $|=1;
        print F $log;
    }
}

close F;
foreach my $h (keys %is_open) {
    close $h;
}

Usually, requests to user directory Web sites are logged in the main server log, with no differentiation between one user’s site and another. This can make it very hard for a user to locate log messages for their personal Web site.

The recipe above allows you to break out those requests into one logfile per user, with requests not going to a userdir Web site going to the main logfile. The log handler can, of course, be modified to put all log messages in the main logfile as well as in the individual logfiles.

In order to lessen the amount of disk activity necessary, file handles are cached, rather than opened and closed with each access. This results in a larger number of file handles which are open at any given time. For sites with a very large number of user Web sites, this may cause you to run out of system resources.

Because Perl buffers output by default, we need to explicitly tell our script not to buffer the output, so that log entries make it into the logfile immediately. This is accomplished by setting the autoflush variable, $|, to a true value. This tells Perl not to buffer output to the most-recently selected file handle. Without this precaution, output will be buffered, and it will appear that nothing is being written to your logfiles.

An alternate approach might involve setting an environment variable using mod_rewrite and then adding that variable to your LogFormat directive:

RewriteRule ^/~([^/]+)/ - [E=userdir:$1]
LogFormat "%{userdir}e %h %l %u %t \"%r\" %>s %b" common

Having done this, you could then use the split-logfile script to split the logfile up into one file per individual user.

You have only one IP address, but you want to support more than one Web site on your system.

Use the NameVirtualHost *:80 directive in conjunction with <VirtualHost> sections:

ServerName 127.0.0.1
NameVirtualHost *:80

<VirtualHost *:80>
    ServerName TheSmiths.name
    DocumentRoot "C:/Apache/Sites/TheSmiths"
</VirtualHost>
        
<VirtualHost *:80>
    ServerName JohnSmith.name
    DocumentRoot "C:/Apache/Sites/JustJohnSmith"
</VirtualHost>

With IP addresses increasingly hard to come by, name-based virtual hosting is the most common way to run multiple Web sites on the same Apache server. The previous recipe works for most users in most virtual hosting situations.

The *:80 in the previous rules means that the specified hosts run on all addresses. For a machine with only a single address, this means that it runs on that address but will also run on the loopback, or localhost address. Thus if you are sitting at the physical server system, you can view the Web site.

The argument to the <VirtualHost> container directive needs to match the argument in a NameVirtualHost directive. Putting the hostname here may cause Apache to ignore the virtual host on server startup, and requests to this virtual host may unexpectedly go somewhere else. If your name server is down or otherwise unresponsive at the time that your Apache server is starting up, then Apache can’t match the particular <VirtualHost> section to the NameVirtualHost directive to which it belongs.

Requests for which there is not a virtual host listed will go to the first virtual host listed in the configuration file. In the case of the previous example, requests coming to the server using hostnames that are not explicitly mentioned in one of the virtual hosts will be served by the TheSmiths.name virtual host.

It is particularly instructive to run httpd -S and observe the virtual host configuration as Apache understands it, to see if it matches the way that you understand it. httpd -S returns the virtual host configuration, showing which hosts are name-based, which are IP-based, and what the defaults are.

Multiple names can be listed for a particular virtual host using the ServerAlias directive, as shown here:

ServerName TheSmiths.name
ServerAlias www.TheSmiths.name Smith.Family.name

It is important to understand that virtual hosts render the server listed in the main body of your configuration file (the “main” or “default” server mentioned earlier) no longer accessible—you must create a virtual host section explicitly for that host. List this host first, if you want it to be the default one.

Adding name-based virtual hosts to your Apache configuration does not magically add entries to your DNS server. You must still add records to your DNS server so that the names resolve to the IP address of the server system. When users type your server name(s) into their browser location bars, their computers first contact a DNS server to look up that name and resolve it to an IP address. If there is no DNS record, then their browsers can’t find your server.

For more information on configuring your DNS server, consult the documentation for the DNS software you happen to be running, or talk to your ISP if you’re not running your own DNS server.

You want all unmatched requests, whether they specify a name or use an IP address, to be directed to a default host, possibly with a “host not found” error message.

Add the following <VirtualHost> section, and list it before all of your other ones:

<VirtualHost *:80>
    ServerName default
    DocumentRoot /www/htdocs
    ErrorDocument 404 /site_list.html
</VirtualHost>

Note that this recipe is used in the context of name-based virtual hosts, so it is assumed that you have other virtual hosts that are also using the <VirtualHost *:80> notation, and that there is also an accompanying NameVirtualHost *:80 appearing above them. We have used the default name for clarity; you can call it whatever you want.

Setting the ErrorDocument 404 to a list of the available sites on the server directs the user to useful content, rather than leaving him stranded with an unhelpful 404 error message. You may wish to set DirectoryIndex to the site list as well, so that users who go directly to the front page of this site also get useful information.

It’s a good idea to list explicitly all valid hostnames either as ServerNames or ServerAliases, so that nobody ever winds up at the default site. However, if someone accesses the site directly by IP address, or if a hostname is added to the address in question before the appropriate virtual host is created, the user still gets useful content.

You have multiple IP addresses assigned to your system, and you want to support one Web site on each.

Create a virtual host section for each IP address you want to list on:

ServerName 127.0.0.1

<VirtualHost 10.0.0.1>
    ServerName Example.Com
    DocumentRoot "C:/Apache/Sites/Example.Com"
</VirtualHost>

<VirtualHost 10.0.0.2>
    ServerName JohnSmith.Example.Com
    DocumentRoot "C:/Apache/Sites/JustJohnSmith"
</VirtualHost>

The virtual hosts defined in this example catch all requests to the specified IP addresses, regardless of what hostname is used to get there. Requests to any other IP address not listed go to the virtual host listed in the main body of the configuration file.

The ServerName specified is used as the primary name of the virtual host, when needed, but is not used in the process of mapping a request to the correct host. Only the IP address (not the Host header field) is consulted to figure out which virtual host to serve requests from.

You want to create a virtual host to catch all requests that don’t map to one of your address-based virtual hosts.

Use the _default_ keyword to designate a default host:

<VirtualHost _default_>
    DocumentRoot /www/htdocs
</VirtualHost>

The _default_ keyword creates a virtual host that catches all requests for any address:port combinations for which there is no virtual host configured.

The _default_ directive may—and should—be used in conjunction with a particular port number, such as:

<VirtualHost _default_:443>

Using this syntax means that the specified virtual host catches all requests to port 443, on all addresses for which there is not an explicit virtual host configured. SSL virtual hosts are usually set up using the _default_ syntax, so you’ll see this syntax used in the default SSL configuration file, along with the necessary directives to enable SSL.

_default_ typically does not work as people expect in the case of name-based virtual hosts. It does not match names for which there are no virtual host sections, only address:port combinations for which there are no virtual hosts configured. If you wish to create a default name-based host, see Recipe 4.2.

You have multiple IP addresses assigned to your system, and you want to support more than one Web site on each address.

Provide a NameVirtualHost directive for each IP address, and proceed as you did with a single IP address:

ServerName 127.0.0.1
NameVirtualHost 10.0.0.1:80
NameVirtualHost 10.0.0.2:80

<VirtualHost 10.0.0.1:80>
    ServerName TheSmiths.name
    DocumentRoot "C:/Apache/Sites/TheSmiths"
</VirtualHost>

<VirtualHost 10.0.0.1:80>
    ServerName JohnSmith.name
    DocumentRoot "C:/Apache/Sites/JustJohnSmith"
</VirtualHost>

<VirtualHost 10.0.0.2:80>
    ServerName Example.Com
    DocumentRoot "C:/Apache/Sites/Example.Com"
</VirtualHost>

<VirtualHost 10.0.0.2:80>
    ServerName DoriFerguson.Example.Com
    DocumentRoot "C:/Apache/Sites/JustDoriFerguson"
</VirtualHost>

Using the address of the server, rather than the wildcard * argument, makes the virtual hosts listen only to that IP address. However, you should notice that the argument to <VirtualHost> still must match the argument to the NameVirtualHost with which the virtual hosts are connected.

You want to host many virtual hosts, all of which have exactly the same configuration.

Use VirtualDocumentRoot and VirtualScriptAlias provided by mod_vhost_alias:

VirtualDocumentRoot /www/vhosts/%-1/%-2.1/%-2/htdocs
VirtualScriptAlias  /www/vhosts/%-1/%-2.1/%-2/cgi-bin

This recipe uses directives from mod_vhost_alias, which you may not have installed when you built Apache, as it is not one of the modules that is enabled by default.

These directives map requests to a directory built up from pieces of the hostname that was requested. Each of the variables represents one part of the hostname, so that each hostname is mapped to a different directory.

In this particular example, requests for content from www.example.com are served from the directory /www/vhosts/com/e/example/htdocs, or from /www/vhosts/com/e/example/cgi-bin (for CGI requests). The full range of available variables is shown in Table 4-1.

M and N may have positive or negative integer values, the meanings of which are shown in Table 4-2.

When the value is placed in the first part of the argument—in the M part of %M.N—it refers to parts of the hostname itself. When used in the second part—the N—it refers to a particular letter from that part of the hostname. For example, in hostname www.example.com, the meanings of the variables are as shown in Table 4-3.

Depending on the number of virtual hosts, you may wish to create a directory structure subdivided alphabetically by domain name, by top-level domain, or simply by hostname.

Note that mod_vhost_alias does not set the DOCUMENT_ROOT environment variable, and so applications that rely on this value may not work in this kind of virtual hosting environment.

Although there is a module—mod_vhost_alias—that is explicitly for the purpose of supporting large numbers of virtual hosts, it is very limiting and requires that every virtual host be configured exactly the same way. You want to support a large number of vhosts, configured dynamically, but at the same time, you want to avoid mod_vhost_alias.

Use directives from mod_rewrite to map to a directory based on the hostname:

RewriteEngine on
RewriteCond   "%{HTTP_HOST}"     "^(www\.)?([^.]+)\.com"
RewriteRule   "^(.*)$"           "/home/%2$1"

mod_vhost_alias is useful, but it is best for settings where each virtual host is identical in every way but the hostname and document tree. Using mod_vhost_alias precludes the use of other URL-mapping modules, such as mod_userdir, mod_rewrite, and mod_alias, and it can be very restrictive. Using mod_rewrite is less efficient, but it is more flexible.

For example, when using mod_vhost_alias, you must do all of your hosts with mod_vhost_alias; with this alternate approach, you can do some of your hosts using the rewrite rules and others using conventional virtual host configuration techniques.

The directives in the Solution map requests for www.something.com (or without the www) to the directory /home/something.

You want each virtual host to have its own logfiles.

Specify Errorlog and CustomLog within each virtual host declaration:

<VirtualHost *:80>
    ServerName   waldo.example.com
    DocumentRoot /home/waldo/www/htdocs

    ErrorLog     /home/waldo/www/logs/error_log
    CustomLog    /home/waldo/www/logs/access_log combined
</VirtualHost>

The various logging directives can be placed either in the main body of your configuration file or within a <VirtualHost> section. When they are placed within a virtual host, log entries for that virtual host go in the specified logfiles, rather than into the logfile(s) defined in the main server configuration.

In the recipe given here, the logfiles are placed within the home directory of a particular user, rather than in the main log directory. This gives you easier access to those files, but you still need to take adequate precautions to set the permissions on the directory in question. Consult Chapter 6 for a discussion on file permissions.

Because of a large number of virtual hosts, you want to have a single logfile for all of them and split it up afterward.

LogFormat "%v %h %l %u %t \"%r\" %>s %b" vhost
CustomLog logs/vhost_log vhost

Then, after rotating your logfile:

split-logfile < logs/vhost_log

The LogFormat directive in this recipe creates a logfile that is similar to the common logfile format but additionally contains the name of the virtual host being accessed. The split-logfile utility splits up this logfile into its constituent virtual hosts.

You want to present different content for HTTP connections on different ports.

Explicitly list the port number in the <VirtualHost> declaration:

Listen 8080

<VirtualHost 10.0.1.2:8080>
    DocumentRoot /www/vhosts/port8080
</VirtualHost>

Listen 9090

<VirtualHost 10.0.1.2:9090>
    DocumentRoot /www/vhosts/port9090
<VirtualHost>

Port-based virtual hosting is somewhat less common than other techniques shown in this chapter. However, there are a variety of situations in which it can be useful. If you have only one IP address, have no ability to add hostnames to DNS, or if your ISP blocks inbound traffic on port 80, it may be useful to run virtual hosts on other ports.

It also may be useful in development environments to run separate httpd instances on different ports for different developers or different setups.

Visitors to your Web site must list the port number in the URL that they use. For example, to load content from the second virtual host previously listed, the following URL might be used:

http://server.example.com:9090

You want to have the same content displayed on two of your addresses.

Specify both addresses in the <VirtualHost> directive:

NameVirtualHost 192.168.1.1:80
NameVirtualHost 172.20.30.40:80

<VirtualHost 192.168.1.1:80 172.20.30.40:80>
    DocumentRoot /www/vhosts/server
    ServerName server.example.com
    ServerAlias server
</VirtualHost>

This setup is most useful on a machine that has addresses that are internal to your network, as well as those that are accessible only from outside your network. If these are the only addresses, you could use the * notation introduced in Recipe 4.1. However, if there are more addresses, this allows you to specify what content appears on what address.

You want to define virtual hosts in a database, rather than having to edit the configuration file each time.

Obtain mod_vhost_dbi from http://www.outoforder.cc/projects/apache/mod_vhost_dbi and use that to configure virtual hosts to be loaded out of the database:

PoolDbiDriver         Server1  mysql
PoolDbiHost           Server1  192.168.1.50
PoolDbiUsername       Server1  datauser
PoolDbiPassword       Server1  password
PoolDbiDBName         Server1  vhosts
PoolDbiConnMin        Server1  1
PoolDbiConnSoftMax    Server1  1
PoolDbiConnHardMax    Server1  5
PoolDbiConnTTL        Server1  30

<VirtualHost *:80>
  VhostDbiEnabled On
  VhostDbiConnName Server1
  VhostDbiQuery "SELECT ServerName, DocumentRoot, Username " \
        FROM vhost_info WHERE ServerName = &{RequestHostname}"
</VirtualHost>

mod_vhost_dbi is a third-party module that provides the ability to put your virtual host configuration in a database and update those virtual hosts without having to modify your configuration file or restart Apache.

The full documentation and code for this module is available from the site http://www.outoforder.cc/projects/apache/mod_vhost_dbi. Using the sample configuration above, you can get virtual hosts running. You’ll need to create records in the vhost_info table for each virtual host, containing ServerName, DocumentRoot, and Username, where ServerName and DocumentRoot have the obvious meanings, and Username refers to the user ID under which suexec will execute CGI processes. Unfortunately, mod_vhost_dbi is a somewhat limited solution because it provides only these three configuration directives.

You want to serve content out of a directory other than the DocumentRoot directory. For example, you may have an existing directory of documents, which you want to have on your Web site but that you do not want to move into the Apache document root.

Alias "/desired-URL-prefix" "/path/to/other/directory"

The given example maps URLs starting with /desired-URL-prefix to files in the /path/to/other/directory directory. For example, a request for the URL:

results in the file /path/to/other/directory/something.html being sent to the client.

This same effect could be achieved on Unixish systems by simply creating a symbolic link from the main document directory to the target directory and turning on the Options +FollowSymLinks directive.^[2] However, using Alias explicitly allows you to keep track of these directories more easily. Creating symlinks to directories makes it hard to keep track of the location of all of your content. Additionally, a stray symlink may cause you to expose a portion of your filesystem that you did not intend to.

You may also need to add a few configuration directives to permit access to the directory that you are mapping to. An error message (in your error_log file) saying that the request was “denied by server configuration” usually indicates this condition. It is fairly common—and recommended in the documentation (http://httpd.apache.org/docs/2.2/misc/security_tips.html#protectserverfiles)—to configure Apache to deny all access, by default, outside of the DocumentRoot directory. Thus, you must override this for the directory in question, with a configuration block as shown below:

<Directory "/path/to/other/directory">
    Order allow,deny
    Allow from all
</Directory>

This permits access to the specified directory.

Note that the Alias is very strict with respect to slashes. For example, consider an Alias directive as follows:

Alias "/puppies/" "/www/docs/puppies/"

This directive aliases URLs starting with /puppies/ but does not alias the URL /puppies (i.e., without the trailing slash). This may result in a “trailing slash problem.” That is, if a user attempts to go to the URL http://example.com/puppies he gets a 404 error, whereas if he goes to the URL http://example.com/puppies/ with the trailing slash, he receives content from the desired directory. To avoid this problem, create Aliases without the trailing slash on each argument.

Finally, make sure that if you have a trailing slash on the first argument to Alias, you also have one on the second argument. Consider the following example:

Alias "/icons/" "/usr/local/apache/icons"

A request for http://example.com/icons/test.gif results in Apache attempting to serve the file /usr/local/apache/iconstest.gif rather than the expected /usr/local/apache/icons/test.gif.

This is called “maintaining slash parity,” which is a fancy way of saying, “If you end the alias with a slash, end the directory with one, too; if the alias doesn’t end with a slash, the directory shouldn’t, either.”

You have an existing directory that you want to access using a different name.

Use an Alias directive in httpd.conf:

Alias "/newurl" "/www/htdocs/oldurl"

Although Alias is usually used to map URLs to a directory outside of the DocumentRoot directory tree, this is not necessarily required. There are many times when it is desirable to have the same content accessible via a number of different names. This is typically the case when a directory has its name changed, and you wish to have the old URLs continue to work, or when different people refer to the same content by different names.

Remember that Alias only affects the mapping of a local URI (the /foo/bar.txt part of http://example.com/foo/bar.txt); it doesn’t affect or change the hostname part of the URL (the http://example.com/ part). To alter that portion of the URL, use the Redirect or RewriteRule directives.

You want to give each user on your system his own Web space.

If you want users’ Web locations to be under their home directories, add this to your httpd.conf file:

UserDir public_html

To put all users’ Web directories under a central location:

UserDir "/www/users/*/htdocs"

If you want to let users access their home directory without having to use a tilde (~) in the URL, you can use mod_rewrite to perform this mapping:

RewriteEngine On
RewriteCond "/home/$1/public_html" -d [NC]
RewriteRule "^/([^/]+)/(.*)" "/home/$1/public_html/$2"

Finally, if you have mod_perl installed, you can do something more advanced like this (again, added to your httpd.conf file):

<Perl>
# Folks you don't want to have this privilege
my %forbid = map { $_ => 1 } qw(root postgres bob);
opendir H, '/home/';
my @dir = readdir(H);
closedir H;
foreach my $u (@dir) {
    next if $u =~ m/^\./;
    next if $forbid{$u};
    if (-e "/home/$u/public_html") {
        push @Alias, "/$u/", "/home/$u/public_html/";
    }
}
</Perl>

The first solution is the simplest and most widely used of the possible recipes we present here. With this directive in place, all users on your system are able to create a directory called public_html in their home directories and put Web content there. Their Web space is accessible via a URL starting with a tilde (~), followed by their usernames. So, a user named bacchus accesses his personal Web space via the URL:

http://www.example.com/~bacchus/

If you installed Apache from the standard source distribution, your default configuration file includes an example of this setup. It also contains a <Directory> section referring to the directory /home/*/public_html, with various options and permissions turned on. You need to uncomment that section in order for anyone to have access to these user Web sites. This section should look something like the following:

<Directory "/home/*/public_html">
    AllowOverride FileInfo AuthConfig Limit
    Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec
    <Limit GET POST OPTIONS PROPFIND>
        Order allow,deny
        Allow from all
    </Limit>
    <LimitExcept GET POST OPTIONS PROPFIND>
        Order deny,allow
        Deny from all
    </LimitExcept>
</Directory>

Make sure you understand what each of these directives is enabling before you uncomment this section in your configuration.

The second solution differs in that the argument to UserDir is given as a full pathname and so is not interpreted as relative to the user’s home directory, but as an actual filesystem path. The * in the file path is replaced by the username. For example, http://example.com/~smith/ is translated to /www/users/smith/htdocs. This directory structure needs to be configured in a manner similar to the previous example.

The third solution is slightly more sophisticated, and provides UserDir functionality without having to use a tilde (~) in the URL.

Using the RewriteCond directive, we first check for the existence of the user’s home directory, and, if it exists, we rewrite requests into that directory. Performing this check first ensures that other URLs continue to work correctly, and only those URLs starting with a valid username are rewritten to a user’s home directory.

This rewrite ruleset takes advantage of a little-known fact about mod_rewrite—in particular, that a RewriteRule is always considered first, and, if it matches, the RewriteCond is evaluated after that. Consequently, we can use $1 in the RewriteCond, even though the value of that variable is set in the RewriteRule appearing on the following line.

The fourth and final solution requires mod_perl and provides alias mappings for all top directories under the /home hierarchy (typically user directories). It differs from the first two by not including the tilde prefix; user smith’s Web location would be specified as http://example.com/smith/ instead of http://example.com/~smith/, but is still the filesystem location /home/smith/public_html.

In each case, the directory in question, and directories in the path leading up to it, need to be readable for the Apache user (usually nobody or www or httpd), and also have the execute bit set for that user, so the Apache server can read content out of that directory. The execute bit is needed in order to get a directory listing. Thus, for user bob, the directories /, /home, /home/bob, and /home/bob/public_html (or the corresponding directory paths for the other solutions) all need to execute access, and the last one also requires read access.

On Unixish systems, you would set these permissions by issuing the following commands:

% chmod o+x / /home /home/bob
%chmod o+rx /home/bob/public_html

The files within the directory need only be readable:

% find /home/bob/public_html -type f | xargs chmod 644

This will recurse through subdirectories, and change the file permission on all files, but not on directories.

If you use the first solution, many users may be concerned about these file permissions, and rightly so, as it usually allows all other users read access to these directories. Make sure that your users are aware of this, and that they keep personal files in directories that are not world readable.

The advantage of the second solution over the previous one is that these files are stored in a location that is not inside the user’s home directory, and so the user may keep sensible file permissions on her home directory. This lets her store personal files there without concern that other users may have free access to them.

The last Solution is completely different and requires that you have mod_perl installed. The list of directives previously mentioned goes in your configuration file, using the <Perl> configuration directive supplied by mod_perl, which allows you to put Perl code in your configuration file to dynamically add things to the configuration file at server startup.

At server startup, the code shown looks in the /home/ directory for any user that has a public_html directory and creates an Alias for them. This has the advantage over the previous two solutions because the URLs no longer contain that annoying tilde character, which people tend to think unprofessional. So user bacchus is now able to access his personal Web space via the URL http://www.example.com/bacchus/.

The %forbid list at the top of the code provides a list of users who should not be given this special alias for one reason or another. This allows you to eliminate users for which this feature may cause a security risk, such as root, or users who have shown that they can’t be trusted with such privileges.

As with the previous examples, this should be accompanied by a <Directory> section that enables read access for the directory /home/*/public_html.

And, of course, you can have this code point these aliases at any location, if you want to serve content out of some other location rather than the home directories of the users.

You want to have more than one URL map to the same directory but don’t want multiple Alias directives.

Use AliasMatch in http.conf to match against a regular expression:

AliasMatch "^/pupp(y|ies)" "/www/docs/small_dogs"
AliasMatch "^/P-([[:alnum:]])([^/]*)" "/usr/local/projects/$1/$1$2"

The AliasMatch directive allows you to use regular expressions to match arbitrary patterns in URLs and map anything matching the pattern to the desired URL. Think of it as Alias with a little more flexibility.

The first AliasMatch causes URLs starting with /puppy, as well as URLs starting with /puppies, to be mapped to the directory /www/docs/small_dogs. The second AliasMatch is designed to map to the appropriate project directory if they’re organised under the first character of their names. For instance, project Example’s URI would be /P-Example/ and would be mapped to /usr/local/projects/E/Example/.

Apache’s regular expression syntax is discussed in much greater detail in Appendix A.

You want to have a number of URLs map to the same CGI directory but don’t want to have multiple ScriptAlias directives.

Use ScriptAliasMatch in httpd.conf to match against a regular expression:

ScriptAliasMatch "^/[sS]cripts?|cgi(-bin)?/" "/www/cgi-bin/"

This is a more complicated recipe than the previous one and may require that you read Appendix A. This directive maps requests starting with /script/, /scripts/, /Script/, /Scripts/, /cgi/, and /cgi-bin/ to the directory /www/cgi-bin/, and it causes all files in that directory to be treated as CGI programs.

This kind of directive is generally used to clean up a mess that you have made. If you design your Web site well from the start, this sort of thing is never necessary, but the first time you redesign, or otherwise rearrange your Web site, you’ll find the necessity for these sorts of contortions.

You want each user to have their own cgi-bin directory rather than giving them all access to the main server CGI directory.

Put this in your httpd.conf:

<Directory "/home/*/public_html/cgi-bin/">
    Options ExecCGI
    SetHandler cgi-script
</Directory>
ScriptAliasMatch "/~([^/]+)/cgi-bin/(.*)" "/home/$1/public_html/cgi-bin/$2"

You can’t use ScriptAlias in this case, because for each user, the first argument to ScriptAlias would be different. The <Directory> container and the ScriptAliasMatch directive are functionally equivalent.

This recipe lets each user put CGI scripts in her own personal Web space. Files accessed via URLs starting with:

http://www.example.com/~username/cgi-bin/

are treated as CGI scripts.

If you have suexec enabled, CGI programs run from this target directory will be run with the user ID of the user specified in the URL. For example, a CGI program accessed via the URL http://www.example.com/~rbowen/cgi-bin/example.cgi would be run as the user rbowen.

You want requests to a particular URL to be redirected to another server.

Use a Redirect directive in httpd.conf, and give an absolute URL on the second argument:

Redirect "/example" "http://www2.example.com/new/location"

Whereas Alias maps a URL to something in the local filesystem, Redirect maps a URL to another URL, usually on another server. The second argument is a full URL and is sent back to the client (browser), which makes a second request for the new URL.

It is also important to know that the Redirect directive preserves path information, if there is any. Therefore, this recipe redirects a request for http://original.example.com/example/something.html to http://other.example.com/new/location/something.html.

Redirections come in several different flavors, too; you can specify which particular type of redirect you want to use by inserting the appropriate keyword between the Redirect directive and the first URL argument. All redirects instruct the client where the requested document is now; the different types of redirection inform where the client should look for the document in the future. If no keyword is specified, the temp meaning is used by default:

By default, if no keyword is present, a temporary redirection is issued.

Here’s an example of the various directive formats, including the HTTP status code number in case you want to use an ErrorDocument to customize the server’s response text:

#
# These are equivalent, and return a response with a 302 status.
#
Redirect      /foo.html http://example.com/under-construction/foo.html
Redirect temp /foo.html http://example.com/under-construction/foo.html
RedirectTemp  /foo.html http://example.com/under-construction/foo.html
#
# These are equivalent to each other as well, returning a 301 status
#
Redirect permanent /foo.html http://example.com/relocated/foo.html
RedirectPermanent  /foo.html http://example.com/relocated/foo.html
#
# This tells the client that the old URL is dead, but the document
# content has been replaced by the specified new document.  It
# returns a 303 status.
#
Redirect seeother /foo.html http://example.com/relocated/bar.html
#
# Returns a 410 status, telling the client that the document has been
# intentionally removed and won't be coming back.  Note that there
# is no absoluteURL argument.
#
Redirect gone     /foo.html

You want to redirect a number of URLs to the same place. For example, you want to redirect requests for /fish and /Fishing to http://fish.example.com/.

Use RedirectMatch in httpd.conf to match against a regular expression:

RedirectMatch "^/[fF]ish(ing)?(/.*)?" "http://fish.example.com/$2"

This recipe redirects requests on one server for URLs starting with fish or fishing, with either an upper-case or lower-case f, to a URL on another server, fish.example.com. As with Redirect, the path information, if any, is preserved. That is, a request for http://original.server/Fishing/tackle.html is redirected to http://fish.example.com/tackle.html so that existing relative links continue to work.

As with several of the earlier examples, RedirectMatch uses regular expressions to provide arbitrary text pattern matching.

You want requested URLs to be valid whether uppercase or lowercase letters are used.

Use mod_speling to make URLs case-insensitive:

CheckSpelling On

The mod_speling module is part of the standard Apache distribution but is not enabled by default, so you need to explicitly enable it.

In addition to making URLs case-insensitive, mod_speling, as the name implies, provides simple spellchecking capability. In particular, in the case of a “not found” error, mod_speling attempts to find files that may have been intended, based on similar spelling, transposed letters, or perhaps letters swapped with similar-looking numbers, like O for 0 and l for 1.

When mod_speling is installed, it may be turned on for a particular scope (such as a directory, virtual host, or the entire server) by setting the CheckSpelling directive to On.

And, yes, that is the correct spelling of the module name.

You want to be able to see the syntax-highlighted source to your PHP scripts without having to set up symbolic links for all of them.

Add a line such as the following to your httpd.conf or .htaccess file:

RewriteRule "^(.+\.php)s$" "$1" [T=application/x-httpd-php-source]

Or, for versions 2.2 and later:

RewriteRule "^(.+\.php)s$" "$1" [H=application/x-httpd-php-source]

Add a line such as the following to your httpd.conf file:

RewriteRule "^(.*\.php)s$" "/cgi-bin/show.php?file=$1" [PT,L]

Create a file named show.php as shown below, and put it in your server’s /cgi-bin/ directory:

<?php
/*
 * Show the highlighted source of a PHP script without a symlink or copy.
 */
if ((! isset($_GET))
    || (! isset($_GET['file']))
    || (! ($file = $_GET['file']))) {
    /*
     * Missing required arguments, so bail.
     */
    return status('400 Bad Request',
                  "Data insufficient or invalid.\r\n");
}

$file = preg_replace('/\.phps$/', '.php', $file);
if (! preg_match('/\.php$/', $file)) {
    return status('403 Forbidden',
                  "Invalid document.\r\n");
}
$docroot = $_SERVER['DOCUMENT_ROOT'];
if ((! preg_match(";^$docroot;", $file))
    || (! preg_match(";^/home/[^/]+/public_html;", $file))) {
    return status('403 Forbidden',
                  "Invalid document requested.\r\n");
}
Header('Content-type: text/html; charset=iso-8859-1');
print highlight_file($file);
return;

function status($msg, $text) {
    Header("Status: $msg");
    Header('Content-type: text/plain; charset=iso-8859-1');
    Header('Content-length: ' . strlen($text));
    print $text;
}
?>

The PHP interpreter has a built-in function to display PHP source code syntax color-coded. Ordinarily, this function is invoked for .phps files when your configuration file contains the following line:

AddHandler application/x-httpd-php-source .phps

However, in order to take advantage of this functionality, you need to make a copy, or symbolic link, of each PHP file you wish to treat this way, replacing the .php file extension with a .phps file extension. This is impractical and inconvenient.

The recipe given removes the need to do this, by rewriting any request for a .phps file to that same filename, but with a .php file extension instead, and associating the php-source handler with that request.

The [H] flag to the RewriteRule directive is new in version 2.2, and so for earlier versions, you will need to use the [T] flag instead, which provides a similar functionality.

The script in the alternate solution uses a built-in PHP function to display the script’s source in highlighted form. It’s a more complex solution than the one-liner change in the first solution, but it allows you to change the behaviour without having to restart the server, just by altering the script. The preg_match against $docroot verifies the requested file is under the server’s DocumentRoot. The next preg_match also permits files in users’ public_html directories.

You want to change all occurrences of string1 to string2 in a request’s URL.

RewriteRule "(.*)string1(.*)" "$1string2$2" [N,PT]

The [N] flag tells Apache to rerun the rewrite rule. This rule will get run repeatedly until the RewriteCond fails. Thus, it will get rerun as long as the URL contains the string that you want to replace. As soon as all occurrences of this string have been replaced, the RewriteCond will fail, and the rule will stop. The [PT] tells mod_rewrite to pass the rewritten URL on to the rest of Apache for any additional processing once the rewriting is done.

Care should be taken to avoid infinite loops, such as might happen if string1 is part of string2.

You want to pass arguments as part of the URL but have these components of the URL rewritten as CGI QUERY_STRING arguments.

This is just an example, of course; make appropriate changes to the RewriteRule line to fit your own environment and needs:

RewriteEngine on
RewriteRule "^/book/([^/]*)/([^/]*)" "/cgi-bin/book.cgi?author=$1&subject=$2" [PT]

One reason you might want or need to do this is if you’re gluing together two legacy systems that do things in different ways, such as a client application and a vendor script.

For example, the RewriteRule in the Solution will cause:

http://www.example.com/book/apache/bowen

to be rewritten as:

http://www.example.com/cgi-bin/book.cgi?subject=apache&author=bowen

The [PT] flag on the RewriteRule directive instructs Apache to keep processing the URL even after it has been modified; without the flag, the server would directly try to treat the rewritten URL as a filename, instead of continuing to the step at which it determines it’s a CGI script. It also allows multiple RewriteRule directives to make additional refinements to the URL.

If the URL being rewritten already has a query string, or might, change the [PT] to [QSA,PT]. The QSA means “query string add” and will cause the query string generated by the rewrite to be added to the query string in the original URL. Without QSA, the original query string will be replaced.

You want to prevent other Web sites from using your images (or other types of documents) in their pages and allow your images to be accessed only if they were referred from your own site.

Put this in your httpd.conf:

RewriteEngine On
RewriteCond "%{HTTP_REFERER}" !=""
RewriteCond "%{HTTP_REFERER}" "!^http://mysite.com/.*$" [NC]
RewriteRule "\.(jpg|gif|png)$ - [F]

This recipe is a series of RewriteCond directives, designed to determine whether an image file is requested from within a document on your site or if it is embedded in a page from another server. If the the latter, then the other site is stealing your images and needs to be stopped.

The first rule checks to see if the referer is even set. Some clients don’t send a referer, and some browsers can be configured not to send referers. If we deny requests from all clients that don’t send a referer, we’ll deny a lot of valid requests; so we let these ones in.

Next, we check to see if the referer appears to be from some site other than our own. If so, we keep going through the rules. Otherwise, we’ll stop processing the rewrite.

Finally, we check to see if this is a request for an image file. If the file is a nonimage file, such as an HTML file, then we want to allow people to link to these files from somewhere offsite.

If we’ve reached this point in the ruleset, we know that we have a request for an image file from within a page on another Web site. The RewriteRule matches a request and returns Forbidden to the client.

When a request comes to your server without a referring URL being mentioned, you want to redirect it to a page that explains why you’re not satisfying it.

Add lines such as the following to your configuration files:

RewriteEngine On
RewriteCond "%{HTTP_REFERER}" "^$"
RewriteRule "(.*)" "/cgi-bin/need-referer" [PT,E=ORIG:$1]

The specific problem being addressed in the Solution is requests coming to your server with no information about where they got the URL (which is passed in the Referer request header field). This frequently indicates someone surfing directly to the page rather than following a link, and in the case of images informaltable may be the result of a “Save As” request on the user’s part. If you want to only serve requests that came from links on your own site, you can change the RewriteCond directive to:

RewriteCond "%{HTTP_REFERER}" "!http://%{SERVER_NAME}/" [NC]

The RewriteRule will pass the request to a CGI script named need-referer, and will put the original URI into the environment variable ORIG for the script to reference. Note that only the local portion of the URI is included; the hostname will not be, for instance.

You want to translate one URI into another based on the value of the query string.

Put this in your httpd.conf:

RewriteCond "%{QUERY_STRING}"   "^user=([^=]*)"
RewriteRule "/people"           "http://%1.users.example.com/" [R]

mod_rewrite does not consider the query string as part of the URI for matching and rewriting purposes, so you need to treat it separately. The given example translates requests of the form:

The [R] tells mod_rewrite to direct the browser to the URL constructed by the RewriteRule directive.

You want certain parts of your non-SSL Web space to be redirected to a secured area.

You can redirect everything that is attached to port 80 with the following RewriteRule:

RewriteCond "%{SERVER_PORT}"       "^80$"
RewriteRule "^(.*)$"               "https://%{SERVER_NAME}$1" [R,L]

You can redirect particular URLs to a secure version:

RewriteRule "^/normal/secure(/.*)" "https://%{HTTP_HOST}$1" [R,L]

You can check to see whether the HTTPS environment variable is set:

RewriteCond "%{HTTPS}" "!=on"
RewriteRule "^(/secure/.*)" "https://%{HTTP_HOST}$1" [R,L]

Or, you can simply use the Redirect directive in the http section of httpd.conf file to to cause a URL to be served as HTTPS:

Redirect "/" "https://secure.example.com/"

Make sure that this appears only in in the http scope and not in the https scope, or all https requests will loop.

The first solution causes all requests that come in on port 80 (normally the unencrypted HTTP port) to be redirected to the same locations on the current server but accessed through SSL. Note the use of SERVER_NAME; because this is a complete site redirection, it’s simplest to use the server’s official name for itself.

The directive shown in the second solution causes all portions of the server’s Web space under http://myhost/normal/secure to be redirected to the SSL location rooted at https://myhost/. The use of HTTP_HOST rather than SERVER_NAME means that only the location and the scheme in the visitor’s browser, not the server name.

Note that using HTTP_HOST might break here, because it may contain the port number, too. If you expect HTTP_HOST to contain the port number, therefore, you’ll need to compensate for this in your ruleset.

Note that the paths to the SSL and non-SSL locations differ; if you want the paths to be the same except for the security, you can use something like the directives given in the third solution.

You want to migrate pathnames under a single hostname to distinct hostnames.

Use RewriteRule in httpd.conf:

RewriteRule "^/(patha|pathb|pathc)(/.*)" "http://$1.example.com$2" [R]
RewriteRule "^/([^./]*)(/.*)" "http://$1.example.com$2" [R]
RewriteRule "^/~([^./]*)(/.*)" "http://$1.example.com$2" [R]

The first recipe redirects requests of the form http://example.com/pathseg/some/file.html to a different host, such as http://pathseg.example.com/some/file.html, but only for those requests in which pathseg is patha, pathb, or pathc.

The second recipe does the same thing, except that any top-level path segment is redirected in this manner.

The third recipe splits the difference, redirecting all “user” requests to distinct hosts with the same name as the user.

You want all requests made of your system to be redirected to a specific host.

Put this in your httpd.conf:

RewriteCond "%{HTTP_HOST}"   "!^www.example.com"        [NC,OR]
RewriteCond "%{SERVER_NAME}" "!^www.example.com"        [NC]
RewriteRule "(.*)"           "http://www.example.com$1" [R]

Any request handled by your server within the scope of the directives in the Solution (which aren’t directed to the www.example.com host) is redirected there.

The two different RewriteCond directives are used to catch all requests made by some host other than www.example.com, regardless of the redirection method.

The NC (No Case) flag makes the regular expression case-insensitive. That is, it makes it match regardless of whether letters are upper- or lowercase.

The OR flag is a logical “or,” allowing the two conditions to be strung together so that either one being true is a sufficient condition for the rule to be applied.

Finally, the R flag causes an actual Redirect to be issued, so that the browser will make another request for the generated URL.

You want to redirect requests for documents to a CGI script, or other handler, that gets the document names as an argument.

Use RewriteRule in httpd.conf:

RewriteRule "^/dir/([^./]*)\.html" "/dir/script.cgi?doc=$1" [PT]

This solution causes all requests for HTML documents in the specified location to be turned into requests for a handler script that receives the document name as an argument in the QUERY_STRING environment variable.

The PT flag should be included to allow any appropriate subsequent URL rewriting or manipulation to be performed.

You want to turn portions of a URL into query-string parameters, or vice versa.

To rewrite http://example.com/path/to/5 to http://example.com/path/to?id=5:

RewriteRule "^(/path/to)/(\d+)" "$1?id=$2" [PT]

To go the other way:

RewriteCond "%{QUERY_STRING}" "\bid=(\d+)\b"
RewriteRule "(/path/to)" "$1/%2" [PT,QSA]

It is quite common for sites to have to change the way URLs are structured, either moving portions into or out of the query-string portion. This often becomed necessary when the underlying software is changed—say, from one blogging package to another. The new software requires URLs to be in a different format, but you don’t want to invalidate all the old-style URLs that might be bookmarked or otherwise spread around the Web.

This is a perfect application for mod_rewrite, and quite simple to accomplish as well, as demonstrated in the solution.

You want requests for http://bogus.example.com/ to be turned into requests for http://example.com/bogus/.

RewriteCond "%{HTTP_HOST}" "^([^.]+)\.example\.com" [NC]
RewriteRule "(.*)" "http://example.com/%1$1" [R]"

To do this transparently, without a redirect:

RewriteCond "%{HTTP_HOST}" "^([^.]+)\.example\.com$" [NC]
RewriteRule "(.*)" "/%1$1" [PT]

This technique is occasionally needed when wildcard hostnames are being supported. It gives the illusion that URLs are pointing to separate hosts rather than subdirectories on only one system. Of course, using the redirection ([R]) flag will void the illusion because the replacement URL will be visible to the end user. If you want it to be completely transparent to the user, you can use the second option to get the equivalent result with a rewrite internal to the server that the client never sees.

You want to turn requests of the form:

          http://example.com/foo/bar.html

into something like this:

          http://example.com/cgi-bin/remap?page=foo/bar.html

Add a line such as the following to your configuration file:

RewriteRule "/(.*)" "/cgi-bin/remap?page=$1" [QSA,PT]

The solution performs the rewrite in a manner completely transparent to the user, since all the changes to how to find things are done within the server rather than giving the users’s client a new URL to load.

The QSA option allows any query-string information that was on the original request to be retained and merged with the one being added by the RewriteRule. The PT option tells the server to continue processing the request rather than treating it as completely finished. This is necessary in order to have the new URL, which involves a CGI script, handled correctly.

You want to use the basic functionality of the Alias or Redirect directives, but you need to handle a bunch of related cases without needing one directive for each.

Use the grouping syntax to manipulate the URI into filesystem paths:

AliasMatch "/users/(.*)/" "/home/webonly/$1/"

Force browsers to learn the new URLs for directories relocated to their own hosts:

RedirectMatch permanent "/projects/([^/]+)(/.*)" "http://$1.projectdomain.com$2"

The AliasMatch, ScriptAliasMatch, and RedirectMatch directives use the same regular expression syntax as the rest of Apache’s configuration language. They’re not as powerful as the mod_rewrite directives, though, because they can’t be made conditional and can’t reference environment variables. However, they tend to have less of a performance impact on the server precisely because of their simpler syntax.

You want all the users on your Unixish system to be able to authenticate themselves over the Web using their already-assigned usernames and passwords.

Set up a realm using mod_auth and name /etc/passwd as the AuthUserFile:

<Directory "/home">
    AuthType Basic
    AuthName HomeDir
    AuthUserFile /etc/passwd
    Require valid-user
    Satisfy All
</Directory>

We must stress that using system account information for Web authentication is a very bad idea, unless your site is also secured using SSL. For one thing, any intruder who happens to obtain one of your users’ credentials not only can access the protected files over the Web, but can actually log onto your system where it’s possible to do significant damage. For another, Web logins don’t have the same security controls as most operating systems; over the Web, an intruder can keep hammering away at a username with password after password without the system taking any defensive measures; all mod_auth will do is record a message in the Apache error log. However, most operating systems will enter a paranoid mode and at least ignore login attempts for a while after some number of failures.

If you still want to do this, either because you consider the risk acceptable or because it doesn’t apply in your situation, the httpd.conf directives in the Solution will do the trick. The syntax and order of the fields in a credential record used by mod_auth happens (and not by accident) to match the standard layout of the /etc/passwd lines. mod_auth uses a simple text file format in which each line starts with a username and password and may optionally contain additional fields, with the fields delimited by colons. For example:

smith:$apr1$GLWeF/..$8hOXRFUpHhBJHpOUdNFe51

mod_auth ignores any additional fields after the password, which is what allows the /etc/passwd file to be used. Note that the password in the example is encrypted.

You can manage Apache mod_auth credential files with the htpasswd utility, but don’t use this utility on the /etc/passwd file! Use the normal account maintenance tools for that.

Note that this technique will not work if shadow passwords are in use, because the password field of /etc/passwd contains nothing useful in that situation. Instead, the passwords are stored in the file /etc/shadow, which is readable only by root, while Apache runs as an unprivileged user. Furthermore, most modern Unixish operating systems use the /etc/shadow means of user authentication by default.

You want to be able to provide credentials that will allow visitors into your site only once.

No solution is available with standard Apache features.

As described in the HTTP, Browsers, and Credentials sidebar, the concept of being “logged in” to a site is an illusion. In order to achieve the desired one-time-only effect, the server needs to complete the following steps:

The last step is not a simple task, and it isn’t a capability provided in the standard Apache distribution. To complicate matters, there is the desire to start a timeout once the credentials have succeeded, so that the user doesn’t authenticate once and then leave his browser session open for days and retain access.

Fulfilling this need would require a custom solution. Unfortunately, we are not aware of any open or public modules that provide this capability; however, search and watch the module registry for possible third-party implementations.

You want a user’s username and password to expire at a particular time or after some specific interval.

No solution is available with standard Apache features, but a few third-party solutions exist.

Refer to the HTTP, Browsers, and Credentials sidebar. In order for Apache to provide this functionality, it would need to store more than just the valid username and password; it would also have to maintain information about the credentials’ expiration time. No module provided as part of the standard Apache distribution does this.

There are several third-party solutions to this problem, including the Perl module Apache::Htpasswd::Perishable and the mod_perl handler Apache::AuthExpire.

There are two slightly different ways to look at this problem, that will influence your choice of a solution. You may want a user’s authentication to be timed out after a certain amount of time, or perhaps after a certain period of inactivity, forcing them to log in again. Or you may want a particular username/password pair to be completely expired after a certain amount of time, so that it no longer works. The latter might be used instead of a single-use password, which is impractical to implement in HTTP.

Apache::Htpasswd::Perishable partially implements the latter interpretation of the problem by adding expiration information to the password file. Inheriting from the Apache::Htpasswd module, it adds two additional methods, expire and extend, which set an expiration date on the password and extend the expiration time, respectively.

For example, the following code will open a password file and set an expiration date on a particular user entry in that file:

use Apache::Htpasswd::Perishable;

my $pass = Apache::Htpasswd::Perishable->new("/usr/local/apache/passwords/user.pass")
    or die "Could not open password file.";
$pass->expire('waldo', 5); # Set the expiration date 5 days in the future

Such a mechanism is only useful if expired passwords are removed from the password file periodically. This can be accomplished by running the following cron script every day. This will delete those users for whom the expiration date has passed:

#! /usr/bin/perl
use Apache::Htpasswd::Perishable;

my $password_file = '/usr/local/apache/passwords/user.pass';

open(F,$password_file) or die "Could not open password file.";
my @users;
while (my $user = <F>) {
    $user =~ s/^([^:])+:.*$/$1/;
    push @users, $user;
}
close F;

my $pass = Apache::Htpasswd::Perishable->new($password_file) or die
    "Could not open password file.";
foreach my $user (@users) {
    $pass->htDelete($user) unless $pass->expire($user) > 0;
}

Apache::AuthExpire, by contrast, implements timeouts on “login sessions.” That is, a user must reauthenticate after a certain period of inactivity. This gives you protection against the user who steps away from her computer for a few moments, leaving herself “logged in.”

As previously discussed, HTTP is stateless so it does not really have a concept of being logged in. However, by watching repeated connections from the same address, such a state can be simulated.

To use the expiring functionality offered by Apache::AuthExpire, download the module from CPAN, and install it:

# perl -MCPAN -e shell
cpan>install Apache::AuthExpire

Then configure your Apache server to use this module for your authentication handler:

PerlAuthenHandler Apache::AuthExpire
PerlSetVar DefaultLimit 7200

The given example will time out idle connections after 7,200 seconds, which is 2 hours.

With more and more Web hosting services allowing customers to upload documents, uploads may become too large. With a little creativity, you can put a limit on uploads by using the security capabilities of the server.

Assume you want to put a limit on uploads of ten thousand (10,000) bytes. The simplest way to do this is to add a LimitRequestBody directive to the appropriate scope:

<Directory "/usr/local/apache2/uploads">
    LimitRequestBody 10000
</Directory>

If a user ties to send a request that’s too large, he’ll get an error message. However, the default Apache message may leave something to be desired so you can either tailor it with a ErrorDocument 413 directive, or with some more complex (and more flexible) jiggery-pokery such as the following:

SetEnvIf Content-Length "^[1-9][0-9]{4,}" upload_too_large=1
<Location /upload>
    Order Deny,Allow
    Deny from env=upload_too_large
    ErrorDocument 403 /cgi-bin/remap-403-to-413
</Location>

You can tailor the response by making the /cgi-bin/remap-403-to-413 script look something like this:

#! /usr/local/bin/perl
#
# Perl script to turn a 403 error into a 413 IFF
# the forbidden status is because the upload was
# too large.
#
if ($ENV{'upload_too_large'}) {
    #
    # Constipation!
    #
    print <<EOHT
Status: 413 Request Entity Too Large
Content-type: text/plain; charset=iso-8859-1
Content-length: 84

Sorry, but your upload file exceeds the limits
set forth in our terms and conditions.
EOHT
}
else {
    #
    # This is a legitimate "forbidden" error.
    #
    my $uri = $ENV{'REDIRECT_REQUEST_URI'};
    my $clength = 165 + length($uri);
    print <<EOHT
Status: 403 Forbidden
Content-type: text/html; charset=iso-8859-1
Content-length: $clength

<html>
 <head>
  <title>Forbidden</title>
 </head>
 <body>
  <h1>Forbidden</h1>
  <p>
  You don't have permission to access $uri
  on this server.
  </p>
 </body>
</html>
EOHT
}
exit(0);

This script is invoked when a request results in a 403 Forbidden error (which is what the Deny directive causes if it’s triggered). It checks to see if it’s a real forbidden condition, or whether the upload file is too large, displaying an appropriate error page.

Note that both paths issue a Status CGI response header field; this is necessary to propagate the correct status back to the client. Without this, the status code would be 200 OK because the script would have been invoked successfully, which is hardly the appropriate status. An incorrect status code may cause the browser to report to the user that the file was uploaded successfully, which might generate confusion, as this may be in conflict with the message of the error page.

Actually, there is a status value that corresponds to “you sent me something too large” (413), so we remap the Deny’s 403 (Forbidden) status to it.

Other sites are linking to images on your system, stealing bandwidth from you and incidentally making it appear as though the images belong to them. You want to ensure that all access to your images is from documents that are on your server.

Add the following lines to the .htaccess file in the directory where the images are, or to the appropriate <Directory> container in the httpd.conf file. Replace the myserver.com with your domain name:

<FilesMatch "\.(jpg|jpeg|gif|png)$">
    SetEnvIfNoCase Referer "^http://([^/]*\.)?myserver.com/" local_referrer=1
    Order Allow,Deny
    Allow from env=local_referrer
</FilesMatch>

In fact, by using the following recipe, you can even go one step further, and return a different image to users accessing your images via an off-site reference:

SetEnvIfNoCase Referer "^http://([^/]*\.)?myserver.com/" local_referrer=1
RewriteCond "%{ENV:local_referer}" "!=1" 
RewriteRule ".*" "/Stolen-100x100.png" [L]

The first solution will cause all requests for image files to be refused with a 403 Forbidden status unless the link leading to the request was in one of your own documents. This means that anyone linking to your images from a different Web site system will get the error instead of the image, because the referer does not match the approved server name.

Note that this technique can cause problems for requests that do not include a Referer request header field, such as people who visit your site through an anonymizing service or who have their browser configured not to send this information.

The second solution is similar to the first, except that it substitutes an image of your choice for the one requested, rather than denying the request. Using the values in the Solution, you can construct a Stolen-100x100.png that has whatever admonitory message or perhaps just some picture that will deter the visitor from “stealing” your images.

You want to require both weak and strong authentication for a particular resource. For example, you wish to ensure that the user accesses the site from a particular location and to require that he provides a password.

Use the Satisfy directive to require both types of authentication:

<Directory /www/htdocs/sensitive>
       
    # Enforce all restrictions
    Satisfy All

    # Require a password
    AuthType Basic
    AuthName Sensitive
    AuthUserFile /www/passwords/users
    AuthGroupFile /www/passwords/groups
    Require group salesmen

    # Require access from a certain network
    Order deny,allow
    Deny from all
    Allow from 192.168.1
</Directory>

In this example, a user must provide a login, identifying him as a member of the salesmen group, and he must also use a machine on the 192.168.1 network.

The Satisfy All directive requires that all access control measures be enforced for the specified scope. A user accessing the resource from a nonmatching IP address will immediately receive a Forbidden error message in his browser, while, in the logfile, the following error message is logged:

[Sun May 25 15:31:53 2003] [error] [client 208.32.53.7] client denied by server 
    configuration: /usr/local/apache/htdocs/index.html

Users who are in the required set of IP addresses, however, receive a password dialog box and are required to provide a valid username and password.

Looking forward a little bit, the syntax of this recipe will change somewhat with the 2.4 release of the Web server. The Allow, Deny, and Satisfy directives have long confused Apache admins, and so this syntax has been revised.

The new syntax will look something like the following, with the old syntax still being available to those who want it by use of the mod_access_compat module.

The Satisfy All command will look like:

<SatisfyAll>
    Require group salesmen
    Require ip 192.168.1
</SatisfyAll>

See also the SatisfyOne directive, which replaces the Satisfy any syntax.

You wish to create password files for use with Basic HTTP authentication.

Use the htpasswd utility to create your password file, as in Table 6-1.

Or, use the Perl module Apache::Htpasswd to manage the file programmatically:

use Apache::Htpasswd;
$pass = new Apache::Htpasswd("/usr/local/apache/passwords/user.pass") or
    die "Couldn't open password file.";

# Add an entry    
$pass->htpasswd("waldo", "emerson");

# Delete entry
$pass->htDelete("waldo");

The htpasswd utility, which comes with Apache, is located in the bin subdirectory.

The first line of the Solution creates a new password file at the specified location. That is, in the example given, it creates a new password file called user.pass, containing a username and password for a user waldo. You will be prompted to enter the desired password, and then prompted to repeat the password for confirmation.

The -c flag creates a new password file, even if a file of that name already exists, so make sure that you only use this flag the first time. After that, using it causes your existing password file to be obliterated and replaced with the (almost empty) new one.

The second line in the Solution adds a password to an existing password file. As before, the user is prompted to enter the desired password, and then prompted to confirm it by typing it again.

The examples given here create a password file using the crypt algorithm by default on all platforms other than Windows, Netware, and TPF. On those platforms, the MD5 algorithm is used by default.

For platforms that use crypt, each line of the password file looks something like:

waldo:/z32oW/ruTI8U

The portion of the line following the username and colon is the encrypted password. Other usernames and passwords appear one per line.

The htpasswd utility provides other options, such as the ability to use the MD5 algorithm to encrypt the password (the -m flag), provide the password on the command line rather than being prompted for it (the -b flag), or print the results to stdout, rather than altering the password file (the -n flag).

The -b flag can be particularly useful when using the htpasswd utility to create passwords in some scripted fashion, rather than from an interactive prompt. The third line of the recipe above illustrates this syntax.

As of Apache 2.0.46, the -D flag lets you delete an existing user from the password file:

% htpasswd -D user.pass waldo

whereas in previous versions, you would need to use some alternate method to remove lines from the file. For example, you could remove a line using grep, or simply open the file in a text editor:

% egrep -v '^waldo:' user.pass >!  user.pass

Apache::Htpasswd, written by Kevin Meltzer, is available from CPAN (http://cpan.org) and gives a Perl interface to Apache password files. This allows you to modify your password files from CGI programs or via other mechanisms, using just a few lines of Perl code as shown in the recipe.

In addition to the methods demonstrated in this recipe, there are also methods for checking a particular password against the contents of the password file, obtaining a list of users from the file, or retrieving the encrypted password for a particular user, among other things. See the documentation for this fine module for the full details on its many features.

One final note about your password file. We strongly recommend that you store your password file in some location that is not accessible through the Web (i.e., outside of your document directory). By putting it in your document directory, you run the risk of someone downloading the file and running a brute-force password-cracking algorithm against it, which will eventually yield your passwords for them to use.

You need to create a password file to be used for Digest authentication.

Use the following command forms to set up a credential file for a realm to be protected by Digest authentication:

% htdigest -c "By invitation only" rbowen
%htdigest "By invitation only" krietz

Digest authorization, implemented by mod_auth_digest, uses an MD5 hash of the username, password, and authentication realm to check the credentials of the client. The htdigest utility, which comes with Apache, creates these files for you.

The syntax for the command is very similar to the syntax for the htpasswd utility, except that you must also specify the authentication realm that the password will be used for. The resulting file contains one line per user, looking something like the following:

rbowen:By invitation only:23bc21f78273f49650d4b8c2e26141a6

Note that, unlike entries in the password files created by htpasswd, which can be used anywhere, these passwords can be used only in the specified authentication realm, because the encrypted hash includes the realm.

As with htpasswd, the -c flag creates a new file, possibly overwriting an existing file. You will be prompted for the password and then asked to type it again to verify it.

htdigest does not have any of the additional options that htpasswd does.

There are times when you might want to apply a tight security blanket over portions of your site, such as with something like:

<Directory /usr/local/apache/htdocs/BoD>
    Satisfy All
    AuthUserFile /usr/local/apache/access/bod.htpasswd
    Require valid-user
</Directory>

Because of Apache’s scoping rules, this blanket applies to all documents in that directory and in any subordinate subdirectories underneath it. But suppose that you want to make a subdirectory, such as BoD/minutes, available without restriction.

The Satisfy directive is the answer. Add the following to either the .htaccess file in the subdirectory or in an appropriate <Directory> container:

Satisfy Any
Order Deny,Allow
Allow from all

This tells Apache that access is granted if the requirements of either the weak (user credentials) or strong protection (IP address) mechanisms are fulfilled. Then it goes on to say that the strong mechanism will always be happy regardless of the visitor’s origin.

Be aware that this sets a new default security condition for all subdirectories below the one affected. In other words, you are not just unlocking the one subdirectory but all of its descendants as well.

You want most documents to be restricted, such as requiring a username and password, but want a few to be available to the public. For example, you may want index.html to be publicly accessible, whereas the rest of the files in the directory require password authentication.

Use the Satisfy Any directive in the appropriate place in your .htaccess or httpd.conf file:

<Files index.html>
    Order Deny,Allow
    Allow from all
    Satisfy Any
</Files>

You can locate this in a .htaccess file, or within a <Directory> container to limit its effect:

<Directory "/usr/local/apache/htdocs">
    Satisfy All
    Order allow,deny
    Deny from all
    <Files *.html>
        Order deny,allow
        Allow from all
        Satisfy Any
    </Files>
</Directory>

Regardless of what sorts of restrictions you may have on other files, or on the directory as a whole, the <Files> container in the solution makes the index.html file accessible to everyone without limitation. Satisfy Any tells Apache that any of the restrictions in place may be satisfied, rather than having to enforce any particular one. In this case, the restriction in force will be Allow from all, which permits access for all clients.

This method can be easily extended to apply to arbitrary filename patterns using shell global characters. To extend it to use regular expressions for the filename, use the <FilesMatch> directive instead.

You wish to require user authentication based on system file ownership. That is, you want to require that the user that owns the file matches the username that authenticated.

Use the Require file-owner directive:

<Directory /home/*/public_html/private>
    AuthType Basic
    AuthName "MyOwnFiles"
    AuthUserFile /some/master/authdb
    Require file-owner
</Directory>

The goal here is to require that username jones must authenticate in order to access the /home/jones/public_html/private directory.

The user does not authenticate against the system password file but against the AuthUserFile specified in the example. Apache just requires that the name used for authentication matches the name of the owner of the file or directory in question. Note also that this is a feature of mod_auth and is not available in other authentication modules.

You wish to use user and password information in your MySQL database for authenticating users.

For Apache 1.3, use mod_auth_mysql:

Auth_MySQL_Info db_host.example.com db_user my_password
Auth_MySQL_General_DB auth_database_name

<Directory /www/htdocs/private>
    AuthName "Protected directory"
    AuthType Basic 
    require valid-user
</Directory>

For Apache 2.2 and later, use mod_authn_dbi:

AuthnDbiDriver Config1 mysql
AuthnDbiHost Config1 db.example.com
AuthnDbiUsername Config1 db_username
AuthnDbiPassword Config1 db_password
AuthnDbiName Config1 auth_database_name
AuthnDbiTable Config1 auth_database_table
AuthnDbiUsernameField Config1 user_field
AuthnDbiPasswordField Config1 password_field
AuthnDbiIsActiveField Config1 is_active_field

AuthnDbiConnMin Config1 3
AuthnDbiConnSoftMax Config1 12
AuthnDbiConnHardMax Config1 20
AuthnDbiConnTTL Config1 600

<Directory  "/www/htdocs/private">
  AuthType Digest
  AuthName  "Protected directory>
  AuthBasicProvider dbi
  AuthnDbiServerConfig Config1
  Require valid-user
</Directory>

There are a number of modules called mod_auth_mysql. The module used in the previous example is the mod_auth_mysql from http://www.diegonet.com/support/mod_auth_mysql.shtml. For the full explanation of the database fields that you will need to create, and the additional options that the module affords, you should consult the documentation on the Web site.

If you are running Apache 2.2 or later, you will want to take advantage of the new authentication framework, and use the module mod_authn_dbi, available from http://open.cyanworlds.com/mod_authn_dbi. Because of the new authentication API in Apache 2.2, a number of things are possible that were not possible in earlier versions. For example, a single module, such as mod_authn_dbi, can be used for either Basic or Digest authentication, by simply changing the AuthType directive from Basic to Digest. (AuthBasicProvider would also become AuthDigestProvider in the previous example.)

mod_authn_dbi uses libdbi, which is a generic database access library, allowing you to use your favorite database server to provide authentication services. libdbi drivers are available for most popular database servers. For a more complete description of mod_authn_dbi, you should consult the documentation on the Web site.

You want to know the name of the user who has authenticated.

Some scripting modules, such as mod_php, provide a standard interface for accessing values set by the server. For instance, to obtain the username that was used to authenticate from within a PHP script, it would access a field in the $_SERVER superglobal array:

$auth_user = $_SERVER['REMOTE_USER'];

For a Perl or mod_perl script, use:

my $username = $ENV{REMOTE_USER};

In a Server-Side Include (SSI) directive, this may look like:

Hello, user <!--#echo var="REMOTE_USER" -->. Thanks for visiting.

Other scripting modules may provide specific means of accessing this information. For those that don’t, or for nonscript applications, use the appropriate means to consult the REMOTE_USER environment variable.

When a user has authenticated, the environment variable REMOTE_USER is set to the name with which she authenticated. You can access this variable in CGI programs, SSI directives, PHP files, and a variety of other methods. The value also will appear in your access_log file.

Note that although it is the convention for an authentication module to set this variable, there are reportedly some third-party authentication modules that do not set it but provide other methods for accessing that information.

You want to get the password the user used to authenticate.

Standard Apache modules do not make this value available. It is, however, available from the Apache API if you wish to write your own authentication methods.

In the Apache 1.3 API, you need to investigate the ap_get_basic_auth_pw function. In the 2.0 API, look at the get_basic_auth function.

If you write an authentication handler with mod_perl, you can retrieve the username and password with the get_username function:

my ($username, $password) = get_username($r);

You can make this information available to CGI scripts executed by the server if you rebuild the package with the appropriate flag. For Apache 1.3 and 2.0:

% CFLAGS="$CFLAGS -DSECURITY_HOLE_PASS_AHTORIZATION"

For security reasons, although the username is available as an environment variable, the password used to authenticate is not available in any simple manner. The rationale behind this is that it would be a simple matter for unscrupulous individuals to capture passwords so that they could then use them for their own purposes. Thus, the decision was made to make passwords near to impossible to obtain.

The only way to change this is to rebuild the server from the sources with a particular (strongly discouraged) compilation flag. Alternately, if you write your own authentication module, you would of course have access to this value, as you would need to verify it in your code.

You want to disable a username when there are repeated failed attempts to authenticate using it, as if it is being attacked by a password-cracker.

There is no way to do this with standard Apache authentication modules. The usual approach is to watch your logfile carefully. Or you can use something like Apache::BruteWatch to tell you when a user is being attacked:

 PerlLogHandler Apache::BruteWatch
 PerlSetVar BruteDatabase     DBI:mysql:brutelog
 PerlSetVar BruteDataUser     username
 PerlSetVar BruteDataPassword password

 PerlSetVar BruteMaxTries     5
 PerlSetVar BruteMaxTime      120
 PerlSetVar BruteNotify       rbowen@example.com

Because of the stateless nature of HTTP and the fact that users are not, technically, “logged in” at all (see the HTTP, Browsers, and Credentials sidebar, earlier in this chapter), there is no connection between one authentication attempt and another. Most Apache auditing tools, such as mod_security, work on a per-request basis, and have no way to compare one request to another to build a profile across multiple requests. This makes it possible to repeatedly attempt to log in with a particular username, without it being easily dectable by any automated tool.

Your best bet is to carefully watch the log files, or to have some process which watches the log files for you.

Apache::BruteWatch is one way to watch the logfile and send notification when a particular account is being targeted for a brute-force password attack. With the configuration shown above, if a given account fails authentication five times in two minutes, the server administrator will be notified of the situation, so that she can take appropriate measures, such as blocking the offending address from the site.

Apache::BruteWatch is available from CPAN (CPAN.org) and requires mod_perl to run.

We are not, at this time, aware of another module that does this in real time.

You want to understand the distinction between the Basic and Digest authentication methods.

Use AuthType Basic and the htpasswd tool to control access using Basic authentication. Use AuthType Digest and the htdigest tool for the Digest method.

Basic Web authentication is exactly that: primitive and insecure. It works by encoding the user credentials with a reversible algorithm (essentially base-64 encoding) and transmitting the result in plaintext as part of the request header. Anyone (or anything) that intercepts the transmission can easily crack the encoding of the credentials and use them later. As a consequence, Basic authentication should only be used in environments where the protected documents aren’t truly sensitive or when there is no alternative.

In contrast, Digest authentication uses a more secure method that is much less susceptible to credential theft, spoofing, and replay attacks. The exact details don’t matter; the essential ingredient is that no username or password traverses the network in plaintext.

Preparing a realm to use Basic authentication consists of simply storing the username/password pair and telling the server where to find them. The password may or may not be encrypted. The same credentials may be applied to any realm on the server, or even copied to a completely different server and used there. They may be stored in a variety of databases; multiple modules exist for storing Basic credentials in flat text files, GDBM files, MySQL databases, LDAP directories, and so on.

Setting up Digest authentication is a little more involved. For one thing, the credentials are not transportable to other realms; when you generate them, you specify the realm to which they apply. For another, the only storage mechanism currently supported directly by the Apache package is flat text files; if you want to keep your Digest credentials in an LDAP directory or Oracle database, you’re going to have to look for third-party modules to do it or else write one yourself.

In addition to the more complex setup process, Digest authentication currently suffers from a lack of market penetration. That is, even though Apache supports it, not all browsers and other Web clients do; so you may end up having to use Basic authentication simply, because there’s nothing else available to your users. However, this is less and less the case with the passage of time, and there are very few Web clients in the wild any more that don’t support Digest authentication.

You know people access your site using URLs with embedded credentials, such as http://user:password@host/, and you want to extract them from the URL for validation or other purposes.

None; this is a nonissue that is often misunderstood.

Embedding the username and password in the URL gives a way to distribute a link to your users to access a password-protected site directly without being prompted for the password. However, what tends to be misunderstood about this is that the username and password are actually sent to the server in the ordinary way (i.e., via the WWW-Authenticate header) and not as part of the URL. The browser dissects the URL and turns it into the appropriate request header fields to send to the server.

You want to allow your users to upload and otherwise manage their Web documents with WebDAV but without exposing your server to any additional security risks.

Require authentication to use WebDAV:

<Directory "/www/htdocs/dav-test">
    Order Allow,Deny
    Deny from all
    AuthDigestFile "/www/acl/.htpasswd-dav-test"
    AuthDigestDomain "/dav-test/"
    AuthName "DAV access"
    Require valid-user
    Satisfy Any
</Directory>

Because WebDAV operations can modify your server’s resources and mod_dav runs as part of the server, locations that are WebDAV-enabled need to be writable by the user specified in the server’s User directive. This means that the same location is writable by any CGI scripts or other modules that run as part of the Apache server. To keep remote modification operations under control, you should enable access controls for WebDAV-enabled locations. If you use weak controls, such as user-level authentication, you should use Digest authentication rather than Basic, as shown in the Solution.

The contents of the <Directory> container could be put into a dav-test/.htaccess file, as well. Note that the authentication database (specified with the AuthDigestFile directive) is not within the server’s URI space, and so it cannot be fetched with a browser nor with any WebDAV tools.

Your authentication database and .htaccess files should not be modifiable by the server user; you don’t want them getting changed by your WebDAV users!

You want to run WebDAV but don’t want to make your document files writable by the Apache server user.

Run two Web servers as different users. The DAV-enabled server, for example, might run as User dav, Group dav, whereas the other server, which is responsible for serving your content, might run as User nobody, Group nobody. Make the Web content writable by the dav user, or the dav group.

A big security concern with DAV is that the content must be modifiable by the Web server user for DAV to be able to update that content. This means that any content also can be edited by CGI programs, SSI directives, or other programs running under the Web server. Although the Apache security guidelines caution against having any files writable by the Web server user, DAV requires it.

By running two Apache servers, you can move around this limitation. The DAV-enabled Web server, running on an alternate port, has the User and Group directives set to an alternate user and group, such as:

User dav
Group dav

which is the owner of the Web content in question. The other Web server, which will be responsible for serving content to users, runs as a user who does not have permission to write to any of the documents.

The DAV-enabled Web server should be well authenticated, so that only those who are permitted to edit the site can access that portion of the server. You should probably also set up this server to be very lightweight, both in the modules that you install as well as in the number of child processes (or threads) that you run.

Finally, it should be noted that the perchild MPM, under Apache 2.0, supports the idea of running different virtual hosts with different user ids, so that this recipe could be accomplished by enabling DAV just for the one particular vhost. However, as of this writing, the perchild MPM is not working yet.

You don’t want people using your proxy server to access particular URLs or patterns of URLs (such as MP3 or streaming video files).

You can block by keyword:

ProxyBlock .rm .ra .mp3

You can block by specific backend URLs:

<Directory proxy:http://other-host.org/path>
    Order Allow,Deny
    Deny from all
    Satisfy All
</Directory>

Or you can block according to regular expression pattern matching:

<Directory proxy:*>
    RewriteEngine On
    #
    # Disable proxy access to Real movie and audio files
    #
    RewriteRule "\.(rm|ra)$" "-" [F,NC]
    #
    # Don't allow anyone to access .mil sites through us
    #
    RewriteRule "^[a-z]+://[-.a-z0-9]*\.mil($|/)" "-" [F,NC]
</Directory>

All of these solutions will result in a client that attempts to access a blocked URL receiving a 403 Forbidden status from the server.

The first solution uses a feature built into the proxy module itself: the ProxyBlock directive. It’s simple and efficient, and it catches the results so that future accesses to the same URL are blocked with less effort; however, the pattern matching it can perform is extremely limited and prone to confusion. For instance, if you specify:

ProxyBlock .mil

the server denies access to both http://www.navy.mil/ and http://example.com/spec.mil/list.html. This is probably not what was intended!

The second method allows you to impose limitations based on the URL being fetched (or gateway, in the case of a ProxyPass directive).

The third method, which allows more complex what-to-block patterns to be constructed, is both more flexible and more powerful, and somewhat less efficient. Use it only when the other methods prove insufficient.

The flags to the RewriteRule directive tell it, first, that any URL matching the pattern should result in the server returning a 403 Forbidden error (F or forbidden), and second that the pattern match is case-insensitive (NC or nocase).

One disadvantage of the mod_rewrite solution is that it can be too specific. The first RewriteRule pattern can be defeated if the client specifies path-info or a query string, or if the origin server uses a different suffix naming scheme for these types of files. A little cleverness on your part can cover these sorts of conditions, but beware of trying to squeeze too many possibilities into a single regular expression pattern. It’s generally better to have multiple RewriteRule directives than to have a single all-singing all-dancing one that no one can read—and is, hence, prone to error.

You have files to which you want to limit access using some method other than standard Web authentication (such as a members-only area).

In httpd.conf, add the following lines to a <Directory> container whose contents should be accessed only through a script:

RewriteEngine On
RewriteRule "\.(dll|zip|exe)$" protect.php [NC]
RewriteCond %{REMOTE_ADDR} "!^my.servers.ip"
RewriteRule "\.cgi$" protect.php [NC]

And an example protect.php that just displays the local URI of the document that was requested:

<?php
/*
 * The URL of the document actually requested is in
 * $_SERVER['REQUEST_URI'].  Appropriate decisions
 * can be made about what to do from that.
 */
Header('Content-type: text/plain');
$body = sprintf("Document requested was: %s\n", $_SERVER['REQUEST_URI']);
Header('Content-length: ' . strlen($body));
print $body;
?>

In the situation that prompted this recipe, authentication and authorization were completed using a cookie rather than the standard mechanisms built into the Web protocols. Any request for a document on the site was checked for the cookie and redirected to the login page if it wasn’t found, was expired, or had some other problem causing its validity to be questioned.

This is fairly common and straightforward. What is needed in addition to this is a way to limit access to files according to the cookie and ensure that no URL-only request could reach them.

To this end, a wrapper is created (called protect.php in the Solution), which is invoked any time one of the protected document types is requested. After validating the cookie, the protect.php script figures out the name of the file from the environment variables, determines the content-type from the extension, and opens the file and sends the contents.

This is illustrated in the Solution. Any time a document ending in one of the extensions .dll, .zip, .exe, or .cgi is requested from the scope covered by the mod_rewrite directives, and the request comes from some system other than the Web server system itself (i.e., from a client system), the protect.php script will be invoked instead. In the Solution, the script simply displays the local URI of the document that is requested; applying additional access control or other functionality is easily developed from the example.

If access control is the main purpose of the wrapper and the access is granted, the wrapper needs to send the requested document to the client. In this case, the wrapper could either determine the filesystem path to the desired document and use the PHP routine fpassthru( ) to open it and send it to the client, or it could access the document using PHP’s ability to open a URL as though it were a file with the fopen("http://docurl") function call. (This latter method is necessary if the document requires server processing, such as if it’s a script.)

This would ordinarily trigger the wrapper on the dynamic document again, causing a loop. To prevent this, the wrapper is only applied to dynamic documents if the requesting host isn’t the server itself. If it is the Web server making the request, we know the wrapper has already been run and you don’t need to run it again. The server processes the document as usual and sends the contents back to the wrapper, which is still handling the original request, and it dutifully passes it along to the client. This is handled by the RewriteCond directive, which says “push requests for scripts through the wrapper unless they’re coming from the server itself.”

This method is perhaps a little less than perfectly elegant and not the best for performance, because each CGI request involves at least two concurrent requests, but it does address the problem.

Scripts running on your Web server may access, modify, or destroy files located on your Web server if they are not adequately protected. You want to ensure that this cannot happen.

Ensure that none of your files are writable by the nobody user or the nobody group, and that sensitive files are not readable by that user and group:

# find / -user nobody
#find / -group nobody

The User and Group directives specify a user and group under whose privileges the Web server will run. These are often set to the values of nobody and nobody, respectively, but they can vary in different setups. It is often advisable to create a completely new user and group for this purpose, so that there is no chance that the user has been given additional privileges of which you are not aware.

Because everything runs with these privileges, any files or directories that are accessible by this user and/or group will be accessible from any script running on the server. This means that a script running under one virtual host may possibly modify or delete files contained within another virtual host, either intentionally or accidentally, if those files have permissions making this possible.

Ideally, no files anywhere on your server should be owned by, or writable by, the server user, unless for the explicit purpose of being used as a datafile by a script. And, even for this purpose, it is recommended that a real database be used, so that the file itself cannot be modified by the server user. And if files simply must be writable by the server, they should definitely not be in some Web-accessible location, such as /cgi-bin/.

You want to set file permissions to provide the maximum level of security.

The bin directory under the ServerRoot should be owned by user root, group root, and have file permissions of 755 (rwxr-xr-x). Files contained therein should also be owned by root.root and be mode 755.

Document directories, such as htdocs, cgi-bin, and icons, will have to have permissions set in a way that makes the most sense for the development model of your particular Web site, but under no circumstances should any of these directories or files contained in them be writable by the Web server user.

The conf directory should be readable and writable only by root, as should all the files contained therein.