Table of Contents for

Squid: The Definitive Guide

The cache_peer directive has quite a few options. I’ll describe some of them here, and the others in the sections relating to specific protocols.

Squid keeps a variety of statistics and state information about each of its neighbors. One of the most important is whether Squid thinks the neighbor is alive (up) or dead (down). The neighbor’s alive/dead state affects many aspects Squid’s selection procedures. The algorithm for determining the alive/dead state is a little bit complicated, so I’ll go through it here. If you want to follow along in the source code, look at the neighborUp( ) function.

Squid uses both TCP (HTTP) and UDP (ICP/HTCP) communication to determine the state. The TCP state defaults to alive, but changes to dead if 10 consecutive TCP connections fail. When this happens, Squid initiates probe connections, no more than once every connect_timeout time period (the global directive, not the cache_peer option). The state remains dead until one of the probe connections succeeds.

If the no-query option isn’t set (meaning Squid is sending ICP/HTCP queries to the neighbor), the UDP layer communication also factors into the alive/dead algorithm. The UDP state defaults to alive, but changes to dead if Squid doesn’t get any ICP/HTCP replies for a certain amount of time—the value of the dead_peer_timeout directive.

Squid also marks a neighbor dead if its hostname doesn’t resolve to any IP addresses. When Squid determines a neighbor is dead, it writes an entry in cache.log. Here’s an example:

2003/09/29 01:13:46| Detected DEAD Sibling: bo2.us.ircache.net/3128/3130

When communication with the neighbor is reestablished, Squid logs a message like this:

2003/09/29 01:13:49| Detected REVIVED Sibling: bo2.us.ircache.net/3128/3130

A neighbor’s state affects neighbor-selection algorithms in the following ways:

There is no way to force Squid to send HTTP requests to a dead neighbor. If all neighbors are dead, Squid will try connecting to the origin server. If you don’t allow Squid to talk to the origin server (with never_direct, for example), Squid returns a cannot forward error message:

This request could not be forwarded to the origin server or to any
parent caches.  The most likely cause for this error is that:

  * The cache administrator does not allow this cache to make
    direct connections to origin servers, and
  * All configured parent caches are currently unreachable.

The neighbor_type_domain directive allows you to change the relationship with your neighbor based on the origin server’s hostname. This is useful, for example, if your neighbor is willing to serve cache hits for any request but misses only for certain nearby domains. The syntax is:

neighbor_type_domain neighbor.host.name 
               relationship [!]domain ...

For example:

cache_peer squid.uk.web-cache.net sibling 3128 3130
neighbor_type_domain squid.uk.web-cache.net parent .uk

Of course, the squid.uk.web-cache.net cache in this example should utilize appropriate miss_access rules to enforce the sibling relationship for non-UK requests. Note that domain names are matched to hostnames as described in Section 6.1.1.2.

The cache_peer_access directive defines an access list for a neighbor cache. That is, it determines which requests may, or may not, be sent to the neighbor.

You can use this, for example, to split the flow of FTP and HTTP requests. You can send all FTP URIs to one parent and all HTTP URIs to another:

cache_peer A-parent.my.org parent 3128 3130
cache_peer B-parent.my.org parent 3128 3130
acl FTP proto FTP
acl HTTP proto HTTP
cache_peer_access A-parent allow FTP
cache_peer_access B-parent allow HTTP

This configuration ensures that A-parent receives only requests for FTP URIs, while B-parent receives only requests for HTTP URIs. This includes ICP/HTCP queries as well.

You might also use cache_peer_access to enable or disable a neighbor cache during certain times of the day:

cache_peer A-parent.my.org parent 3128 3130
acl DayTime time 07:00-18:00
cache_peer_access A-parent.my.org deny DayTime

The cache_peer_domain directive is an earlier form of cache_peer_access. Rather than using the full access control feature set, it only uses domain names in URIs. It is often used to partition a group of parent caches by domain name. For example, if you have a global intranet, you may want to send requests to caches located on each continent:

cache_peer europe-cache.my.org parent 3128 3130
cache_peer asia-cache.my.org   parent 3128 3130
cache_peer aust-cache.my.org   parent 3128 3130
cache_peer africa-cache.my.org parent 3128 3130
cache_peer na-cache.my.org     parent 3128 3130
cache_peer sa-cache.my.org     parent 3128 3130
cache_peer_domain europe-cache.my.org parent .ch .dk .fr .uk .nl .de .fi ...
cache_peer_domain asia-cache.my.org parent   .jp .kr .cn .sg .tw .vn .hk ...
cache_peer_domain aust-cache.my.org parent   .nz .au .aq ...
cache_peer_domain africa-cache.my.org parent .dz .ly .ke .mz .ma .mg ...
cache_peer_domain na-cache.my.org parent     .mx .ca .us ...
cache_peer_domain sa-cache.my.org parent     .br .cl .ar .co .ve ...

Of course, this scheme doesn’t address the popular global top-level domains, such as .com.

The never_direct directive is an access list for requests that must never be sent directly to an origin server. When a request matches this access list, it must be sent to a neighbor (usually parent) cache.

For example, if Squid is behind a firewall, it may be able to talk to your “internal” servers directly but must send all requests for external servers via the firewall proxy (a parent). You can tell Squid “never connect directly to sites outside the firewall.” To do so, tell Squid what is inside the firewall:

acl InternalSites dstdomain .my.org
never_direct allow !InternalSites

The syntax is a little strange. never_direct allow foo means Squid will not go directly for requests that match “foo.” Since the set of internal sites is easy to specify, I used the negation operator (!) to match external sites, which Squid must never directly contact.

Note that this example doesn’t force Squid to connect directly to sites that match the InternalSites ACL. The never_direct access rule can only force Squid not to contact certain origin servers. You must use the always_direct rule to force direct connections to origin servers.

You must take care when using never_direct in combination with the other directives that control request routing. You can easily create an impossible situation. Here’s an example:

cache_peer A-parent.my.org parent 3128 3130
acl COM dstdomain .com
cache_peer_access A-parent.my.org deny COM
never_direct allow COM

This configuration creates a contradiction because any request whose domain name ends with .com must go through a neighbor cache. However, I defined only one neighbor cache, and don’t allow the .com requests to go there. When this happens, Squid emits the “cannot forward” error message mentioned earlier in Chapter 10.

As you can probably guess, the list of always_direct rules tell Squid that some requests must be forwarded directly to the origin server. For example, many organizations want to keep their local traffic local. An easy way to do this is to define an IP address-based ACL and put it in the always_direct rule list:

acl OurNetwork src 172.16.3.0/24
always_direct allow OurNetwork

Internally, Squid flags each client request as either hierarchical or nonhierarchical. A nonhierarchical request is one that is unlikely to result in a cache hit. For example, responses to POST requests are almost never cachable. Forwarding requests for uncachable objects to neighbors is a waste of resources when Squid can simply connect to the origin server.

Some of the rules for differentiating hierarchical and nonhierarchical requests are hardcoded in Squid. For example, the POST and PUT methods are always nonhierarchical. However, the hierarchy_stoplist directive allows you to customize the algorithm. It contains a list of strings that, when found in a URI, make the request nonhierarchical. The default list is:

hierarchy_stoplist ? cgi-bin

Thus, any request that contains a question mark or the cgi-bin string matches the stoplist and becomes nonhierarchical.

By default, Squid prefers to send nonhierarchical requests directly to origin servers. Because they are unlikely to result in cache hits, they are generally an extra burden on neighbor caches. However, the never_direct access control rules override hierarchy_stoplist. In particular, Squid:

This directive controls the way that Squid forwards nonhierarchical (i.e., probably uncachable) requests. By default, Squid prefers to send nonhierarchical requests directly to origin servers. This is because such requests are unlikely to result in cache hits. I feel it is always better to get them directly from the origin server, rather than waste time looking for them in neighbor caches. If, for some reason, you want to route such requests through the hierarchy, disable this directive:

nonhierarchical_direct off

This directive controls the way that Squid forwards hierarchical (i.e., probably cachable) requests. By default, Squid prefers to send such requests to a neighbor cache first and then directly to the origin server. You can reverse this behavior by enabling the directive:

prefer_direct on

In this way, your neighbor caches become a backup if communication with the origin server fails.

When you use the icp_port directive, Squid automatically becomes an ICP server. That is, it listens for ICP messages on the port you’ve specified, or port 3130 by default. Be sure to tell your sibling and/or child caches if you decide to use a nonstandard port.

By default, Squid denies all ICP queries. You must use the icp_access rule list to allow queries from your neighbors. It’s usually easiest to do this with src ACLs. For example:

acl N1 src 192.168.0.1
acl N2 src 172.16.0.2
acl All src 0/0
icp_access allow N1
icp_access allow N2
icp_access deny All

Note that only ICP_QUERY messages are subject to the icp_access rules. ICP client functions, such as sending queries and receiving replies, don’t require any special access controls. I also recommend that you take advantage of your operating system’s packet filtering features (e.g., ipfw, iptables, and pf) if possible. Allow UDP messages on the ICP port from your trusted neighbors and deny them from all other hosts.

When Squid denies an ICP query due to the icp_access rules, it sends back an ICP_DENIED message. However, if Squid detects that more than 95% of the recent queries have been denied, it stops responding for an hour. When this happens, Squid writes a message in cache.log:

WARNING: Probable misconfigured neighbor at foo.web-cache.com
WARNING: 150 of the last 150 ICP replies are DENIED
WARNING: No replies will be sent for the next 3600 seconds

If you see this message, you should contact the administrator responsible for the misconfigured cache.

Squid was designed to answer ICP queries immediately. That is, Squid can tell whether or not it has a fresh, cached response by checking the in-memory index. This is also why Squid is a bit of a memory hog. When an ICP query comes in, Squid calculates the MD5 hash of the URI and looks for it in the index. If not found, Squid sends back an ICP_MISS message. If found, Squid checks the expiration time. If the object isn’t fresh, Squid returns ICP_MISS. For fresh objects, Squid returns ICP_HIT.

By default, Squid logs all ICP queries (but not responses) to access.log. If you have a lot of busy neighbors, your log file may become too large to manage. Use the log_icp_queries directive to prevent logging of these queries. Although you’ll lose the detailed logging for ICP, you can still get some aggregate stats via the cache manager (see Section 14.2.1.24).

If you have sibling neighbors, you’ll probably want to use the miss_access directive to enforce the relationship. It specifies an access rule for cache misses. It is similar to http_access but is checked only for requests that must be forwarded. The default rule is to allow all cache misses. Unless you add some miss_access rules, any sibling cache can become a child cache and forward cache misses through your network connection, thus stealing your bandwidth.

Your miss_access rules can be relatively simple. Don’t forget to include your local clients (i.e., web browsers) as well. Here’s a simple example:

acl Browsers src 10.9.0.0/16
acl Child1 src 172.16.3.4
acl Child2 src 192.168.2.0/24
acl All src 0/0
miss_access allow Browsers
miss_access allow Child1
miss_access allow Child2
miss_access deny All

Note that I haven’t listed any siblings here. The child caches are allowed to request misses through us, but the siblings are not. Their cache miss requests are denied by the deny All rule.

First, you must use the cache_peer directive to define your neighbor caches. See the section Section 10.3.

Second, you must also use the icp_port directive, even if your Squid is only an ICP client. This is because Squid uses the same socket for sending and receiving ICP messages. It is perhaps a bad design decision in retrospect. If you are a client only, use icp_access to block queries. For example:

acl All src 0/0
icp_access deny All

Squid sends ICP queries to its neighbors for most requests by default. See Section 10.10 for a complete description of the way that Squid decides when, and when not, to query its neighbors.

After sending one or more queries, Squid waits some amount of time for ICP replies to arrive. If Squid receives an ICP_HIT from one of its neighbors, it forwards the request there immediately. Otherwise, Squid waits until all replies arrive or until a timeout occurs. The timeout is calculated dynamically, based on the following algorithm.

Squid knows the average round-trip time between itself and each neighbor, taken from recent ICP transactions. When querying a group of neighbors, Squid calculates the mean of all the neighbor ICP RTTs, and then doubles it. In other words, the query timeout is twice the mean of RTTs for each neighbor queried. Squid ignores neighbors that appear to be down when calculating the timeout.

In some cases, the algorithm doesn’t work well, especially if you have neighbors with widely varying RTTs. You can change the upper limit on the timeout using the maximum_icp_query_timeout directive. Alternatively, you can make Squid always use a constant timeout value with the icp_query_timeout directive.

As you already know, ICP has poor scaling properties. The number of messages is proportional to the number of neighbors. Because Squid sends identical ICP_QUERY messages to each neighbor, you can use multicast to reduce the number of messages transmitted on the network. Rather than send N messages to N neighbors, Squid sends one message to a multicast address. The multicast routing infrastructure makes sure each neighbor receives a copy of the message. See the book Interdomain Multicast Routing: Practical Juniper Networks and Cisco Systems Solutions by Brian M. Edwards, Leonard A. Giuliano, and Brian R. Wright (Addison Wesley) for more information on the inner workings of multicast.

Note that ICP replies are always sent via unicast. This is because ICP replies may be different (e.g., hit versus miss) and because the unicast and multicast routing topologies may differ. Because ICP is also used to indicate network conditions, an ICP reply should follow the same path an HTTP reply takes. The bottom line is that multicast only reduces message counts for queries.

Historically, I’ve found multicast infrastructure unstable and unreliable. It seems to be a low priority for many ISPs. Even though it works one day, something may break a few days or weeks later. You’re probably safe using multicast entirely within your own network, but I don’t recommend using it for ICP on the public Internet.

mcast_groups 224.11.22.45

cache_peer 224.0.14.1 multicast 3128 3130 ttl=32

cache_peer 172.16.3.2 parent 3128 3130 multicast-responder

mcast_groups 224.0.14.255

cache_peer 224.0.14.225 multicast 3128 3130 ttl=16
cache_peer parent1.yourisp.net parent 3128 3130 multicast-responder
cache_peer parent2.yourisp.net parent 3128 3130 multicast-responder
cache_peer parent3.yourisp.net parent 3128 3130 multicast-responder
mcast_icp_query_timeout 2 sec

First of all, you must compile Squid with the Cache Digest code enabled. Simply add the —enable-cache-digests option when running ./configure. Taking this step causes two things to happen when you run Squid:

If you don’t want to request digests for a particular neighbor, use the no-digest option on the cache_peer line. For example:

cache_peer neighbor.host.name parent 3128 3130 no-digest

Squid stores its own digest under the following URL: http://my.host.name:port/squid-internal-periodic/store_digest. When Squid requests a neighbor’s digest, it simply requests http://neighbor.host.name:port/squid-internal-periodic/store_digest. Obviously, this naming scheme is specific to Squid. If you have a non-Squid neighbor that supports Cache Digests, you may need to tell your Squid that the neighbor’s digest has a different address. The digest-url=url option to cache_peer allows you to configure the URL for the neighbor’s Cache Digest. For example:

cache_peer neighbor.host.name parent 3128 3130 digest-url=http://blah/digest

squid.conf has a number of directives that control the way in which Squid generates its own Cache Digest. First, the digest_generation directive controls whether or not Squid generates a digest of its cache. You might want to disable digest generation if your cache is a child to a parent, but not a parent or sibling to any other caches. The remaining directives control low-level underlying details of digest generation. You should change them only if you fully understand the Cache Digest implementation.

The digest_bits_per_entry determines the size of the digest. The default value is 5. Increasing the value results in larger digests (consuming more memory and bandwidth) and lower false-hit probabilities. A lower setting results in smaller digests and more false hits. I feel that the default setting is a very nice tradeoff. A setting of 3 or lower has too many false hits to be useful, and a setting of 8 or higher simply wastes bandwidth.

Squid uses a two-step process to create a cache digest. First, it builds the cache digest data structure. This is basically a large Bloom filter and small header that contains the digest parameters. Once the data structure is filled, Squid creates a cached HTTP response for the digest. This simply involves prepending some HTTP headers and storing the response on disk with the other cached responses.

A Cache Digest represents a snapshot in time of the cache’s contents. The digest_rebuild_period controls how frequently Squid rebuilds the digest data structure (but not the HTTP response). The default is once per hour. More frequent rebuilds mean Squid’s digest is more up to date, at the expense of higher CPU utilization. The rebuild procedure is relatively CPU-intensive. Your users may experience a slowdown while Squid rebuilds its digest.

The digest_rebuild_chunk_percentage directive controls how much of the cache to add to the digest each time the rebuild procedure is called. The default is 10%. During each invocation of the rebuild function, Squid adds some percentage of the cache to the digest. Squid doesn’t process user requests while this function runs. After adding the specified percentage, the function reschedules itself and then exits so that Squid can process normal HTTP requests. After processing pending requests, Squid returns to the rebuild function and adds another chunk of the cache to the digest. Decreasing this value should give better response time to your users, while increasing the total time needed to rebuild the digest.

The digest_rewrite_period directive controls how often Squid creates an HTTP response from the digest data structure. In most cases, this should match the digest_rebuild_period value. The default is one hour. The rewrite procedure consists of numerous calls to a function that simply appends some amount of the digest data structure to the cache entry (as though Squid were reading an origin server response from the network). Each time this function is called, Squid appends digest_swapout_chunk_size bytes of the digest.

To use HTCP, you must configure Squid with the —enable-htcp option. With this option enabled, Squid becomes an HTCP server by default. The htcp_port specifies the HTCP port number, which defaults to 4827. Setting the port to 0 disables the HTCP server mode.

To become an HTCP client, you need to add the htcp option to a cache_peer line. When you add this option, Squid always sends HTCP messages, instead of ICP, to the neighbor. You can’t use both HTCP and ICP with a single neighbor. The ICP port number field actually becomes an HTCP port number, so you need to change that as well. For example, let’s say you want to convert an ICP neighbor to HTCP. Here’s the neighbor configured for ICP:

cache_peer neighbor.host.name parent 3128 3130

To switch over to HTCP, the line becomes:

cache_peer neighbor.host.name parent 3128 4827 htcp

Sometimes people forget to change the port number, and they end up sending HTCP messages to the ICP port. When this happens, Squid writes warnings to cache.log:

2003/09/29 02:28:55| WARNING: Unused ICP version 23 received from 64.216.111.20:4827

Squid doesn’t currently log HTCP queries as it does for ICP queries. HTCP queries aren’t tracked in the client_list page either. However, when you enable HTCP for a peer, the cache manager server_list page (see Section 14.2.1.50) shows the count and percentage of HTCP replies that were hits and misses:

Histogram of PINGS ACKED:
        Misses      5085  98%
        Hits          92   2%

Note that none of the current Squid versions support HTCP authentication yet.

To use CARP in Squid, you must first run the ./configure script with the —enable-carp option. Next, you must add carp-load-factor options to the cache_peer lines for parents that are members of the array. The following is an example.

cache_peer neighbor1.host.name parent 3128 0 carp-load-factor=0.3
cache_peer neighbor2.host.name parent 3128 0 carp-load-factor=0.3
cache_peer neighbor3.host.name parent 3128 0 carp-load-factor=0.4

Note that all carp-load-factor values must add up to 1.0. Squid checks for this condition and complains if it finds a discrepancy. Additionally, the cache_peer lines must be listed in order of increasing load factor values. Only recent versions of Squid check that this condition is true.

Remember that CARP is treated somewhat specially with regard to a neighbor’s alive/dead state. Squid normally declares a neighbor dead (and ceases sending requests to it) after 10 failed connections. In the case of CARP, however, Squid skips a parent that has one or more failed connections. Once Squid is working with CARP, you can monitor it with the carp cache manager page. See Section 14.2.1.49 for more information.

The first step is to determine if the request may, must, or must not be sent directly to the origin server. Squid evaluates the never_direct and always_direct access rule lists for the request. The goal is to set a flag to one of three values: DIRECT_YES, DIRECT_MAYBE, or DIRECT_NO. This flag later determines whether Squid should, or should not, try to select a neighbor cache for the request. Squid checks the following conditions in order. If any condition is true, it sets the direct flag and proceeds to the next step. If you’re following along in the source code, this step corresponds to the beginning of the peerSelectFoo( ) function:

If the direct flag is set to DIRECT_YES, the selection process is complete. Squid forwards the request directly to the origin server and skips the remaining steps in this section.

Here Squid uses one of the hierarchical protocols to select a neighbor cache. As before, once Squid selects a neighbor in this step, it exits the routine and proceeds to Step 3. This step roughly corresponds to the peerGetSomeNeighbor( ) function:

If Squid sends out any ICP or HTCP queries, it waits for some number of replies. Just after transmitting the queries, Squid knows how many replies to expect and the maximum amount of time to wait for them. Squid expects a reply from every alive neighbor queried. If you’re using multicast, Squid adds the current group size estimate to the expected reply count. While waiting for replies, Squid schedules a timeout, in case one or more of the replies don’t arrive.

When Squid receives an ICP/HTCP reply from a neighbor, it takes the following actions:

Squid may not receive any ICP/HTCP replies from parent caches, either because they weren’t queried or because the network dropped some packets. In this case, Squid relies on the secondary parent (or direct) selection algorithm described in the next section.

If the ICP/HTCP query timeout occurs before receiving the expected number of replies, Squid prepends the string TIMEOUT_ to the result code in access.log.

This step is a little tricky. Remember that if the direct flag is DIRECT_YES, Squid never executes this step. If the flag is DIRECT_NO, Squid calls the getSomeParent( ) function (described subsequently) to select a backup parent, in case Step 2 failed to select one. Following that, Squid adds to the list all parents it believes are alive. Thus, it tries all possible parent caches before returning an error message to the user.

In the case of DIRECT_MAYBE, Squid adds both a parent cache, and the origin server. The order, however, depends on the prefer_direct setting. If prefer_direct is enabled, Squid inserts the origin server into the list first. Next, Squid calls getSomeParent( ) if the request is hierarchical or if the nonhierarchical_direct directive is disabled. Finally, Squid adds the origin server last if prefer_direct is disabled.

The getSomeParent( ) function selects one of the parents based on the following criteria. In each case, the parent must be alive and allowed to handle the request according to the cache_peer_access and cache_peer_domain rules:

Occasionally, Squid’s attempt to forward a request to an origin server or neighbor may fail for one reason or another. This is why Squid creates a list of appropriate next-hop locations during the neighbor selection procedure. When one of the following types of errors occurs, Squid can retry the request at the next server in the list:

Squid’s algorithm for retrying failed requests is relatively aggressive. It is better for Squid to keep trying (causing some extra delay), rather than return an error to the user.

You simply need to define a parent and tell Squid it isn’t allowed to connect directly to origin servers. For example:

cache_peer parent.host.name parent 3128 0
acl All src 0/0
never_direct allow All

The drawback to this configuration is that Squid can’t forward cache misses if the parent goes down. If that happens, your users receive the “cannot forward” error message.

Try this configuration:

nonhierarchical_direct off
prefer_direct off
cache_peer parent.host.name parent 3128 0 default no-query

Or, if you’d like to use ICP with the other proxy:

nonhierarchical_direct off
prefer_direct off
cache_peer parent.host.name parent 3128 3130 default

With this configuration, Squid forwards all cache misses to the parent as long as it is alive. Using ICP should cause Squid to detect a dead parent quickly, but at the same time may incorrectly declare the parent dead on occasion.

Define an ACL to match the special request:

cache_peer parent.host.name parent 3128 0
acl Special dstdomain special.server.name
always_direct allow Special

In this case, cache misses for requests in the special.server.name domain are always sent to the origin server. Other requests may, or may not, go through the parent cache.

Some ISPs (and other organizations) have upstream providers that force HTTP traffic through a filtering proxy (perhaps with HTTP interception). You might be able to get around their filters if you can use a different proxy beyond their network. Here’s how you can send only special requests to the far-away proxy:

cache_peer far-away-parent.host.name parent 3128 0
acl BlockedSites dstdomain www.censored.com
cache_peer_access far-away-parent.host.name allow BlockedSites
never_direct allow BlockedSites