Backend Selection

In previous chapters, I mentioned that there are two ways to announce the backend to Varnish:

• By adding a backend to your VCL file.

• Or by omitting VCL completely and using the -b flag at startup time.

That’s how you do it automatically. But there are situations where there are multiple backends and you want to control which request goes to which backend. You can define multiple backends and use req.backend_hint to assign a backend other than the default one.

Here’s an example:

vcl 4.0;

backend public {
  .host = "web001.example.com";
}

backend admin {
  .host = "web002.example.com";
}

sub vcl_recv {
    if(req.url ~ "^/admin(/.*)?") {
        set req.backend_hint = admin;
    } else {
        set req.backend_hint = public;
    }
}

You notice that we defined two backends:

• A backend that handles the public traffic and that resides on web001.example.com.

• A backend that serves the admin panel and that resides on web002.example.com.

By incorporating the req.backend_hint variable in our VCL logic, we can perform content-aware load balancing. Each backend could be tuned to its specific task.

Backend Health

In “Backends and Health Probes”, we discussed how health checks can be performed using health probes. Let’s take our previous example and add a health probe:

vcl 4.0;

probe healthcheck {
   .url = "/";
   .interval = 5s;
   .timeout = 1s;
   .window = 10;
   .threshold = 3;
   .initial = 1;
   .expected_response = 200;
}

backend public {
  .host = "web001.example.com";
  .probe = healthcheck;
}

backend admin {
  .host = "web002.example.com";
  .probe = healthcheck;
}

sub vcl_recv {
    if(req.url ~ "^/admin(/.*)?") {
        set req.backend_hint = admin;
    } else {
        set req.backend_hint = public;
    }
}

The health checks in this example are done based on an HTTP request to the homepage. This check is done every five seconds with a timeout of one second. We expect an HTTP 200 status code. The backends are only considered healthy if 3 out of 10 health checks succeed.

You can view the health of your backends by executing the following command:

varnishadm backend.list

This could be the output of that command:

Backend name                   Admin      Probe
boot.public                    probe      Healthy 10/10
boot.admin                     probe      Healthy 10/10

Both backends are listed and their health is automatically checked by a probe (see the Admin column). It is considered healthy because 10 out of 10 checks were successful. As a quick reminder: only three need to succeed to have a healthy backend.

To get more verbose output, use the -p parameter:

varnishadm backend.list -p

This could be the output:

Backend name                   Admin      Probe
boot.public                    probe      Healthy 10/10
  Current states  good: 10 threshold:  3 window: 10
  Average response time of good probes: 0.575951
  Oldest ================================================== Newest
  4444444444444444444444444444444444444444444444444444444444444444 Good IPv4
  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Good Xmit
  RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR Good Recv
  HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH Happy

boot.admin                     probe      Healthy 3/10
  Current states  good:  3 threshold:  3 window: 10
  Average response time of good probes: 0.642713
  Oldest ================================================== Newest
  4444444444444444444444444444444444444444444444---------------444 Good IPv4
  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX---------------XXX Good Xmit
  RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR---------------RRR Good Recv
  HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH---------------HHH Happy

You see that the admin backend’s last three checks were successful, and those three are enough to be considered healthy. This is a backend that recovered from being “sick.”

If you continuously run this command (the watch command on Linux can help you with that), you can see the evolution of the health checks (Xmit), the responses (Recv), and the happiness of your server.

You can also manually override the probe’s decision on the backend health by issuing the following commands:

varnishadm backend.set_health boot.public sick
varnishadm backend.set_health boot.admin healthy

The output is the following:

Backend name                   Admin      Probe
boot.public                    sick       Healthy 10/10
boot.admin                     healthy    Healthy 10/10

As you can see, the Admin column no longer contains the value probe, but sick and healthy. To give the control back to the health probes, simply issue the following commands:

varnishadm backend.set_health boot.public auto
varnishadm backend.set_health boot.admin auto

The std.healthy() function that is part of the vmod_std can tell you whether or not a backend is healthy. To check the health of the current backend, just use std.healthy(req.backend_hint).

Directors

A director is a built-in VMOD that groups several backends and presents them as one. Directors offer different decision-making strategies to decide which request is handled by which backend.

The goal of directors is to avoid backend latency by distributing requests across multiple backends: by balancing out backend requests, backend servers will experience less load, which reduces latency. This is basically a horizontal scalability strategy.

Besides pure load balancing, directors also make sure that unhealthy backend nodes are not used; instead, another healthy node is used to handle those requests. This is essentially a high availability strategy.

In order to use directors, you first need to import the directors VMOD and initialize it in vcl_init. Here’s an example:

vcl 4.0;
import directors;

backend web001 {
  .host = "web001.example.com";
  .probe = healthcheck;
}

backend web002 {
  .host = "web002.example.com";
  .probe = healthcheck;
}

sub vcl_init {
    new loadbalancing = directors.round_robin();
    loadbalancing.add_backend(web001);
    loadbalancing.add_backend(web002);
}

sub vcl_recv {
    set req.backend_hint = loadbalancing.backend();
}

Let’s take this example step by step:

1. First we import the directors VMOD.

2. We declare two backends.

3. We initialize the director and decide on the load-balancing strategy (in this case, it’s “round robin”).

4. We assign both backends to the director.

5. We now have a new backend that we named loadbalancing.

6. We assign the loadbalancing backend to Varnish by issuing set req.backend_hint = loadbalancing.backend();.

More information about directors can be found on the Varnish documentation page about vmod_directors.

sub vcl_init {
    new loadbalancing = directors.round_robin();
    loadbalancing.add_backend(backend1);
    loadbalancing.add_backend(backend2);
    loadbalancing.add_backend(backend3);
}

sub vcl_recv {
    set req.backend_hint = loadbalancing.backend();
}

sub vcl_init {
    new loadbalancing = directors.random();
    loadbalancing.add_backend(backend1,1.0);
    loadbalancing.add_backend(backend2,2.0);
}

sub vcl_recv {
    set req.backend_hint = loadbalancing.backend();
}

sub vcl_init {
    new loadbalancing = directors.hash();
    loadbalancing.add_backend(backend1,1.0);
    loadbalancing.add_backend(backend2,1.0);
}

sub vcl_recv {
    set req.backend_hint = loadbalancing.backend(client.ip);
}

sub vcl_recv {
    set req.backend_hint = loadbalancing.backend(regsuball(req.http.Cookie,
    "^.*;? ?PHPSESSID=([a-zA-Z0-9]+)( ?|;| ;).*$","\1"));
}

sub vcl_recv {
    set req.backend_hint = loadbalancing.backend(req.url);
}

vcl 4.0;

import directors;

probe healthcheck {
   .url = "/";
   .interval = 2s;
   .timeout = 1s;
   .window = 3;
   .threshold = 2;
   .initial = 1;
   .expected_response = 200;
}

backend web001 {
  .host = "web001.example.com";
  .probe = healthcheck;
}

backend web002 {
  .host = "web002.example.com";
  .probe = healthcheck;
}

sub vcl_init {
    new loadbalance = directors.fallback();
    loadbalance.add_backend(web001);
    loadbalance.add_backend(web002);
}

sub vcl_recv {
    set req.backend_hint = loadbalance.backend();
}

Grace Mode

Throughout this chapter, we’ve focused on providing a stable backend service so that Varnish can access backend data without the slightest hiccup:

• Measuring backend health

• Adding health check probes

• Offering directors to distribute load

• Leveraging directors to use healthy nodes instead of unhealthy ones

But one important question remains unanswered: what do we do if no backends are available?

We can either live with it or compensate for it. I’d rather go for the latter, and that’s where grace mode comes into play.

If we assign a certain amount of grace time, we’re basically telling Varnish that it can serve objects beyond their time-to-live. These objects are considered “stale” and are served as long as there’s no updated object for a duration defined by the grace time.

The built-in VCL hit logic explains it best:

sub vcl_hit {
    if (obj.ttl >= 0s) {
        // A pure unadultered hit, deliver it
        return (deliver);
    }
    if (obj.ttl + obj.grace > 0s) {
        // Object is in grace, deliver it
        // Automatically triggers a background fetch
        return (deliver);
    }
    // fetch & deliver once we get the result
    return (miss);
}

• If the object hasn’t expired (obj.ttl >= 0s), keep on serving the object

• If the object has expired, but there is some grace time left (obj.ttl + obj.grace > 0s), keep serving the object but fetch a new version asynchronously

• Otherwise, fetch a new version and queue the request

sub vcl_backend_response {
    set beresp.grace = 30s;
}

Conclusion

You’ve spent a lot of time learning how Varnish works and how to get data in the cache. A lot of attention has gone to the client-side aspect: serving cached data to the end user. But let’s not forget that Varnish relies on a healthy backend to do its work.

At this point, you know how to link Varnish to one or multiple backends. You can configure all kinds of timeouts, and you’re able to check the health of a backend. You should also feel confident circumventing unhealthy backends with directors or by using grace mode.

By using some of the tips and tricks from this chapter, you’ll have more uptime. Regardless of your hit rate, you will still need your backend for content revalidation, so be sure to think about the availability of your backend servers.