24

Recently I added a Varnish instance to a Rails application stack. Varnish in it's default configuration can be convinced from caching a certain resource using the Cache-Control Header like so:

Cache-Control: max-age=86400, public=true

I achieved that one using the expires_in statement in my controllers:

def index
  expires_in 24.hours, public: true
  respond_with 'some content'
end

That worked well. What I did not expect is, that the Cache-Control header ALSO affects the browser. That leads to the problem that both - Varnish and my users browser cache a certain resource. The resource is purged from varnish correctly, but the browser does not attempts to request it again unless max-age is reached.

So I wonder wether I should use 'expires_in' in combination with Varnish at all? I could filter the Cache-Control header in a Nginx or Apache instance in front of Varnish, but that seems odd.

Can anyone enlighten me?

Regards Felix

GeorgieF
  • 2,687
  • 5
  • 29
  • 43

1 Answers1

15

That is actually a very good and valid question, and a very common one with reverse proxies.

The problem is that there's only one Cache-Control property and it is intended for the client browser (private cache) and/or a proxy server (shared cache). If you don't want 3rd party proxies to cache your content at all, and want every request to be served by your Varnish (or by your Rails backend), you must send appropriate Cache-Control header from Varnish.

Modifying Cache-Control header sent by the backend is discussed in detail at https://www.varnish-cache.org/trac/wiki/VCLExampleLongerCaching

You can approach the solution from two different angles. If you wish to define max-age at your Rails backend, for instance to specify different TTL for different objects, you can use the method described in the link above.

Another solution is to not send Cache-Control headers at all from the backend, and instead define desirable TTLs for objects in varnish vcl_fetch(). This is the approach we have taken.

We have a default TTL of 600 seconds in Varnish, and define longer TTLs for pages that are definitely explicitly purged when changes are made. Here's our current vcl_fetch() definition:

sub vcl_fetch {
  if (req.http.Host ~ "(forum|discus)") {
    # Forum pages are purged explicitly, so cache them for 48h
    set beresp.ttl = 48h;
  }

  if (req.url ~ "^/software/") {
    # Software pages are purged explicitly, so cache them for 48h
    set beresp.ttl = 48h;
  }

  if (req.url ~ "^/search/forum_search_results" ) {
    # We don't want forum search results to be cached for longer than 5 minutes
    set beresp.ttl = 300s;
  }

  if(req.url == "/robots.txt") {
    # Robots.txt is updated rarely and should be cached for 4 days
    # Purge manually as required
    set beresp.ttl = 96h;
  }

  if(beresp.status == 404) {
    # Cache 404 responses for 15 seconds
    set beresp.http.Cache-Control = "max-age=15";
    set beresp.ttl = 15s;
    set beresp.grace = 15s;
  }
}

In our case we don't send Cache-Control headers at all from the web backend servers.

Ketola
  • 2,767
  • 18
  • 21
  • 3
    thank you for the detailed answer. We solved the problem by simply changing the Cache-Control header to: Cache-Control: max-age=0 s-maxage=86400, public=true so the Browser does not cache the resource but shared caches like Varnish do. Is that wrong? – GeorgieF May 01 '12 at 18:25
  • 4
    You can use s-maxage as well. However if your users sit behind a transparent proxy (some ISPs still use them), they will see the cached version for up to 24 hours as well. If this is desired, s-maxage can be used. If not, then you should beresp.unset the cache-control header from the response in your VCL. I will edit my answer to mention s-maxage as well. – Ketola May 03 '12 at 10:09
  • Thx for your info regarding that. We will definitely refactor this and change according to your solution. – GeorgieF May 03 '12 at 21:11
  • 3
    Indeed. s-maxage is definitely the easiest way. Make sure you remove s-maxage from the Cache-Control header in vcl_deliver to keep downstream caches from using it. You could use regsub to filter it. If you set both s-maxage and max-age in your backend you'll get a very flexible system that allows you to set different TTL's for Varnish and the client, yet avoid hardcoded TTL's in your VCL code. – Martijn Heemels Jan 17 '13 at 08:47