13

isvarnishworking.com is letting me know that

Varnish appears to be responding at that url, but the Cache-Control header's "max-age" value is less than 1, which means that Varnish will never serve content from cache at this url.

The max-age value appears to be: 0

And this header info

The url we checked: myDomainHere.com
    HTTP/1.1 200 OK
Server: Apache/2.4.7 (Ubuntu)
X-Powered-By:   PHP/5.5.9-1ubuntu4.5
Set-Cookie: PHPSESSID=vgk7db66kh7nce8lpe5789u105; path=/
Expires:    Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control:  max-age=60, private, proxy-revalidate
Pragma: no-cache
Vary:   Accept-Encoding,User-Agent
Content-Encoding:   gzip
Content-Type:   text/html
Content-Length: 14192
Accept-Ranges:  bytes
Date:   Sat, 18 Jul 2015 09:31:55 GMT
X-Varnish:  324589322
Age:    0
Via:    1.1 varnish
Connection: keep-alive

I have this in .htaccess

 <FilesMatch "\.(js|css)$">
    Header set Cache-Control "max-age=604800, public"
</FilesMatch>
<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|swf)$">
    Header set Cache-Control "max-age=604800, public"
</FilesMatch>
<FilesMatch "\.(html|htm|php)$">
    Header set Cache-Control "max-age=60, private, proxy-revalidate"
</FilesMatch>

So my question, Do I really have to change that max-age=0 in order to varnish perform better? If so, where would I Do this? I am using apache2 on a ubuntu digitalocean's droplet

-edit-

This is my /etc/varnish/default.vcl

# This is a basic VCL configuration file for varnish.  See the vcl(7)
# man page for details on VCL syntax and semantics.
#
# Default backend definition.  Set this to point to your content
# server.
#
backend default {
    .host = "127.0.0.1";
    .port = "8080";
}
#
# Below is a commented-out copy of the default VCL logic.  If you
# redefine any of these subroutines, the built-in logic will be
# appended to your code.
# sub vcl_recv {
#     if (req.restarts == 0) {
#       if (req.http.x-forwarded-for) {
#           set req.http.X-Forwarded-For =
#               req.http.X-Forwarded-For + ", " + client.ip;
#       } else {
#           set req.http.X-Forwarded-For = client.ip;
#       }
#     }
#     if (req.request != "GET" &&
#       req.request != "HEAD" &&
#       req.request != "PUT" &&
#       req.request != "POST" &&
#       req.request != "TRACE" &&
#       req.request != "OPTIONS" &&
#       req.request != "DELETE") {
#         /* Non-RFC2616 or CONNECT which is weird. */
#         return (pipe);
#     }
#     if (req.request != "GET" && req.request != "HEAD") {
#         /* We only deal with GET and HEAD by default */
#         return (pass);
#     }
#     if (req.http.Authorization || req.http.Cookie) {
#         /* Not cacheable by default */
#         return (pass);
#     }
#     return (lookup);
# }
#
# sub vcl_pipe {
#     # Note that only the first request to the backend will have
#     # X-Forwarded-For set.  If you use X-Forwarded-For and want to
#     # have it set for all requests, make sure to have:
#     # set bereq.http.connection = "close";
#     # here.  It is not set by default as it might break some broken web
#     # applications, like IIS with NTLM authentication.
#     return (pipe);
# }
#
Toni Michel Caubet
  • 19,333
  • 56
  • 202
  • 378

2 Answers2

4

Do I really have to change that max-age=0 in order to varnish perform better?

Yes. You need to do it if you want it to perform at all.

If so, where would I Do this?

It seems you got everything right for caching static content, but you also seem to want to cache PHP script execution responses:

You are just missing session_cache_limiter and session_cache_expire, which should be the first lines executed on the requested PHP script, even before session_start().

If, for example, the content returned is private and you want to refresh it every minute, try something like:

session_cache_limiter('private');
session_cache_expire(1);

After that you should see a proper Cache-Control: max-age=60, private value on Varnish responses for requests for PHP generated content including those lines at the beginning of the script.

[update] Seeing your problem isn't solved yet, maybe you should try to use Apache mod_expires instead of manually setting the Cache-Control header: http://httpd.apache.org/docs/2.4/mod/mod_expires.html

That should answer your questions, but allow me to give you a few extra notes that may be useful to you and help you get a better understanding on how Varnish and the Cache-Control header relate to each other:

  • With the default configuration max-age=0 on the response indicates both to Varnish and to the browser that the response is not cacheable. Unless configured otherwise or explicitly allowed on the client's request, Varnish will not serve stale content:

A stale cache item will not be returned by any cache (proxy cache or client cache).

From https://www.rfc-editor.org/rfc/rfc7234#section-5.3:

If a response includes a Cache-Control field with the max-age
directive (Section 5.2.2.8), a recipient MUST ignore the Expires
field. Likewise, if a response includes the s-maxage directive
(Section 5.2.2.9), a shared cache recipient MUST ignore the Expires
field. In both these cases, the value in Expires is only intended
for recipients that have not yet implemented the Cache-Control field.

From http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.3:

If a cache returns a stale response, either because of a max-stale directive on a request, or because the cache is configured to override the expiration time of a response, the cache MUST attach a Warning header to the stale response, using Warning 110 (Response is stale).

A cache MAY be configured to return stale responses without validation, but only if this does not conflict with any "MUST"-level requirements concerning cache validation (e.g., a "must-revalidate" cache-control directive).

If both the new request and the cached entry include "max-age" directives, then the lesser of the two values is used for determining the freshness of the cached entry for that request.

  • The main purpose of Varnish is to store in-memory cacheable responses (while they are fresh) so they can be sent to the same OR different clients without regenerating them, also facilitating load balancing if required.

  • Varnish will not serve stale content when requested fresh content (most common scenario) and even if it did, the browser would not be saving it and would generate a new request for a fresh page every time the user asked for that content, unnecessarily hitting the cache (with all the network activity involved) instead of rendering a locally stored copy, which would be quite inefficient and noticeably slower!

max-age Indicates that the client is willing to accept a response whose age is no greater than the specified time in seconds. Unless max- stale directive is also included, the client is not willing to accept a stale response.

Now, if you still see the browser sends a new request every time the content is loaded (check the network tab of your browser's developer tools or varnish logs), recheck everything. As a last resort you can also try to set the proper html meta tags, although html5 deprecates them and they shouldn't be necessary for any modern browser and it is not best practice so I would advise against it.

Also here is some good reading on doing proper cache control on content generated by PHP scripts that might interest you.

Community
  • 1
  • 1
NotGaeL
  • 8,344
  • 5
  • 40
  • 70
  • Thank your for such an elaborated answer! I tried `session_cache_limiter('private'); session_cache_expire(1);` and it didn't change any of the isvarnishworking.com results... also the html `` tags... any other thing you think I could try? thanks again! – Toni Michel Caubet Jul 26 '15 at 12:50
  • are you calling `session_start();` afterwards? If not, you cannot set the limiter to private. Also you may try to manually set the Cache-Control header instead, by replacing both calls with `header("Cache-Control: max-age=60, private, proxy-revalidate");` – NotGaeL Jul 26 '15 at 17:01
  • 1
    More importantly, recheck your Apache configuration files. The origin of the error is most likely there (maybe you are overriding your configuration somewhere?). A good place to start would be using mod_expires instead of manually setting Cache-Control headers: http://httpd.apache.org/docs/2.4/mod/mod_expires.html – NotGaeL Jul 26 '15 at 17:08
  • I awared you the bounty even that my problem is not resolved, but it's indeed a great answer! – Toni Michel Caubet Jul 30 '15 at 18:56
  • I work with varnish regularly and I have had to solve a few problems already, although it is usually quite simple to set up and it does a pretty good job once you get it to work. I will let you know if I come up with any other possible cause for this outcome. Now I've got so invested I really need to know what's happening! Please tell me if you find out :-) – NotGaeL Jul 30 '15 at 22:01
  • I apreciate your time a lot... this is my whole htacess (just with not rewrite rules) http://pastebin.com/LCenyNFh In case anything pops out your mind, cheers! – Toni Michel Caubet Aug 01 '15 at 09:23
  • 1
    I've noticed a couple of things: isvarnishworking.com you doesn't seem to be actually getting a `max-age=0`. The dump you posted has a `max-age=60`. Then, why the error? Well, you also have an `Expires` header, which should be overriden by `max-age` so its value is not a big deal but anyway it reads `Thu, 19 Nov 1981 08:52:00 GMT`. This is probably due to an unset system clock in your Apache/Varnish server machine, and it is probably causing an incorrect `Age:0 `header which is probably interpreted by isvarnishworking.com as something being wrong with the response caching. – NotGaeL Aug 01 '15 at 10:30
  • 1
    isvarnishworking.com is telling you something like: "come on, you are telling me you have been caching this content for a whole of 0 seconds every time I ask for it? That's a little suspicious!". Although for private content if a session cookie is not provided or is not validated, starting a new session implies generating and sending Age:0 content. A lot of maybes, but that's all I've got. I suggest you sync the problematic system clock using ntp: https://help.ubuntu.com/lts/serverguide/NTP.html and then recheck to see if that fixes the problem. – NotGaeL Aug 01 '15 at 10:34
  • Something else: your .htaccess is wrong. Remove the `Header set Cache-Control "max-age=60, private, proxy-revalidate"` directive from .htaccess. You don't need it if you are already using `ExpiresByType`/`ExpiresDefault` (Check the examples on http://httpd.apache.org/docs/2.4/mod/mod_expires.html to see how to use the directives. The apache mime types configuration file, located in /etc/apache2/mods-enabled/mime.conf in ubuntu can also be useful for determine which mime types are associated with the extension you want to set caching times for) – NotGaeL Aug 01 '15 at 10:43
  • Also, I don't know which is the highest load peak you are expecting to serve, but even the cheapest server (let's say, a 6USD/month EC2 instance), with pretty much a standard configuration, can serve up to 40-50 simultaneous requests for a typical PHP page (or even a kinda heavy one with DB access and all such as a wordpress front page) without significant impact in the page load times (this is from my personal blog: https://plus.google.com/u/0/+GaelAbadin/posts/2km96GkBQcS?pid=6096084785794746850&oid=100532267028694326619). Consider that, just in case it is not worth your time and trouble. – NotGaeL Aug 01 '15 at 11:04
  • 1
    I tried `ntpdate -s ntp.ubuntu.com` and restarting apache2 and varnish, and same result... Yes, I know that there is not a real need, but just like you I'm curious, I hope to get bigger traffic and I might need it then.. ;) Really, thanks for every tip! – Toni Michel Caubet Aug 01 '15 at 12:16
1

Your Varnish VCL is not sufficient. Search and use corresponding VCL 3 or VCL 4 template, according to your Varnish version.

Check your Varnish version: varnishd -V

If you use version 2, upgrade to version 3 or 4.

VCL for Varnish 3: https://github.com/dreamhost/varnish-vcl-collection

VCL for Varnish 4: https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/blob/master/default.vcl

Domi
  • 156
  • 3