117

I'm trying to speed up my benchmark (3 tier web architecture), and I have some general questions related to Memcache(d) and Varnish.

  • What is the difference?
    It seems to me that Varnish is behind the web server, caching web pages and doesn't require change in code, just configuration.
    On the other side, Memcached is general purpose caching system and mostly used to cache result from database and does require change in get method (first cache lookup).

  • Can I use both? Varnish in front web server and Memcached for database caching?

  • What is a better option?

    (scenario 1 - mostly write,
    scenario 2 - mostly read,
    scenario 3 - read and write are similar)

Community
  • 1
  • 1
user449219
  • 1,333
  • 2
  • 9
  • 7

2 Answers2

279
  • Varnish is in front of the webserver; it works as a reverse http proxy that caches.
  • You can use both.
  • Mostly write -- Varnish will need to have affected pages purged. This will result in an overhead and little benefit for modified pages.
  • Mostly read -- Varnish will probably cover most of it.
  • Similar read & write -- Varnish will serve a lot of the pages for you, Memcache will provide info for pages that have a mixture of known and new data allowing you to generate pages faster.

An example that could apply to stackoverflow.com: adding this comment invalidated the page cache, so this page would have to be cleared from Varnish (and also my profile page, which probably isn't worth caching to begin with. Remembering to invalidate all affected pages may be a bit of an issue). All the comments, however, are still in Memcache, so the database only has to write this comment. Nothing else needs to be done by the database to generate the page. All the comments are pulled by Memcache, and the page is recached until somebody affects it again (perhaps by voting my answer up). Again, the database writes the vote, all other data is pulled from Memcache, and life is fast.

Memcache saves your DB from doing a lot of read work, Varnish saves your dynamic web server from CPU load by making you generate pages less frequently (and lightens the db load a bit as well if not for Memcache).

Brennan
  • 5,632
  • 2
  • 17
  • 24
Jeff Ferland
  • 17,832
  • 7
  • 46
  • 76
  • 5
    this: "(perhaps by voting my answer up)" made me vote up your answer – Cesc Jan 08 '16 at 08:03
  • so it is not recommended to sites like stackoverflow (or site like facebook with activity feed page) to use varnish. is it? – Mohammad Reza Esmaeilzadeh Apr 17 '16 at 10:35
  • @jbferland My question in your example is that the top of this page has a photo of me with the number of points I have received, how can we cache with Varnish but show that area dynamic? – Hossj Sep 09 '16 at 13:44
  • @Hossj using ESI tags. – Johny Pie Oct 28 '16 at 04:45
  • Varnish might be in front of a web server but also behind web server. It can also be at the same time in front and behind the same web server ;-) For example Varnish doesn't support SSL so it's not uncommon to have a web server in front of Varnish to handle HTTPS. Traffic is then proxied to Varnish which then communicates with application web server - which might be the same server as the one handling HTTPS traffic. – matt Nov 24 '16 at 23:25
34

My experience comes from using Varnish with Drupal. In as simple terms as possible, here's how I'd answer:

In general, Varnish works for unauthenticated (via cookie) traffic and memcached will cache authenticated traffic.

So use both.

Joe Hyde
  • 999
  • 7
  • 17