1

We're running a website which performs financial modeling and it takes a while for memcache to build its cache. For instance, after 1 week the number of hits is only at 48% and the cache used is 2GB (out of 5GB allocated). Since we don't want to loose that cache should the server crash or need to be restarted, we would like to save it somewhere.

Q: What are the best options for storing the content of the memcache cache somewhere permanent (and restoring that content)?

So far we haven't seen memcache reach the point whereby % of hits doesn't improve. We know we quickly get to 30% hits with 300MB of data, which corresponds to caching of shared content. Afterwards, objects become much bigger and are created less frequently. By looking at our munin graphs, I would say we could reach our best % of hits within 2 to 3 months. I really think we have a case for saving our memcache data.

enter image description here

FYI I'm not adding the graph showing the % of hits/misses because it evolves so slowly that it's not really readable.

Max
  • 12,794
  • 30
  • 90
  • 142
  • How much RAM are you allocating for memcached? Is it the full 2GB you mention? If the cache is full, and the hit rate is low, then you may be running out of cache space, and memcached is purging the oldest entries (which could still be valid). – bhamby Mar 09 '12 at 16:16
  • Filesystem or Berlkey-DB would be better. Also you may use `Redis`, which is like memcache, but handles persistent storage as well automatically. – kirilloid Mar 09 '12 at 16:18
  • @galador: we've allocated `5GB` to memcache and right now it's at `48%` hits with `2GB` used. – Max Mar 09 '12 at 16:22
  • You should get a measure of how much time it takes to get to that 48%, and what is the most it ever gets to? If it takes 10 minutes to get to 45% and only ever goes as high as 50%, then you'll be wasting your time to add persistence. – DampeS8N Mar 09 '12 at 16:34
  • @DampeS8N: see edited question with munin data. – Max Mar 09 '12 at 17:04
  • i don t know how to perform a financial model, but i guess your data change frequently in time..right? – andrea.marangoni Mar 14 '12 at 13:39

2 Answers2

1

I'd not suggest using Redis in a financial production environment, it's to unstable for it at this time. It is something to keep an eye out for in the future.

CouchBase is a solution.

You can also use repcached (http://repcached.lab.klab.org/) to replicate the memcache data to a 2nd memcache daemon in case the first one crashes, you still have all your memcache data. You can also use this type of setup to loadbalance your memcache usage if you want, i've been using it for quite some time now and are very happy with it's performance.

MarkNL
  • 11
  • 1
1

You might try:

Or you may find more tips on StackOverflow here: alternative to memcached that can persist to disk

Community
  • 1
  • 1
Gábor Nagy
  • 179
  • 3
  • 11