Rails: how best to re-populate cache when invalidating

Question

for a costly-to-generate page or action, the standard Rails approach to caching calls for invalidating the page/action when its underlying data goes sufficiently stale. Then, the next user who requests that page has to sit there for N seconds when the cache miss happens and the page is generated. If N is sufficiently large, this will make that user sad. Sure, it's only one user, but if invalidations are frequent enough, we could be making lots of sad users.

One solution is to, whenever I invalidate some pages/actions, also trigger the rendering of the new versions of the pages/actions and put them into the cache. It'll be a few hours coding to build the version of this that I need.

But certainly someone else has already tackled this problem? I looked around and couldn't find any suitable gems or answers.

Thanks!

score 0 · Accepted Answer · answered Mar 05 '12 at 16:51

0

I assume that the expensive computation isn't the rendering of the HTML page, but the queries and/or calculations that you need in order to populate the page.

As you said, you should trigger this calculation/query when it becomes invalidated so that it will be instantly available for the next time the page is accessed.

You can store the data in a variety of formats: YAML, a Marshal dump, JSON, CSV, etc. Also if you go this route, I would store it someplace permanent like the filesystem or your database, rather than the Rails cache because the Rails cache isn't assumed to always have your data in it.

answered Mar 05 '12 at 16:51

nicholaides

19,211
12
66
82

This is a promising lead. Have you seen any good patterns/gems for organizing the query caching? Randomly calling Rails.cache.fetch() all through controller code seems like a quick path to spaghetti code and confusion. – dsjoerg Mar 05 '12 at 17:39
No, I don't know if any gems or patterns, but it shouldn't be too hard to figure out a good pattern for your use case yourself. Just remember to refactor and you'll end up with something reasonable. – nicholaides Mar 05 '12 at 17:42

score 0 · Answer 2 · edited May 23 '17 at 12:12

Finally found a question and answer:
Q: "Warm Up Cache" on deployment
A: https://stackoverflow.com/a/942774/593053

Leaving my question here so that others can follow the trail and learn from my pain.

Also relevant: Rails 3.2: Pre-render (bake) a new page cache immediately after expiry?

My full solution was to adopt the convention that if cache=regen is in a requested URL, that means that the generated page should be stuffed into the cache.

For controllers for which I want caching, include CacheRegen. CacheRegen causes the controller to not read from cache when cache=regen, and to not put cache=regen into the key when storing in the cache.

The code for which is:

module CacheRegen
  def read_fragment(key, options = nil)
    if /cache=regen/.match(key)
      logger.info("forcing cache miss due to param cache=regen, key=#{key}")
      return nil
    end
    super(key, options)
  end

  def write_fragment(key, content, options = nil)
    unless key.sub!(/cache=regen/, '').nil?
      key.sub!(/\?\&/, '?')
      key.sub!(/\&\&/, '&')
      key.sub!(/\?$/, '')
      key.sub!(/\&$/, '')
      logger.info("wrote page to cache with key #{key}")
    end
    super(key, content, options)
  end
end

Finally, I put the following code in new_pages.rake:

require 'action_dispatch'

def get_url(sess, url)
      uri = "http://YOURSITE.com/" + url + "cache=regen"
      puts "retrieving " + uri
      foo = sess.get(uri)
      puts "got it. #{foo}, #{sess.response.body.length} bytes"
end

desc "If necessary, generate new versions of the most expensive pages"
task :new_pages => :environment do
    puts "Updating pages..."
    sess = ActionDispatch::Integration::Session.new(Rails.application)
    ["controller1", "controller2", "controller3"].each { |noun|
      get_url(sess, noun + "?")
    }

    puts "done."
end

And in my environment, I have a deploy task which depends on the new_pages task.

Is there some gem out there that makes all this more automagical?

score -1 · Answer 3 · answered Mar 05 '12 at 17:18

-1

typical solution for cache invalidation in rails is sweepers. It's technic for automatic cache invalidation. you can read more about sweepers in this guide

answered Mar 05 '12 at 17:18

VVN

144
1
3

2

sweepers streamline the process of invalidation, but don't appear to help in re-population of the cache, which is what this question is about. – dsjoerg Mar 05 '12 at 17:33

Rails: how best to re-populate cache when invalidating

3 Answers3