Avoid repeated calls to an API in Jekyll Ruby plugin

Question

I have written a Jekyll plugin to display the number of pageviews on a page by calling the Google Analytics API using the garb gem. The only trouble with my approach is that it makes a call to the API for each page, slowing down build time and also potentially hitting the user call limits on the API.

It would be possible to return all the data in a single call and store it locally, and then look up the pageview count from each page, but my Jekyll/Ruby-fu isn't up to scratch. I do not know how to write the plugin to run once to get all the data and store it locally where my current function could then access it, rather than calling the API page by page.

Basically my code is written as a liquid block that can be put into my page layout:

 class GoogleAnalytics < Liquid::Block
    def initialize(tag_name, markup, tokens)
      super # options that appear in block (between tag and endtag)
      @options = markup # optional optionss passed in by opening tag
    end
    def render(context)
      path = super
      # Read in credentials and authenticate 
      cred = YAML.load_file("/home/cboettig/.garb_auth.yaml")
      Garb::Session.api_key = cred[:api_key]
      token = Garb::Session.login(cred[:username], cred[:password])
      profile = Garb::Management::Profile.all.detect {|p| p.web_property_id == cred[:ua]}

      # place query, customize to modify results
      data = Exits.results(profile, 
                           :filters => {:page_path.eql => path}, 
                           :start_date => Chronic.parse("2011-01-01"))

      data.first.pageviews
    end

Full version of my plugin is here

How can I move all the calls to the API to some other function and make sure jekyll runs that once at the start, and then adjust the tag above to read that local data?

EDIT Looks like this can be done with a Generator and writing the data to a file. See example on this branch Now I just need to figure out how to subset the results: https://github.com/Sija/garb/issues/22

Why reference google at all? Store your page counts locally. They'll be more accurate anyway. — NotMe, Mar 05 '13 at 22:05
I'm afraid you'll have to clarify. I'm using google analytics to record pageviews in the first place. How else would I do that (that would not be functionally equivalent?) Any method would have to get the current number of pageviews when Jekyll runs, and then add it to each page, so the problem remains the same. — cboettig, Mar 05 '13 at 22:14
Actually, the problem is different. The problem you have is that you are pulling from google on every page load. By just storing them locally you accomplish two things. First, you'll have slightly more accurate data; but more importantly if you still query on every page load then it's no big deal. As it stands if you merge the two: only query google on startup but maintain a local cache, you will see your numbers diverge slightly. — NotMe, Mar 05 '13 at 22:45
No I am not. Jekyll is a *static* site generator. Please see https://github.com/mojombo/jekyll/ for background. the data is only as current as the last build, which is fine for me. I want to store data at time of build to speed building and reduce API calls. — cboettig, Mar 05 '13 at 23:15

score 0 · Accepted Answer · edited Mar 25 '16 at 09:02

To store the data, I had to:

Write a Generator class (see Jekyll wiki plugins) to call the API.

Convert data to a hash (for easy lookup by path, see 5):

result = Hash[data.collect{|row| [row.page_path, [row.exits, row.pageviews]]}]

Write the data hash to a JSON file.
Read in the data from the file in my existing Liquid block class.

Note that the block tag works from the _includes dir, while the generator works from the root directory.
Match the page path, easy once the data is converted to a hash:
```
result[path][1]
```

Code for the full plugin, showing how to create the generator and write files, etc, here

And thanks to Sija on GitHub for help on this.

Your problem seems to be related to a problem I currently have (see [this comment](https://gist.github.com/yeban/2290195#comment-966668)). Do you know how to fix it? — Martin Thoma, Dec 10 '13 at 13:08

Avoid repeated calls to an API in Jekyll Ruby plugin

1 Answers1

Linked