Composing json from cached strings in ruby

Question

Consider the following scenario, I have quite a few big hashes that I need to put in an array and then convert it to json:

hash1 = { ... big hash ... }
hash2 = { ... big hash ... }
hash3 = { ... big hash ... }
array = [hash1, hash2, hash3]

json = JSON.dump(array)

The problem is that generating json from those hashes takes a long time, so I would like to cache it. However, I can't cache the whole array, only separate items. Obviously putting cached json string in array gives bad results:

hash1 = {:a => 1}
hash1json = JSON.dump(hash1)
array = [hash1json]
JSON.generate(array)
==> ["{\"a\":1}"]

while I need

==> [{"a":1}]

The only way I can think of is doing something like this:

"[#{[hash1json].join(",")}]"
==> [{"a":1}]

which might be enough for this specific case, but it will be much harder if one wanted to cache some deep structure instead of simple array.

I checked my previous answer with JSON.dump, and it didn't work - sorry! — Neil Slater, Mar 07 '13 at 20:13

score 2 · Accepted Answer · answered Apr 23 '13 at 22:37

Turns out this is actually dead simple:

class CachedJson
  def initialize(str)
    @str = str
  end

  def to_json
    @str
  end
end

puts Yajl::Encoder.encode(:data => [{:a => 1}, '{"b":2}'])
# => {"data":[{"a":1},"{\"b\":2}"]}

puts Yajl::Encoder.encode(:data => [{:a => 1}, CachedJson.new('{"b":2}')])
# => {"data":[{"a":1},{"b":2}]}

Under the hood yajl calls to_json on every object, and this method must return string, so it's just a matter of wrapping cached json string with CachedJson object

fmendez · Answer 2 · 2013-03-07T21:38:29.083

0

EDIT

My previous answer missed the performance aspect of the question completely (sorry about that), so this is my finding. Perhaps it can help you a little.

Apparently in these situations using yajl-ruby, which is a binding to the C yajl library, seems to improve performance while doing the transformation. For instance, here I'm generating a hash with 10,000 entries:

  require 'json'
  require 'yajl'
  require 'benchmark'
  tmp = "" 
  10000.times do |i|
   tmp += "\"#{i}\" => \"#{i}\", " 
  end

 domains = eval("{#{tmp}}")

 puts "JSON DUMP #{Benchmark.measure { JSON.dump(domains) }} "

 puts "Yajl::Encoder #{Benchmark.measure { Yajl::Encoder.encode(domains)}}"

And these are the results:

JSON DUMP   0.010000   0.000000   0.010000 (  0.007495)

Yajl::Encoder   0.000000   0.000000   0.000000 (  0.003542)

Consistenly im halving the time for the task of transforming to json. Hope it helps!

edited Mar 07 '13 at 21:38

answered Mar 07 '13 at 21:03

fmendez

7,250
5
36
35

What is the point of parsing cached json string just to convert it to json again? I'm talking about performance optimisation here. – teamon Mar 07 '13 at 21:04
@teamon Sorry about that, I updated my answer. Hope is a bit more helpful. – fmendez Mar 07 '13 at 21:39
Thanks for the effort, but no, it doesn't help at all. I'm already using yajl (I have tested all available json libs for ruby) and the only way to make things faster is to cache hashes as rendered json. I'm already caching hashes, but that is not enough. – teamon Mar 07 '13 at 21:47

Composing json from cached strings in ruby

2 Answers2