Deserialize JSON primitives with the built-in Ruby JSON library

Question

Why can Ruby's built-in JSON not deserialize simple JSON primitives, and how do I work around it?

irb(main):001:0> require 'json'
#=> true

irb(main):002:0> objects = [ {}, [], 42, "", true, nil ]
#=> [{}, [], 42, "", true]

irb(main):012:0> objects.each do |o|
irb(main):013:1*   json = o.to_json
irb(main):014:1>   begin
irb(main):015:2*     p JSON.parse(json)
irb(main):016:2>   rescue Exception => e
irb(main):017:2>     puts "Error parsing #{json.inspect}: #{e}"
irb(main):018:2>   end
irb(main):019:1> end
{}
[]
Error parsing "42": 706: unexpected token at '42'
Error parsing "\"\"": 706: unexpected token at '""'
Error parsing "true": 706: unexpected token at 'true'
Error parsing "null": 706: unexpected token at 'null'
#=> [{}, [], 42, "", true, nil]

irb(main):020:0> RUBY_DESCRIPTION
#=> "ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.7.0]"
irb(main):022:0> JSON::VERSION
#=> "1.4.2"

Answer 1

RFC 4627: The application/json Media Type for JavaScript Object Notation (JSON) has this to say:

2.  JSON Grammar

   A JSON text is a sequence of tokens.  The set of tokens includes six
   structural characters, strings, numbers, and three literal names.

   A JSON text is a serialized object or array.

      JSON-text = object / array

[...]

2.1.  Values

   A JSON value MUST be an object, array, number, or string, or one of
   the following three literal names:

      false null true

If you call to_json on your six sample objects, we get this:

>> objects = [ {}, [], 42, "", true, nil ]
>> objects.map { |o| puts o.to_json }
{}
[]
42
""
true
null

So the first and second are valid JSON texts whereas the last four are not valid JSON texts even though they are valid JSON values.

JSON.parse wants what it calls a JSON document:

Parse the JSON document source into a Ruby data structure and return it.

Perhaps JSON document is the library's term for what RFC 4627 calls a JSON text. If so, then raising an exception is a reasonable response to an invalid input.

If you forcibly wrap and unwrap everything:

objects.each do |o|
    json = o.to_json 
    begin
        json_text = '[' + json + ']'
        p JSON.parse(json_text)[0]
    rescue Exception => e 
        puts "Error parsing #{json.inspect}: #{e}"    
    end    
end

And as you note in your comment, using an array as the wrapper is better than an object in case the caller wants to use the :symbolize_names option. Wrapping like this means that you'll always be feeding JSON.parse a JSON text and everything should be fine.

Answer 2

This is quite an old question but I think it worths to have a proper answer to prevent hair loss for the ones who just encountered with the problem and still searching for a solution :)

To be able to parse "JSON primitives" with JSON gem below version 2, you can pass quirks_mode: true option like so;

JSON::VERSION # => 1.8.6

json_text = "This is a json primitive".to_json
JSON.parse(json_text, quirks_mode: true)

With the JSON gem version greater or equals to 2, the quirks_mode is not necessary anymore.

JSON::VERSION # => 2.0.0

json_text = "This is a json primitive".to_json
JSON.parse(json_text)

Before parsing the JSON, you can check the version of the JSON gem that you are using in your project with bundle show json or gem list | grep json and then use the corresponding one.

Happy JSON parsing!

Answer 3

Use JSON.load instead of JSON.parse to handle primitives:

e.g.

JSON.load('true') # => true
JSON.load('false') # => false
JSON.load('5150') # => 5150
JSON.load('null') # => nil

Answer 4

It appears that the built-in JSON parser intentionally fails on anything but objects and arrays. My current workaround is the following:

# Work around a flaw in Ruby's built-in JSON parser
# not accepting anything but an object or array at the root level.
module JSON
  def self.parse_any(str,opts={})
    parse("[#{str}]",opts).first
  end
end

Answer 5

I think you are right...whether it is a bug or not, there is some wonky logic going on with the implementation. If it can parse arrays, and hashes it should be able to parse everything else.

Because JSON.parse seems geared for objects and arrays, I would try to pass your data one of those ways if you can, and if you can't, stick with the workaround you have.