What is wrong with my JSON object?

Question

I've written a JSON object (generated by someone else, in code I have no access to) to a file named kommscache.json, and now I'm trying to read it in again in Python.

This is what I do:

import json
from pprint import pprint

with open('kommscache.json') as data_file:
    data = json.load(data_file)

pprint(data)

On the call to json.load(), I get the following error message:

Traceback (most recent call last):
  File "./kladd.py", line 7, in <module>
    data = json.load(data_file)
  File "/usr/lib/python2.7/json/__init__.py", line 280, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 328, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 381, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 1 column 1 (char 1)

The first few lines of the JSON file look like this (I've tried a non-prettyprinted version too, with the same result):

{u'filtered': 458,
 u'items': [{u'comment_count': 0,
             u'current_revision': {u'created_by': {u'avatar': 19435601,

Unfortunately, I can't show you the entire file because it contains some sensitive data (and it's over 6000 lines long...), but if I'm reading the error message correctly the error is already at the start of the file. However, I can't see why this JSON syntax isn't correct. I've double-checked, and all the braces and brackets in this snippet have matching closing ones.

Why can't I load this JSON object?

@JensSchauder: Those are Python unicode string literals. `u''` creates a unicode string, vs. `''` creating a byte string. This is Python, not JSON. — Martijn Pieters, Mar 05 '13 at 13:25

score 3 · Answer 1 · answered Mar 05 '13 at 13:22

3

You saved the literal representation of python objects, not JSON.

You can decode those with ast.literal_eval() but you may want to fix your JSON writing code instead. Make sure you use json.dumps() on your structure when saving.

Using ast.literal_eval() (after closing all the braces):

>>> import ast
>>> ast.literal_eval('''\
... {u'filtered': 458,
...  u'items': [{u'comment_count': 0,
...              u'current_revision': {u'created_by': {u'avatar': 19435601,
... }}}]}''')
{u'filtered': 458, u'items': [{u'comment_count': 0, u'current_revision': {u'created_by': {u'avatar': 19435601}}}]}

answered Mar 05 '13 at 13:22

Martijn Pieters

1,048,767
296
4,058
3,343

Don't let him get such ideas... – Markus Unterwaditzer Mar 05 '13 at 13:35
@MarkusUnterwaditzer: Why not? `ast.literal_eval()` is perfectly safe. It won't perform as well as a JSON de-serialisation, but there are no security risks. Like JSON, only primitive python types are supported. It'll even support numeric keys (which JSON doesn't). – Martijn Pieters Mar 05 '13 at 13:36
That's not my point. I just think it's a bad idea to use it for data serialization. – Markus Unterwaditzer Mar 05 '13 at 13:39
@MarkusUnterwaditzer: *Why* do you think it is a bad idea? I happen to agree, but a motivation would be interesting. The primary problem of course is that the data is not serialized as JSON, but if the OP has no other means to fix the saving process or needs to recover that data then `ast.literal_eval` is a great way to load the data again. – Martijn Pieters Mar 05 '13 at 13:43
Actually, I might have been a little misleading there - I do have control over printing to the file, just not over generating the object. Since I thought the object was already in a JSON format, I assumed printing it to stdout and piping to a file was enough. Instead, I now use `json.dumps()` as you suggested and everything works. Thanks! – Tomas Aschan Mar 05 '13 at 13:56

score 2 · Accepted Answer · answered Mar 05 '13 at 13:23

2

That is not a JSON file. It appears to be a file created by simply writing the string version of a Python dictionary. Although the formats are similar, they are not the same.

You should show the code that creates the file. You need to use json.dump(data) rather than simply calling str().

answered Mar 05 '13 at 13:23

Daniel Roseman

588,541
66
880
895

Thanks! I was simply doing `pprint(myobj)` and piping the output to a file, but changing to `print json.dumps(myobj)` fixed the problem. – Tomas Aschan Mar 05 '13 at 13:55

What is wrong with my JSON object?

2 Answers2