Issue in reading JSON file in python

Question

>>> import json
>>> d2 = json.loads(open("t.json").read())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/json/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.6/json/decoder.py", line 319, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.6/json/decoder.py", line 336, in raw_decode
    obj, end = self._scanner.iterscan(s, **kw).next()
  File "/usr/lib64/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib64/python2.6/json/decoder.py", line 185, in JSONObject
    raise ValueError(errmsg("Expecting object", s, end))
ValueError: Expecting object: line 1 column 11 (char 11)
[ RHEL - ~/testing ]$ cat t.json
{"us": u"OFF", "val": u"5"}

Here is what I have in my JSON file and when I try to read it using open and json.load and json.loads it fails.

After using json.load

>>> import json
>>> d2 = json.load(open("t.json"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/json/__init__.py", line 267, in load
    parse_constant=parse_constant, **kw)
  File "/usr/lib64/python2.6/json/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.6/json/decoder.py", line 319, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.6/json/decoder.py", line 336, in raw_decode
    obj, end = self._scanner.iterscan(s, **kw).next()
  File "/usr/lib64/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib64/python2.6/json/decoder.py", line 185, in JSONObject
    raise ValueError(errmsg("Expecting object", s, end))
ValueError: Expecting object: line 1 column 11 (char 11)
>>>

you have the answer in your own question: `TypeError: expected string or buffer`. try to read error message first. — Jason Hu, Mar 12 '15 at 13:30
@HuStmpHrrr i have updated the output for json.load , sorry for missing it earlier — pkm, Mar 12 '15 at 13:39
this is what i have got as return value from api call , anyway to parse it — pkm, Mar 12 '15 at 14:39

Martijn Pieters · Answer 1 · 2015-03-12T15:13:32.920

2

You are using the wrong function. Use json.load() (no s!) to load data from an open file object:

d2 = json.load(open("t.json"))

The json.loads() function expects you to pass in a string, not a file object. You'd have to read your file in that case, returning the read data:

d2 = json.loads(open("t.json").read())

Next, you have invalid JSON in that file:

{"us": u"OFF", "val": u"5"}
#      ^              ^

JSON is not Python; those u prefixes are not supported nor needed. You'll need to remove those from the file before it'll load.

If you have an API producing that format, it is not giving you JSON. It could be that it is producing a (strange form of) Python syntax instead; Python itself would produce {'us': u'OFF', 'val': u'5'} (single quotes). You can have Python interpret that as Python literals with ast.literal_eval():

import ast

with open('t.json') as fileobj:
    d2 = ast.literal_eval(fileobj.read())

but it could be that the format is broken in other ways we cannot determine from a single isolated sample. It could be using true and false for boolean values, like in JSON, for example.

Better to have the API fixed rather that try and work around this broken-ness.

edited Mar 12 '15 at 15:13

answered Mar 12 '15 at 13:28

Martijn Pieters

1,048,767
296
4,058
3,343

i have updated the output for json.load , sorry for missing it earlier – pkm Mar 12 '15 at 13:37
You don't have valid JSON in your file, however. – Martijn Pieters Mar 12 '15 at 13:46
this is what i have got as return value from api call , anyway to parse it – pkm Mar 12 '15 at 14:41
@mkp: any way to fix the API instead? You can try and interpret it as Python literals instead: `import ast` then `d2 = ast.literal_eval(open('t.json').read())` but that's slower and may break if there are nulls or booleans. – Martijn Pieters Mar 12 '15 at 14:45
will it work if the data have much more than simple two element , like having multiple layered dict type format of data – pkm Mar 12 '15 at 14:53
@mkp: `ast.literal_eval()` will work with valid Python syntax, however deeply nested. – Martijn Pieters Mar 12 '15 at 15:50

score 0 · Answer 2 · answered Mar 12 '15 at 13:31

0

You are using the json.loads method. More documentation here. This method is used for string arguments only. Luckily, there is a similarly named json.load method documented here. This one can be used directly on a file object.

d2 = json.load(open("t.json"))

answered Mar 12 '15 at 13:31

backtrack

7,996
5
52
99

i have updated the output for json.load , sorry for missing it earlier – pkm Mar 12 '15 at 13:35

score 0 · Answer 3 · answered Mar 12 '15 at 13:43

0

Your issue is that the JSON is not valid.

It looks like it is a python dictionnary. u'string' is a python 2 unicode string.

If you remove the u from your strings, it works fine.

>>> import json
>>> json.load(open('i.json'))
{u'val': u'5', u'us': u'OFF'}

Here is the json file:

$ cat i.json
{"us": "OFF", "val": "5"}

answered Mar 12 '15 at 13:43

Paco

4,520
3
29
53

this is what i have got as return value from api call , anyway to parse it – pkm Mar 12 '15 at 14:39

Issue in reading JSON file in python

3 Answers3