Reading JSON from a file

Question

A simple looking, easy statement is throwing some errors in my face.

I have a JSON file called strings.json like this:

"strings": [{"-name": "city", "#text": "City"}, {"-name": "phone", "#text": "Phone"}, ...,
            {"-name": "address", "#text": "Address"}]

I want to read the JSON file, just that for now. I have these statements which I found out, but it's not working:

import json
from pprint import pprint

with open('strings.json') as json_data:
    d = json.loads(json_data)
    json_data.close()
    pprint(d)

The error displayed on the console was this:

Traceback (most recent call last):
  File "/home/.../android/values/manipulate_json.py", line 5, in <module>
    d = json.loads(json_data)
  File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer
[Finished in 0.1s with exit code 1]

If I use json.load instead of json.loads, I get this error:

Traceback (most recent call last):
  File "/home/.../android/values/manipulate_json.py", line 5, in <module>
    d = json.load(json_data)
  File "/usr/lib/python2.7/json/__init__.py", line 278, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 369, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 829 column 1 - line 829 column 2 (char 18476 - 18477)
[Finished in 0.1s with exit code 1]

See also: [Read & Write example for JSON](http://stackoverflow.com/a/37795053/562769) — Martin Thoma, Mar 09 '17 at 10:50
Your file is an invalid json format. Change it to: `{"strings": [{"-name": "city", "#text": "City"}, {"-name": "phone", "#text": "Phone"}, ..., {"-name": "address", "#text": "Address"}]}` — krizex, Nov 27 '18 at 09:11
...i.e. the thing that's invalid about the input is that it's missing the surrounding {} braces. — smci, Mar 11 '22 at 20:22
If you are reading the data from the Internet instead, the same techniques can generally be used with the response you get from your HTTP API (it will be a file-like object); however, it is heavily recommended to use the third-party Requests library instead, which includes built-in support for JSON requests. See https://stackoverflow.com/questions/6386308/http-requests-and-json-parsing-in-python for details; note that multiple answers all mention Requests. — Karl Knechtel, Jul 02 '22 at 02:12
Do we really need the `json_data.close()` here? It would be called automatically in the end of the scope of `with` statement. — Vlad Nikiforov, Dec 06 '22 at 12:47
Someone accidentally showed me a much better canonical today, which I further improved by editing the question for style, cleaning up the title, and adding my own comprehensive answer. The question is here: [How can I parse and use JSON?](/questions/7771011) and it is properly **focused** on dealing with compliant data. I closed this as a duplicate because it is not suitable for use as a canonical, since it conflates two issues - the actual reading and parsing, and a defect in the data. We are still missing a good canonical to explain common problems in JSON data. — Karl Knechtel, Jan 19 '23 at 05:25

score 805 · Accepted Answer · edited Jul 05 '22 at 16:00

The json.load() method (without "s" in "load") can read a file directly:

import json

with open('strings.json') as f:
    d = json.load(f)
    print(d)

You were using the json.loads() method, which is used for string arguments only.

The error you get with json.loads is a totally different problem. In that case, there is some invalid JSON content in that file. For that, I would recommend running the file through a JSON validator.

There are also solutions for fixing JSON like for example How do I automatically fix an invalid JSON string?.

score 121 · Answer 2 · edited Jul 05 '22 at 16:01

121

Here is a copy of code which works fine for me,

import json

with open("test.json") as json_file:
    json_data = json.load(json_file)
    print(json_data)

with the data

{
    "a": [1,3,"asdf",true],
    "b": {
        "Hello": "world"
    }
}

You may want to wrap your json.load line with a try catch, because invalid JSON will cause a stacktrace error message.

edited Jul 05 '22 at 16:01

Peter Mortensen

30,738
21
105
131

answered Nov 25 '13 at 18:46

user1876508

12,864
21
68
105

score 49 · Answer 3 · edited Jul 05 '22 at 16:01

49

The problem is using the with statement:

with open('strings.json') as json_data:
    d = json.load(json_data)
    pprint(d)

The file is going to be implicitly closed already. There is no need to call json_data.close() again.

edited Jul 05 '22 at 16:01

Peter Mortensen

30,738
21
105
131

answered Sep 22 '14 at 22:38

Zongjun

627
5
8

3

to pretty-print, I had to use: `print(json.dumps(d,sort_keys=True,indent=2))` – Mike D Mar 14 '16 at 22:46
What about the invalid JSON input in the question? – Peter Mortensen Jul 05 '22 at 16:02

score 40 · Answer 4 · edited Jul 05 '22 at 16:14

In Python 3, we can use the below method.

Read from a file and convert to JSON

import json
from pprint import pprint

# Considering "json_list.json" is a JSON file

with open('json_list.json') as fd:
     json_data = json.load(fd)
     pprint(json_data)

The with statement automatically closes the opened file descriptor.

String to JSON

import json
from pprint import pprint

json_data = json.loads('{"name" : "myName", "age":24}')
pprint(json_data)

score 3 · Answer 5 · edited Jul 05 '22 at 16:15

3

To add on this, today you are able to use pandas to import JSON: pandas.read_json

You may want to do a careful use of the orient parameter.

edited Jul 05 '22 at 16:15

Peter Mortensen

30,738
21
105
131

answered Jan 17 '18 at 10:17

Ando Jurai

1,003
2
14
29

1

This answer would be better if you add code examples as well as the url... – James McCorrie Oct 15 '18 at 11:58

score 3 · Answer 6 · edited Jul 05 '22 at 21:20

3

You can use the Pandas library to read the JSON file.

import pandas as pd
df = pd.read_json('strings.json', lines=True)
print(df)

edited Jul 05 '22 at 21:20

Peter Mortensen

30,738
21
105
131

answered Mar 15 '20 at 04:35

drorhun

500
7
22

score -4 · Answer 7 · answered Mar 19 '21 at 15:09

-4

def read_JSON():
    with open("FILE PATH", "r") as i:
        JSON_data = i.read()
    print(JSON_data)

answered Mar 19 '21 at 15:09

Pushparaj

92
4

3

All this does is open the JSON file and read it into a variable, not parsing it; are you sure this is what OP wants? – TheTechRobo the Nerd Mar 19 '21 at 15:55
Yes, this will help to read JSON objects to utilize them as a request payload or for any other purpose. I am using this for my current API request implementation and it is working fine for me. – Pushparaj Mar 19 '21 at 22:59
If you have to parse the file, then `.read` would be a bad way of doing it, since you'll have to reimplement what the json module already does. – Gino Mempin Mar 20 '21 at 01:20
I think LOAD will reimplement the JSON with python dictionary but READ helps to load the JSON directly to the data frame – Pushparaj Mar 20 '21 at 12:34

Reading JSON from a file

7 Answers7

Linked

Related