Pretty-Print JSON Data to a File using Python

Question

A project for class involves parsing Twitter JSON data. I'm getting the data and setting it to the file without much trouble, but it's all in one line. This is fine for the data manipulation I'm trying to do, but the file is ridiculously hard to read and I can't examine it very well, making the code writing for the data manipulation part very difficult.

Does anyone know how to do that from within Python (i.e. not using the command line tool, which I can't get to work)? Here's my code so far:

header, output = client.request(twitterRequest, method="GET", body=None,
                            headers=None, force_auth_header=True)

# now write output to a file
twitterDataFile = open("twitterData.json", "wb")
# magic happens here to make it pretty-printed
twitterDataFile.write(output)
twitterDataFile.close()

Note I appreciate people pointing me to simplejson documentation and such, but as I have stated, I have already looked at that and continue to need assistance. A truly helpful reply will be more detailed and explanatory than the examples found there. Thanks

Also: Trying this in the windows command line:

more twitterData.json | python -mjson.tool > twitterData-pretty.json

results in this:

Invalid control character at: line 1 column 65535 (char 65535)

I'd give you the data I'm using, but it's very large and you've already seen the code I used to make the file.

I was taught this was necessary for Windows machines and thus far has worked for all of my assignments. If you can offer documentation as to why this might be incorrect, I'd be happy to look at it. — Zelbinian, Feb 07 '12 at 03:01
It's only necessary if you're working with binary files, or other cases where the specific form of line ending (e.g. `\r\n` vs `\n`) is important. See http://stackoverflow.com/questions/3257869/difference-between-binary-and-text-i-o-in-python-on-windows. In your case, you want windows friendly line endings, but you might not get that from the twitter endpoint, so you should open in text mode. — Hamish, Feb 07 '12 at 06:18
Does this answer your question? [How to prettyprint a JSON file?](https://stackoverflow.com/questions/12943819/how-to-prettyprint-a-json-file) — wesinat0r, Dec 23 '19 at 19:25

score 165 · Accepted Answer · edited May 09 '20 at 02:34

165

You should use the optional argument indent.

header, output = client.request(twitterRequest, method="GET", body=None,
                            headers=None, force_auth_header=True)

# now write output to a file
twitterDataFile = open("twitterData.json", "w")
# magic happens here to make it pretty-printed
twitterDataFile.write(simplejson.dumps(simplejson.loads(output), indent=4, sort_keys=True))
twitterDataFile.close()

edited May 09 '20 at 02:34

barshopen

1,190
2
15
28

answered Feb 07 '12 at 03:14

mattbornski

11,895
4
31
25

1

Thank you, that worked **perfectly**. Can you explain why "sort_keys" needs to be in there? – Zelbinian Feb 07 '12 at 03:22
1

It doesn't need to be there but it makes things very pretty and alphabetically sorted. I tend to use it when I want human readable output. – mattbornski Feb 07 '12 at 03:25
10

Well explained thank you -however not trying to be a &$ but open/close to write a file isn't encourage, the with structure is generally preferable: `with open("name_of_file.json", "w") as f: f.write(my_formatted_json_var)` Advantage being you're sure the file will close, say on bigger snippets... – logicOnAbstractions Mar 14 '16 at 21:41
2

`with` syntax is definitely nicer, but I try to scale my answers to my audience – mattbornski Mar 15 '16 at 20:54

dkamins · Answer 2 · 2012-02-07T21:45:25.047

128

You can parse the JSON, then output it again with indents like this:

import json
mydata = json.loads(output)
print json.dumps(mydata, indent=4)

See http://docs.python.org/library/json.html for more info.

edited Feb 07 '12 at 21:45

answered Feb 07 '12 at 02:41

dkamins

21,450
7
55
59

@Zelbinian: yes it works for simplejson as well.Take a look here http://simplejson.googlecode.com/svn/tags/simplejson-1.9.1/docs/index.html – RanRag Feb 07 '12 at 02:53
This results in an empty file. `header, output = client.request(twitterRequest, method="GET", body=None, headers=None, force_auth_header=True) twitterDataFile = open("twitterData.json", "wb") json.dumps(json.loads(output), twitterDataFile, indent=4) twitterDataFile.close()` – Zelbinian Feb 07 '12 at 02:58
7

@Zelbinian - `json.dumps` returns a string. `json.dump` writes to a file. – dkamins Feb 07 '12 at 17:31

score 114 · Answer 3 · answered Jan 28 '18 at 12:13

import json

with open("twitterdata.json", "w") as twitter_data_file:
    json.dump(output, twitter_data_file, indent=4, sort_keys=True)

You don't need json.dumps() if you don't want to parse the string later, just simply use json.dump(). It's faster too.

score 16 · Answer 4 · answered Feb 07 '12 at 02:42

16

You can use json module of python to pretty print.

>>> import json
>>> print json.dumps({'4': 5, '6': 7}, sort_keys=True, indent=4)
{
    "4": 5,
    "6": 7
}

So, in your case

>>> print json.dumps(json_output, indent=4)

answered Feb 07 '12 at 02:42

RanRag

48,359
38
114
167

Tried that route and that unfortunately doesn't work as well as you'd think. – Zelbinian Feb 07 '12 at 02:50
@Zelbinian: Exactky what do you mean by `doesn't work as well`.? – RanRag Feb 07 '12 at 02:54
1

It outputted the data in a single line in what looked to be Python dict syntax instead of pretty-printed Json syntax – Zelbinian Feb 07 '12 at 02:56
Include the output in you question as an edit.So,we can see it. – RanRag Feb 07 '12 at 02:57
using this, arrays are listed out as many lines of each value, it'd be nice to keep the array on one line. – scape Dec 15 '12 at 02:07

score 9 · Answer 5 · answered Sep 16 '19 at 05:53

If you are generating new *.json or modifying existing josn file the use "indent" parameter for pretty view json format.

import json
responseData = json.loads(output)
with open('twitterData.json','w') as twitterDataFile:    
    json.dump(responseData, twitterDataFile, indent=4)

score 4 · Answer 6 · answered Feb 21 '19 at 14:50

4

If you already have existing JSON files which you want to pretty format you could use this:

    with open('twitterdata.json', 'r+') as f:
        data = json.load(f)
        f.seek(0)
        json.dump(data, f, indent=4)
        f.truncate()

answered Feb 21 '19 at 14:50

locke14

1,335
3
15
36

score 3 · Answer 7 · answered Dec 02 '19 at 20:38

3

import json
def writeToFile(logData, fileName, openOption="w"):
  file = open(fileName, openOption)
  file.write(json.dumps(json.loads(logData), indent=4)) 
  file.close()

answered Dec 02 '19 at 20:38

Peter

202
2
5

While this code may answer the question, providing additional context regarding *why* and/or *how* this code answers the question improves its long-term value. – Tân Dec 03 '19 at 03:31

score -1 · Answer 8 · edited Aug 28 '18 at 04:48

-1

You could redirect a file to python and open using the tool and to read it use more.

The sample code will be,

cat filename.json | python -m json.tool | more

edited Aug 28 '18 at 04:48

OneCricketeer

179,855
19
132
245

answered Jun 04 '15 at 21:26

Ramya

21
4

Pretty-Print JSON Data to a File using Python

8 Answers8

Linked

Related