Read JSON Lines file as String in Python

Question

I have a JSON Lines file that I would like to read as a string in Python. The file contains multiple JSON objects in this format:

{"Data1": "Value1"}
{"Data2": "Value2"}
{"Data3": "Value3"}

I tried the following code in Python but it returned an error. I was able to load the file as a list of dictionaries using lines = [] but apprently it doesn't work for a string. How can I read the whole file as a string?

import json

lines = ''

with open('file.json', 'r') as f:
    for line in f:
        lines.append(json.loads(line))

Please share the contents of `links.jl` (or at least a relevant part of it) in your question. — Grismar, Jan 06 '22 at 03:41
Can you explain, in your own words, what you think `json.loads()` does? Why are you trying to load a single line of JSON (which should fail as a single line wouldn't necessarily be syntactically-valid JSON) and subsequently append the resulting object to a string? Why not just read the entire file in with `readlines()` then use `json.loads()` on the resulting string to serialize it into Python data structures? It's really not clear to me where you got the idea that this methodology should work - can you link to the example or documentation on which you're basing your attempt here? — esqew, Jan 06 '22 at 03:46
and then load a string once, something like: `json_str=‘[‘ + f.read().replace(‘\n’, ‘,\n’) + ‘]’; json.loads(json_str)` — rv.kvetch, Jan 06 '22 at 03:46
@esqew well he mentioned it’s in jsonl format, where each line is basically a valid map type. I mean it’s not valid jaon as a whole, since there’s no open close braces [] in the file presumably. — rv.kvetch, Jan 06 '22 at 03:49
@esqew Since the file is in JSONL format, it needs to be parsed line by line as explained here: https://stackoverflow.com/questions/12451431/loading-and-parsing-a-json-file-with-multiple-json-objects. I wanted to achieve the same but loading the file as a string as opposed to a list. — elideli, Jan 06 '22 at 03:55

score 2 · Answer 1 · answered Jan 06 '22 at 04:01

2

You can simply use f.read() if you aren't worried about the memory usage in case of large files or else your implementation seems fine, except the part which you are trying to use append() with string. This can be achieved through simple modification

lines = ""
with open("links.jl", "r") as f:
    for line in f:
        lines += line

answered Jan 06 '22 at 04:01

W4RR10R

101
1
4

This worked! Can you explain what `lines += line` is doing? – elideli Jan 06 '22 at 04:21
It will add both operands if they're integers or concatenate them if they're strings and assign the result to the left side operand, which can also be written as `lines = lines + line` – W4RR10R Jan 06 '22 at 06:59

score 0 · Accepted Answer · answered Jan 09 '22 at 04:17

0

The best way to read a JSON Lines document as a string would be to use the read() function as follow:

with open("file.json", "r") as file:
    data_str = file.read()

answered Jan 09 '22 at 04:17

elideli

179
3
15

score -1 · Answer 3 · edited Jan 06 '22 at 20:18

-1

you can use following template to load your JSON string into the DataFram

import pandas as pd
df = pd.read_json (r'C:\Users\Ron\Desktop\data.json')
print (df)

edited Jan 06 '22 at 20:18

S4eed3sm

1,398
5
20

answered Jan 06 '22 at 04:27

Vineet

70
7

it should be the following for json lines file: df = pd.read_json('C:\Users\Ron\Desktop\data.json', lines=True) – Suben Saha Jan 01 '23 at 10:31

Read JSON Lines file as String in Python

3 Answers3