1

I have a huge JSON file in the list form

{data 1}
{data 2}
{data 3}
...

I don't want to load the whole file but only each part one by one to parse, i.e. I first want to parse data1, then data2, ...

I know json.loads() in Python 3 but it seems to me that it needs an entire JSON string from the file.

mommomonthewind
  • 4,390
  • 11
  • 46
  • 74
  • In `.load`, [the source](https://github.com/python/cpython/blob/3.7/Lib/json/__init__.py) shows it indeed uses `return loads(fp.read())` – it reads the whole file at once. But (being unfamiliar with JSON *at all*) isn't it a problem that the *root object* must be parsed in its entirety anyway? You cannot do that if you can't read it entirely first. – Jongware Nov 21 '18 at 09:51
  • 1
    In this case I know that the JSON file is stored in the list form, i.e. there is no root object. Not sure if I can integrate this knowledge to read each object one by one? – mommomonthewind Nov 21 '18 at 10:17
  • Is each JSON object on a separate line? Then you could just parse each line separately with `json.loads`. – Janne Karila Nov 21 '18 at 10:30
  • Hi @JanneKarila: no, each object is in multiple lines. – mommomonthewind Nov 21 '18 at 10:34
  • 1
    See https://stackoverflow.com/questions/6886283/how-i-can-i-lazily-read-multiple-json-values-from-a-file-stream-in-python – Janne Karila Nov 21 '18 at 13:30
  • If @JanneKarila's (very good) suggestion works, we can close this as a duplicate. – Jongware Nov 21 '18 at 22:06
  • Yes I think it is a duplicated one. – mommomonthewind Nov 22 '18 at 07:43

0 Answers0