1

I have a json file, it was generated using rest api call as below:

import json
resp = requests.get('https://....offset=0&limit=500&where=....', headers=headers)
json_data = json.loads(resp.text)

with open('strings.json') as f:
    d = json.load(f)
    print(d)

After reading it in variable d, I get following structure data:

d = [{}, {}, {},{}]

I want to convert above data structure into following format and store it in json file which is also called as jsonlines https://jsonlines.org/examples/:

required_format = {}{}{}{}

In a way, I want to remove outer square brackets and also space, comma between adjacent dictionaries and then again store it in json or any other file format or even without extension. this file it should have this structure after opening it in the chrome or any other browser.

 {}{}{}{}
dan
  • 373
  • 2
  • 5
  • 24
  • How is the required_format a valid json? – shahkalpesh Mar 17 '22 at 18:48
  • `{}{}{}{}` is not a valid JSON, so you can't, end. Why do you nead that ? The purpose of JSON is to be handled by machines, so you don't care is there is spaces of not in the file, or whatever – azro Mar 17 '22 at 18:50
  • I need this format in order to have processed by glue job in aws. The glue job only handles it well if data is in this format, which is the case if it is generated by SNS topic. When it is generated by rest api, structure is different than the above. – dan Mar 17 '22 at 18:53
  • Then you could write this to a file: `str(d)[1:-1]` – RJ Adriaansen Mar 17 '22 at 18:55
  • @RJAdriaansenWould it not convert all the data in that to str format? it would be problematic to bring data in it to original form, which consist of ints, floats, and strings as these dictionaries are nested. – dan Mar 17 '22 at 18:57
  • This required format isn't JSON anymore though – Luatic Mar 17 '22 at 18:57
  • No you want to break the json, so you end up with a string of broken json that you can convert back in a hacky way if you need to. But I'd just stick to the json format – RJ Adriaansen Mar 17 '22 at 18:58
  • @azro But would this be possible to store this data in required format in the file (with no extension), does not need to be json may be? – dan Mar 17 '22 at 19:01
  • @RJAdriaansenCould you write sample code, if required final file does not need to be json, as long as I get required data format. – dan Mar 17 '22 at 19:03
  • I have updated the questions detials. – dan Mar 17 '22 at 19:04
  • 1
    It sounds like a poor description of [JSON Lines](https://jsonlines.org) format. – Mark Tolonen Mar 17 '22 at 19:06
  • thank you very much for giving this link, wanted to know what it was called. – dan Mar 17 '22 at 19:08
  • 1
    @MarkTolonen after findind proper name to this format, found out there is already anwer available to this question. – dan Mar 17 '22 at 19:20
  • Pretty much a duplicate for https://stackoverflow.com/questions/32242647/pattern-for-saving-newline-delimited-json-aka-linejson-jsonlines-jsonl-files isn't it? I'd suggest to remove the question. – N1ngu May 24 '22 at 23:07

3 Answers3

5
[{}, {}, {},{}]

is valid JSON.

{}{}{}{}

is not. It is a different, JSON-based serialization format ("JSON lines" using lines as delimiters apparently). You should be able to simply generate this using a list comprehension and some JSON serialization for the objects:

"".join([json.dumps(obj) for obj in d])

This will yield a string. If you want to write directly to a file, use json.dump:

with f as io.open(..., "w"):
    for obj in d:
        json.dump(obj, f)

Edit: As the desired format is apparently the "JSON lines" format, just emit newlines when writing to a file or concat using them when stringifying (do you want a trailing newline BTW?). The code then becomes the following:

"\n".join([json.dumps(obj) for obj in d])

and

with f as io.open(..., "w"):
    for obj in d:
        json.dump(obj, f)
        f.write("\n")

respectively.

Luatic
  • 8,513
  • 2
  • 13
  • 34
2

A jsonlines python library exists, so you could go ahead and try to use that.

import json
import jsonlines

with open('strings.json', 'r') as f:
    d = json.load(f)

with jsonlines.open('strings.jsonl', 'w') as f:
    f.write_all(d)

See https://jsonlines.readthedocs.io/en/latest/#user-guide

N1ngu
  • 2,862
  • 17
  • 35
1

The simplest way of doing it:

import json

with open('strings.json', 'r') as f:
    d = json.load(f)

output_path = "your/output/path/result.jsonl" 
with open(output_path, 'w') as f:
    for line in d:
        f.write(json.dumps(line) + '\n')
Tau n Ro
  • 108
  • 8