0

i want to use jq to split a very large json (>80GB) file into smaller parts (<1GB or a fixed number of lines). I have the necessary statements together... I thought. What do I do so far?

jq ". | length" z:\DOWNLOAD\rows.json

works!

Under Windows this should output the first two lines.

jq ".[0:1]" z:\DOWNLOAD\rows.json

but I'm getting an error

jq: error (at z:\DOWNLOAD\rows.json:589): Cannot index object with object

What I also haven't understood the --Steam switch

yes, there are a bunch answers, but they do not work under windows (double quotes instead of apostrophe, but see error above)

[{"node":"http://www.wikidata.org/entity/Q952111","Unterklasse_von":"http://www.wikidata.org/entity/Q2095"},{"node":"http://.....
ozz
  • 175
  • 1
  • 11
  • 1
    Since `jq length` works, `jq type` should also work and may explain why you are seeing the reported error. Also, since `jq length` works, chances are you won't need the --stream option, but it would probably be helpful to see a snippet that's representative of your JSON. – peak May 26 '20 at 22:24
  • You may use `split` command to get it done. Check this https://stackoverflow.com/questions/34609650/how-to-split-file-in-windows-just-like-linux – Ajay Kr Choudhary May 27 '20 at 09:10
  • this splitting does not respect the structure of a json file. the splitting is done line by line. this may only work with jsonl files. but unfortunately the file is not a.... ;-( – ozz May 27 '20 at 10:55

0 Answers0