Stack Overflow
Stack Overflow

Questions tagged [jsonlines]

JSON Lines is a format for storing structured data that may be processed one record at a time. It is a convenient format for storing structured data that may be processed one record at a time. It works well with Unix-style text processing tools and shell pipelines.

This text format is documented at http://jsonlines.org/.

156 questions
146
votes
6 answers

Loading and parsing a JSON file with multiple JSON objects

I am trying to load and parse a JSON file in Python. But I'm stuck trying to load the file: import json json_data = open('file') data = json.load(json_data) Yields: ValueError: Extra data: line 2 column 1 - line 225116 column 1 (char 232 -…
Pi_
  • 2,010
  • 5
  • 22
  • 24
74
votes
5 answers

Loading JSONL file as JSON objects

I want to load a JSONL file as JSON objects in python. Is there an easy way to do so?
MBT
  • 21,733
  • 19
  • 84
  • 102
52
votes
6 answers

Python conversion from JSON to JSONL

I wish to manipulate a standard JSON object to an object where each line must contain a separate, self-contained valid JSON object. See JSON Lines JSON_file = [{u'index': 1, u'no': 'A', u'met': u'1043205'}, {u'index': 2, u'no': 'B', …
LearningSlowly
  • 8,641
  • 19
  • 55
  • 78
42
votes
3 answers

How to use jq to output JSONL (one independent JSON object per line)

My request sounds trivial but I could not find a way to do it. I have as input an array of JSON objects: [ { "foo": 1, "bar": 2 }, { "foo": 3, "bar": 4 }, (...) ] and I want as output the JSONL…
giacecco
  • 661
  • 1
  • 5
  • 8
28
votes
4 answers

Convert JSON lines to JSON array using jq

Firstly, I'm new to jq, like 1 day new, I'm also new to JSON, I'm an SQL guy so I'm learning fast but can't get my head around this ... so please bear with me. I'm running Windows, using jq v1.5 on PowerShell. I have multiple JSON files downloaded…
JayBay2279
  • 395
  • 1
  • 3
  • 9
28
votes
5 answers

how to parse a large, Newline-delimited JSON file by JSONStream module in node.js?

I have a large json file, its is Newline-delimited JSON, where multiple standard JSON objects are delimited by extra newlines, e.g. {'name':'1','age':5} {'name':'2','age':3} {'name':'3','age':6} I am now using JSONStream in node.js to parse a…
user824624
  • 7,077
  • 27
  • 106
  • 183
24
votes
3 answers

Pandas dataframe to JSONL (JSON Lines) conversion

I need to convert pandas data frame to JSONL format. I couldn't find a good package to do it and tried to implement myself, but it looks a bit ugly and not efficient. For example, given a pandas df: label pattern 0 DRUG …
Arnold Klein
  • 2,956
  • 10
  • 31
  • 60
23
votes
3 answers

JSON lines Mime type

I want to know what Content-Type to set for JSON lines (http://jsonlines.org/)? I tried searching. Its not really application/json as the entire content is not JSON (each line is). Thanks
codesalsa
  • 882
  • 5
  • 18
19
votes
2 answers

Parsing JSON record-per-line with jq?

I've got a tool that outputs a JSON record on each line, and I'd like to process it with jq. The output looks something like…
Roger Lipscombe
  • 89,048
  • 55
  • 235
  • 380
16
votes
4 answers

Filter empty and/or null values with jq

I have a file with jsonlines and would like to find empty values. {"name": "Color TV", "price": "1200", "available": ""} {"name": "DVD player", "price": "200", "color": null} And would like to output empty and/or null values and their…
Fernando César
  • 681
  • 1
  • 8
  • 16
13
votes
1 answer

Create JSONL with Python

I can't figure out how to create JSONL using Python3. test = [{'a': 'b'}, {'a': 'b'}, {'a': 'b'}] with open("data.json", 'w') as f: for item in test: json.dump(item, f) with open("data.json") as f: for line in f: // only 1…
rix
  • 10,104
  • 14
  • 65
  • 92
13
votes
2 answers

Use jq to convert json array to jsonl format

I have json like this: [ {"one": 1}, {"two": 2}] and wish to convert it to this format: {"one": 1} {"two": 2} to facilitate indexing it into ElasticSearch. (latter is called 'jsonl' format). JQ is my tool of preference but I can't figure out how…
David
  • 485
  • 1
  • 5
  • 16
10
votes
1 answer

pattern for saving newline-delimited json (aka linejson, jsonlines, .jsonl files) with python

With Python, I'm saving json documents onto separate lines like this: from bson import json_util # pymongo with open('test.json', 'ab') as f: for document in documents: f.write(json_util.dumps(document)+'\n') and then reading like…
scharfmn
  • 3,561
  • 7
  • 38
  • 53
9
votes
2 answers

How to add syntax highlighting for a data-format, JSON Lines, that doesn't yet have syntax-highlighting support?

I would like to gain syntax highlighting support for the JSONL data format — "JSON-Lines" — as the format is not yet supported in 'V.S. Code'. The JSONL website recommends that I search for an extension that adds support for JSON Lines, however, all…
Max Heiber
  • 14,346
  • 12
  • 59
  • 97
7
votes
1 answer

json format each object in a line

cat 2.txt | ./jq '{(.id): .custom}' above command outputs { "1": { "results": "Success" } } { "2": { "input method": "touch", "from": "Prescription Center", } } { "3": { "entry point": "|All" } } Expected output : I…
user2711819
  • 960
  • 3
  • 16
  • 29
1
2 3
…
10 11