4

I have a ndjson (newline delimited JSON) file, I need to parse it and get the data for some logical operation. Is there any good method for parsing ndjson files using golang. A sample ndjson is given below

{"a":"1","b":"2","c":[{"d":"100","e":"10"}]}
{"a":"2","b":"2","c":[{"d":"101","e":"11"}]}
{"a":"3","b":"2","c":[{"d":"102","e":"12"}]}
ASHWIN RAJEEV
  • 2,525
  • 1
  • 18
  • 24

1 Answers1

6

The encoding/json Decoder parses sequential JSON documents with optional or required whitespace depending on the value type. Because newlines are whitespace, the decoder handles ndjson. 

d := json.NewDecoder(strings.NewReader(stream))
for {
    // Decode one JSON document.
    var v interface{}
    err := d.Decode(&v)

    if err != nil {
        // io.EOF is expected at end of stream.
        if err != io.EOF {
            log.Fatal(err)
        }
        break
    }

    // Do something with the value.
    fmt.Println(v)
}

Run it on the playground.

Peter
  • 29,454
  • 5
  • 48
  • 60
Charlie Tumahai
  • 113,709
  • 12
  • 249
  • 242
  • FTR: Decoder doesn't require any kind of separation between documents: https://play.golang.org/p/oHc5xaegQuC. – Peter Jan 14 '20 at 06:27
  • 1
    @Peter I updated answer regarding whitespace. Separation is required for some value types: https://play.golang.org/p/ztaeA5UYKlo – Charlie Tumahai Jan 14 '20 at 06:50