1

I have a very large JSON file (about a gigabyte) which I want to parse.

I tried the JsonSlurper, but it looks like it tries to load the whole file into memory which causes out of memory exception.

Here is a piece of code I have:

def parser = new JsonSlurper().setType(JsonParserType.CHARACTER_SOURCE);
def result = parser.parse(new File("equity_listing_full_201604160411.json"))

result.each{
println it.Listing.ID
}

And Json is something like this but much longer with more columns and rows

[
{"Listing": {"ID":"2013056","VERSION":"20160229:053120:000","NAME":"XXXXXX","C_ID":["1927445"],}},
{"Listing": {"ID":"2013057","VERSION":"20160229:053120:000","NAME":"XXXXXX","C_ID":["1927446"],}},
{"Listing": {"ID":"2013058","VERSION":"20160229:053120:000","NAME":"XXXXXX","C_ID":["1927447"],}}
]

I want to be able to read it row by row. I can probably just parse each row separately, but was thinking that there might be something for parsing as you read.

tim_yates
  • 167,322
  • 27
  • 342
  • 338
Roman Goyenko
  • 6,965
  • 5
  • 48
  • 81
  • even though this is a very old question check http://stackoverflow.com/questions/19522919/reading-json-object-from-txt-file-in-groovy – Chanaka Perera Feb 16 '17 at 09:49

1 Answers1

0

Suggest using GSON by Google. There is a streaming Parsing Option here: https://sites.google.com/site/gson/streaming

RvK
  • 234
  • 2
  • 6