Efficient parsing of first four elements of large JSON arrays

Question

I am using Jackson to parse JSON from a json inputStream which looks like following:

[
      [ 36,
        100,
        "The 3n + 1 problem",
         56717,
         0,
         1000000000,
         0,
         6316,
         0,
         0,
         88834,
         0,
         45930,
         0,
         46527,
         5209,
         200860,
         3597,
         149256,
         3000,
         1
      ],
      [
         ........
      ],
      [
         ........
      ],
         .....// and almost 5000 arrays like above
]

This is the original feed link: http://uhunt.felix-halim.net/api/p

I want to parse it and keep only the first 4 elements of every array and skip other 18 elements.

36
100
The 3n + 1 problem
56717

Code structure I have tried so far:

while (jsonParser.nextToken() != JsonToken.END_ARRAY) {

        jsonParser.nextToken(); // '['
        while (jsonParser.nextToken() != JsonToken.END_ARRAY) {
            // I tried many approaches here but not found appropriate one
         }

}

As this feed is pretty big, I need to do this efficiently with less overhead and memory. Also there are three models to procress JSON: Streaming API, Data Binding and Tree Model. Which one is appropriate for my purpose?

How can I parse this json efficiently with Jackson? How can I skip those 18 elements and jump to next array for better performance?

Edit: (Solution)

Jackson and GSon both works in almost in the same mechanism (incremental mode, since content is read and written incrementally), I am switching to GSON as it has a function skipValue() (pretty appropriate with name). Although Jackson's nextToken() will work like skipValue(), GSON seems more flexible to me. Thanks @Kowser bro for his recommendation, I came to know about GSON before but somehow ignored it. This is my working code:

reader.beginArray();
while (reader.hasNext()) {
   reader.beginArray(); 
   int a = reader.nextInt(); 
   int b = reader.nextInt();
   String c = reader.nextString();
   int d = reader.nextInt();
   System.out.println(a + " " + b + " " + c + " " + d);
   while (reader.hasNext())
      reader.skipValue();
   reader.endArray();
} 
reader.endArray();
reader.close();

Good luck with GSON, it's a decent library. I do not however see the point wrt `skipValue()` -- Jackson has `skipChildren()` that does the same, skipping the whole logical value (which for structured values consists of multiple tokens). — StaxMan, Aug 05 '13 at 22:37
Please make sure that you put relevant details into the title. "Parsing JSON with Jackson" was in no way describing the specific problem you were having. — oberlies, May 06 '19 at 09:39
Also, when you [answer your own question](https://stackoverflow.com/help/self-answer), please post it as answer instead of editing it into the question. — oberlies, May 06 '19 at 09:40

Kowser · Accepted Answer · 2013-08-04T08:08:15.783

3

This is for Jackson

Follow this tutorial.

Judicious use of jasonParser.nextToken() should help you.

while (jasonParser.nextToken() != JsonToken.END_ARRAY) { // might be JsonToken.START_ARRAY?

The pseudo-code is

find next array
1. read values
2. skip other values
3. skip next end token

This is for gson. Take a look at this tutorial. Consider following second example from the tutorial.

Judicious use of reader.begin* reader.end* and reader.skipValue should do the job for you.

And here is the documentation for JsonReader

edited Aug 04 '13 at 08:08

answered Aug 04 '13 at 07:53

Kowser

8,123
7
40
63

vai, thanks for your answer! now it works! I edited my answer and added my solution, I used GSON at last. – Kaidul Aug 04 '13 at 08:57

Efficient parsing of first four elements of large JSON arrays

1 Answers1

Linked