0

I'm working on an android project and I have a json response to parse.

{"key1":"val1","key2":"val2","key3":[{...}, {...}, {...}... upto 5000 elements]}

This is how my current json looks.

I'm using HttpUrlConnection for parsing response as a stream.

I'm also using gson to parse this response using custom deserializers.

Everything works fine, but the problem comes one heap starts growing for those 5000 objects.

I'm looking for a way where I can take objects individually and store them in local db.

Any suggestions?

Ankit Garg
  • 709
  • 1
  • 11
  • 26

1 Answers1

1

If you're having memory issues, it means you're attempting to deserialize the entire stream at once into memory.

Gson provides the JsonStreamParser but unfortunately this isn't well suited for your use case - you have exactly one object in the input stream. Calling .next() would read the entire object including that array.

Jackson, on the other had, has more fine-grained streaming support via its JsonParser class. You can read each field from the stream using the tokenization:

MappingJsonFactory jfactory = new MappingJsonFactory();
JsonParser jParser = jfactory.createJsonParser(inputStream);

while (jParser.nextToken() != JsonToken.END_OBJECT)
{
    String fieldName = jParser.getCurrentName();

    if ("key3".equals(fieldname)) 
    {
        jParser.nextToken(); // current token is "[", move next

        // key3 is array, loop until token equal to "]"
        while (jParser.nextToken() != JsonToken.END_ARRAY) 
        {
            MyClass mc = jParser.readValueAs(MyClass.class);
            // store in your DB
        }
    }
}
jParser.close()

Note, the above is completely untested; it's just to show the concept. It should point you in the right direction.

Brian Roach
  • 76,169
  • 12
  • 136
  • 161
  • Hi Brian, I tried gson for individual field parsing on the input stream using JsonReader and using beginObject(), beginArray() methods available in gson lib. I found that heap wasn't increasing by bounds, but the time to parse one such stream was huge. So considering this scenario for android app, I can't compromise on CPU cycles as well. Some middle path has to be found, which could be asking my API team to provide fewer objects in response and writing custom gson deserializer for array objects. – Ankit Garg Feb 05 '14 at 05:49