OutOfMemoryError when Jackson deserializes JSON object

Question

I have a problem with a very large JSON file that is too large to use ObjectMapper.readValue() into a JsonNode. I would like to use the solution from Out of memory error while parsing a large JSON using Jackson library on Android, except the JSON file is a single object with field names that are not known ahead of time, so I can't create a model POJO to deserialize to.

Each property inside the object has the same format, and I can ignore many of the properties of those inner objects (I already have a POJO class to model that). It would be easier for me to solve this problem if the JSON file was an array instead of an object. (I'm not the one creating the file, just reading from it.)

(I'm posting my solution below, but I hope there's a better one!)

score 0 · Answer 1 · answered Jun 07 '19 at 20:49

Without being able to load the original file in an ObjectMapper, I decided to parse the JSON file and rewrite it as an array. Reading and writing line-by-line, I converted a file that looks like this (but much larger):

{
    "Unexpected Monkey" : {
        "name" : "UnexpectedMonkey",
        "age" : 7
    },
    "Another Unexpected Name" : {
        "name" : "Another Unexpected Name",
        "age" : 2
    }
}

into:

[
    {
        "name" : "UnexpectedMonkey",
        "age" : 7
    },
    {
        "name" : "Another Unexpected Name",
        "age" : 2
    }
]

Then I could parse the file a la How to parse a JSON string to an array using Jackson

@zerohedge After pre-processing the file, it was smaller and I could open it without causing the OOM error. — Kyle, Aug 06 '23 at 14:44

OutOfMemoryError when Jackson deserializes JSON object

1 Answers1