Parse only one field in a large JSON string

Question

I have a JSON string of the following format:

{

  "foo": "small_vale"  
  "baz": "large_value"
  "bar": "another_large_value"

}

How can I efficiently extract foo while ignoring the rest of the fields? Basically, I'm using Gson and I defined a "lean class" like that:

MyClass {
  private String foo;
}

If I'm ensuring that foo appears first in the JSON string, would Gson still scan the whole string, or is it smart enough to stop?

Should I use JsonPath instead?

easy way to do it, in api accept MyClass. and your MyClass add only which you want. — Akash Shah, Feb 24 '19 at 14:10

score 1 · Answer 1 · answered Feb 24 '19 at 16:23

Your json file is not valid. Commas are missing. It should look like this:

{
  "foo":"small_value",
  "baz":"large_value",
  "bar":"another_large_value"
}

This blog post says that Jackson or simple JSON are the fastest ways to parse big json data. Se the chapter "Big File Results" for reference.

Example code for Jackson: Jackson JSON – Read Specific JSON Key

It shows how to parse a json file ang get a value of a specific element.

//read json file data to String
byte[] jsonData = Files.readAllBytes(Paths.get("data.json"));

//create ObjectMapper instance
ObjectMapper objectMapper = new ObjectMapper();

//read JSON like DOM Parser
JsonNode rootNode = objectMapper.readTree(jsonData);
JsonNode fooNode = rootNode.path("foo");
System.out.println("foo value = "+fooNode.asText());

A few years later that link is now redirecting to a different blog post, but I believe this is the post you are referring to: https://www.harness.io/blog/ultimate-json-library-comparison — user313, Sep 01 '23 at 12:35

score 1 · Accepted Answer · answered Feb 24 '19 at 17:15

To answer on this question we need to see how do you parse JSON. I assume that you are using simplest:

Test test = gson.fromJson(new FileReader(jsonFile), Test.class);

If this is your case, answer for your question is Gson is not smart enough to do that. If you check implementation of this method, you will find out:

public <T> T fromJson(Reader json, Class<T> classOfT) throws JsonSyntaxException, JsonIOException {
    JsonReader jsonReader = newJsonReader(json);
    Object object = fromJson(jsonReader, classOfT);
    assertFullConsumption(object, jsonReader);
    return Primitives.wrap(classOfT).cast(object);
}

Before method returns value, it checks whether whole JSON was consumed and in case not, JsonIOException is thrown. Gson internally uses TypeAdapter implementation for given type. For your custom MyClass it will use ReflectiveTypeAdapterFactory.Adapter class which will consume the whole JSON payload. To avoid this situation you can write your own TypeAdapter:

class TestTypeAdapter extends TypeAdapter<Test> {

    @Override
    public void write(JsonWriter out, Test value) throws IOException {
        throw new IllegalStateException("Implement me!");
    }

    @Override
    public Test read(JsonReader in) throws IOException {
        if (in.peek() == JsonToken.NULL) {
            in.nextNull();
            return null;
        }

        Test test = new Test();

        try {
            in.beginObject();
            while (in.hasNext()) {
                String name = in.nextName();
                if (name.equals("foo")) {
                    test.setFoo(in.nextString());
                    break;
                }
            }
        } catch (IllegalStateException e) {
            throw new JsonSyntaxException(e);
        }

        return test;
    }
}

Simple usage:

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonSyntaxException;
import com.google.gson.TypeAdapter;
import com.google.gson.stream.JsonReader;
import com.google.gson.stream.JsonToken;
import com.google.gson.stream.JsonWriter;

import java.io.File;
import java.io.FileReader;
import java.io.IOException;

public class GsonApp {

    public static void main(String[] args) throws Exception {
        File jsonFile = new File("./resource/test.json").getAbsoluteFile();
        Gson gson = new GsonBuilder().create();

        Test test = gson.fromJson(new FileReader(jsonFile), Test.class);
        Test test1 = new TestTypeAdapter().fromJson(new FileReader(jsonFile));

        System.out.println(test);
        System.out.println(test1);
    }
}

class Test {

    private String foo;

    public String getFoo() {
        return foo;
    }

    public void setFoo(String foo) {
        this.foo = foo;
    }

    @Override
    public String toString() {
        return "Test{" +
                "foo='" + foo + '\'' +
                '}';
    }
}

Above code prints:

Test{foo='small_value'}
Test{foo='small_value'}

As, you can see, in both cases we parsed our small value. You can test this code and calculate how faster custom TypeAdapter is for your JSON payload.

But in case, you have much complex situation and you need to parse much more JSON to find your value, try to use JSONPath solution. You can start from this question: how to parse a huge JSON file without loading it in memory.

Parse only one field in a large JSON string

2 Answers2

Linked