How to compare JSON documents and return the differences with Jackson or Gson?

Question

I am using spring-boot to develop backend services. There is a scenario to compare 2-beans(one is the DB object and another one is the client requested object) and return the "new element","modified element" and if there is no change then return false. The 2-beans are in a below format

"sampleList":{
     "timeStamp":"Thu, 21 Jun 2018 07:57:00 +0000",
     "id":"5b19441ac9e77c000189b991",
     "sampleListTypeId":"type001",
     "friendlyName":"sample",
     "contacts":[
        {
           "id":"5b05329cc9e77c000189b950",
           "priorityOrder":1,
           "name":"sample1",
           "relation":"Friend",
           "sampleInfo":{
              "countryCode":"91",
              "numberType":"MOBILE",
              "numberRegion":"IN"
           }
        },
        {
           "id":"5b05329cc9e77c000189b950",
           "priorityOrder":1,
           "name":"sample2",
           "relation":"Friend",
           "sampleInfo":{
              "countryCode":"91",
              "numberType":"MOBILE",
              "numberRegion":"IN"
           }
        }
     ]
  }

I have browsed internet about bean comparison for this scenario in java but I couldn't find any simpler solution but found some cool solution for JSON. I can see some solution for GSON but it will not return the client object contains "new element" and the "changes element". Is there any way to return the newer and modified element in JSON or JAVA? Your help should be appreciable. Even a hint will be a great start for me.

score 79 · Accepted Answer · edited Oct 07 '21 at 07:34

Reading the JSON documents as `Map`s and comparing them

You could read both JSON documents as Map<K, V>. See the below examples for Jackson and Gson:

ObjectMapper mapper = new ObjectMapper();
TypeReference<HashMap<String, Object>> type = 
    new TypeReference<HashMap<String, Object>>() {};

Map<String, Object> leftMap = mapper.readValue(leftJson, type);
Map<String, Object> rightMap = mapper.readValue(rightJson, type);

Gson gson = new Gson();
Type type = new TypeToken<Map<String, Object>>(){}.getType();

Map<String, Object> leftMap = gson.fromJson(leftJson, type);
Map<String, Object> rightMap = gson.fromJson(rightJson, type);

Then use Guava's Maps.difference(Map<K, V>, Map<K, V>) to compare them. It returns a MapDifference<K, V> instance:

MapDifference<String, Object> difference = Maps.difference(leftMap, rightMap);

If you are not happy with the result, you can consider flattening the maps and then compare them. It will provide better comparison results especially for nested objects and arrays.

Creating flat `Map`s for the comparison

To flat the map, you can use:

public final class FlatMapUtil {

    private FlatMapUtil() {
        throw new AssertionError("No instances for you!");
    }

    public static Map<String, Object> flatten(Map<String, Object> map) {
        return map.entrySet().stream()
                .flatMap(FlatMapUtil::flatten)
                .collect(LinkedHashMap::new, (m, e) -> m.put("/" + e.getKey(), e.getValue()), LinkedHashMap::putAll);
    }

    private static Stream<Map.Entry<String, Object>> flatten(Map.Entry<String, Object> entry) {

        if (entry == null) {
            return Stream.empty();
        }

        if (entry.getValue() instanceof Map<?, ?>) {
            return ((Map<?, ?>) entry.getValue()).entrySet().stream()
                    .flatMap(e -> flatten(new AbstractMap.SimpleEntry<>(entry.getKey() + "/" + e.getKey(), e.getValue())));
        }

        if (entry.getValue() instanceof List<?>) {
            List<?> list = (List<?>) entry.getValue();
            return IntStream.range(0, list.size())
                    .mapToObj(i -> new AbstractMap.SimpleEntry<String, Object>(entry.getKey() + "/" + i, list.get(i)))
                    .flatMap(FlatMapUtil::flatten);
        }

        return Stream.of(entry);
    }
}

It uses the JSON Pointer notation defined in the RFC 6901 for the keys, so you can easily locate the values.

Example

Consider the following JSON documents:

{
  "name": {
    "first": "John",
    "last": "Doe"
  },
  "address": null,
  "birthday": "1980-01-01",
  "company": "Acme",
  "occupation": "Software engineer",
  "phones": [
    {
      "number": "000000000",
      "type": "home"
    },
    {
      "number": "999999999",
      "type": "mobile"
    }
  ]
}

{
  "name": {
    "first": "Jane",
    "last": "Doe",
    "nickname": "Jenny"
  },
  "birthday": "1990-01-01",
  "occupation": null,
  "phones": [
    {
      "number": "111111111",
      "type": "mobile"
    }
  ],
  "favorite": true,
  "groups": [
    "close-friends",
    "gym"
  ]
}

And the following code to compare them and show the differences:

Map<String, Object> leftFlatMap = FlatMapUtil.flatten(leftMap);
Map<String, Object> rightFlatMap = FlatMapUtil.flatten(rightMap);

MapDifference<String, Object> difference = Maps.difference(leftFlatMap, rightFlatMap);

System.out.println("Entries only on the left\n--------------------------");
difference.entriesOnlyOnLeft()
          .forEach((key, value) -> System.out.println(key + ": " + value));

System.out.println("\n\nEntries only on the right\n--------------------------");
difference.entriesOnlyOnRight()
          .forEach((key, value) -> System.out.println(key + ": " + value));

System.out.println("\n\nEntries differing\n--------------------------");
difference.entriesDiffering()
          .forEach((key, value) -> System.out.println(key + ": " + value));

It will produce the following output:

Entries only on the left
--------------------------
/address: null
/phones/1/number: 999999999
/phones/1/type: mobile
/company: Acme


Entries only on the right
--------------------------
/name/nickname: Jenny
/groups/0: close-friends
/groups/1: gym
/favorite: true


Entries differing
--------------------------
/birthday: (1980-01-01, 1990-01-01)
/occupation: (Software engineer, null)
/name/first: (John, Jane)
/phones/0/number: (000000000, 111111111)
/phones/0/type: (home, mobile)

Thanks a lot for your clear explanation Cassio...I will check and let you know — VelNaga, Jun 21 '18 at 21:01
Here it throws nullPointerException "return Stream.of(entry);" if the entry is null. — VelNaga, Jul 20 '18 at 14:08
ah, Exception comes from "Map.Entry::getValue". so we should have a null check there because value can be null — VelNaga, Jul 20 '18 at 14:13
Do you have any suggestion? Meanwhile i am also working on this — VelNaga, Jul 20 '18 at 14:18
With the following piece of code it works fine if(entry.getValue() instanceof String || entry.getValue() == null) { entry.setValue(defaultString(String.valueOf(entry.getValue()))); } — VelNaga, Jul 20 '18 at 14:37
But i'm not sure whether it's an right approach...But anyway Have a wonderful weekend. Thanks a lot for your message — VelNaga, Jul 20 '18 at 14:38
@VelNaga Looks like my code doesn't handle `null` values properly. I've just added `.filter(entry -> entry.getValue() != null)` until I can think about something better. It handles `null` values as missing properties. — cassiomolin, Jul 20 '18 at 15:04
But it will not collect the attribute whose values are null so i don't think this logic will work. Also, one of the Map I'm comparing contains some additional attributes i would like to remove those attributes and then compare these 2-Maps.This is someThing we can do via "Flattening" the map. — VelNaga, Jul 20 '18 at 15:11
@VelNaga Give it a try with the JSON documents I provided as example and set some properties to `null`. — cassiomolin, Jul 20 '18 at 15:12
@VelNaga `.collect(LinkedHashMap::new, (m, e) -> m.put("/" + e.getKey(), e.getValue()), LinkedHashMap::putAll);` will collect `null` values properly. So `.filter(entry -> entry.getValue() != null)` is no longer needed. See [this](https://stackoverflow.com/q/24630963/1426227) for reference. — cassiomolin, Jul 20 '18 at 15:41
It works like a charm. Thanks a lot. But I also have another challenge. In the right json i have extra attribute called "code"..Before doing MapsDifference i would like to exclude that attribute otherwise the maps are always "notEqual". Is that something we can while doing "Flatten"? — VelNaga, Jul 21 '18 at 11:22
@VelNaga If it answers your original question, please accept my answer. And, yes, you can customize the code as per your needs. You can use `.filter()` to remove the keys you don't want to compare while processing the streams or simply remove the key from the comparison result. But it goes beyond the scope of your original question. — cassiomolin, Jul 21 '18 at 12:33
@CassioMazzochiMolin -- I am getting an error The method difference(Map extends K,? extends V>, Map extends K,? extends V>) in the type Maps is not applicable for the arguments (Iterators.Map, Iterators.Map) for the code MapDifference difference = Maps.difference(left, right); Can you help me with this? — Kapil, Jan 24 '19 at 08:39
Looks like your solution can't handle different order of an array, like those two are the same : [ {f:"a"},{f:"b"}] == [ {f:"b"},{f:"a"}], but your code said it is different — Justin, Jul 08 '19 at 20:54
@Justin As per the [RFC 8259](https://tools.ietf.org/html/rfc8259) (the document that defines the JSON format), the order of the elements of the array matters (highlight is mine): _An **object** is an **unordered** collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array. An **array** is an **ordered** sequence of zero or more values._ So `[{"f":"a"},{"f":"b"}]` and `[{"f":"b"},{"f":"a"}]` are different. — cassiomolin, Jul 08 '19 at 21:23
@Justin To handle such situation, you may need to implement your own comparison for the maps. But I don't have any solution upfront, I'm afraid. — cassiomolin, Jul 08 '19 at 21:24
@cassiomolin thanks what an amazing answer, very clearly written. Worked for me at the first run — kunal, Dec 26 '21 at 01:41

score 43 · Answer 2 · edited Oct 07 '21 at 07:34

Creating a JSON Patch document

Alternatively to the approach described in the other answer, you could use the Java API for JSON Processing defined in the JSR 374 (it doesn't use on Gson or Jackson). The following dependencies are required:

<!-- Java API for JSON Processing (API) -->
<dependency>
    <groupId>javax.json</groupId>
    <artifactId>javax.json-api</artifactId>
    <version>1.1.2</version>
</dependency>

<!-- Java API for JSON Processing (implementation) -->
<dependency>
    <groupId>org.glassfish</groupId>
    <artifactId>javax.json</artifactId>
    <version>1.1.2</version>
</dependency>

Then you can create a JSON diff from the JSON documents. It will produce a JSON Patch document as defined in the RFC 6902:

JsonPatch diff = Json.createDiff(source, target);

When applied to the source document, the JSON Patch yields the target document. The JSON Patch can be applied to the source document using:

JsonObject patched = diff.apply(source);

Creating a JSON Merge Patch document

Depending on your needs, you could create a JSON Merge Patch document as defined in the RFC 7396:

JsonMergePatch mergeDiff = Json.createMergeDiff(source, target);

When applied to the source document, the JSON Merge Patch yields the target document. To patch the source, use:

JsonValue patched = mergeDiff.apply(source);

Pretty printing JSON documents

To pretty print the JSON documents, you can use:

System.out.println(format(diff.toJsonArray()));
System.out.println(format(mergeDiff.toJsonValue()));

public static String format(JsonValue json) {
    StringWriter stringWriter = new StringWriter();
    prettyPrint(json, stringWriter);
    return stringWriter.toString();
}

public static void prettyPrint(JsonValue json, Writer writer) {
    Map<String, Object> config =
            Collections.singletonMap(JsonGenerator.PRETTY_PRINTING, true);
    JsonWriterFactory writerFactory = Json.createWriterFactory(config);
    try (JsonWriter jsonWriter = writerFactory.createWriter(writer)) {
        jsonWriter.write(json);
    }
}

Example

Consider the following JSON documents:

{
  "name": {
    "first": "John",
    "last": "Doe"
  },
  "address": null,
  "birthday": "1980-01-01",
  "company": "Acme",
  "occupation": "Software engineer",
  "phones": [
    {
      "number": "000000000",
      "type": "home"
    },
    {
      "number": "999999999",
      "type": "mobile"
    }
  ]
}

{
  "name": {
    "first": "Jane",
    "last": "Doe",
    "nickname": "Jenny"
  },
  "birthday": "1990-01-01",
  "occupation": null,
  "phones": [
    {
      "number": "111111111",
      "type": "mobile"
    }
  ],
  "favorite": true,
  "groups": [
    "close-friends",
    "gym"
  ]
}

And the following code to produce a JSON Patch:

JsonValue source = Json.createReader(new StringReader(leftJson)).readValue();
JsonValue target = Json.createReader(new StringReader(rightJson)).readValue();

JsonPatch diff = Json.createDiff(source.asJsonObject(), target.asJsonObject());
System.out.println(format(diff.toJsonArray()));

It will produce the following output:

[
    {
        "op": "replace",
        "path": "/name/first",
        "value": "Jane"
    },
    {
        "op": "add",
        "path": "/name/nickname",
        "value": "Jenny"
    },
    {
        "op": "remove",
        "path": "/address"
    },
    {
        "op": "replace",
        "path": "/birthday",
        "value": "1990-01-01"
    },
    {
        "op": "remove",
        "path": "/company"
    },
    {
        "op": "replace",
        "path": "/occupation",
        "value": null
    },
    {
        "op": "replace",
        "path": "/phones/1/number",
        "value": "111111111"
    },
    {
        "op": "remove",
        "path": "/phones/0"
    },
    {
        "op": "add",
        "path": "/favorite",
        "value": true
    },
    {
        "op": "add",
        "path": "/groups",
        "value": [
            "close-friends",
            "gym"
        ]
    }
]

Now consider the following code to produce a JSON Merge Patch:

JsonValue source = Json.createReader(new StringReader(leftJson)).readValue();
JsonValue target = Json.createReader(new StringReader(rightJson)).readValue();

JsonMergePatch mergeDiff = Json.createMergeDiff(source, target);
System.out.println(format(mergeDiff.toJsonValue()));

It will produce the following output:

{
    "name": {
        "first": "Jane",
        "nickname": "Jenny"
    },
    "address": null,
    "birthday": "1990-01-01",
    "company": null,
    "occupation": null,
    "phones": [
        {
            "number": "111111111",
            "type": "mobile"
        }
    ],
    "favorite": true,
    "groups": [
        "close-friends",
        "gym"
    ]
}

Different results when applying the patches

When the patch document is applied, the results are slightly different for the approaches described above. Consider the following code that applies JSON Patch to a document:

JsonPatch diff = ...
JsonValue patched = diff.apply(source.asJsonObject());
System.out.println(format(patched));

It produces:

{
    "name": {
        "first": "Jane",
        "last": "Doe",
        "nickname": "Jenny"
    },
    "birthday": "1990-01-01",
    "occupation": null,
    "phones": [
        {
            "number": "111111111",
            "type": "mobile"
        }
    ],
    "favorite": true,
    "groups": [
        "close-friends",
        "gym"
    ]
}

Now consider the following code that applies JSON Merge Patch to a document:

JsonMergePatch mergeDiff = ...
JsonValue patched = mergeDiff.apply(source);
System.out.println(format(patched));

It produces:

{
    "name": {
        "first": "Jane",
        "last": "Doe",
        "nickname": "Jenny"
    },
    "birthday": "1990-01-01",
    "phones": [
        {
            "number": "111111111",
            "type": "mobile"
        }
    ],
    "favorite": true,
    "groups": [
        "close-friends",
        "gym"
    ]
}

In the first example, the occupation property is null. In the second example, it's omitted. It's due to the null semantics on JSON Merge Patch. From the RFC 7396:

If the target does contain the member, the value is replaced. Null values in the merge patch are given special meaning to indicate the removal of existing values in the target. [...]

This design means that merge patch documents are suitable for describing modifications to JSON documents that primarily use objects for their structure and do not make use of explicit null values. The merge patch format is not appropriate for all JSON syntaxes.

Looks like your solution can't handle different order of an array, like those two are the same : [ {f:"a"},{f:"b"}] == [ {f:"b"},{f:"a"}], but your code said it is different — Justin, Jul 08 '19 at 20:53
@Justin As per the [RFC 8259](https://tools.ietf.org/html/rfc8259) (the document that defines the JSON format), the order of the elements of the array matters (highlights are mine): _An **object** is an **unordered** collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, `null`, object, or array. An **array** is an **ordered** sequence of zero or more values._ So `[{"f":"a"},{"f":"b"}]` and `[{"f":"b"},{"f":"a"}]` are _different_. — cassiomolin, Jul 09 '19 at 12:49

mkfl3x · Answer 3 · 2023-05-23T15:18:48.337

You can try my library - json-delta.
It's based on Gson and can be configured with specific purposes like ignoring fields or consider/not consider missed/unexpected fields.

Example

expected:
{
  "type": "animal",
  "info": {
    "id": 123,
    "subtype": "Cat",
    "timestamp": 1684852390
  }
}

actual:
{
  "type": "animal",
  "info": {
    "id": 123,
    "subtype": "Tiger",
    "timestamp": 1684852399
  }
}

Comparison and printed result:

// Third parameter 'ignoredFields' is vararg  
// Here 'timestamp' field is ignored because dynamic
JsonDeltaReport report = new JsonDelta().compare(expected, actual, "root.info.timestamp");
System.out.println(report);

The output will look like this:

Status: failed
Mismatches:
"root.info.subtype": Value mismatch. Expected: "Cat"; Actual: "Tiger"

JsonDeltaReport object has following fields:

success (Boolean): Comparison result (success if JSONs are equals)
mismatches (List): List of all mismatches

How to compare JSON documents and return the differences with Jackson or Gson?

3 Answers3

Reading the JSON documents as Maps and comparing them

Creating flat Maps for the comparison

Example

Creating a JSON Patch document

Creating a JSON Merge Patch document

Pretty printing JSON documents

Example

Different results when applying the patches

Linked

Reading the JSON documents as `Map`s and comparing them

Creating flat `Map`s for the comparison