Parsing a json which contains duplicate keys

Question

I'm trying to parse json with invalid structure which has duplicate keys using Jackson library. If a json has duplicate keys, I would like to extract them as a Collection.

Example of what I'm trying to parse (the actual json that I'm trying to parse comes from Wireshark json export):

{
    "a": "a",
    "a": {
        "b": {

        },
        "b": true
    }
}

However, since this json has duplicate keys, only the last value is retained:

JsonNode tree = new ObjectMapper().readTree(json);
System.out.println(tree); // {"a":{"b":true}}

I've also tried the Guava module which has Multimap support, however it doesn't work as expected for nested json objects.

Example using Guava module for the json that I have shown before:

ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new GuavaModule());
mapper.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true);

Multimap read = mapper.readValue(json, Multimap.class);
System.out.println(read); // {a=[a, {b=true}]}

How should I tackle this problem using Jackson library? Are there any other libraries which would support parsing of such json structure for java?

Valid JSON doesn't have duplicate keys, so a parsing library shouldn't work the way you're asking — OneCricketeer, Mar 03 '17 at 07:34
duplicate keys is valid json. https://tools.ietf.org/html/rfc7159 "The names within an object SHOULD be unique", not required to be unique. — Justin Dennahower, Feb 17 '20 at 20:42
@OneCricketeer I dunno. ECMA-404 (linked from json.org) says: "The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs. These are all semantic considerations that may be defined by JSON processors or in specifications defining specific uses of JSON for data interchange." — Robert Dodier, Jan 25 '21 at 07:46

Ravi MCA · Accepted Answer · 2021-01-25T07:34:25.940

If you are flexible with json library you can make use of net.sf.json.JSONObject.

This library will retain the duplicated values by storing them into arrays. If multiple same keys are available it will create one key with all the values as Array.

And also the coding part is just a single line. Once you parsed the json using net.sf.json.JSONObject then you can supply this to jackson library.

JSONObject jsonObject = JSONObject.fromObject( "{ \"a\": \"a\", \"a\": { \"b\": {},\"b\": true}}" );

System.out.println( "net.sf.json.JSONObject: " + jsonObject );

JsonNode jsonNode = new ObjectMapper().readTree( jsonObject.toString() );

System.out.println( "com.fasterxml.jackson.databind.JsonNode" + jsonNode );

Output:

net.sf.json.JSONObject: {"a":["a",{"b":[{},true]}]}
com.fasterxml.jackson.databind.JsonNode{"a":["a",{"b":[{},true]}]}

Maven dependency of net.sf.json

<dependency>
    <groupId>net.sf.json-lib</groupId>
    <artifactId>json-lib</artifactId>
    <version>2.4</version>
    <classifier>jdk15</classifier>
</dependency>

Is there a way we can remove the duplicates and keep it as a single object rather than an array — Atul Soman, Sep 11 '17 at 05:50
@AtulSoman Not possible as long as you are using duplicate keys. — Ravi MCA, Sep 11 '17 at 06:13

score 3 · Answer 2 · answered Mar 03 '17 at 07:56

3

You can use "jackson" library to read the json message token by token in a streaming way, then it would not eat anything form the message.

answered Mar 03 '17 at 07:56

Krzysztof Cichocki

6,294
1
16
32

1

I believe that to be the only viable approach as well. There is some information to get started in this answer: http://stackoverflow.com/a/25426227/13075 – Henrik Aasted Sørensen Mar 03 '17 at 08:13

Parsing a json which contains duplicate keys

2 Answers2

Linked