How can I parse a response with multiple identical keys to json?

Question

I am working with an API (which I have no control over) that returns a response with hundreds of duplicate keys:

{
  "user": {
    //...
  },
  "user": {
    //...
  },
  //...
}

Using response.json() will only return one object, since it will overwrite any duplicate keys. Is there any way I can take the full response from response.text and turn it into something that I can parse to json?

*Returns a dict with duplicate keys*? Dicts can't have duplicate keys. So in what form is this 'dict' returned? — Ronald, Jul 08 '20 at 11:13
It sounds like you've got your concepts of "dict" and "JSON" backward. — user2357112, Jul 08 '20 at 11:27
Have a look at https://stackoverflow.com/questions/29321677/python-json-parser-allow-duplicate-keys — phhu, Jul 08 '20 at 11:33
Does this answer your question? [Python json parser allow duplicate keys](https://stackoverflow.com/questions/29321677/python-json-parser-allow-duplicate-keys) — Booboo, Jul 08 '20 at 12:02

score 0 · Accepted Answer · answered Jul 08 '20 at 11:38

Based on the answer to Python json parser allow duplicate keys, this seems to work:

from json import JSONDecoder

def parse_object_pairs(pairs):
    return pairs

data = """
{
  "user": {
    "val":[1,2,3]
  },
  "user": {
    "val":[1,2,3]
  }
}
"""

decoder = JSONDecoder(object_pairs_hook=parse_object_pairs)
obj = decoder.decode(data)
print (obj)
# [('user', [('val', [1, 2, 3])]), ('user', [('val', [1, 2, 3])])]

score 0 · Answer 2 · edited Jul 08 '20 at 11:49

You can use the object_pairs_hook argument to json.loads().

text = """
{
    "user": "Adam",
    "user": "Bill"
}
"""

import json

def hook(key_value_pairs):
    users = []
    for key, value in key_value_pairs:
        if key == "user":
            users.append(value)
    return {
        "users": users
    }

data = json.loads(text, object_pairs_hook=hook)
print(data)
# {'users': ['Adam', 'Bill']}

How can I parse a response with multiple identical keys to json?

2 Answers2