2

I have a json file in this format,

{
  "details": {

    "hawk_branch": {
      "tandem": {
        "value": "4210bnd72"
      }
    },
    "uclif_branch": {
      "tandem": {
        "value": "e2nc712nma89",
        "value": "23s24212",
        "value": "12338cm82",
      }
    }
    }
}

The problem is, I need to keep all the value, however when i use json.load to load this file i only get one value, which make sense since dict can keep only unique keys.

Here is the expected output,

{ "hawk_branch": ["4210bnd72"] }
{ "uclif_branch": ["e2nc712nma89" , "23s24212", "12338cm82"] }

I have read this answer, Python json parser allow duplicate keys to use object_pairs_hook like this,

def parse_object_pairs(pairs):
    return pairs

# f is file
json.load(f, object_pairs_hook=parse_object_pairs)

but it returns entire json file as list.

I think its possible to do it using lambda as object_pairs_hook but i can't understand how can I use it.

Can someone please guide me

MaverickD
  • 1,477
  • 1
  • 13
  • 27

1 Answers1

4

You can use a custom duplicate key resolver function that turns the values of the value keys into a list:

def value_resolver(pairs):
    if all(k == 'value' for k, _ in pairs):
        return [v for _, v in pairs]
    return dict(pairs)

so that:

json.load(f, object_pairs_hook=value_resolver)

returns:

{'details': {'hawk_branch': {'tandem': ['4210bnd72']}, 'uclif_branch': {'tandem': ['e2nc712nma89', '23s24212', '12338cm82']}}}

And to dump the new data structure back to the original JSON format by converting lists to dicts with duplicate value keys, you can use a custom json.JSONEncoder subclass:

class restore_value(json.JSONEncoder):
    def encode(self, o):
        if isinstance(o, dict):
            return '{%s}' % ', '.join(': '.join((json.encoder.py_encode_basestring(k), self.encode(v))) for k, v in o.items())
        if isinstance(o, list):
            return '{%s}' % ', '.join('"value": %s' % self.encode(v) for v in o)
        return super().encode(o)

so that:

d = {'details': {'hawk_branch': {'tandem': ['4210bnd72']}, 'uclif_branch': {'tandem': ['e2nc712nma89', '23s24212', '12338cm82']}}}
print(json.dumps(d, cls=restore_value))

would output:

{"details": {"hawk_branch": {"tandem": {"value": "4210bnd72"}}, "uclif_branch": {"tandem": {"value": "e2nc712nma89", "value": "23s24212", "value": "12338cm82"}}}}

blhsing
  • 91,368
  • 6
  • 71
  • 106
  • 1
    thank you, can you please explain it a little bit?, this works but i don't know why – MaverickD Oct 22 '18 at 02:30
  • Glad to be of help. This works because it selectively converts the list of tuples passed in by `object_pairs_hook` to a list by testing if all of the keys (i.e. the first items of the tuples) are equal to `'value'` first, before falling back to the default behavior of converting the list of tuples to a dict using the `dict()` constructor. – blhsing Oct 22 '18 at 02:33
  • Is it possible to dump it back to the file in the same format ? I added few more `value` and I need to dump it back to file with new data in same format – MaverickD Oct 22 '18 at 04:40
  • if works when i do `json.dumps(d, cls=restore_value)` but when i use `json.dump(d, opfile, cls=restore_value)` it still write to file as list. Do you have any idea why ? – MaverickD Oct 22 '18 at 21:57