Python best way to unflat json?

Question

I have a function that makes JSON flat:

def flat_json(z, sep):
    val = {}
    for i in z.keys():
        if isinstance(z[i], dict):
            get = flat_json(z[i], sep)
            for j in get.keys():
                val[i + sep + j] = get[j]
        else:
            val[i] = z[i]

    return val


flat_json({"a": "b", "c": {"d": "g"}}, '__')
# returns {'a': 'b', 'c__d': 'g'}

But I have not yet figured out how to convert the flat JSON back

Why do you want to flattent it like this in the first place? — Olivier Melançon, Oct 03 '18 at 12:23

Martijn Pieters · Accepted Answer · 2018-10-03T13:19:38.167

Unflattening back to arbitrarily nested dictionaries requires that you create new dictionaries for arbitrary paths; c__d means that there must be a nested dictionary associated with the c key, so you can set d in there. c__d__e means c is a dictionary with a d key pointing to another dictionary, were e is set, etc. Essentially, everything but the last element in key.split('__') is a key in a dictionary pointing to another dictionary, and you need to create those dictionaries if they have not yet been created, in a loop.

You can use functools.reduce() to do this, see this previous answer of mine for an explanation of the get_nested_default() and set_nested() functions:

from functools import reduce

def get_nested_default(d, path):
    return reduce(lambda d, k: d.setdefault(k, {}), path, d)

def set_nested(d, path, value):
    get_nested_default(d, path[:-1])[path[-1]] = value

def unflatten(d, separator='__'):
    output = {}
    for k, v in d.items():
        path = k.split(separator)
        set_nested(output, path, v)
    return output

get_nested_default() takes care of creating those nested dictionaries if they don't yet exist, for any number of keys. set_nested() takes care of separating out the last key in a path from the remainder, and setting the value in the innermost dictionary.

This assumes that the input never used __ in keys originally, of course. But as long as you pick a path separator that's not used as a substring of any of the keys in the input dictionary, you can un-flatten back to the original form.

Demo on your example:

>>> unflatten({'a': 'b', 'c__d': 'g'})
{'a': 'b', 'c': {'d': 'g'}}

and a more complex one using a different separator:

>>> from pprint import pprint
>>> flat = {
...     'foo::bar::baz': 'Eric Idle',
...     'foo::bar::spam': 'John Cleese',
...     'foo::spam::ham': 'Terry Gilliam',
...     'spam::ham': 'Vikings singing'
... }
>>> nested = unflatten(flat, '::')
>>> pprint(nested)
{'foo': {'bar': {'baz': 'Eric Idle', 'spam': 'John Cleese'},
         'spam': {'ham': 'Terry Gilliam'}},
 'spam': {'ham': 'Vikings singing'}}

Python best way to unflat json?

1 Answers1