how to transfer this flat dict to a nested dict?

Question

There is a flat dict like this:

{"a.b": "foo", "a.c": "bar", "d.e.f":"baz"}

And how to use Python to transfer the dict to this:

{
    "a": 
        {
            "b": "foo",
            "c": "bar"
        },
    "d": 
        {
            "e":
                {
                    "f": "baz"
                }
        }
}

Answer 1

You can split the your keys on . into the last value and everything before with:

*parents, key =  k.split('.')

You can dig down into a nested dictionary (creating them if necessary) with something like:

end = reduce(lambda x, y: x.setdefault(y, {}) , parents, new_d) 

here end will be the dictionary at the end of the chain. So at that point you can just assign the value. Something like:

from functools import reduce

d = {"a.b": "foo", "a.c": "bar", "d.e.f":"baz", "d.e.g":"M", 'l':"single_key_test" }

new_d = {}

for k in d:
    *parents, key =  k.split('.')
    end = reduce(lambda x, y: x.setdefault(y, {}) , parents, new_d)
    end[key] = d[k]

print(new_d)

result:

{'a': {'b': 'foo', 'c': 'bar'}, 'd': {'e': {'f': 'baz', 'g': 'M'}}, 'l': 'single_key_test'}

Answer 2

You would first have to split your keys by '.' to get the paths.

Then you can create a function that creates a nested dictionary from one path:

def make_nested_dict(iterable, final):
    """Make nested dict from iterable"""
    if iterable:
        head, *tail = iterable
        return {head: make_nested_dict(tail, final)}
    else:
        return final

Which works as follows:

d = {"a.b": "foo", "a.c": "bar", "d.e.f":"baz"}    

for key in d:
    paths = key.split('.')
    nested_path = make_nested_dict(paths, d[key])
    print(nested_path)

And gives the following paths:

{'a': {'b': 'foo'}}
{'a': {'c': 'bar'}}
{'d': {'e': {'f': 'baz'}}}

Then you can create a function that merges two nested dictionaries together recursively:

def merge_nested_dicts(d1, d2):
    """Merge two nested dictionaries together"""
    for key in d2:
        if key in d1:
            if isinstance(d2[key], dict) and isinstance(d1[key], dict):
                merge_nested_dicts(d1[key], d2[key])
        else:
            d1[key] = d2[key]

    return d1

Which you can merge by updating a resultant nested dict:

nested_dict = {}

for key in d:
    paths = key.split('.')
    nested_path = make_nested_dict(paths, d[key])
    nested_dict = merge_nested_dicts(nested_dict, nested_path)

print(nested_dict)

And now this gives the following:

{'a': {'b': 'foo', 'c': 'bar'}, 'd': {'e': {'f': 'baz'}}}

Full code with some comments:

def make_nested_dict(iterable, final):
    """Make nested dictionary path with a final attribute"""

    # If iterable, keep recursing
    if iterable:

        # Unpack key and rest of dict
        head, *tail = iterable

        # Return new dictionary, recursing on tail value
        return {head: make_nested_dict(tail, final)}

    # Otherwise assign final attribute
    else:
        return final

def merge_nested_dicts(d1, d2):
    """Merge two nested dictionaries together"""

    for key in d2:

        # If we have matching keys
        if key in d1:

            # Only merge if both values are dictionaries
            if isinstance(d2[key], dict) and isinstance(d1[key], dict):
                merge_nested_dicts(d1[key], d2[key])

        # Otherwise assign normally
        else:
            d1[key] = d2[key]

    return d1

if __name__ == "__main__":
    d = {"a.b": "foo", "a.c": "bar", "d.e.f":"baz", "d.e.g":"M", 'l':"single_key_test" }  

    nested_dict = {}

    for key in d:

        # Create path
        paths = key.split('.')

        # Create nested path
        nested_path = make_nested_dict(paths, d[key])

        # Update result dict by merging with new dict
        nested_dict = merge_nested_dicts(nested_dict, nested_path)

    print(nested_dict)