Flatten nested dictionary using Pandas

Question

I would like to flatten a nested dictionary. A solution for such a problem was suggested here: https://stackoverflow.com/a/41801708/8443371. Problem: I would like to obtain a keys identical to the keys in the last layer only. For an input:

d = {'a': 1,
     'c': {'b': {'x': 5,
                 'y' : 10}},
     'd': [1, 2, 3]}

I would like to have an output:

{'a': 1, 'x': 5, 'y': 10, 'd': [1, 2, 3]}

Suggestions using python only should be probably slower than Pandas, which is based on a C implementation.

Note: Assuming max two layer dictionary, I have a python solution, which seems to be very slow:

for key in dict.keys():
    if '.' in key:
       dict[key.split('.')[-1]] = dict.pop(key)

Doing it with pandas might be an overkill. Plus I don't know if there is a function for this case. — Mohit Motwani, Feb 21 '19 at 12:10
You are most probably correct. timeit stats can back you up. — Nihal Sangeeth, Feb 21 '19 at 12:14
Yes. I can time it but I am pretty sure that it is slower... — Gideon Kogan, Feb 21 '19 at 12:15
In any case your examples tells me that it is not standard flattening. So you will have to write a parser. — Nihal Sangeeth, Feb 21 '19 at 12:17

Mohit Motwani · Answer 1 · 2019-02-21T12:23:51.300

Here's my solution in pure python for the dictionary you've provided:

d = {'a': 1,
     'c': {'b': {'x': 5,
                 'y' : 10}},
     'd': [1, 2, 3]}

def flatten_dict(dic):
    result = {}
    for key in dic.keys():
        if isinstance(dic[key], dict):
            result.update(flatten_dict(dic[key]))
        else:
            result[key] = dic[key]
return result

flatten_dict(d)
{'a': 1, 'x': 5, 'y': 10, 'd': [1, 2, 3]}

%%timeit
flatten_dict(d)
2.45 µs ± 72.5 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

Flatten nested dictionary using Pandas

1 Answers1