How to convert nested dictionaries into Python tuples?

Question

I would like to convert nested dictionaries into Python tuples but all examples I have found don't fit my case. This is how the dictionary looks like :

d = {'default': {'USD': {'value': 490.0, 'quantity': 490.0}},
     'expert': {'USD': {'value': 500.0, 'quantity': 500.0}}}

Desired result:

{('default', 'value'): {'USD': 490.0},
 ('default', 'quantity'): {'USD': 490.0},
 ('expert', 'value'): {'USD': 500.0},
 ('expert', 'quantity'): {'USD': 500.0}
}

The idea is to create a multiindexes dataframe like this :

pd.DataFrame(tuple)

        default             expert
        value   quantity    value   quantity
USD     490.0   490.0       500.0   500.0

What I did

A slightly different results can be obtain with (see this question) :

{(outerKey, innerKey): values for outerKey, innerDict in d.items() 
 for innerKey, values in innerDict.items()}

Which gives:

{('default', 'USD'): {'value': 490.0, 'quantity': 490.0},
 ('expert', 'USD'): {'value': 500.0, 'quantity': 500.0}}

But as you can see the dataframe doesn't have the desired structure :

            default expert
            USD     USD
value       490.0   500.0
quantity    490.0   500.0

score 1 · Accepted Answer · answered Mar 17 '21 at 09:56

You can use smth like this, it looks meaty but works like expected result:

d = {'default': {'USD': {'value': 490.0, 'quantity': 490.0}},
     'expert': {'USD': {'value': 500.0, 'quantity': 500.0}}}

new_d = {}

for key, value in d.items():
    for key1, value1 in value.items():
        for key2, value2 in value1.items():
            new_d[(key, key2)] = {key1: value2}

score 1 · Answer 2 · answered Mar 17 '21 at 10:27

For a general way, I would flatten the initial dict, load it into a DataFrame and then pivot the DataFrame:

# flatten:
def flatten(val):
    if isinstance(val, list):
        return val
    elif isinstance(val, dict):
        return [(k, *v) for k, value in val.items()
                 for v in flatten(value)]
    return ((val,),)

# print(flatten(d))          # uncomment for traces

# load into a raw DataFrame:
df = pd.DataFrame(flatten(d))
# print(df)                  # uncomment for traces

# We can now choose what should be the index, columns and values
df = df.pivot(1, [0,2], 3).rename_axis(index=None, columns=(None, None))

It gives as expected:

    default          expert         
      value quantity  value quantity
USD   490.0    490.0  500.0    500.0

And the traces are (when uncommented):

[('default', 'USD', 'value', 490.0), ('default', 'USD', 'quantity', 490.0),
 ('expert', 'USD', 'value', 500.0), ('expert', 'USD', 'quantity', 500.0)]

         0    1         2      3
0  default  USD     value  490.0
1  default  USD  quantity  490.0
2   expert  USD     value  500.0
3   expert  USD  quantity  500.0

For some reason the pivot function returns `IndexError: Too many levels: Index has only 2 levels, not 3`. This is the trace I have `[('default', 'USD', 'value', 490.0), ('default', 'USD', 'quantity', 490.0), ('expert', 'USD', 'value', 500.0), ('expert', 'USD', 'quantity', 500.0)]` — Florent, Mar 17 '21 at 17:55
I forgot to mention that this is also part of the error message `ValueError: 2 is not in list` — Florent, Mar 17 '21 at 18:08

score 0 · Answer 3 · answered Mar 17 '21 at 10:00

I found the answer to my own question by adding a nested level with secInnerDict to the example in the question :

{(outerKey, secinnerKey): {innerKey:secvalues} for outerKey, innerDict in total.items() for innerKey, secInnerDict in innerDict.items() for secinnerKey, secvalues in secInnerDict.items()}

Returns :

{('default', 'value'): {'USD': 490.0},
 ('default', 'quantity'): {'USD': 490.0},
 ('expert', 'value'): {'USD': 500.0},
 ('expert', 'quantity'): {'USD': 500.0}}

I have not provided nested list comprehension due to hearing about it's not recommendation because of reading complexity — Vova, Mar 17 '21 at 12:15

How to convert nested dictionaries into Python tuples?

3 Answers3