How can i use dictionary comprehensions where nested dict or list may occur

Question

How can I use dictionary comprehension for values that are literals or list

Right now I'm able to iterate through a nested dictionary and get as a result dict with nested values, but I would like to include in the output dict, list, and literals (int, str)

Here is my example ( I know that isinstance is not needed here)

nested_dict = {'first':{'a':1}, 'second':{'b':2}, 'third': 3, 'fourth': [1, 2, 3, 4]}

float_dict = {
    outer_k: { float(inner_v)
        for (inner_k, inner_v) in outer_v.items()}
            for (outer_k, outer_v) in nested_dict.items()
                if isinstance(outer_v, dict) 
}

print(float_dict)

Expected output:

{'first': {'a': 1.0}, 'second': {'b': 2.0}, 'third': 3.0, 'fourth': [1.0, 2.0, 3.0, 4.0]}

same dictionary but only each value converted to floating point — Andrew, Sep 15 '21 at 10:24
You mean that the `third` and `fourth` key also become floats? — U13-Forward, Sep 15 '21 at 10:25
If the input isn't uniform, you can't really do it (reasonably) with a single comprehension. You'll want a recursive function which specifically checks whether it's dealing with lists, dicts or single values and treats them accordingly (recursively). E.g.: https://stackoverflow.com/q/55704719/476 — deceze, Sep 15 '21 at 10:27

deceze · Accepted Answer · 2021-09-15T10:42:30.807

7

It's not (reasonably) possible using a single comprehension, you want a recursive function like this:

def floatify(v):
    if isinstance(v, list):
        return list(map(floatify, v))
    if isinstance(v, dict):
        return {k: floatify(_v) for k, _v in v.items()}
    return float(v)

>>> floatify(nested_dict)
{'first': {'a': 1.0}, 'second': {'b': 2.0}, 'third': 3.0, 'fourth': [1.0, 2.0, 3.0, 4.0]}

Note that you can make this function even more generic:

def anyify(v, f):
    if isinstance(v, list):
        return [anyify(_v, f) for _v in v]
    if isinstance(v, dict):
        return {k: anyify(_v, f) for k, _v in v.items()}
    return f(v)

anyify(nested_dict, float)

edited Sep 15 '21 at 10:42

answered Sep 15 '21 at 10:32

deceze

510,633
85
743
889

@deceze Really nice. Thank you! One more question for a kind of similar task, if I need to process key instead of value, for example, i need to lowercase all dict keys in case they are uppercased, now most probably I will not be able to access nested dict if `key` will be lowered – Andrew Sep 15 '21 at 11:37
@Andrew Basically the same thing, but you'd process the `k` in the dict comprehension. I'm sure you can figure it out with a bit of work. – deceze Sep 15 '21 at 11:44
Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackoverflow.com/rooms/237139/discussion-on-answer-by-deceze-how-can-i-use-dictionary-comprehensions-where-nes). – deceze Sep 15 '21 at 15:12

score 2 · Answer 2 · answered Sep 15 '21 at 10:33

2

Or without recursion you could kinda make it in one-liner:

{outer_k: ({inner_k: float(inner_v) for (inner_k, inner_v) in outer_v.items()} if isinstance(outer_v, dict) else ([float(i) for i in outer_v] if isinstance(outer_v, list) else float(outer_v))) for (outer_k, outer_v) in nested_dict.items()}

Ex:

nested_dict = {'first':{'a':1}, 'second':{'b':2}, 'third': 3, 'fourth': [1, 2, 3, 4]}

float_dict = {outer_k: ({inner_k: float(inner_v) for (inner_k, inner_v) in outer_v.items()} if isinstance(outer_v, dict) else ([float(i) for i in outer_v] if isinstance(outer_v, list) else float(outer_v))) for (outer_k, outer_v) in nested_dict.items()}
print(float_dict)

Output:

{'first': {'a': 1.0}, 'second': {'b': 2.0}, 'third': 3.0, 'fourth': [1.0, 2.0, 3.0, 4.0]}

answered Sep 15 '21 at 10:33

U13-Forward

69,221
14
89
114

1) This is horrible. 2) It doesn't work for arbitrarily nested data structures… – deceze Sep 15 '21 at 10:34
1

@deceze Yeah, but for this scenario of the OP's dictionary it works :P – U13-Forward Sep 15 '21 at 10:34
Call me old fashioned, but I prefer [readable and generically applicable solutions](https://stackoverflow.com/a/69191414/476)… – deceze Sep 15 '21 at 10:35
2

@deceze True, but it is horrible in an impressive sort of way and makes a good motivation for your answer. +1 to both. – John Coleman Sep 15 '21 at 10:36
@deceze Me too :P I was ready to provide a solution with recursion as well, but since you already did, thought to do this crazy dictionary comprehension – U13-Forward Sep 15 '21 at 10:36
1

@deceze I guess this is close to the limit of how ugly python syntax can become haha :) – U13-Forward Sep 15 '21 at 10:37
1

Do not tempt the daemons… ಠ_ಠ – deceze Sep 15 '21 at 10:37
@deceze Haha! Exactly! – U13-Forward Sep 15 '21 at 10:38
1

It's not _quite_ as ugly with a few well-placed linebreaks and shorter names for `k/v`, though... – tobias_k Sep 15 '21 at 10:46
@tobias_k Yah, that's true. – U13-Forward Sep 15 '21 at 10:54

Niel Godfrey Pablo Ponciano · Answer 3 · 2021-09-15T11:06:56.997

Here is a solution that uses a queue. By taking advantage of the mutability of dicts and lists, we can enqueue those objects (even inner ones and even how deeply nested) and still be able to update the source data. This will traverse and enqueue each inner element of the data. If the element is an int, it will convert it to float.

import copy

nested_dict = {'first':{'a':1}, 'second':{'b':2}, 'third': 3, 'fourth': [4, {"fifth": 5}, 6, [{"sixth": [7, 8]}, 9], 10]}

float_dict = copy.deepcopy(nested_dict)

queue = [float_dict]
while queue:
    data = queue.pop()
    items = data.items() if isinstance(data, dict) else enumerate(data)
    for key, value in items:
        if isinstance(value, int):
            data[key] = float(value)
        elif isinstance(value, (dict, list)):
            queue.append(value)

print(float_dict)

Output

{'first': {'a': 1.0}, 'second': {'b': 2.0}, 'third': 3.0, 'fourth': [4.0, {'fifth': 5.0}, 6.0, [{'sixth': [7.0, 8.0]}, 9.0], 10.0]}

This is exactly *not* recursive. You've unrolled the recursion into a loop. Which may be beneficial for very very *very* deeply nested data structures… — deceze, Sep 15 '21 at 10:45

How can i use dictionary comprehensions where nested dict or list may occur

3 Answers3