Complicated list comprehension from dict

Question

After Googling a bit on this and reading a few other SO posts (this one in particular), I can't seem to find a working solution.

I have a loop as follows:

    json_preferred = []
    for score, terms in profile.preferred.items():
        for term in terms:
            json_preferred.append({"key": term, "value": score})

Basically I have a dictionary with values that are lists. For each key in the dictionary, I need to create a dictionary in another list that puts the dictionary key and each value as values with fixed keys.

It's kind of difficult to put into words, but hopefully the code above is a bit more self-explanatory. For context, the output of all of this is a JSON structure that will be used later on in a REST API request.

I want to replace the code above with some kind of list comprehension. However, my attempt at this originally has failed:

json_preferred = [{"key": term, "value": score} for term in terms for score, terms in profile.preferred.items()]

This fails with:

NameError: name 'terms' is not defined

dict/list comprehensions greater than 1 dimension are very confusing for me. So I'm not sure what I'm doing wrong here. I'm using Python 3.9.1.

Yes it seems like you are right. I wasn't aware that you had to think of comprehensions in reverse order. EDIT: I mean, I *was* thinking of it in reverse order, and you shouldn't. My bad. My head is spinning! — void.pointer, Feb 04 '21 at 17:20
`[{"key": term, "value": score} for score, terms in profile.preferred.items() for term in terms ]` — Epsi95, Feb 04 '21 at 17:20
Here's a tutorial: https://spapas.github.io/2016/04/27/python-nested-list-comprehensions/ — Barmar, Feb 04 '21 at 17:20
Another one: https://www.geeksforgeeks.org/nested-list-comprehensions-in-python/ — Barmar, Feb 04 '21 at 17:20
Does this answer your question? [Nested For Loops Using List Comprehension](https://stackoverflow.com/questions/3633140/nested-for-loops-using-list-comprehension) — Tomerikoo, Feb 04 '21 at 17:26
In hindsight, yes most of these questions do technically answer mine; however what would have been *more* helpful is to find an explicit mention on not doing the reverse order of the loops. It's more about the thought process for me as opposed to the solution itself, if that makes sense. Thanks to everyone. — void.pointer, Feb 04 '21 at 20:49

score 2 · Accepted Answer · answered Feb 04 '21 at 17:19

To form a comprehension, flatten the nested loops and move its body to the beginning of the comprehension:

json_preferred = []
for score, terms in profile.preferred.items():
    for term in terms:
        json_preferred.append({"key": term, "value": score})

...becomes:

json_preferred = [
    {"key": term, "value": score}
    for score, terms in profile.preferred.items()
    for term in terms
]

score 0 · Answer 2 · answered Feb 04 '21 at 17:19

0

You'd need itertools.chain() or maybe functools.reduce() to express that nested loop as a list comprehension that'd flatten into one list.

That's a fancy way of saying that you probably shouldn't try to express this as a list comprehension if you want to keep your code readable.

answered Feb 04 '21 at 17:19

AKX

152,115
15
115
172

I realize this is likely off topic, but I'm interested: Is it considered bad practice to use nested list comprehension? I won't lie that I was thinking this doesn't seem as readable as the for loop version. – void.pointer Feb 04 '21 at 17:22
For generating a list of lists, a nested list comprehension _could_ be okay, but if it's hard enough to think about when writing it, it won't be easy to read (by you or others) either. – AKX Feb 04 '21 at 17:24
@AKX he simply miswrote the order of the loops... It should be `for score, terms in profile.preferred.items() for term in terms` and not `for term in terms for score, terms in profile.preferred.items()` – Tomerikoo Feb 04 '21 at 17:26

score 0 · Answer 3 · answered Feb 04 '21 at 17:33

Taken from the question you linked (with some modifications):

lst = []
for i, j in s1:
 for k in s2:
   lst.append({i: k})

Becomes:

lst = [{i: k} for i, j in s1 for k in s2]

And in your case we have:

i, j = score, terms
s1 = profile.preferred.items()
k = term
s2 = j = terms

Which gives according to the above:

lst = [{score: term} for score, terms in profile.preferred.items() for term in terms]

Complicated list comprehension from dict

3 Answers3