Understanding python nested iteration inside a list comprehension

Question

I am having a hard time interpreting nested iteration inside a list comprehension. Below code

movie = []
genre = []
for movie_ids in ratings["movie_id"].values:
  movie.append(movie_ids)
  genre.append([x for movie_id in movie_ids for x in genres_dict[movie_id]])

If I print genre[0] suppose I see a result [a,b,c]

My understanding from this and many others on the internet was that I can replace the line

genre.append([x for movie_id in movie_ids for x in genres_dict[movie_id]])

With

for movie_id in movie_ids:
    if movie_id in genres_dict:
      genre.append(genres_dict[movie_id])

But that gives a different result when I print genre[0]

What am I missing?

Please suggest

Hephaistos-plus · Accepted Answer · 2020-06-02T00:35:50.577

With the list comprehension, you are also iterating over the contents of genres_dict[movie_id]. This code would not quite translate into the for loop you provided, but instead into

for movie_id in movie_ids:
    for x in genres_dict[movie_id]
        genre.append(x)

I have omitted the if statement because, although it is not a bad idea to test whether the key exists, the nested iteration does not perform this test either. Nested iteration essentially performs the for loops you provide, nested, from the left to the right, and then appends the result of the expression.

Furthermore, appending the result of a list comprehension does not add every element of the list comprehension individually, but instead the entire list as one element. So the code actually performs

list_out = []
for movie_id in movie_ids:
    for x in genres_dict[movie_id]
        list_out.append(x)
genre.append(list_out)

If you want to add all of the elements of the list comprehension, you should instead use list.extend, which takes an iterable as an argument and adds every element to the list. In your original code this would be genre.extend([x for movie_id in movie_ids for x in genres_dict[movie_id]]).

score 0 · Answer 2 · answered Jun 02 '20 at 00:32

The list comprehension is actually flattening the array for you as it iterates over the inner content also.

The equivalent code block would be this. Notice the extend vs append.

for movie_id in movie_ids:
    if movie_id in genres_dict:
      genre.extend(genres_dict[movie_id])

The extend takes care of iterating over each item and adding it to the genre list. See docs for append vs extend. https://docs.python.org/3/tutorial/datastructures.html

Understanding python nested iteration inside a list comprehension

2 Answers2