Understanding nested list comprehension

Question

I want to understand nested list comprehension. Below, I listed a list comprehension expression and their for loop equivalent.
I wonder if my understanding is correct on those.

For example,

[(min([row[i] for row in rows]),max([row[i] for row in rows])) 
for i in range(len(rows[0]))]

is equivalent to

result=[]
for i in range(len(rows[0])):
  innerResult=[]
  for row in rows:
    innerResult.append(row[i])
  innerResult2=[]
  for row in rows:
    innerResult2.append(row[i])
  tuple=(min(innerResult), max(innerResult2))
  result.append(tuple)

If I may generalize, I guess

[exp2([exp1 for x in xSet]) for y in ySet]

form can be translated to the following. (I hope I'm correct on this)

result=[]
for y in ySet:
  innerResult =[]
  for x in xSet:
    innerResult.append(exp1)
  exp2Result = exp2(innerResult)
  result.append(exp2Result)

For simpler case,

[exp1 for x in xSet for y in ySet]

is equal to

result=[] 
for x in xSet:
  for y in ySet: 
    result.append(exp1)

whereas,

[[exp1 for x in xSet] for y in ySet]

is equal to

result=[]
for y in ySet:
  innerResult=[]
  for x in xSet:
    innerResult.append(exp1)
  result.append(innerResult)

I asked a similar question on Equivalent for loop expression for complex list comprehension
The answers given there reconstruct the form after understanding what it does internally.
I'd like to know how it works systematically so I can apply the concept to other slightly varying examples.

Side note: In your example, you could have [transposed](http://en.wikipedia.org/wiki/Transpose) the rows into columns using `cols = zip(*rows)`, after which you could have simply used `min(col)` and `max(col)` for each column: `[(min(c), max(c)) for c in cols]`. Or in one short line: `[(min(c), max(c)) for col in zip(*rows)]`. — taleinat, Nov 08 '11 at 12:36
Would it be true, and improve clarity, if you were to write exp1(x,y) instead of just exp1 ? I _think_ it's implied, but would like to be sure? — GreenAsJade, Mar 01 '16 at 22:26
@GreenAsJade well, yes that would work, exp1 was meant to be expression1 — eugene, Mar 03 '16 at 10:03
"Here's a detailed mental model; is my understanding correct?" generally doesn't make a good question, because it's mostly shoving the answer (or an attempt at one) into the question. — Karl Knechtel, Jan 26 '23 at 10:30

score 78 · Answer 1 · answered Nov 08 '11 at 12:32

78

Indeed, you are correct. This is described in detail in the Expressions section in the Python Language Reference.

Note especially the order of nesting of several fors in a single list comprehension, which is always left-to-right:

>>> matrix = [[1, 2], [3, 4]]
>>> [item for item in row for row in matrix] # oops!
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    [item for item in row for row in matrix]
NameError: name 'row' is not defined
>>> [item for row in matrix for item in row] # nesting is in left-to-right order
[1, 2, 3, 4]

answered Nov 08 '11 at 12:32

taleinat

8,441
1
30
44

I've looked at the link. Indeed, there's an explanation for [item for row in matrix for item in row] but couldn't find explanation for [[exp for item in row] for row in matrix] format. Am I missing something from the grammar listed there? – eugene Nov 08 '11 at 12:44
You're not missing anything, but that's because your example doesn't use any more special grammar, it is just two simple list comprehensions! The first `[exp for item in row]` will create a list given a row. The "outer" list comprehension will create a list, in which each item is a list created by the "inner" list-comp, one for each row in the matrix. – taleinat Nov 08 '11 at 12:54
1

Oh. As you pointed out in your answer, the order is important. And, I was wondering if there's a section on library reference which talks about ordering of [[list comprehension] list comprehension] format. – eugene Nov 08 '11 at 13:03
3

There in no such section in the docs that I know of. The reason is that in a list comprehension, the expression evaluated for each item can be any valid Python expression. In this context, an "inner" list comprehension is just like any other Python expression. – taleinat Nov 08 '11 at 13:28
2

wow whee.. am I the only one surprised by this ordering: now an explanation why the macro level (matrix in this case..) occurs before the nested/micro level (row level..) might be in order.. – WestCoastProjects Jun 28 '17 at 03:59
@javadba If I remember correctly, the reasoning is that it is like the order of nested for loops in code. – taleinat Aug 17 '17 at 09:24
That actually helps .. Guido's thinking is typically somewhere between 90 to 180 degrees to mine: no exception here. – WestCoastProjects Aug 17 '17 at 10:25

mac · Accepted Answer · 2011-11-08T12:27:13.223

The short answer is: yes, you are correct in your understanding.

There's only a catch: the way you normally use nested list comprehension in python code is to operate on multidimensional arrays.

A typical example is when you operate on matrices:

>>> matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> [[el - 1 for el in row] for row in matrix]
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]

As you can see the "nesting" works by operating on each dimension of the matrix.

In the examples you provided, it seems that ySet [unfortunate name btw, as sets are one of the types provided with python] is just a generic counter, which makes a bit harder to follow what is going on under the hood.

As for your first example:

>>> rows = ([1, 2, 3], [10, 20, 30])
>>> [(min([row[i] for row in rows]),max([row[i] for row in rows])) for i in range(len(rows[0]))]
[(1, 10), (2, 20), (3, 30)]

You might wish to look into the zip built-in function:

>>> zip(rows[0], rows[1])
[(1, 10), (2, 20), (3, 30)]

or for maximum brevity and elegance:

>>> zip(*rows)
[(1, 10), (2, 20), (3, 30)]

HTH!

Understanding nested list comprehension

2 Answers2

Linked

Related