Why is this list comprehension valid

Question

I was surprised that [x for x in range(10) if x is 5 not in range(10) if True] is valid (Python 3.6). How is it parsed?

Does this answer your question? [What does "list comprehension" mean? How does it work and how can I use it?](https://stackoverflow.com/questions/34835951/what-does-list-comprehension-mean-how-does-it-work-and-how-can-i-use-it) — CDJB, Dec 06 '19 at 14:17
Its gibberish. It depends on execution order of things . x is 5 is only True for 5. if True in range(10) is always False. result is an empty list. why formulate such a thing? — Patrick Artner, Dec 06 '19 at 14:22
@PatrickArtner What depends on the execution order here…? The list is empty because `5 not in range(10)` is always false… (and `x is 5` depends on the small integer interning of your Python flavour…) — deceze, Dec 06 '19 at 14:33
@deceze I ment operation order - the `is` is evaluated first, True or False in range(10) is always False — Patrick Artner, Dec 06 '19 at 16:58

score 3 · Answer 1 · answered Dec 06 '19 at 14:29

Common syntax elements for comprehensions are:
comprehension ::=  expression comp_for
comp_for      ::=  ["async"] "for" target_list "in" or_test [comp_iter]
comp_iter     ::=  comp_for | comp_if
comp_if       ::=  "if" expression_nocond [comp_iter]
The comprehension consists of a single expression followed by at least one for clause and zero or more for or if clauses. In this case, the elements of the new container are those that would be produced by considering each of the for or if clauses a block, nesting from left to right, and evaluating the expression to produce an element each time the innermost block is reached.

https://docs.python.org/3/reference/expressions.html#displays-for-lists-sets-and-dictionaries

There can be more than one for or if clause, which each just form nested for/if blocks, so this really just unrolls to:

res = []
for x in range(10):
    if x is 5 not in range(10):
        if True:
            res.append(x)

The x is 5 not in range(10) is a chained comparison, equal to x is 5 and 5 not in range(10).

Izaak van Dongen · Answer 2 · 2019-12-06T14:32:14.367

2

You can think of it as the following "unrolled" structure:

my_list = []
for x in range(10):
    if x is 5 not in range(10):
        if True:
            my_list.append(x)

x is 5 not in range(10) just a weird condition that is read as (x is 5) and (not (5 in range(10))) (credit to Matthias :)), but 5 is always in range(10), so this is kind of weird.

When you have multiple if clauses like this, you just think of them as nested if statements, so it's equivalent to

[x for x in range(10) if x is 5 not in range(10) and True]

Their actual use-case is when you also want to have another inner loop inside an if clause, eg a contrived example:

[i + j for i in range(10) if i & 1 for j in range(4) if j & 1]

which is the same as

my_list = []
for i in range(10):
    if i & 1:
        for j in range(4):
            if j & 1:
                my_list.append(i + j)

edited Dec 06 '19 at 14:32

answered Dec 06 '19 at 14:26

Izaak van Dongen

2,450
13
23

1

According to the [documentation](https://docs.python.org/3/reference/expressions.html#comparisons) I would read `x is 5 not in range(10)` as `(x is 5) and (5 not in range(10))` – Matthias Dec 06 '19 at 14:29
@Matthias Good point! Thank you for spotting that, I'm clearly rusty – Izaak van Dongen Dec 06 '19 at 14:30
1

I didn't trust my ability to understand the documentation and tested with `5 == 5 in range(2, 10)` and `(5 == 5) in range(2, 10)` and got different results. – Matthias Dec 06 '19 at 14:33
Of course the usage of `is` is the next crucial point. If we have `(x is y) in range(1, 2000)` and define `x = 5` and `y = 5` we get `True`. Using `x = 1000` and `y = 1000` we get `False` (due to caching of small integers in CPython). But OK, this is commenting on code that we all agree to be rubbish anyway. – Matthias Dec 06 '19 at 14:38
@Matthias You're making that even trickier with the added parentheses there. With `(x is y) in range(1, 2000)`, you depend on small integer interning of your Python flavour to end up with `True/False in range(1, 2000)`. Now, `True` happens to be represented as `1` internally, so results in `True`, while `False` (`0`) is not in that range. Try `True in range(2, 2000)`… However, `x is y in range(1, 2000)` is the entirely different `x is y and y in range(1, 2000)`. – deceze Dec 06 '19 at 14:42
@deceze I know. This is beyond the basic question. My main point was the usage of `is` in the context of the caching (interning) of small integers. I could have reduced the example to `x == y` vs. `x is y`. – Matthias Dec 06 '19 at 14:43

Why is this list comprehension valid

2 Answers2