18

Why does list(next(iter(())) for _ in range(1)) return an empty list rather than raising StopIteration?

>>> next(iter(()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> [next(iter(())) for _ in range(1)]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> list(next(iter(())) for _ in range(1))  # ?!
[]

The same thing happens with a custom function that explicitly raises StopIteration:

>>> def x():
...     raise StopIteration
... 
>>> x()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in x
StopIteration
>>> [x() for _ in range(1)]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in x
StopIteration
>>> list(x() for _ in range(1))  # ?!
[]
sophros
  • 14,672
  • 11
  • 46
  • 75
1''
  • 26,823
  • 32
  • 143
  • 200
  • 1
    Wow, I really like the answer in the original "duplicate" question! – wheaties Aug 29 '16 at 20:56
  • 3
    This is a bug, it is being addressed with [PEP 479](http://legacy.python.org/dev/peps/pep-0479/) where the `StopIteration` will be converted into a `RuntimeError` so that the list doesn't incorrectly stop iterating as you are expecting it to. – Tadhg McDonald-Jensen Aug 29 '16 at 20:57
  • 1
    @TadhgMcDonald-Jensen please give that as an answer so I may vote it up. – wheaties Aug 29 '16 at 21:02
  • 1
    For posterity, here is the ["duplicate" question](http://stackoverflow.com/questions/14413969/why-does-next-raise-a-stopiteration-but-for-do-a-normal-return) wheaties mentioned. – 1'' Aug 29 '16 at 21:09

2 Answers2

9

assuming all goes well, the generator comprehension x() for _ in range(1) should raise StopIteration when it is finished iterating over range(1) to indicate that there are no more items to pack into the list.

However because x() raises StopIteration it ends up exiting early meaning this behaviour is a bug in python that is being addressed with PEP 479

In python 3.6 or using from __future__ import generator_stop in python 3.5 when a StopIteration propagates out farther it is converted into a RuntimeError so that list doesn't register it as the end of the comprehension. When this is in effect the error looks like this:

Traceback (most recent call last):
  File "/Users/Tadhg/Documents/codes/test.py", line 6, in <genexpr>
    stuff = list(x() for _ in range(1))
  File "/Users/Tadhg/Documents/codes/test.py", line 4, in x
    raise StopIteration
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/Tadhg/Documents/codes/test.py", line 6, in <module>
    stuff = list(x() for _ in range(1))
RuntimeError: generator raised StopIteration
Tadhg McDonald-Jensen
  • 20,699
  • 5
  • 35
  • 59
7

The StopIteration exception is used to tell the underlying mechanism of the list function when to actually stop iterating on the iterable that has been passed to it. In your case, you're telling Python that the thing that has been passed into list() is a generator. So when the generator throws a StopIteration before generating any items, it outputs an empty list because nothing has been accumulated.

Tadhg McDonald-Jensen
  • 20,699
  • 5
  • 35
  • 59
wheaties
  • 35,646
  • 15
  • 94
  • 131
  • Wrong, the stop iteration from `for _ in range(1)` should raise the `StopIteration` for the list to finish but it doesn't, instead the `x()` raises an error that is **silently suppressed** causing it to accidentally stop. This is being changed in python3.6 and can be used with `from __future__ import generator_stop` in python 3.5. see [PEP 479](http://legacy.python.org/dev/peps/pep-0479/) – Tadhg McDonald-Jensen Aug 29 '16 at 21:03
  • 2
    @TadhgMcDonald-Jensen I'm glad to see it being fixed but just because it's a bug, doesn't change the fact that that is exactly what's happening. – wheaties Aug 29 '16 at 21:05
  • @TadhgMcDonald-Jensen I wouldn't say wheaties's answer is wrong. However, if you could post that as an answer I'd gladly accept it. – 1'' Aug 29 '16 at 21:06
  • in retrospect I think I was a little harsh on this. You are right that a generator throwing `StopIteration` is absolutely expected. But letting `StopIteration` propagate from somewhere not expecting to throw it in a way that silently exits early what ever iterator intercepts it is behaviour that should be snuffed out. ( I guess I still feel a little strong about that part :P) – Tadhg McDonald-Jensen May 12 '19 at 19:06