Why isn't a StopIteration raised here?

Question

Take a look at this, the crux of the question is at the bottom:

>>> scan = iter('FHUR203459')
>>> while True:
        print(next(scan))


F
H
U
R
2
0
3
4
5
9
Traceback (most recent call last):
  File "<pyshell#11>", line 2, in <module>
    print(next(scan))
StopIteration
>>> scan = iter('FHUR203459')
>>> for i in range(12):  # 12 * 2 for each join is 24, much longer than the string; should raise error.
        print(''.join(next(scan) for i in range(2)))


FH
UR
20
34
59







>>>

In other words, we can see that the iterator reaches its end in both situations, however it only raises a StopIteration in the first, even though next() is used in both situations after it reached the end. Why does using it in join seem to evade the error? Or is this a bug?

Martijn Pieters · Answer 1 · 2014-10-24T16:42:00.843

2

str.join() calls list() on the generator, and that call swallows the StopIteration.

Anything that consumes an iterator must catch StopIteration; it doesn't matter then exactly what raised the exception; the generator expression or anything used by the generator expression:

>>> def raises_stopiteration(): raise StopIteration
... 
>>> next(raises_stopiteration() for _ in range(10))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <genexpr>
  File "<stdin>", line 1, in raises_stopiteration
StopIteration
>>> list(raises_stopiteration() for _ in range(10))
[]

edited Oct 24 '14 at 16:42

answered Oct 24 '14 at 16:36

Martijn Pieters

1,048,767
296
4,058
3,343

really? `str.join` listifies`?! Then why use a generator at all? What would be the advantage of using a generator over a list comprehension within `str.join`? – inspectorG4dget Oct 24 '14 at 16:38
1

@inspectorG4dget: yes, it listifies because it has to pre-calculate the output size before copying in the strings. Using a list comprehension instead of a generator expression is therefor *faster* when using `str.join()` on the result. – Martijn Pieters Oct 24 '14 at 16:39
@inspectorG4dget `str.join` has to know the length of the string in advance to join. So, if we pass a list, it just has to do one iteration otherwise, it converts it to a list and does another pass to find the size of the joined string. – thefourtheye Oct 24 '14 at 16:40
1

@inspectorG4dget: see [Raymond Hettinger's answer here](http://stackoverflow.com/a/9061024). – Martijn Pieters Oct 24 '14 at 16:40

thefourtheye · Accepted Answer · 2014-10-24T16:42:39.320

In the first case, the StopIteration is not handled anywhere. But in the second case,

''.join(next(scan) for i in range(2))

we pass a generator expression to the ''.join, which handles the StopIteration raised by next(scan) and exits every time. That is why ''.join produces empty strings.

You can slightly modify the same, and pass a list to ''.join and see the exception being raised, yourself, like this

>>> scan = iter('FHUR203459')
>>> for i in range(12):
...     print(''.join([next(scan) for i in range(2)]))
... 
FH
UR
20
34
59
Traceback (most recent call last):
  File "<input>", line 2, in <module>
  File "<input>", line 2, in <listcomp>
StopIteration

It shows that the StopIteration is actually raised, and List Comprehension takes the hit, this time.

Why isn't a StopIteration raised here?

2 Answers2