3

I would like to know what's the general behaviour of an iterator if the underlaying object changes during the iteration.

Using a simple mutable list, it seems obvious: the iterator will try to follow on the next element if any, and send StopIteration if the end is reached.

>>> l = range(10)
>>> a = iter(l)
>>> a.next()
0
>>> a.next()
1
>>> a.next()
2
>>> l[3]='a'
>>> a.next()
'a'
>>> a.next()
4
>>> del l[5]
>>> a.next()
6
>>> a.next()
7
>>> a.next()
8
>>> a.next()
9
>>> a.next()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

This is self-explanatory so far. What I don't understand is, that if I append a new element, the iterator will still return StopIteration.

>>> l.append(11)
>>> a.next()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

If I do the same before reaching the end:

>>> l=[1]
>>> a=iter(l)
>>> a.next()
1
>>> l.append(2)
>>> a.next()
2

How is this working under the hood, and what is the expected behaviour of a more complex mutable iterable object? (e.g. think of an object representing a graph, which then can be iterated over, using a traversal algorithm. What is supposed to happen then if nodes are added/removed while iterating?)

Mkoch
  • 1,994
  • 2
  • 13
  • 21
  • 2
    Answer: don't. You ask what is supposed to happen in these conditions, but really you've stumbled across the answer yourself: such behaviour is typically ill-defined. – dvntehn00bz Apr 10 '14 at 12:15

3 Answers3

2

There's a comment on that particular issue in PEP 234 (Iterators):

Once a particular iterator object has raised StopIteration, will it also raise StopIteration on all subsequent next() calls?

Some say that it would be useful to require this, others say that it is useful to leave this open to individual iterators. Note that this may require an additional state bit for some iterator implementations (e.g. function-wrapping iterators).

Resolution: once StopIteration is raised, calling it.next() continues to raise StopIteration.

Note: this was in fact not implemented in Python 2.2; there are many cases where an iterator's next() method can raise StopIteration on one call but not on the next. This has been remedied in Python 2.3.

Phillip
  • 13,448
  • 29
  • 41
  • Is this like a suggestion or enforced by the language? If I implement an iterator class, do I need to take care that once __next__() returns StopIteration, it doesn't return anything else? – Mkoch Apr 10 '14 at 13:53
  • No, it's not enforced. You'll have to implement that behaviour yourself. – Phillip Apr 10 '14 at 14:02
0

There are three typical behaviours of changing an object while iterating over it:

  • the new data will be returned
  • the new data will be ignored
  • old data will be skipped

In other words: the actual behaviour is undefined.

Changing objects while iterating over them was such a common problem that in Python 3 the types set and dict (and possibly others) were changed to immediately raise an error if additions or removals were detected during iteration.

Ethan Furman
  • 63,992
  • 20
  • 159
  • 237
0

There are some iterators that can yield more data even after a StopIteration has been raised; such iterators are considered broken.

This does not mean the iterator is at fault -- it means that if you use such an iterator without caution you can end up with bugs and broken code.

Ethan Furman
  • 63,992
  • 20
  • 159
  • 237