9

I was trying to do a challenge on codeeval in python3 and got stuck trying to improve my solution. Every time i tried to iterate (or print, or some other action) two times consecutively over the same iterator, the second loop came up empty. Here is a minimal example that produces this behavior, although I tried several different combinations with lists etc. that gave me the same result:

numbers = ('1','2','3','4','5')
numbers = map(int, numbers)                                                    
print(list(numbers))                                                          
print(list(numbers))

results in:

[1, 2, 3, 4, 5]
[]

Why does print (in this case) delete the content of numbers?

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Guerki
  • 93
  • 3

1 Answers1

11

This is exactly how iterators work. They're designed to generate data on the fly exactly one time, no more. If you want to get data out of it a second time, you either have to save all the iterated data to another list, or initiate the iterator again. In cases where you need to read from files or do other annoying things to re-obtain that iterator, it's probably best to just store that data in a list when it's generated.

>>> numbers = ('1','2','3','4','5')
>>> ints = [x for x in map(int, numbers)]
>>> print(list(ints))
[1, 2, 3, 4, 5]
>>> print(list(ints))
[1, 2, 3, 4, 5]

https://docs.python.org/2/library/stdtypes.html#iterator-types

The intention of the protocol is that once an iterator’s next() method raises StopIteration, it will continue to do so on subsequent calls. Implementations that do not obey this property are deemed broken. (This constraint was added in Python 2.3; in Python 2.2, various iterators are broken according to this rule.)

I should note that I ran the exact code you gave on Python 2.4.3, and it printed out the list every time. So it's a version dependent thing, I suppose.

TheSoundDefense
  • 6,753
  • 1
  • 30
  • 42
  • You can just call the `map` function again. Though if you're going to do that, it's best not to do `numbers = map(int, numbers)`. Try giving it another name, like `ints = map(int, numbers)`. That way you're not destroying the original data. – TheSoundDefense Jul 30 '14 at 14:33
  • What would be the way to re-initiate an iterator? Do I really have to map it for each print? Also, in my specific cast (but not the minimal example) the initial list comes from an open('file') statement. Would I have to do that twice too? – Guerki Jul 30 '14 at 14:34
  • @Guerki you might have to, yes. In that case, you should iterate over the data and save it to a list; that will keep the data there permanently. I'll update my answer. – TheSoundDefense Jul 30 '14 at 14:37
  • 1
    @TheSoundDefense: In Python 3.x, many functions that previously returned lists now return iterators. In this case, `map()` has been converted in that way. So the equivalent py3k code is: `ints = list(map(int, numbers))` – Bill Lynch Jul 30 '14 at 14:39
  • 1
    One other thing worth noting is [`itertools.tee()`](https://docs.python.org/3/library/itertools.html#itertools.tee), which can be helpful if you don't think you're going to iterate through the entire iterator, but you may need to iterate up to that point multiple times. – Bill Lynch Jul 30 '14 at 14:42