33

I have the following code:

s = [1,2,3]
t = reversed(s)

for i in t:
    print(i)
# output: 3,2,1

If I pop one element from s (original), then the t (reversed) is emptied:

s = [1,2,3]
t = reversed(s)
s.pop()

for i in t:
    print(i)
# expected output: 2, 1
# actual output (nothing):

Why does this happen?

user2357112
  • 260,549
  • 28
  • 431
  • 505
shelper
  • 10,053
  • 8
  • 41
  • 67
  • can not reproduce your issue – billz Jan 04 '21 at 02:41
  • 3
    @billz I have just run this using 3.8.7 in the command line and it does output nothing for me. – M Z Jan 04 '21 at 02:43
  • 2
    `reversed(s)` is an iterator, not a reversed list. – user2357112 Jan 04 '21 at 02:47
  • @blitz - reproducible with python 3.6.9. `next(t)` throws a `StopIteration` exception, which explains why nothing is printed. But I don't know why `list(t)` is `[]`. I think this is a very good question. – Michael Szczesny Jan 04 '21 at 02:50
  • @MichaelSzczesny: It's interesting, but not too surprising. `t` isn't a reversed list, it's basically just a wrapped iterator around `s`. Don't modify an iterator before you've consumed it, otherwise you get "surprising" results. – Eric Duminil Jan 04 '21 at 17:41

2 Answers2

30

Taking a look at the cpython code on GitHub, we can get some intuition as to why it no longer works.

The iterator that is returned essentially requires knowing the position of the last index and the length of the array. If the size of the array is changed, the iterator will no longer work.

Test 1: Increasing the array length

This will not produce the correct results either, but the iterator does run:

s = [1,2,3]
t = reversed(s)
s.append(4)

for i in t:
    print(i)
# output: [3, 2, 1]

Test 2: Decreasing, then increasing the length

s = [1,2,3]
t = reversed(s)
s.pop()
s.append(4)

for i in t:
    print(i)
# output: [4, 2, 1]

It still works!

So there's an internal check to see whether or not the last index is still valid, and if it is, it's a simple for loop down to index 0.

If it doesn't work, the iterator returns empty.

Michael Szczesny
  • 4,911
  • 5
  • 15
  • 32
M Z
  • 4,571
  • 2
  • 13
  • 27
  • 2
    The check is in `line 3339`: `if (index>=0 && index < PyList_GET_SIZE(seq))` where `seq` is a pointer to the original `list` object. If the `PyList_GET_SIZE` check fails the reference is removed and `NULL` is returned. – Michael Szczesny Jan 04 '21 at 03:15
  • @MichaelSzczesny Feel free to add that to the answer if you want, instead of just a comment, since you were the one that pointed it out :D. – M Z Jan 04 '21 at 03:17
  • 1
    I updated the link to github to point in the current revision to the relevant line numbers. – Michael Szczesny Jan 04 '21 at 03:34
  • 1
    @MZ I just created pure python [implementation](https://gist.github.com/abdulniyaspm/b116215839c5ad1751923278bf10f518)(for **readability**) of list reverse iterator. Would you mind adding on to this answer? – Abdul Niyas P M Jan 04 '21 at 03:53
  • @AbdulNiyasPM: It looks good, and seems to show the same behaviour as the cpython code. – Eric Duminil Jan 04 '21 at 17:38
9

calling reversed return a iterator over that list, which a special object that allow you iterate in reverse order over the original list, is not a new list and is a one time use only

>>> s= [1,2,3]
>>> t = reversed(s)
>>> t
<list_reverseiterator object at 0x00000261BE8F0C40>
>>> list(t)
[3, 2, 1]
>>> list(t)
[]
>>>

and because this iterator reference the original list, any change on it is reflected when you iterate over the iterator later.

Update

In particular and as MZ explain, if that change is such that the state of the list is different from when the iterator was created you get nothing if the size decreases or an incomplete version of the list if increased

>>> s= [1,2,3]
>>> t = reversed(s)
>>> s.insert(0,23)
>>> s
[23, 1, 2, 3]
>>> list(t)
[2, 1, 23]
>>> t = reversed(s)
>>> s.append(32)
>>> list(t)
[3, 2, 1, 23]
>>> s
[23, 1, 2, 3, 32]
>>> t = reversed(s)
>>> s.pop()
32
>>> list(t)
[]
>>>
Copperfield
  • 8,131
  • 3
  • 23
  • 29
  • 1
    This does not answer the question. The item is only removed from the list and the iterator is untouched. – Deadbeef Jan 04 '21 at 15:07
  • 3
    @Deadbeef The iterator is backed by the original list, though, so changing the state of the list affects the iterator in the sense that it is not necessarily valid after the alteration. In most languages, it's usually a Bad Idea to alter a collection while you're iterating over it using an iterator pattern. – sfdcfox Jan 04 '21 at 15:46
  • @Deadbeef: It does answer the question. `reversed(s)` is basically just a wrapper around `s`, so it's fast to initialize and doesn't require much memory. But... – Eric Duminil Jan 04 '21 at 17:35
  • at first I get the full extend of the question, but by then MZ put his, I leave mine as a little complement to that – Copperfield Jan 04 '21 at 21:40