19
from pprint import *

sites = [['a','b','c'],['d','e','f'],[1,2,3]]

pprint(sites)

for site in sites:
        sites.remove(site)

pprint(sites)

outputs:

[['a', 'b', 'c'], ['d', 'e', 'f'], [1, 2, 3]]
[['d', 'e', 'f']]

why is it not None, or an empty list [] ?

Alexander Cameron
  • 309
  • 1
  • 4
  • 11
  • 5
    I don't work in python so I'm not sure, but it's probably because you're modifying the list while iterating over it. Lots of weird things tend to happen when you do that. – Paul Phillips Aug 26 '11 at 21:10

3 Answers3

54

It's because you're modifying a list as you're iterating over it. You should never do that.

For something like this, you should make a copy of the list and iterate over that.

for site in sites[:]:
    sites.remove(site)
  • 15
    'You should never do that' deserves more emphasis. – job Aug 26 '11 at 21:22
  • 8
    Thank you. I will never do this. – Alexander Cameron Aug 26 '11 at 22:01
  • hey i used your code but i still get the same problem! i do not know if i can ask here? '#len(MAC1_list) = 1845 for key in MAC1_list[:]: if(condition): #it does enter about 10 times to the condition (tried using print) MAC1_list.remove(key) #len(MAC1_list) = 1845' – Gonzalo Dec 04 '18 at 15:34
  • 1
    wow this is interesting behavior. although you are iterating over the list you are ediing,.. you are not doing it by index! there is inherently a shallow copy of the data being created here just for the implementation of 'in'. – roberto tomás Apr 23 '19 at 01:00
  • It is very strange because it works sometimes... but other times, the remove doesn't work when iterating over itself. I will never edit a list I am iterating over and always make a copy going forward. Thanks! – embulldogs99 Mar 22 '21 at 17:11
12

Because resizing a collection while iterating over it is the Python equivalent to undefined behaviour in C and C++. You may get an exception or subtly wrong behaviour. Just don't do it. In this particular case, what likely happens under the hood is:

  • The iterator starts with index 0, stores that it is at index 0, and gives you the item stored at that index.
  • You remove the item at index 0 and everything afterwards is moved to the left by one to fill the hole.
  • The iterator is asked for the next item, and faithfully increments the index it's at by one, stores 1 as the new index, and gives you the item at that index. But because of said moving of items caused by the remove operation, the item at index 1 is the item that started out at index 2 (the last item).
  • You delete that.
  • The iterator is asked for the next item, but signals end of iteration as the next index (2) is out of range (which is now just 0..0).
  • Nothing in Python is really equivalent to C or C++'s undefined behaviour, except maybe playing around with the bytecode manually via deep magic. Modifying a collection while iterating over it behaves in pretty predictable ways. It's just that none of them are particularly useful, and even if they were, you could write the same thing much more clearly in other ways. – Karl Knechtel Aug 27 '11 at 06:11
  • @Karl: (1) As far as I know, the result is not documented anywhere, it just says it's "not safe". That would make the behaviour an implementation detail. (2) Some undefined behaviour in C and C++ is pretty predictable too as long as you stick to one version of one implementation ;) But yes, if you feel like nitpicking, you're right that it isn't quite as bad as serious undefined behaviour. –  Aug 27 '11 at 12:24
3

Normally I would expect the iterator to bail out because of modifying the connected list. With a dictionary, this would happen at least.

Why is the d, e, f stuff not removed? I can only guess: Probably the iterator has an internal counter (or is even only based on the "fallback iteration protocol" with getitem).

I. e., the first item yielded is sites[0], i. e. ['a', 'b', 'c']. This is then removed from the list.

The second one is sites[1] - which is [1, 2, 3] because the indexes have changed. This is removed as well.

And the third would be sites[2] - but as this would be an index error, the iterator stops.

glglgl
  • 89,107
  • 13
  • 149
  • 217