Python list.remove() skips next element in list

Question

Python, but not programming, newbie here. I'm programming with lists and have run into an interesting problem.

width = 2
height = 2

# Traverse the board
def traverse(x, y):
    # The four possible directions
    squares = [(x - 1, y), (x + 1, y), (x, y - 1), (x, y + 1)]
    print squares

    # Remove impossible squares
    for square in squares:
        print "now accessing", square
        if (square[0] < 1 or square[0] > width or 
            square[1] < 1 or square[1] > height or
            square == (1, height)):
            squares.remove(square)
            print "removed", square
    print(squares)

# Testing traverse
traverse(1,1)

This results in the following output:

[(0, 1), (2, 1), (1, 0), (1, 2)]
now accessing (0, 1)
removed (0, 1)
now accessing (1, 0)
removed (1, 0)
[(2, 1), (1, 2)]

It completely skips the elements (2,1) and (1,2) -- without even checking them! I found answers here saying I shouldn't be modifying a list while traversing it, and yup, that definitely makes sense. Rookie mistake. But can someone tell me WHY it doesn't work? What's behind the veil of Python lists?

I would assume Python internally keeps track of the ordinal position in the list, and removing an item shifts that position down by one. — Mike Christensen, Jan 10 '13 at 21:40
I'm surprised this works at all. I could have sworn that altering an iterable while iterating it raises an exception. — Silas Ray, Jan 10 '13 at 21:41
I think you have the same issue as covered in this SO question: [Remove items from a list while iterating in Python](http://stackoverflow.com/questions/1207406/remove-items-from-a-list-while-iterating-in-python) — fiz, Jan 10 '13 at 21:43
@sr2222 For some types, such as `dict`, it does (though I think it's only a heuristic that can fail). Detecting this, without slowing down to a crawl, is pretty hard. — , Jan 10 '13 at 21:44
@sr2222: you can occasionally get `Set changed size during iteration` errors; likewise with `dict`s and `deque`s. Maybe that's what you're thinking of. — DSM, Jan 10 '13 at 21:45
@DSM That's exactly it. I have a bunch of 2-pass or index-based iterations in my code specifically because I was getting those exceptions. — Silas Ray, Jan 10 '13 at 21:47

score 19 · Answer 1 · answered Jan 11 '13 at 17:25

Others have explained that you should not remove elements from an array you are iterating over; however, if you traverse the array backwards, there is no problem.

The most compact way to solve this problem (assuming you don't need a copy of the original array after you are done) is to use the reversed() function, as in

for square in reversed(squares)

This will start iterating from the end of the array, and work backwards. Taking out elements in this way will not affect the rest of the code, since you are not changing the order of elements you have not yet visited. I think this is the most elegant way to solve this. I learnt this trick here

score 16 · Answer 2 · answered Jan 10 '13 at 21:39

16

The statement for square in squares just visits each item in the list in order: squares[0], then squares[1], then squares[2], and so on until it runs out of squares.

Removing squares[0] shifts all the other items in the list to the left one slot; the original squares[1] is now squares[0], so the for loop will skip over it.

answered Jan 10 '13 at 21:39

Jason Orendorff

42,793
6
62
96

2

That definitely makes sense. Something I should've forseen. :) Thanks for the quick response! To fix, instead of removing from this list, I just add the ones I want to keep to a new list. Problem solved! – yuechen Jan 10 '13 at 21:43

score 2 · Answer 3 · answered Jan 10 '13 at 21:39

2

You are removing from a list while iterating over it. This is not a good idea.
There are other ways to do what you are trying, one of which is to track the list indices of all items that you want to remove from the list and then remove them outside your for-loop

answered Jan 10 '13 at 21:39

inspectorG4dget

110,290
27
149
241

this is the most practical solution of the problem as we may some time need to traverse list in natural order not in reverse order. – The EasyLearn Academy Dec 31 '21 at 03:55

Floris · Answer 4 · 2013-01-11T17:24:04.473

When you are accessing a list in Python and remove an element, the list becomes shorter. Simple example: take the list 1,2,3,4,5, and remove the prime numbers > 1.
If you look at the second element (2) and decide to remove it, you end up with a list that goes 1,3,4,5 Now you look at the third element (you have already looked at the second one), and you find it's 4. That's not prime, so you go to the fifth element - 5. Remove it.
Result: you have 1,3,4,5. If you had started from the other end of the array, things would work just fine. Make sense?

EDIT: I have created another answer below - other posters gave a better explanation of the problem than I gave here, but I think I have a more elegant solution.

score 1 · Answer 5 · answered Jan 10 '13 at 21:58

1

Don't modify lists that you are iterating over!

Instead, do something like:

for square in squares[:]:

so that you're iterating over a copy of the list while still changing the original.

answered Jan 10 '13 at 21:58

muad_dib

143
1
8

score 0 · Answer 6 · answered Sep 09 '20 at 17:09

Removing array elements inside for loop is not advisable because for loop checks indexes in a order and removing an element inside loop shifts proceeding elements left one index, if you want to do it then use reversed()

arr = [1,2,3,4,5]

for ele in reversed(arr): condition: arr.remove(ele)

Python list.remove() skips next element in list

6 Answers6

Linked

Related