Weird 'partial completion' of loop

Question

Okay this may be really trivial (I am pretty new to this), but I have been stuck at this for a while and I do not understand why my function below is returning such a weird result.

testx = [(1,2), (1,1), (2,2), (3,5), (4,4), (5,5)]

def test_loop(interval_set):
for item in interval_set:
    if item[0] == item[1]:
        interval_set.remove(item)
return interval_set

print test_loop(testx)

>>>[(1, 2), (2, 2), (3, 5), (5, 5)]

If you notice, only repetitive sets (1,1) and (4,4) got removed while (2,2) and (5,5) remained in the list. This is almost illogical, please assist.

http://stackoverflow.com/questions/1637807/modifying-list-while-iterating — NPE, Aug 10 '14 at 15:47
http://stackoverflow.com/questions/10812272/modifying-a-list-while-iterating-over-it-why-not — NPE, Aug 10 '14 at 15:47

score 4 · Answer 1 · answered Aug 10 '14 at 15:55

Iterating over a collection (dict, lists, sets) while removing things from it is bad as you will remove items, shortening the collection and breaking out of the loop earlier.

I would recommend doing a list comprehension, like this:

[item for item in testx if item[0] != item[1]]

to obtain the result you expect.

You can read the docs about the list comprehension here https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions

miindlek · Answer 2 · 2015-05-05T18:10:45.113

2

You shouldn't remove items from a list while you are iterating over the same list. Try to iterate over a copy of that list, like this:

testx = [(1,2), (1,1), (2,2), (3,5), (4,4), (5,5)]

def test_loop(interval_set):
    for item in list(interval_set):
        if item[0] == item[1]:
            interval_set.remove(item)
    return interval_set

print test_loop(testx)

Outputs:

[(1, 2), (3, 5)]

edited May 05 '15 at 18:10

answered Aug 10 '14 at 15:55

miindlek

3,523
14
25

Building a new list of the values to keep, instead of making a copy to iterate so you can delete from the original, is usually simpler. Especially for lists, because otherwise you have to do things backward. (For sets, it's not _as much_ of a problem, because you can just use `remove`, but it's still not the best way to do things.) – abarnert Aug 10 '14 at 16:08
@abarnert Right. I would also prefer doing a list comprehension. Just wanted to show the OP how he could minimally change his code and get it to work. – miindlek Aug 10 '14 at 16:17
Omg thanks. I can't believe one adding list() managed to change the result. Not sure I understand why python could not iterate directly, but know how to avoid it now! Thanks! – Vicknesh Kodak Manoselvam Aug 10 '14 at 16:42
1

@VickneshKodakManoselvam: It's very important to understand what adding one `list()` does (it makes a copy of the list), and why it makes a difference (which is explained in this answer and the other one, and probably in the links NPE gave you, so hopefully if you read all of them you can understand it). – abarnert Aug 10 '14 at 17:03
@VickneshKodakManoselvam Hey, I would still recommend reading about list comprehensions, that is a more pythonic way to achieve what you're trying. – Rafael Barros Aug 10 '14 at 20:14
Hey guys, yep I read up about list comprehensions from the link, it is a much faster way to do what I was trying to do. Will try to use that next time, Thanks! – Vicknesh Kodak Manoselvam Aug 11 '14 at 15:14

score 0 · Accepted Answer · answered Aug 10 '14 at 17:09

Just to add some illustration as to why modifying a sequence you are iterating over is a bad idea. Here is a list, the arrow indicates where the iterator is currently:

 v
[0, 1, 2, 3, 4, 5]

Now lets say you remove 0, what happens is the list shifts towards the front so it now looks like this, take note of where the iterator is:

 v
[1, 2, 3, 4, 5]

Now the iterator advances to the next item without ever evaluating/comparing/whatever 1:

    v
[1, 2, 3, 4, 5]

Now your code is back up at the first line of your for loop and it will execute your code on the value 2 but 1 was never evaluated.

Using a list comprehension is usually the best method for this however sometime a separate function is required due to the limitations of a comprehensions syntax.

def test_loop(interval_set):
    return [item for item in interval_set if item[0] != item[1]]

Also as tip for the future if you are building a set in the mathematical sense (no duplicates) you can always:

>>> interval_set = [(1,1), (1,1), (2,2), (3,5), (4,4), (4,4)]
>>> set(interval_set)
set([(1,1), (2,2), (3,5),(4,4)])

Weird 'partial completion' of loop

3 Answers3