Why strange list comprehension behavior with side effects?

Question

I know that using side effects in Python list comprehensions is not good practice. But I can't understand why something like the following happens:

In [66]: tmp = [1,2,3,4,5]; [tmp.remove(elem) for elem in tmp]
Out[66]: [None, None, None]

In [67]: tmp
Out[67]: [2, 4]

Whether or not this is good practice, shouldn't the interior of the list comprehension do something predictable? If the above is predictable, can someone explain why only three remove operations occurred, and why the even entries are the ones that remain?

http://stackoverflow.com/questions/2442651/how-to-delete-an-element-from-a-list-while-iterating-it-in-python?rq=1 — Josh Lee, Oct 16 '12 at 23:26
@JoshLee I don't need to delete the elements, per se, I just wanted to understand why the `None or...` stuff wasn't working. It turns out it was working and I didn't realize the index was changing. — ely, Oct 16 '12 at 23:27
Also, in the example this derives from, the part where I am putting `None` is actually an expensive computation that may result in returning `None`, in which case deleting the items from `tmp` where that occurs *may* be a viable choice for me. But either way, it won't be good to also include an `if` clause at the end of the list comprehension because it would require evaluating that expensive function again to know what to keep. — ely, Oct 16 '12 at 23:29
`None` may be returned by an expensive function in my actual code. — ely, Oct 16 '12 at 23:31
Of course. But to distill this example down to its essence, it could help to realize that every `None` in your result is actually the return value from `tmp.remove`. — Josh Lee, Oct 16 '12 at 23:32
That's a good point. I'll edit. But I think this is still different from the linked question in which a simple conditional in the comprehension solves the problem. — ely, Oct 16 '12 at 23:34

score 6 · Accepted Answer · answered Oct 16 '12 at 23:21

This isn't about listcomps, it's about removing from lists you're iterating over:

>>> tmp = [1,2,3,4,5]
>>> for elem in tmp:
...     tmp.remove(elem)
... 
>>> tmp
[2, 4]

It goes something like this:

>>> tmp = [1,2,3,4,5]
>>> for elem in tmp:
...     print elem, tmp
...     tmp.remove(elem)
...     print elem, tmp
... 
1 [1, 2, 3, 4, 5]
1 [2, 3, 4, 5]
3 [2, 3, 4, 5]
3 [2, 4, 5]
5 [2, 4, 5]
5 [2, 4]

First it's looking at the 0th element, and 1 is removed. So on the next iteration, it wants to remove the 1st element, which is now the 3, etc.

Gotcha, so if I make it `[None or tmp.remove(elem) for elem in list(tmp)]` or `[None or tmp.remove(elem) for elem in tmp[:]]` then it should work "as expected" (still being bad style and all). — ely, Oct 16 '12 at 23:25

Santiclause · Answer 2 · 2012-10-16T23:26:47.157

It's never a good idea to remove from a list as you're iterating over it directly. Edit: whoops, I was confusing lists with dictionaries. Dictionaries have an iteritems() and iterkeys() method which is what you would use when you're iterating over a dictionary and removing items from it.

For a list, you would probably want to copy the list if you want to do this inside a list comp. Or, alternately:

[None and tmp.pop(0) for i in xrange(len(tmp))]

Why strange list comprehension behavior with side effects?

2 Answers2