3

I tried this:

list = [[1], [2], [], [4], [5], [], [], [], [9]]
for e in list:
    if e == []:
        list.remove(e)

Then I got 

>>> list
[[1], [2], [4], [5], [], [9]]

What happened?!

thefourtheye
  • 233,700
  • 52
  • 457
  • 497
ghchoi
  • 4,812
  • 4
  • 30
  • 53
  • possible duplicate of [Loop "Forgets" to Remove Some Items](http://stackoverflow.com/questions/17299581/loop-forgets-to-remove-some-items) – Ashwini Chaudhary Feb 21 '15 at 09:49

2 Answers2

2
for e in list:
    if e == []:
        list.remove(e)

In this case, when e is an empty list, that particular value is removed from the list. So, the next element in the list becomes the current element. In the next iteration, next element is taken from the list. Since the actual next element has been shifted backwards, the next to actual next element will be taken. So, there is a chance that elements are skipped, if there are more than one continuous matches.

In your case, lets say e is at [] after the [2]

[[1], [2], [], [4], [5], [], [], [], [9]]
            ^

Since it is an empty list, it will be removed and the list becomes 

[[1], [2], [4], [5], [], [], [], [9]]
            ^

In the next iteration, next element will be picked, so 

[[1], [2], [4], [5], [], [], [], [9]]
                 ^

In this case, we skipped [4] silently. But, since it is expected in the output, it didn't make any difference. But consider the [] after [5],

[[1], [2], [4], [5], [], [], [], [9]]
                      ^

That will be removed, so the list becomes like this

[[1], [2], [4], [5], [], [], [9]]
                      ^

Now, in the next iteration, e will refer the next element, which would be

[[1], [2], [4], [5], [], [], [9]]
                          ^

So, the current empty list is removed and the [9] becomes the current element. That is how, one of the empty lists is skipped.

Note: The bottom line is, never remove an item from a container, while you are iterating it.

Solution 1:

The most common technique is to use a copy of the list being iterated, for the iteration and the actual list will be modified in the loop. For example,

for e in list[:]:       # `list[:]` is a shallow copy of `list`
    if e == []:
        list.remove(e)

Solution 2: (Preferred way to do this)

Use list comprehension, with a filtering condition to create a new list, like this

>>> [e for e in list if e]
[[1], [2], [4], [5], [9]]

Note the way empty lists are eliminated. Empty lists in Python are considered as Falsy values. So, if e will evaluate to Truthy only when e has atleast one element in it.

thefourtheye
  • 233,700
  • 52
  • 457
  • 497
  • The built-in function `filter` also provides this functionality. As an alternative to the list comprehension you can use `filter(None, list)`. – nelfin Feb 21 '15 at 06:26
  • @nelfin True, but [LC is preferred over `filter`](http://www.artima.com/weblogs/viewpost.jsp?thread=98196) in Python – thefourtheye Feb 21 '15 at 06:28
  • @nelfin List comprehension is faster than the `filter` and `map` methods, and should be preferred where possible. – aruisdante Feb 21 '15 at 06:29
  • Taking microbenchmarks with a grain of salt, list comprehensions are in fact slower in this case: `~ $ python -m timeit -s 'import random; l = [[] if random.random() < 0.5 else [1] for _ in range(1000000)]' '[x for x in l if x]'` `10 loops, best of 3: 44.9 msec per loop` `~ $ python -m timeit -s 'import random; l = [[] if random.random() < 0.5 else [1] for _ in range(1000000)]' 'filter(None, l)'` `10 loops, best of 3: 25.7 msec per loop` – nelfin Feb 21 '15 at 06:43
  • @nelfin Aren't those two tests running on two different random lists? – thefourtheye Feb 21 '15 at 06:44
  • Law of large numbers. Scale it down to 10,000 elements and you see that `filter` is still twice as fast as a list comprehension even over many trials. Sure, they're different lists, but the distribution of elements is similar. I'm not advocating this, list comprehensions are preferable, but you (well @ariusdante in this case) should always be willing to defend statements like "list comprehension is faster than `filter` and `map`" – nelfin Feb 21 '15 at 06:49
  • @nelfin Be aware that in python 3, filter returns an iterator, so filtering hasn't occurred yet after calling filter. If you're using python 3, you'd need `list(filter(None, l))` for the benchmark to make any sense. – user2313067 Feb 21 '15 at 08:42
0

Language independent solution: iterating backwards

l = [[1], [2], [], [4], [5], [], [], [], [9]]
for e in reversed(l):
    if e == []:
        l.remove(e)
ghchoi
  • 4,812
  • 4
  • 30
  • 53