97

I've got this piece of code:

numbers = list(range(1, 50))

for i in numbers:
    if i < 20:
        numbers.remove(i)

print(numbers)

But, the result I'm getting is:
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]

Of course, I'm expecting the numbers below 20 to not appear in the results. Looks like I'm doing something wrong with the remove.

Super Kai - Kazuya Ito
  • 22,221
  • 10
  • 124
  • 129
Finger twist
  • 3,546
  • 9
  • 42
  • 52
  • See also: https://stackoverflow.com/questions/1207406/how-to-remove-items-from-a-list-while-iterating. I reconsidered and decided that this is not a duplicate; this question is about understanding the failure of one specific wrong way to approach the problem, while the other question is about finding correct ways. – Karl Knechtel Jul 30 '22 at 00:29

12 Answers12

147

You're modifying the list while you iterate over it. That means that the first time through the loop, i == 1, so 1 is removed from the list. Then the for loop goes to the second item in the list, which is not 2, but 3! Then that's removed from the list, and then the for loop goes on to the third item in the list, which is now 5. And so on. Perhaps it's easier to visualize like so, with a ^ pointing to the value of i:

[1, 2, 3, 4, 5, 6...]
 ^

That's the state of the list initially; then 1 is removed and the loop goes to the second item in the list:

[2, 3, 4, 5, 6...]
    ^
[2, 4, 5, 6...]
       ^

And so on.

There's no good way to alter a list's length while iterating over it. The best you can do is something like this:

numbers = [n for n in numbers if n >= 20]

or this, for in-place alteration (the thing in parens is a generator expression, which is implicitly converted into a tuple before slice-assignment):

numbers[:] = (n for n in numbers if n >= 20)

If you want to perform an operation on n before removing it, one trick you could try is this:

for i, n in enumerate(numbers):
    if n < 20 :
        print("do something")
        numbers[i] = None
numbers = [n for n in numbers if n is not None]
    
Suyog Shimpi
  • 706
  • 1
  • 8
  • 16
senderle
  • 145,869
  • 36
  • 209
  • 233
  • 1
    Related note on `for` keeping an index from the Python docs https://docs.python.org/3.9/reference/compound_stmts.html#the-for-statement: "*There is a subtlety when the sequence is being modified by the loop (this can only occur for mutable sequences, e.g. lists). An internal counter is used to keep track of which item is used next, and this is incremented on each iteration. ... This means that if the suite deletes the current (or a previous) item from the sequence, the next item will be skipped (since it gets the index of the current item which has already been treated).*" – Gino Mempin Jan 31 '22 at 04:03
  • 1
    This is a good answer, but with the final solution, "if you want to perform an operation...", is slightly unsatisfactory because 1) in fact there is no need to include that qualification: it is just a waste of effort to try and remove elements while iterating in a single operation, so this 2-stage solution applies in all cases, and 2) because there should be a warning that setting to `None` won't always be appropriate (if some elements are *meant* to be `None`): some conventional "poison pill" value, appropriate to the case in hand, is needed. – mike rodent Mar 16 '22 at 12:08
  • When I say that about the "waste of effort", I am referring to the "general" solution by the way, i.e. when random elements may need removing, rather than the "specialist" case of removing elements only at the start (or only at the end), which lends itself to something simple, like your list comprehension solution... – mike rodent Mar 16 '22 at 12:27
21

Begin at the list's end and go backwards:

li = list(range(1, 15))
print(li)

for i in range(len(li) - 1, -1, -1):
    if li[i] < 6:
        del li[i]
        
print(li)

Result:

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14] 
[6, 7, 8, 9, 10, 11, 12, 13, 14]
Boris Verkhovskiy
  • 14,854
  • 11
  • 100
  • 103
eyquem
  • 26,771
  • 7
  • 38
  • 46
  • 2
    How I wish I could +2 this answer! Elegant, easy...not entirely obfuscated. – David Hempy Dec 08 '21 at 04:17
  • 1
    This is a very specialised answer: it is not in fact clear whether we're meant to be looking for a general solution to the problem of how to remove elements while iterating, or how to do this exclusively when we just want to remove the first n elements of a list. The chosen answer provides the former, which is infinitely more helpful, but also the latter, in the shape of a one-line list comprehension solution. – mike rodent Mar 16 '22 at 12:22
  • 2
    @mikerodent no it's not. It's pretty common when you want to modify a list while iterating over it that going backwards works – Boris Verkhovskiy Apr 07 '22 at 11:49
  • 1
    @Boris you haven't understood my comment. The OP's question does not specify that we are removing contiguous elements (either from the start or end of the list). – mike rodent Apr 07 '22 at 12:56
  • 1
    I still don't understand your comment then because it doesn't matter whether the list is shuffled or ordered, this code will still work. – Boris Verkhovskiy Apr 07 '22 at 12:57
11

@senderle's answer is the way to go!

Having said that to further illustrate even a bit more your problem, if you think about it, you will always want to remove the index 0 twenty times:

[1,2,3,4,5............50]
 ^
[2,3,4,5............50]
 ^
[3,4,5............50]
 ^

So you could actually go with something like this:

aList = list(range(50))
i = 0
while i < 20:
    aList.pop(0)
    i += 1

print(aList) #[21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]

I hope it helps.


The ones below are not bad practices AFAIK.

EDIT (Some more):

lis = range(50)
lis = lis[20:]

Will do the job also.

EDIT2 (I'm bored):

functional = filter(lambda x: x> 20, range(50))
Sneha Valabailu
  • 115
  • 2
  • 5
Trufa
  • 39,971
  • 43
  • 126
  • 190
3

So I found a solution but it's really clumsy...

First of all you make an index array, where you list all the index' you want to delete like in the following

numbers = range(1, 50)
index_arr = []

for i in range(len(numbers):
    if numbers[i] < 20:
        index_arr.append(i)

after that you want to delete all the entries from the numbers list with the index saved in the index_arr. The problem you will encounter is the same as before. Therefore you have to subtract 1 from every index in the index_arr after you just removed a number from the numbers arr, like in the following:

numbers = range(1, 50)
index_arr = []

for i in range(len(numbers):
    if numbers[i] < 20:
        index_arr.append(i)

for del_index in index_list:
    numbers.pop(del_index)

    #the nasty part
    for i in range(len(index_list)):
        index_list[i] -= 1

It will work, but I guess it's not the intended way to do it

1

As an additional information to @Senderle's answer, just for records, I thought it's helpful to visualize the logic behind the scene when python sees for on a "Sequence type".

Let's say we have :

lst = [1, 2, 3, 4, 5]

for i in lst:
    print(i ** 2)

It is actually going to be :

index = 0
while True:
    try:
        i = lst.__getitem__(index)
    except IndexError:
        break
    print(i ** 2)
    index += 1

That's what it is, there is a try-catch mechanism that for has when we use it on a Sequence types or Iterables(It's a little different though - calling next() and StopIteration Exception).

*All I'm trying to say is, python will keep track of an independent variable here called index, so no matter what happens to the list (removing or adding), python increments that variable and calls __getitem__() method with "this variable" and asks for item.

S.B
  • 13,077
  • 10
  • 22
  • 49
1

Building on and simplying the answer by @eyquem ...

The problem is that elements are being yanked out from under you as you iterate, skipping numbers as you progress to what was the next number.

If you start from the end and go backwards, removing items on-the-go won't matter, because when it steps to the "next" item (actually the prior item), the deletion does not affect the first half of the list.

Simply adding reversed() to your iterator solves the problem. A comment would be good form to preclude future developers from "tidying up" your code and breaking it mysteriously.

for i in reversed(numbers): # `reversed` so removing doesn't foobar iteration
  if i < 20:
    numbers.remove(i)
David Hempy
  • 5,373
  • 2
  • 40
  • 68
0

You could also use continue to ignore the values less than 20

mylist = []

for i in range(51):
    if i<20:
        continue
    else:
        mylist.append(i)
print(mylist)
Moses
  • 1,391
  • 10
  • 25
0

Since Python 3.3 you may use the list copy() method as the iterator:

numbers = list(range(1, 50))

for i in numbers.copy():
    if i < 20:
        numbers.remove(i)
print(numbers)

[20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
Cosmittus
  • 637
  • 6
  • 19
  • This looks like a neat suggestion to a classic problem, but I'm not sure that in practice it will prove better than the 2-stage solution suggested at the end of the chosen answer: firstly, what is the cost of the `copy` operation? Secondly, you have to concern yourself with whether this is a deep or shallow copy. In this trivial case the question doesn't arise but that won't always be so. – mike rodent Mar 16 '22 at 12:13
0

You can use list() for numbers to create a different copied numbers as shown below:

numbers = list(range(1, 50))
       # ↓ ↓ Here ↓ ↓
for i in list(numbers):
    if i < 20:
        numbers.remove(i)

print(numbers) # [20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 
               #  31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 
               #  f42, 43, 44, 45, 46, 47, 48, 49]
Super Kai - Kazuya Ito
  • 22,221
  • 10
  • 124
  • 129
0

Making a shallow copy of numbers for iteration, will ensure that the list use for iteration is not modified. The shallow copy can be made using list(), or copy.copy(), and will ensure that the identify of the elements to remove is the same, and reduce the size of the temporary list. Code:

...
for i in list(numbers):
    if i < 20:
        numbers.remove(i)
...
Morten Zilmer
  • 15,586
  • 3
  • 30
  • 49
0

Well, I needed a sort of this.
The answer was to use while statement.
It really helped me in my case. So I'd like to post it as an answer.

numbers = list(range(1, 50))
while numbers:
    current_number = numbers[0]
    if current_number < 20:
        numbers.remove(current_number)
    else:
        break
numbers
>>> [20, 21, 22, 23, 24, 25, ..., 49]

I also wanted to add my special case using while and for using [:], but it might be out of the scope, so I'll keep it.

HyeonPhil Youn
  • 428
  • 4
  • 11
0

A bit late but I feel like adding my answer. I found the following trick which is fairly simply

for x in list[::-1]:
    if validate(x):
        list.pop(x)

This way, you iterate starting from the end. When you remove the n-th element, the elements n+1,n+2... decrease their index by one, but because you are going backwards this won't intact the following (or preceding, in this case) elements..