Unexpected output of for.. in loop in Python

Question

I wanted to run a for loop for checking every letter in a string and if wanted list contained the letter, then I wanted the letter to stay and if wanted did not contain the letter, then I wanted to remove it.

phrase = "Don't panic"
pList = list(phrase)
print(pList)
wanted = ["o","n","t","a","p"]
for anything in pList:
    print("Now checking", anything)
    if anything not in wanted:
        print("Removed",anything) 
        pList.remove(anything)
    elif anything in wanted:
        print("Didn't remove", anything)

print(pList)

The following code returns this output-

['D', 'o', 'n', "'", 't', ' ', 'p', 'a', 'n', 'i', 'c']                                                   

Now checking D                                                                                       
Removed D                                                                                                                                                                                           
Now checking n                                                                                          
Didn't remove n                                                                                         
Now checking '                                                                                           
Removed '                                                                                                                                                                                               
Now checking                                                                                              
Removed                                                                                                                                                                                                 
Now checking a                                                                                           
Didn't remove a                                                                                         
Now checking n                                                                                           
Didn't remove n                                                                                           
Now checking i                                                                                          
Removed i                                                                                                                                                                                       
['o', 'n', 't', 'p', 'a', 'n', 'c']

The code doesn't remove the last letter (i.e. "c") and letters "o", "t", "p" and "c" didn't display the Now checking output.

I tried removing and some lines and just ran -

phrase = "Don't panic"
pList = list(phrase)
print(pList)
wanted = ["o","n","t","a","p"]
for anything in pList:
    print("Now checking", anything)

This time the output made sense -

Now checking D                                                                                            
Now checking o                                                                                            
Now checking n                                                                                            
Now checking '                                                                                            
Now checking t                                                                                            
Now checking                                                                                              
Now checking p                                                                                            
Now checking a                                                                                            
Now checking n                                                                                            
Now checking i                                                                                            
Now checking c

So why doesn't this happen when the full code is run?

never change the list you are iterating over (`pList`)! (except you really know what you are doing). — hiro protagonist, Jun 12 '17 at 05:51
it certainliy is; this may help: https://stackoverflow.com/questions/1207406/remove-items-from-a-list-while-iterating — hiro protagonist, Jun 12 '17 at 05:54
Possible duplicate of [Remove items from a list while iterating](https://stackoverflow.com/questions/1207406/remove-items-from-a-list-while-iterating) — hiro protagonist, Jun 12 '17 at 05:54
You can simply do `print([x for x in pList if x in wanted])`. Modifying the list while iterating over it might provide unexpected result. (unless we know what exactly is being done) — voidpro, Jun 12 '17 at 06:00

score 1 · Accepted Answer · answered Jun 12 '17 at 05:53

You're editing the list as you iterate over it. Because of this, the size will change midway and you end up getting weird behaviour. You can instead create a new list to hold your filtered chars.

phrase = "Don't panic"
pList = list(phrase)
new_plist = []
wanted = ["o","n","t","a","p"]
for i, anything in enumerate(pList):
    print("Now checking", anything)
    if anything in wanted:
        print("Didn't remove", anything)
        new_plist.append(anything)
    else:
        print("Removed", anything)

print(new_plist)

A simpler way using list comprehensions is

pList = ''.join([i for i in pList if i not in wanted])

score 1 · Answer 2 · answered Jun 12 '17 at 06:05

Other answers and comments are 100% correct - never modify a list while iterating over it! Something that may help with these sort of problems is thinking about the problem in the inverse: instead of removing from a list, invert the conditions and add to a new list. This way it builds up a new output based on the input rather than mutating the input = less side effects, cleaner code, functional programming principles, etc. @coldspeed's list comprehension is an elegant solution in this way.

An alternative solution with minimal changes to your original code:

phrase = "Don't panic"
pList = list(phrase)
print(pList)
wanted = ["o","n","t","a","p"]
finalList = []
for anything in pList:
    print("Now checking", anything)
    if anything in wanted:
        finalList.append(anything)
        print("Didn't remove", anything)
    else:
        print("Removed",anything)

print(finalList)

Now pList doesn't get modified, and the new finalList contains the required result. (EDIT: realized @coldspeed's code is now almost exactly the same as this - wasn't that way when I originally submitted.)

Unexpected output of for.. in loop in Python

2 Answers2