Unexpected behavior with remove items from list

Question

For starters, I'm new to python and coding in general.

I'm trying to remove desktop.ini files form a zip folder I am creating. I'm using the following to do this

if 'desktop.ini' in item:
     contents.remove(item)
     continue

However, the item in the list after desktop.ini is also getting removed and I have no idea why. Any ide why this is happening?

Here is the entire script if more context is needed

import shutil
import zipfile
import tempfile
from distutils.dir_util import copy_tree

def fileList(fileName):
    files = list()
    for (dirpath, dirnames, filenames) in os.walk(fileName):
        files += [os.path.join(dirpath, file) for file in filenames]
    return files

print("What version are these macros? ")
versNo = str(input())
path = os.path.dirname(os.path.realpath(__file__))
mac = 'macros'

with tempfile.TemporaryDirectory() as macfolder:
    copy_tree(os.path.join(path,mac),macfolder)
    with zipfile.ZipFile(os.path.join(path, "Release Assets",'Nautilus_Macros_'+versNo+'.zip'),'a') as maczip:
        contents = fileList(macfolder)
        #print(contents)
        for item in contents:
            #print(item)
            if 'desktop.ini' in item:
                    contents.remove(item)
                    continue
            #print(item)
            f = open(item,'r+')
            lines = f.readlines()
            f.seek(0)
            f.write('; Macro for the Nautilus 3D printer by Hydra Research \n')
            f.write('; Version: '+versNo+' \n \n')
            for line in lines:
                f.write(line)
            f.close()
            print(item)
            maczip.write(os.path.join(macfolder, item), os.path.relpath(item, macfolder))
    maczip.close()

The linked question marked as a duplicate is not really a duplicate; the solution is not to remove the item from the list at all. Just `continue` without removing it. — kaya3, Nov 14 '19 at 01:07
@kaya3 Good call. I've reopened the issue so you can point that out in an answer. — Jonathon Reinhart, Nov 14 '19 at 01:09

score 0 · Accepted Answer · answered Nov 14 '19 at 01:17

Sirens should blare and warning lights should flash any time you remove something from a list (or other collection) while you are iterating over it.

Consider how iterating over a list actually works: there is some variable (let's call it i) which keeps track of the index of the current item. On each iteration of the loop, i increases by 1, and the loop stops when i >= len(contents).

When you remove an element from the list at the index i, all the elements after it are shifted left by 1, to fill the "gap" where the removed element was. The element previously at index i + 1 is now at index i. But i still gets increased by 1, ready for the next iteration, and the next element you read comes from index i + 1. That element used to be at index i + 2, so you've skipped an element.

For this kind of reason, you should never add or remove elements from a collection while you're iterating over that collection in a for loop.

In this case, just using continue is enough to make sure that element doesn't get written to maczip; it skips the loop to the next element from contents, so you don't also need to remove the current element from contents. Change the code this way:

if 'desktop.ini' in item:
    # don't do contents.remove(item)
    continue

Unexpected behavior with remove items from list

1 Answers1