python iterating over list

Question

I try to iterate over each item in the list. But iterator does not go through all objects. Here is my function code(dirs and root comes from os.walk)

def remove_hidden_dirs(dirs,root):
    """
    Function remove hidden directories . .. in unix
    """
    logging.debug("\n\n remove hidden" )
    logging.debug('Start removing hidden dirs for %s',root)
    logging.debug("Length of dirs %s",len(dirs))
    logging.debug('File list before removing')
    dirs.sort()
    for d in dirs:
        logging.debug(d)
    i=0
    for d in dirs:
        i=i+1
        logging.debug("Iterating over %s",d)
        if d[0:1] == '.' or d[0:2] =='..':
            dirs.remove(d)
            logging.debug("Dir %s is being removed because of being hidden dir",d)
        else:
            logging.debug("Dir %s is NOT being removed because of being hidden dir",d)
    logging.debug("Iterate on %s", i)
    logging.debug('File list after removing')
    for d in dirs:
        logging.debug(d)
    logging.debug('end remove hidden files\n\n')
    return dirs

And here is part of my log.file

DEBUG:root:Start removing hidden dirs for /home/artur/
DEBUG:root:Length of dirs 38
DEBUG:root:File list before removing
DEBUG:root:.adobe
DEBUG:root:.android
DEBUG:root:.cache
DEBUG:root:.config
DEBUG:root:.dbus
...
DEBUG:root:Iterate on 22 dirs
DEBUG:root:File list after removing
DEBUG:root:.adobe
DEBUG:root:.cache
DEBUG:root:.dbus
...

Thanks in advance for any help

score 9 · Answer 1 · answered Jun 18 '11 at 14:58

9

Don't modify lists while iterating over them. Instead, produce a new list with just the items that you want to keep.

dirs = [d for d in dirs if not d.startswith('.')]

Is an efficient simple and readable technique for doing it.

answered Jun 18 '11 at 14:58

Winston Ewert

44,070
10
68
83

score 4 · Accepted Answer · edited May 23 '17 at 11:55

4

Very simple: you are removing the current item from dirs while iterating over it. That means you skip two forward on the next loop. See here.

If what I said wasn't clear, say you iterate over this list of numbers:

[1, 2, 3, 4, 5, 6...]
 ^

That's the state of the list initially; then 1 is removed and the loop goes to the second item in the list:

[2, 3, 4, 5, 6...]
    ^

Since you've removed 1 and moved forward by one in the list, now you're at 3, not 2.

[2, 4, 5, 6...]

Then you remove 3... and so on.

edited May 23 '17 at 11:55

Community

1
1

answered Jun 18 '11 at 14:56

senderle

145,869
36
209
233

1

A quick fix is to duplicate the list by slicing it: for d in dirs[:] – Peter Lyons Jun 18 '11 at 14:58
More importantly: this means the loop skips certain entries, because they've all shifted one place. – Jasmijn Jun 18 '11 at 14:59
@Robin, that's what I said. Isn't it? – senderle Jun 18 '11 at 15:00
@senderle, the answer didn't say that when I commented, but now it does. – Jasmijn Jun 18 '11 at 15:02
@Robin, I'll clarify: that's what I meant when I said "That means you skip two forward on the next loop," which _was_ in the very first version of my post. – senderle Jun 18 '11 at 15:11
Thanks for explaining this. Thanks Peter for sugestion it is worth to remember – John Jun 18 '11 at 15:18

python iterating over list

2 Answers2