How to ignore hidden files using os.listdir()?

Question

My python script executes an os.listdir(path) where the path is a queue containing archives that I need to treat one by one.

The problem is that I'm getting the list in an array and then I just do a simple array.pop(0). It was working fine until I put the project in subversion. Now I get the .svn folder in my array and of course it makes my application crash.

So here is my question: is there a function that ignores hidden files when executing an os.listdir() and if not what would be the best way?

you may also want to avoid folders with `os.path.isdir()` – Kermit Dec 09 '20 at 14:45 — Kermit, Dec 09 '20 at 14:45

score 143 · Accepted Answer · edited Oct 15 '21 at 12:36

143

You can write one yourself:

import os

def listdir_nohidden(path):
    for f in os.listdir(path):
        if not f.startswith('.'):
            yield f

Or you can use a glob:

import glob
import os

def listdir_nohidden(path):
    return glob.glob(os.path.join(path, '*'))

Either of these will ignore all filenames beginning with '.'.

edited Oct 15 '21 at 12:36

zoltron

67
6

answered Aug 17 '11 at 20:53

Adam Rosenfield

390,455
97
512
589

1

the proposed function `listdir_nohidden` is not quite compatible with `os.listdir`, since the use of `yield` makes it a generator. Instead it should run through the output list from `os.listdir` and remove entries that start with '.' – Milo Wielondek Mar 15 '12 at 20:26
3

@0sh: Why does it have to remove things in-place? Just define a new function that does `list(listdir_nohidden(path))` and that new function is exactly compatible with `os.listdir`. – abarnert Mar 05 '13 at 23:08
So in other words, there's no builtin regex in `listdir()` that allows you to customise your list. – DryLabRebel Feb 22 '23 at 02:22

Joshmaker · Answer 2 · 2021-12-22T20:38:08.190

71

This is an old question, but seems like it is missing the obvious answer of using list comprehension, so I'm adding it here for completeness:

[f for f in os.listdir(path) if not f.startswith('.')]

As a side note, the docs state listdir will return results in 'arbitrary order' but a common use case is to have them sorted alphabetically. If you want the directory contents alphabetically sorted without regards to capitalization, you can use:

sorted((f for f in os.listdir() if not f.startswith(".")), key=str.lower)

(Edited to use key=str.lower instead of a lambda)

edited Dec 22 '21 at 20:38

answered Oct 24 '14 at 19:27

Joshmaker

4,068
3
27
29

5

`key=lambda f: f.lower()` can be written without lambda: `key=str.lower` – Jean-François Fabre Jan 21 '19 at 14:12
2

To combine both: `sorted([f for f in os.listdir('./') if not f.startswith('.')], key=str.lower)` – rbv Mar 27 '20 at 10:00

score 29 · Answer 3 · edited May 25 '16 at 14:00

29

On Windows, Linux and OS X:

if os.name == 'nt':
    import win32api, win32con


def folder_is_hidden(p):
    if os.name== 'nt':
        attribute = win32api.GetFileAttributes(p)
        return attribute & (win32con.FILE_ATTRIBUTE_HIDDEN | win32con.FILE_ATTRIBUTE_SYSTEM)
    else:
        return p.startswith('.') #linux-osx

edited May 25 '16 at 14:00

tmsblgh

517
5
21

answered Dec 28 '12 at 00:26

cle

291
3
2

2

should also work on a Mac, hidden files start there with '.' as well. – Verena Haunschmid Jun 05 '13 at 09:58
3

This is the only portable answer, great job, but the other answers provide a full wrapper for os.listdir so... `[f for f in os.listdir(path) if not folder_is_hidden(f)]` – SensorSmith May 01 '16 at 20:45
Why do you check the `FILE_ATTRIBUTE_SYSTEM` flag too? Are system files hidden by default on Windows? – Jake Ireland Sep 23 '22 at 12:09

scum · Answer 4 · 2020-07-20T13:12:25.973

19

Joshmaker has the right solution to your question.
How to ignore hidden files using os.listdir()?

In Python 3 however, it is recommended to use pathlib instead of os.

from pathlib import Path 
visible_files = [
    file for file in Path(".").iterdir() if not file.name.startswith(".")
]

edited Jul 20 '20 at 13:12

answered Apr 29 '13 at 22:29

scum

3,202
1
29
27

Josh Lee · Answer 5 · 2013-10-29T13:55:06.207

17

glob:

>>> import glob
>>> glob.glob('*')

(glob claims to use listdir and fnmatch under the hood, but it also checks for a leading '.', not by using fnmatch.)

edited Oct 29 '13 at 13:55

answered Aug 17 '11 at 20:51

Josh Lee

171,072
38
269
275

user 923227 · Answer 6 · 2019-09-06T05:01:50.063

8

I think it is too much of work to go through all of the items in a loop. I would prefer something simpler like this:

lst = os.listdir(path)
if '.DS_Store' in lst:
    lst.remove('.DS_Store')

If the directory contains more than one hidden files, then this can help:

all_files = os.popen('ls -1').read()
lst = all_files.split('\n')

for platform independence as @Josh mentioned the glob works well:

import glob
glob.glob('*')

edited Sep 06 '19 at 05:01

answered May 18 '18 at 18:04

user 923227

2,528
4
27
46

That works only if you have _one_ hidden file, **and** you know the name of it. What if the directory contains dozens of hidden files, with arbitrary names that you can't know in advance? – FeRD Jul 21 '19 at 03:20
Hi @FeRD, Yes. When I am doing batch/backlog processing on a mac, I put all the files in new folder and `.DS_Store` gets created automatically. When I zip all the files and push it to a server `.DS_Store` also gets added. If there are various hidden files then may be you can try `os.system('ls -1')` – user 923227 Jul 22 '19 at 16:49
Not cross-platform. `os.popen('ls -1').read()` won't work on Windows. That's the whole point of `os.listdir()`. – FeRD Jul 24 '19 at 03:50

score 1 · Answer 7 · answered Dec 09 '17 at 05:15

1

filenames = (f.name for f in os.scandir() if not f.name.startswith('.'))

answered Dec 09 '17 at 05:15

hrsma2i

4,045
6
15
24

score -2 · Answer 8 · answered Sep 20 '21 at 12:11

You can just use a simple for loop that will exclude any file or directory that has "." in the front.

Code for professionals:

import os

directory_things = [i for i in os.listdir() if i[0] != "."] # Exclude all with . in the start

Code for noobs

items_in_directory = os.listdir()
final_items_in_directory = []

for i in items_in_directory:
    if i[0] != ".": # If the item doesn't have any '.' in the start
        final_items_in_directory.append(i)

How to ignore hidden files using os.listdir()?

8 Answers8

Linked

Related