Automated way to list files from multiple directories in Python

Question

I need to create a list holding all files from multiple directories.

I have all_dir which contains dir1, dir2, dir3.... Each directory contains multiple files ['text1.txt','text2.txt'...]. While I'm capable of creating list of single directories, I can't find the way to automate.

This is what I have and it work for the single directory.

path = '../all_dir'
list1 = [f for f in os.listdir(os.path.join(path, 'dir1')
list2 = [f for f in os.listdir(os.path.join(path,'dir1') 
#etc...

This would be the code I'm thinking of:

all_list = []

for dir1 in os.listdir(path):
    current = os.listdir(os.path.join(path,dir1))
    all_list.append(current)

But this for loop raise: NotADirectoryError

To fix this I've tried

all_list = []

for dir1 in os.listdir(path):
    current = os.walk(os.path.join(path,dir1))
    all_list.append(current)

But this loop raises a list of <generator object walk at 0x100ca4e40>

Could you help?

Does this answer your question? [Python recursive folder read](https://stackoverflow.com/questions/2212643/python-recursive-folder-read) — Tomerikoo, Oct 04 '21 at 08:19
Not sure what you need here but it looks like you're hoping to get a list of all files in the sub-directories of ../all_dir If that is the case then consider using *glob* with appropriate wildcards — , Oct 04 '21 at 08:26

matebende · Accepted Answer · 2021-10-05T08:55:27.697

3

listdir also gives back files, so in the for loop you should do a check, if it is directory. You can use os.path.isdir()

for dir1 in os.listdir(path):
    full_path = os.path.join(path, dir1)
    if os.path.isdir(full_path):
        current = os.listdir(full_path)
        all_list += current

edited Oct 05 '21 at 08:55

answered Oct 04 '21 at 08:22

matebende

543
1
7
21

If you want to check whether it is a file that you found in the subdirectory, you should use `os.path.isfile(os.path.join(full_path, current))` on it – matebende Oct 04 '21 at 09:29

score 0 · Answer 2 · answered Oct 19 '21 at 05:23

#Navigate to the location where all_dir is
os.chdir('../all_dir')

#l is the list of paths of each directory in all_dir  
l = []
for folder in os.listdir(os.getcwd()):
    path = os.path.join(os.getcwd(), folder)
    l.append(path)

#all files in all directories will be stored in all_list
all_list=[]    
for i in li:
    os.chdir(i)
    #'*.*' prints files only (ended with any extension), if you want to print everything (including) in each directory use '*'
    all_files = glob.glob('*.*')
    for f in all_files:
        all_list.append(f)  

    
#number of all files
len(all_list)
#print all files
print(all_list)

Automated way to list files from multiple directories in Python

2 Answers2