Extract the csv files from the sub directory python

Question

I have the local path (parent directory) and I would like to extract only the paths that contain csv and saves them in a csv file.

What I tried so far?

import os

directory = os.path.join("path/to/dir","path")
for root,dirs,files in os.walk(directory): 
    for file in files: 
        if file.endswith(".csv"): 
            f=open(file, 'r')
            f.close()

This does not extract all csv and saves it. How do I do that?

import csv and https://stackoverflow.com/questions/37289951/how-to-write-to-a-csv-line-by-line — Yuri, Jul 13 '21 at 14:44
Well, what do you want in your CSV? Show your desired output as text. — , Jul 13 '21 at 14:48
From your script, it seems you only open the CSV file and close it. I'm not sure what you're trying to achieve. — Alexper, Jul 13 '21 at 15:43

score 1 · Accepted Answer · answered Jul 13 '21 at 19:08

I think you don't really need to use os.walk function. Instead glob has the recursive functionality that can get you exactly what you want.

from glob import glob
import csv
import os

parent_directory = "/parent/directory/"
save_file = "/save/directory/csv_of_csvs.csv"
csv_files_list = glob(pathname=parent_directory + "**/*.csv", recursive=True)
folder_list = [os.path.dirname(i) for i in csv_files_list]
folder_list = set(folder_list)

with open(save_file, 'w', newline ='') as csv_file:
    write = csv.writer(csv_file, delimiter=',')
    for i in folder_list:
        write.writerow([i])
exit()

score 0 · Answer 2 · answered Jul 13 '21 at 16:01

If I understand, you only want to log the path of CSV files in the folder.

This should do, for all the recursive paths:

import os

directory = os.path.join("path/to/dir","path")

with open("all_files.csv", "w") as f:
    for root,dirs,files in os.walk(directory): 
        for file in files: 
            if file.endswith(".csv"): 
                f.write(f"{root}/{file}")

In case you want one CSV file per directory:

for root,dirs,files in os.walk(directory): 
    with open(f"{root}/files.csv", "w") as f:
        for file in files: 
            if file.endswith(".csv"): 
                f.write(f"{root}/{file}")

Extract the csv files from the sub directory python

2 Answers2