Use Every CSV file in a Folder

Question

I'm trying to have this program use all of the CSV files inside of a folder. The folder is on my desktop, and it's called "Web_Scraper" (I'm on a mac btw).

This is how the code looks; what do I replace whatever is inside of the "enumerate" with (the 1.csv, 2.csv, 3.csv, etc)?

from openpyxl import Workbook
import csv
import os

wb = Workbook()
ws = wb.worksheets[0]

header_keys = []
for n, fName in 

enumerate(['3.csv','4.csv','5.csv','6.csv','7.csv','8.csv','9.csv','10.csv','11.csv','12.csv',]):
with open(fName) as fh:
    csv_reader = csv.DictReader(fh, fieldnames=['header', 'data'], delimiter=',')
    if n == 0:
        for values in csv_reader:

I think I'm supposed to use something like os.listdir but I'm not exaclty sure what the syntax should be.

Possible duplicate of [How do I list all files of a directory?](https://stackoverflow.com/questions/3207219/how-do-i-list-all-files-of-a-directory) — user1427026, Aug 03 '17 at 05:16
Possible duplicate of [Find all files in a directory with extension .txt in Python](https://stackoverflow.com/questions/3964681/find-all-files-in-a-directory-with-extension-txt-in-python) — SwiftsNamesake, Aug 03 '17 at 05:16

score 1 · Answer 1 · answered Aug 03 '17 at 05:07

1

You can take advantage of the glob module

import glob

scraper_files = glob.glob('*.csv') //returns an array of filenames

answered Aug 03 '17 at 05:07

Josh Hamet

957
8
10

score 1 · Answer 2 · answered Aug 03 '17 at 05:12

you can use something like below,

import os
from pathlib import Path

rootdir = 'xx/xx/x'

for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        if ('.csv' in Path.suffix)
            csv_reader = csv.DictReader(file, fieldnames=['header', 'data'], delimiter=',')

score 0 · Answer 3 · answered Aug 03 '17 at 05:10

0

You can use os module

import os
os.listdir('~/Desktop/Web_Scraper') # Returns list of files or directory  names inside that dir

And you can apply filter on file name extension to filter out only csv files

answered Aug 03 '17 at 05:10

viveksyngh

777
7
15

Om Prakash · Answer 4 · 2017-08-03T05:25:07.640

You can use python's inbuilt os module.

import os
os.listdir("this-directory")  # returns list of files in this-directory

E.g.

import os
for file in os.listdir("."):  # reading from current directory, replace it with your directory.
   if file.endswith(".csv"):
      with open(file) as fh:
          csv_reader = csv.DictReader(fh, fieldnames=['header', 'data'], delimiter=',')
              if n == 0:
                  for values in csv_reader:

Hope it helps!

Use Every CSV file in a Folder

4 Answers4