walking along and processing files in directory in python

Question

I have huge data in a directory tree format like:
c:/user/name/class/std/section I to VI [all section has individual folder i.e. 6 folder in total and all folder have 100+ files to be processed]
I wrote a script in which if I give the folder containing the files e.g.Section I then it would process files inside using glob.iglobfunction.
Is it possible to write a script which can walk along directories by just entering one directory > processing file > leaving directory > entering different directory > and so on.
please help.

Take a look at the os.walk function http://docs.python.org/library/os.html?highlight=os.walk#os.walk — Levon, May 16 '12 at 11:40
@Levon thanx for your link...actually I'm new to Python, so it was quite helpfull. — diffracteD, May 27 '12 at 08:03

urcm · Answer 1 · 2012-05-16T13:45:22.137

4

you can try this code :

import os
folder = 'C:'

for root, dirs, files in os.walk(folder):
    for name in files:
        print os.path.join(root, name)
    for name in dirs:
        print os.path.join(root, name)

UPDATE:

import os folder = 'C:'

for root, dirs, files in os.walk(folder):
    for name in files:
        nm, ext = os.path.splitext(name)
        if ext == ".csv":
            print os.path.join(root, name)

edited May 16 '12 at 13:45

answered May 16 '12 at 11:45

urcm

2,274
1
19
45

but suppose if i want to find and process *.csv files inside each folder, then what will be the easiest way..?? – diffracteD May 16 '12 at 11:58

score 0 · Answer 2 · answered May 16 '12 at 11:49

import os
for (dirpath, dirnames, filenames) in os.walk(directory):
    # Do some processing

That will iterate through the root of the directory specified, for eg. c:/user/name/class/std and enter every folder contained in it, and give you the folders and files contained in them. With that you should be able to do what you need to processing wise.

score 0 · Accepted Answer · answered May 16 '12 at 12:28

0

Addressing Abhisek's comment on Aragon's solution:

import os folder = 'C:'

for root, dirs, files in os.walk(folder):
    for name in files:
        (base, ext) = os.path.splitext(name)
        if ext in "csv":
            print os.path.join(root, name)

answered May 16 '12 at 12:28

Simon Peverett

4,128
3
32
37

walking along and processing files in directory in python

3 Answers3

Linked