Reading multiple files from a same folder using python

Question

I have thousands of text files which I want to read using python. I have successfully read one file and now I am not sure how to read multiple files using loop or any other command

I tried to split the name and type of the file by saving the variable character of the text files saved in a string. e.g I have 3 files named as file_1, file_2, file_3

I took one variable and limit ="1" and conciniate it with the full address of the file. now I want to know how can I access the other two files. provided I don't want to use same line of code multiple times because in reality I have
thousands of files

import os
from os import path
limit1 = "1"
strpath = r"C:/Users/saqibshakeel035/Desktop/SP/text_file_r_w"
print("Your current directory is : %s"  %path.abspath(strpath))
f = open("C:/Users/saqibshakeel035/Desktop/SP/text_file_r_w/file_" + 
limit1 + ".txt", "r")
print(f.read())

This code is working fine for 1 file. Now i want my code to read multile files and later on I will transfer my files to somewhere else.

What exactly prevents you from using a loop? – DYZ Feb 01 '19 at 01:03 — DYZ, Feb 01 '19 at 01:03

score 7 · Answer 1 · answered Feb 01 '19 at 01:05

You can use glob.glob to access all file paths of the folder, and read each file using for loop.

files = [file for file in glob.glob("../somefolder/*")]
for file_name in files:
    with io.open(file_name, 'rb') as image_file:
        content = image_file.read()

score 3 · Answer 2 · answered Feb 01 '19 at 01:07

You can use os.walk to scan all files in a directory.

import os

for root, dirs, files in os.walk('lang/'):
    for file in files:
        filename, extension = os.path.splitext(file)
        if extension == '.txt':
            # Do Some Task

Note that if you can filter by filename or file extension.

For example, if you want to get files that contains 'hello' in filename, you can also add

if 'hello' in filename:
    # Do Some Task

score 0 · Answer 3 · answered Feb 01 '19 at 01:06

Try using a loop for this:

import os
def main():
    for dirName, subDirList, fileList in os.walk('path'):
        for subDir in subDirList:
            for file in fileList:
                #Do something with file
main()

This recursively goes through your files in a given directory. Note that this also goes through files in any other subdirectory in the parent directory. To instead ignore subdirectories do this:

import os
def main():
    for dirName, subDirList, fileList in os.walk('path'):
        for file in fileList:
            #Do something with file
main()

Of course you also have other options.

Reading multiple files from a same folder using python

3 Answers3