Reading multiple csv file with specificing character in naming

Question

I have multiple csv file in a folder and some of them end with 'xxxxx.t01.csv'. Now what I want to do is to create a list of data frames in panda which contains all these files with '.t01.csv' ending. I tried to use fnmatch and globe2 but none of them worked. (I use python 3.7). Here are my codes:

import os, fnmatch
import pandas as pd

list_of_files = os.listdir('C:\\Users\\My_PC\\Desktop\\new folder')   
for entry in list_of_files:  
    if fnmatch.fnmatch(entry, '*t01.csv'):
       df=pd.read_csv(entry)

       print(df[])

and I got this error:

then I tried with glob2 as well but I got nowhere.

import pandas as pd
import glob2

path = r''C:\\Users\\My_PC\\Desktop\\new folder''
all_files = glob2.glob(path + "*t01.csv")


for filename in all_files:

    df = pd.read_csv(filename, index_col=None, header=0)
    print(df)

but it will show nothing. Could someone tell me what the problem is?

You have to append the base path with the file name. `all_files` contains the filename in string format and not actual file objects. — mad_, Mar 13 '19 at 14:17
@mad_ thanks for your quick reply but i just append the base path but nothing happend. I guess somthing wrong with logic of my code ????!! what do you think — Saeid Hedayati, Mar 13 '19 at 15:05
the path is correct, because when i deactivate the line ´ df=pd.read_csv(entry)´ in fnmatch part, it should a list of the names correctly but i keep constantly get this mentioend error or error with encoding (i use windows) — Saeid Hedayati, Mar 13 '19 at 15:10
Then I will point you to have a look at documentation before moving forward https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html. There is `encoding` parameter that can be useful — mad_, Mar 13 '19 at 15:24

score 0 · Answer 1 · answered Mar 13 '19 at 14:16

Use os.chdir() and glob:

os.chdir('C:\\Users\\My_PC\\Desktop\\new folder\\')
all_files = glob.glob("*t01.csv")

for filename in all_files:
    df = pd.read_csv(filename, index_col=None, header=0)
    print(df)
    print("---------seperator--------------")

For appending them together in one dataframe: refer to this lnk

mad_ · Answer 2 · 2019-03-13T14:25:56.783

0

The error is self-explanatory that it is unable to find the filename in the current directory. To resolve the issue pass in the complete filepath or change the current working directory using os.chdir(path) which is not recommended. Replace filename as below using os.path.join()

pd.read_csv(os.path.join(path,filename), index_col=None, header=0)

edited Mar 13 '19 at 14:25

answered Mar 13 '19 at 14:20

mad_

8,121
2
25
40

Reading multiple csv file with specificing character in naming

2 Answers2