12

I have a folder with lots of .txt files. How can I read all the files in the folder and get the content of them with pandas?. I tried the following:

import pandas as pd
list_=pd.read_csv("/path/of/the/directory/*.txt",header=None)
print list_
newWithPython
  • 853
  • 3
  • 9
  • 20

2 Answers2

24

Something like this:

import glob

l = [pd.read_csv(filename) for filename in glob.glob("/path/*.txt")]
df = pd.concat(l, axis=0)

You have to take into account the header, for example if you want to ignore it take a look at the skiprows option in read_csv.

elyase
  • 39,479
  • 12
  • 112
  • 119
1

I used this in my project for merging the csv files

import pandas as pd
import os
path = "path of the file"
files = [file for file in os.listdir(path) if not file.startswith('.')]
all_data = pd.DataFrame()

for file in files:
current_data = pd.read_csv(path+"/"+file ,  encoding = "ISO-8859-1")  
all_data = pd.concat([all_data,current_data])
Abhi Singh
  • 19
  • 2
  • 1
    The question already has an accepted answer - consider adding an explanation of why you believe your answer is more helpful. – gshpychka Aug 20 '21 at 06:17
  • This answer provides value as it is an alternative for developers that does not have the formatting into account. – NuValue Nov 28 '21 at 09:52