How to concatenate numerous df's pandas

Question

This is similar to a few questions regarding this topic.

I have a df for each day in the month. the df's are exported daily and I want to concatenate them to create a monthly df. I'm currently performing this via the following:

import pandas as pd

df1 = pd.read_csv('df1.csv')#1st
df2 = pd.read_csv('df2.csv')#2nd
df3 = pd.read_csv('df3.csv')#3rd ect

concat_df = pd.concat([df1, df2, df3])

I repeat this 30 odd times. Is there a more efficient way to concatenate these df's if the initial df in the directory is the first day of the month and the final df is the last day of the month.

As in;

For directory 'month' concatenate all files?

Rather than reading each individual df and concatenating 30 odd df's

Use a list comprehension like `concat_df = pd.concat([pd.read_csv('df{}.csv'.format(i) for i in range(1,31)])` — Bharath M Shetty, Oct 03 '18 at 04:40

score 1 · Answer 1 · answered Oct 03 '18 at 04:41

1

I hope this helps.

import os
import glob
import pandas as pd

allFiles = glob.glob(os.getcwd()+ "/*.csv")
list_ = []

for file_ in allFiles:
    df = pd.read_csv(file_,index_col=None, header=0)
    list_.append(df)

frame = pd.concat(list_)
print(frame)

Similar - Import multiple csv files into pandas and concatenate into one DataFrame

answered Oct 03 '18 at 04:41

Ekagra Singh

11
2

Use this op, I use it almost everyday! – Umar.H Oct 03 '18 at 06:34

How to concatenate numerous df's pandas

1 Answers1