How to convert all CSV files in a folder to data frames and append them in a single data frame

Question

I have multiple CSV files in C:\data\ folder. I want to know how to convert all of them into data frames and append them to one single data frame. They all have the same column names. Thanks!

Does this answer your question? [How to join (merge) data frames (inner, outer, left, right)](https://stackoverflow.com/questions/1299871/how-to-join-merge-data-frames-inner-outer-left-right) — Joe Ferndz, Mar 19 '21 at 22:16
I'm trying to use append function to append dataframes from the bottom, not trying to lookup values. — kalapangha, Mar 19 '21 at 22:21

score 2 · Accepted Answer · answered Mar 19 '21 at 22:24

This will combine all the CSV files in 'C:\Data\Folder' into one dataframe.

import os
import glob
import pandas as pd

os.chdir(r'C:\Data\Folder')

files = glob.glob('*.csv')

combined_csv = pd.concat([pd.read_csv(file) for file in files ]).reset_index()

score 2 · Answer 2 · answered Mar 19 '21 at 22:30

2

import pandas as pd
combined_csv = pd.concat( [ pd.read_csv(f) for f in filenames ] )

If you want everything in one file, this should work;

combined_csv.to_csv( "combined_csv.csv", index=False )

answered Mar 19 '21 at 22:30

Nisarg Bhatt

379
1
13

Thank you. For some reason it doesn't work, I'll have to try again. – kalapangha Mar 19 '21 at 23:06
@kalapangha Could you describe the error in detail? – Nisarg Bhatt Mar 19 '21 at 23:08

How to convert all CSV files in a folder to data frames and append them in a single data frame

2 Answers2