append last 5 lines of all CSV files in a directory to 1 Dataframe

Question

I need to create a dataframe to push to database containing the last 5 lines (all columns) of all CSV files in directory.

So far here is what I have, but it is not adding Data and just says Empty DataFrame

#blank Dataframe
cols = ['ticker', 'date', 'open', 'high', 'low', 'close', 'volume', 'time']
insert_list = pd.DataFrame(columns = cols)

#select csvs to merge
listdrs = os.listdir(string)
listdrs_path = [ string + x for x in listdrs]
#loop through csv's
for file_path in listdrs_path:
    data = pd.read_csv(file_path, index_col=0)
    insert_list.append(data.tail(5))
    
#save list as new csv
insert_list.to_csv('compressed_update.csv')

`append()` is not an inplace operation you should store the result back in `insert_list` — deadshot, Sep 18 '20 at 01:33

score 0 · Answer 1 · answered Sep 18 '20 at 01:56

0

Code updated based on resources shared in the comments:

for file_path in listdrs_path:
    data = pd.read_csv(file_path, index_col=0).tail(5)
    insert_list = insert_list.append(data)

answered Sep 18 '20 at 01:56

Jason Maynard

359
1
3
15

append last 5 lines of all CSV files in a directory to 1 Dataframe

1 Answers1