Use a loop to stack data frames generated from a loop

Question

I'm using a loop to generate data frames with all the same column names and I wanted to stack them together. (like this web page showing)


#here's a list I use to read my data frames 
#(the csv files I'm working on have a name convention that I need to follow.)
title_cn = [1, 2, 3]
title_tn = [22, 27, 31]

#here's the for loop I used to generate data frames called df 
for (i, j) in zip(title_cn, title_tn): 
    filename = '{}_Cycle_{}_Test.csv'.format(i, j)
    reading = pd.read_csv(filename)
    df = pd.DataFrame(reading)  

#now I want to stack the 'df' together in 'Joined_dataframe'  
for k in title_cn:
    Joined_dataframe = pd.concat(df)

From the code above I have 3 data frames. I want to stick them all together as a new one using pd.concat

I think my problems is that I don't know how to write the second loop properly. So far I only got the last data frame generated by the for loop. Otherwise I got repetitive data frames as the remaining of the first for loop.

I tried to search but most of the tutorials are talking about using one loop to generate the stitched data frame (without the previous for loop). I feel I'm on the right track but I don't know what to do to move on yet.

I'm looking to use a loop because in reality I have 20+ dataframes that I'd like to stack together.

Look at your first loop, you're losing all but the last DataFrame. — AMC, Mar 04 '20 at 02:14

score 2 · Accepted Answer · answered Mar 04 '20 at 01:57

Consider building a list of data frames with list comprehension and then pass list into concat once outside any loop:

df_list = [pd.DataFrame('{}_Cycle_{}_Test.csv'.format(i, j)) \
              for (i, j) in zip(title_cn, title_tn)]

final_df = pd.concat(df_list, ignore_index = True)

Use a loop to stack data frames generated from a loop

1 Answers1