0

I'm trying to concatenate a dataframe created in a loop to a main df. It seems that using .copy() or not doesn't change the fact that every time the loop start again, the main df reset to the looping df. I've based my script on Appending pandas dataframes generated in a for loop but it doesn't work.

Here's my script:

firstloop = True
for x in x_list:

    loop_db = myFunctionReturnADataframe(x)
    
    if firstloop:
        main_db = loop_db.copy()
        firstloop == False
    else:
        main_db = pd.concat([main_db.copy(), loop_db.copy()])

Each loop the main_db doesn't add loop_db, instead it change and equal loop_db.

What am I doing wrong?

Thanks for your time

Vincent Coulombe
  • 37
  • 6

3 Answers3

3

Neither .copy() nor a loop with conditional is needed in your case. Just concat dataframes:

main_df = pd.concat(map(myFunctionReturnADataframe, x_list))
RomanPerekhrest
  • 88,541
  • 4
  • 65
  • 105
2

In your loop, you have a tiny typo.

firstloop == False is a comparison, which returns a boolean value (False in this case). It is not setting firstloop value to False.

You need to change it to:
firstloop = False

epursiai
  • 36
  • 5
  • oh my god.... I can't anymore lmao, I realy need to go outside ... Thanks @epursiai – Vincent Coulombe Aug 26 '23 at 16:32
  • You're welcome. But I would also encourage you to consider a more straightforward approach, like the one that RomanPerekhrest suggested, unless you have some other specific reason to use loop like that. – epursiai Aug 26 '23 at 17:20
0

I usually start with an empty list (this is also done in the example you mention) and then add the dataframes one by one to the list using list append. Finally, I concatenate the list items using pandas concat.

import pandas as pd

lst = list()
for file in filenames:
    df_tmp = pd.read_csv(file)
    lst.append(df_tmp)

df = pd.concat(lst, ignore_index=True)
Laconico
  • 1
  • 3