I have a for loop that generates a new dataframe, then concatenate into a final one, somethink like this:
final_df = pd.DataFrame()
for i in range(2):
new = pd.DataFrame({'col':[1]})
final_df = pd.concat([final_df, new])
In my real case, there are millions of "new" dataframe, so this process takes too long. Is there any way to make it more efficient?
Thank you.