1

I have a pandas dataframe created . Now, I have to keep incrementally adding rows to this dataframe . What should I use concat, append or any other? . I am asking this for performance considerations to be taken into account when doing this operation.

And what are the thumb rules to be taken into account when using concat or append operations ?

Andriy Ivaneyko
  • 20,639
  • 6
  • 60
  • 82
Nishant Sharma
  • 341
  • 2
  • 5
  • 17
  • 1
    Do you have jupyter/ipython installed? If so, the best answer you'll get is by using the `%%timeit` magic command to compare different techniques. Other than that, appending row-by-row is almost never optimally performant. – Paul H Mar 12 '16 at 06:29
  • 1
    I think you can add code, sample of data and desired output. Maybe your code can be more optimalized. The best (but the most impossible) is share your real data for testing. – jezrael Mar 12 '16 at 06:47
  • Thanks... I will check... I have jupyter installed... Why does pandas have two different techniques for doing the same thing ? In my case, I have a initial data frame of 5 rows,.. Then after few milli seconds, I have to add a row to that data frame... So should I be using concat by creating a new data frame with one row and appending it to the previous one ? – Nishant Sharma Mar 12 '16 at 10:55

1 Answers1

1

Concat is faster or equal to append. See existed SO question Pandas DataFrame concat vs append

It contains timeit metrics, code sample and description of why concat is faster.

Community
  • 1
  • 1
Andriy Ivaneyko
  • 20,639
  • 6
  • 60
  • 82