58

The pandas documentation includes a note:

Note Unlike list.append method, which appends to the original list and returns nothing, append here does not modify df1 and returns its copy with df2 appended.

How can I append to an existing DataFrame without making a copy? Or in the terms of the note how can I modify df1 in place by appending df2 and return nothing?

cameron.bracken
  • 1,236
  • 1
  • 9
  • 14
  • 1
    It looks like there is not actually a performance gain by modifying in place: https://github.com/pydata/pandas/issues/2801 – cameron.bracken Aug 12 '13 at 21:40
  • 1
    This is not currently supported in `pandas`. I'm not sure it's worth the trouble either. Did you have a particular use case in mind? – Phillip Cloud Aug 12 '13 at 21:59
  • heres a related question: http://stackoverflow.com/questions/16740887/using-pandas-dataframe-with-incoming-real-time-data. What is your end goal here? – Jeff Aug 13 '13 at 00:09
  • 13
    I am reading in a many large data files from an external source and building a DataFrame piece by piece, that I can then write out to a database all at once. The DataFrame will get very large (many GBs) and I want to avoid making a copy each time I add new data. – cameron.bracken Aug 13 '13 at 01:12

1 Answers1

13

See How to add an extra row to a pandas dataframe

Upcoming pandas 0.13 version will allow to add rows through loc on non existing index data.

Description is here and this new feature is called Setting With Enlargement.

Nasser Al-Wohaibi
  • 4,562
  • 2
  • 36
  • 28
  • 6
    the solution from _Setting With Enlargement_ works only for 1 row. The questions was "appends row**s** ..." – 2diabolos.com Dec 01 '16 at 12:32
  • 1
    Also, quoting from the [linked answer](https://stackoverflow.com/questions/19365513/how-to-add-an-extra-row-to-a-pandas-dataframe/19368360#19368360): "However, be aware that under the hood, this creates a copy of the entire DataFrame so it is not an efficient operation." – Leland Hepworth Sep 16 '20 at 16:02
  • Yes, worked with `df.loc[some_index from a loop counter] = a row of a df (= a Series)` or `df.loc[some indices from a range] = df2.values`, see the latter example at [Python pandas insert empty rows after each row](https://stackoverflow.com/a/66466675/11154841). – questionto42 Dec 06 '21 at 19:13