64

If I have an empty dataframe as such:

columns = ['Date', 'Name', 'Action','ID']
df = pd.DataFrame(columns=columns)

is there a way to append a new row to this newly created dataframe? Currently I have to create a dictionary, populate it, then append the dictionary to the dataframe at the end. Is there a more direct way?

Cleb
  • 25,102
  • 20
  • 116
  • 151
Ahdee
  • 4,679
  • 4
  • 34
  • 58

3 Answers3

113

Try this:

df.loc[len(df)]=['8/19/2014','Jun','Fly','98765']

Warning: this method works only if there are no "holes" in the index. For example, suppose you have a dataframe with three rows, with indices 0, 1, and 3 (for example, because you deleted row number 2). Then, len(df) = 3, so by the above command does not add a new row - it overrides row number 3.

Erel Segal-Halevi
  • 33,955
  • 36
  • 114
  • 183
Jun
  • 1,131
  • 2
  • 7
  • 2
  • 17
    `df.loc[df.shape[0]]` (even `df.loc[len(df.index)]`) is much faster than `df.loc[len(df)]`. – Nuno André Aug 06 '16 at 06:32
  • 3
    one element >>> timeit.timeit(lambda : len(flows)) 2.4147243930055993 >>> timeit.timeit(lambda : flows.shape[0]) 3.4786632809991715 4k elements: >>> timeit.timeit(lambda : len(flows)) 2.520429938987945 >>> timeit.timeit(lambda : flows.shape[0]) 3.6195146050013136 – eri Dec 18 '16 at 22:19
  • 5
    If you're using `len(df)`, why do we use `loc` instead of `iloc`? Doesn't it have the danger of matching a member of the index, rather than a row number? – Ken Williams Apr 28 '17 at 18:15
  • 1
    Goodness, this took a long time to figure out! Much appreciated! – Abe Hoffman May 27 '17 at 07:50
  • 5
    This is an INCREDIBLY SLOW method of appending a row to a dataframe. As a practical example, I created a dataframe from parsing a file have 1.5 million lines using this method and it took over 7 hours. The same thing when done with a dictionary and then put into a dataframe took about 10 seconds. This internally happens because of the slow appending to a Series. See https://stackoverflow.com/a/37992805/1224075 for understanding what happens internally. – tinkerbeast Aug 28 '17 at 10:39
  • Most intuitive I have found, even if slow... – jtlz2 May 16 '18 at 13:35
  • If it works and it's slow, it still works. – Rob Truxal Mar 13 '19 at 20:49
49

Upcoming pandas 0.13 version will allow to add rows through loc on non existing index data. However, be aware that under the hood, this creates a copy of the entire DataFrame so it is not an efficient operation.

Description is here and this new feature is called Setting With Enlargement.

Aidan Feldman
  • 5,205
  • 36
  • 46
Zeugma
  • 31,231
  • 9
  • 69
  • 81
  • 1
    enlargement only allowed thru ``loc`` (``iloc`` could add not-at-the-end so its a bit ambiguous) – Jeff Oct 14 '13 at 20:17
  • typo, corrected, thanks. – Zeugma Oct 14 '13 at 20:17
  • 1
    I played around with doing this on an empty DataFrame, and I found that it did not work. Seems to work only if the frame has at least one row.... – Dan Allan Oct 14 '13 at 21:49
  • not cool for the completeness of pandas apis... – Zeugma Oct 14 '13 at 21:57
  • @DanAllan why don't u post an issue on github with some examples – Jeff Oct 14 '13 at 23:19
  • [Done.](https://github.com/pydata/pandas/issues/5226) – Dan Allan Oct 15 '13 at 01:57
  • this is fixed in master see [here](https://github.com/pydata/pandas/issues/5226). @Boud always welcome bug reports and/or PR's for new features which are not always complete the second they are out. – Jeff Oct 15 '13 at 13:12
  • 1
    I've found that this will drop any columns from the added Series that are not already in the DataFrame. In this case, the documentation is misleading in that it states that "Setting With Enlargement" is like an "append" operation, even though "append" will add any new columns to the DataFrame. – adam.r Jan 22 '15 at 17:12
  • 4
    Isn't this a lot slower than creating a dictionary and then adding the entire dictionary to `df`? – max Feb 28 '15 at 00:38
  • Isn't [this](http://stackoverflow.com/q/32749104/938408) how it is supposed to work? Then why doesn't it? – László Sep 23 '15 at 20:56
5

A different approach that I found ugly compared to the classic dict+append, but that works:

df = df.T

df[0] = ['1/1/2013', 'Smith','test',123]

df = df.T

df
Out[6]: 
       Date   Name Action   ID
0  1/1/2013  Smith   test  123
Zeugma
  • 31,231
  • 9
  • 69
  • 81