pandas append not working; always return empty dataframe

Question

I am trying to combine two tables row wise (stack on top of each other, like using rbind in R). I've followed steps mentioned in:

Pandas version of rbind

how to combine two data frames in python pandas

But none of the "append" or "concat" are working for me.

About my data

I have two panda dataframe objects (type class 'pandas.core.frame.DataFrame'), both have 19 columns. when i print each dataframe they look fine.

The problem

So I created another panda dataframe using:

query_results = pd.DataFrame(columns=header_cols)

and then in a loop (because sometimes i may be combining more than just 2 tables) I am trying to combine all the tables:

for CCC in CCCList:
    query_results.append(cost_center_query(cccode=CCC))

where cost_center_query is a customized function and returns pandas dataframe objects with same column names as the query_results.

however, with this, whenever i print "query_results" i get empty dataframe.

any idea why this is happening? no error message as well, so i am just confused. Thank you so much for any advice!

`.append` returns a *new data frame* which you would need to assign to something, or else the result simply gets discarded. — juanpa.arrivillaga, Apr 10 '17 at 20:19
After append store result into a variable and that will hold values you want. — Vipin, Apr 10 '17 at 20:20
Note, in your quoted question, the accepted answer has `bigdata = data1.append(data2, ignore_index=True) ` — juanpa.arrivillaga, Apr 10 '17 at 20:20
ah! thank you so much! that was quite a simple fix! feeling embarrassed to ask such beginner question but thank you all for your quick help! — alwaysaskingquestions, Apr 10 '17 at 20:21
Possible duplicate of [Appending to an empty data frame in Pandas?](http://stackoverflow.com/questions/16597265/appending-to-an-empty-data-frame-in-pandas) — juanpa.arrivillaga, Apr 10 '17 at 20:21
@juanpa.arrivillaga ah yes, i thought they just wanted to save the result as a different name, but i can see the point now. — alwaysaskingquestions, Apr 10 '17 at 20:22
As a rule, `pandas` methods don't operate in-place. If it can, there will be a default argument `inplace=False` which you can set to `True`. — juanpa.arrivillaga, Apr 10 '17 at 20:23
@juanpa.arrivillaga Thanks so much for the education! Will keep that in mind from now on. — alwaysaskingquestions, Apr 10 '17 at 20:27

score 2 · Answer 1 · answered Apr 10 '17 at 20:58

Consider the concat method on a list of dataframes which avoids object expansion inside a loop with multiple append calls. Even consider a list comprehension:

query_results = pd.concat([cost_center_query(cccode=CCC) for CCC in CCCList], ignore_index=True)

pandas append not working; always return empty dataframe

1 Answers1