0

I want to append a row to a pandas data frame without using df.loc. I am looking for something similar to the .append() method used in lists.

I currently do this, but I'm looking for a different solution:

stupid_table = pd.DataFrame(columns=['max_depth', 'features'])
for max_depth in np.arange(50, 101):
    new_row = pd.DataFrame({'max_depth':[max_depth], 'features':[10]})
    stupid_table = pd.concat([stupid_table, new_row], axis=0)

I would like to declare new_row as a list or pandas Series rather than a pandas data frame. Is there a way to make the following syntax work?

stupid_table = pd.DataFrame(columns=['max_depth', 'features'])
for max_depth in np.arange(50, 101):
    new_row = [max_depth, 10]
    stupid_table = pd.concat([stupid_table, new_row], axis=0)

I specifically want to do this because the real data frame that I want to append new rows to has tons of columns and I am too lazy to write down the new row as a dictionary. I'd rather type the numbers in a list. I do this very often and a more convenient syntax would do wonders for my mental health.

EDIT: I prefer not using .loc because my for loop is taking each value in the numpy arrays. Switching the current syntax to for value in range(len(np.arange(start, stop))) to make the for loop to use indexes, then I'll have to translate the combinations into numeric values because, in the real example, I have tons of combinations. I'd rather not do that.

Arturo Sbr
  • 5,567
  • 4
  • 38
  • 76
  • Does this answer your question? [Add one row to pandas DataFrame](https://stackoverflow.com/questions/10715965/add-one-row-to-pandas-dataframe). The short version is that you can `stupid_table.append([{'max_depth': max_depth, 'features': 10}])`. The longer version is that this is _very inefficient_ and you should probably either (1) use `loc`, despite your preference not to, or (2) build a list of dicts, then convert those to a `pd.DataFrame` at the end. – mcskinner Apr 18 '20 at 19:38
  • [JFYI](https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples) – Danila Ganchar Apr 18 '20 at 20:09
  • Can you explain what that operation is meant to do? – AMC Apr 19 '20 at 03:08

1 Answers1

1

A solution that is a close as possible to your description is:

import numpy as np
import pandas as pd

stupid_table = pd.DataFrame(columns=['max_depth', 'features'])
for max_depth in np.arange(50, 101):
    new_row = [max_depth, 10]
    new_df = pd.DataFrame({head: [val] for head, val in zip(stupid_table.columns, new_row)})
    stupid_table = pd.concat([stupid_table, new_df], axis=0)

However, this is not efficient as the main dataframe object stupid_table is being re-assigned inside your main loop. You could ideally save all the new dataframes to append in a new list and do the concatenation in the end. This does not include an index reset for the newly created dataframe, but it might be a good idea to do so.

import numpy as np
import pandas as pd

stupid_table = pd.DataFrame(columns=['max_depth', 'features'])
data_frames = [stupid_table]
for max_depth in np.arange(50, 101):
    new_row = [max_depth, 10]
    data_frames.append(pd.DataFrame({head: [val] for head, val in zip(stupid_table.columns, new_row)}))
stupid_table = pd.concat(data_frames, axis=0)
KBoug
  • 26
  • 2