Appending rows in a dataframe in a for loop

Question

I have a Dataframe1 that is with a length of 4000, and I have a list with a length of 600.

I want to compare the list with a column from the Dataframe1 and if they have the same value I want to move the row of the Dataframe1 to Dataframe2 and if not I want to move it to Dataframe3

so this was my code but it takes forever to run, can anyone recommend a faster solution.

for i in range(len(Dataframe1)):
    for j in range(len(list)):
         if Dataframe1['Column'][i] == list[j]:
              Datafram2 = Dataframe2.append(Dataframe1.loc[[i]])
         else:                  
              Datafram3 = Dataframe3.append(Dataframe1.loc[[i]])

i think @HoaNguyen's solution should suffice; also it is helpful if u share data with expected output. it helps a whole lot more than just words — sammywemmy, May 15 '20 at 10:53
Does this answer your question? [How to filter Pandas dataframe using 'in' and 'not in' like in SQL](https://stackoverflow.com/questions/19960077/how-to-filter-pandas-dataframe-using-in-and-not-in-like-in-sql) — Arne, May 15 '20 at 11:16

score 0 · Answer 1 · answered May 15 '20 at 10:43

I think you can use .isin() instead of doing the loop. To learn more, you can see: pandas.Series.isin, and pandas.DataFrame.isin

Here is my suggestion:

Add rows that have Column in the list from Dataframe1 to DataFrame2

Dataframe2 = pd.concat([Dataframe2, Dataframe1[Dataframe1["Column"].isin(list)]],ignore_index=True)

Add rows that have Column not in the list from Dataframe1 to DataFrame3

Dataframe3 = pd.concat([Dataframe3, Dataframe1[~Dataframe1["Column"].isin(list)]],ignore_index=True)

Appending rows in a dataframe in a for loop

1 Answers1