Drop rows on multiple conditions in pandas dataframe

Question

My df has 3 columns

df = pd.DataFrame({"col_1": (0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0), 
                   "col_2": (0.0, 0.24, 1.0, 0.0, 0.22, 3.11, 0.0),
                    "col_3": ("Mon", "Tue", "Thu", "Fri", "Mon", "Tue", "Thu")})

I want to drop rows where df.col_1 is 1.0 and df.col_2 is 0.0. So, I would get:

df = pd.DataFrame({"col_1": (0.0, 0.0, 1.0, 0.0, 1.0), 
                   "col_2": (0.0, 0.24, 1.0, 0.22, 3.11),
                    "col_3": ("Mon", "Tue", "Thu", "Mon", "Tue")})

I tried:

df_new = df.drop[df[(df['col_1'] == 1.0) & (df['col_2'] == 0.0)].index]

It gives me the error:

'method' object is not subscriptable

Any idea how to solve the above problem?

Dani Mesejo · Accepted Answer · 2022-06-16T15:16:25.137

59

drop is a method, you are calling it using [], that is why it gives you:

'method' object is not subscriptable

change to () (a normal method call) and it should work:

import pandas as pd

df = pd.DataFrame({"col_1": (0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0),
                   "col_2": (0.0, 0.24, 1.0, 0.0, 0.22, 3.11, 0.0),
                   "col_3": ("Mon", "Tue", "Thu", "Fri", "Mon", "Tue", "Thu")})

df_new = df.drop(df[(df['col_1'] == 1.0) & (df['col_2'] == 0.0)].index)
print(df_new)

Output

   col_1  col_2 col_3
0    0.0   0.00   Mon
1    0.0   0.24   Tue
2    1.0   1.00   Thu
4    0.0   0.22   Mon
5    1.0   3.11   Tue

edited Jun 16 '22 at 15:16

answered Sep 22 '18 at 13:03

Dani Mesejo

61,499
6
49
76

Just to complement this answer: `&` is the logical `AND` operator, `|` is the logical `OR` and `~` is the logical `NOT` – Daniel Lavedonio de Lima May 24 '23 at 06:25
Why was the .index method needed at the end? I see that it doesn't work without it. – Yogesh Riyat Jul 17 '23 at 20:30

score 19 · Answer 2 · edited Mar 31 '22 at 07:38

19

Try to filter your df with loc. It's so powerful! The "~" means you want to keep those with the opposite of your condition. The ":" means you want to keep all the columns

df = df.loc[~((df['col_1'] == 1.0) & (df['col_2'] == 0.0)),:]

edited Mar 31 '22 at 07:38

Mechi

3
4

answered Sep 22 '18 at 13:00

Charles R

1,621
1
8
25

score 10 · Answer 3 · answered Nov 22 '18 at 18:03

10

You can use or (|) operator for this , Refer this link for it pandas: multiple conditions while indexing data frame - unexpected behavior

i.e dropping rows where both conditions are met

 df = df.loc[~((df['col_1']==1) | (df['col_2']==0))]

answered Nov 22 '18 at 18:03

Saurabh

7,525
4
45
46

The "|" operator is for "OR", while the "&" operator is for "AND" conditions. I think what is needed here is the & operator – Web Ster Aug 04 '21 at 13:59

score 9 · Answer 4 · edited Sep 27 '21 at 00:23

9

mask = df['Product_Code'].isin(['filter1', 'filter2', 'filter3'])
df = df[~mask]
df.head()

.isin() allows you to filter the entire dataframe based on multiple values in a series. This is the least amount of code to write, compared to other solutions that I know of.

Adding the ~ inside the column wise filter reverses the logic of isin().

edited Sep 27 '21 at 00:23

zr0gravity7

2,917
1
12
33

answered Sep 30 '20 at 03:08

Alex Slessor

91
1
4

score 1 · Answer 5 · edited Oct 28 '19 at 18:58

1

Put the location of row which you want remove at "location".

df = df.drop(['location' axix=1, inplace=True]

edited Oct 28 '19 at 18:58

xandermonkey

4,054
2
31
53

answered Oct 28 '19 at 16:39

aliasgar paloda

19
1

score 0 · Answer 6 · answered Jul 30 '22 at 14:53

You can also use query here:

In [4]: df.query('~(col_1 == 1 & col_2 == 0)')
Out[4]: 
   col_1  col_2 col_3
0    0.0   0.00   Mon
1    0.0   0.24   Tue
2    1.0   1.00   Thu
4    0.0   0.22   Mon
5    1.0   3.11   Tue

With ~ the query is negated, returning those observations where the condition col_1 == 1 & col_2 == 0 does not hold.

Drop rows on multiple conditions in pandas dataframe

6 Answers6

Linked

Related