Drop columns if rows contain a specific value in Pandas

Question

I am starting to learn Pandas. I have seen a lot of questions here in SO where people ask how to delete a row if a column matches certain value.

In my case it is the opposite. Imagine having this dataframe:

Where you want to know is, if any column has in any of its row the value salty, that column should be deleted, having as a result:

I have tried with several similarities to this:

if df.loc[df['A'] == 'salty']:
   df.drop(df.columns[0], axis=1, inplace=True)

But I am quite lost at finding documentation onto how to delete columns based on a row value of that column. That code is a mix of finding a specific column and deleting always the first column (as my idea was to search the value of a row in that column, in ALL columns in a for loop.

@cs95 I'm afraid you misread the question... – IanS Jun 25 '19 at 13:12 — IanS, Jun 25 '19 at 13:12
@IanS Indeed the code was confusing. – cs95 Jun 25 '19 at 13:13 — cs95, Jun 25 '19 at 13:13

score 16 · Accepted Answer · answered Jun 25 '19 at 13:14

16

Perform a comparison across your values, then use DataFrame.any to get a mask to index:

df.loc[:, ~(df == 'Salty').any()]

If you insist on using drop, this is how you need to do it. Pass a list of indices:

df.drop(columns=df.columns[(df == 'Salty').any()])

df = pd.DataFrame({
    'A': ['Mountain', 'Salty'], 'B': ['Lake', 'Hotty'], 'C': ['River', 'Coldy']})
df
          A      B      C
0  Mountain   Lake  River
1     Salty  Hotty  Coldy

(df == 'Salty').any()
A     True
B    False
C    False
dtype: bool

df.loc[:, ~(df == 'Salty').any()]
       B      C
0   Lake  River
1  Hotty  Coldy

df.columns[(df == 'Salty').any()]
# Index(['A'], dtype='object')

df.drop(columns=df.columns[(df == 'Salty').any()])
       B      C
0   Lake  River
1  Hotty  Coldy

answered Jun 25 '19 at 13:14

cs95

379,657
97
704
746

1

This: `df.loc[:, ~(df == 'Salty').any()]`worked just fine! Thank you! – M.K Jun 25 '19 at 13:19
@cs95 What if I wanted to check for a particular substring instead of a specific string? – Mega_Noob May 19 '20 at 18:59
2

@Mega_Noob You might find my answer on substring matching useful: https://stackoverflow.com/questions/11350770/select-by-partial-string-from-a-pandas-dataframe/55335207#55335207 – cs95 May 19 '20 at 20:15

score 3 · Answer 2 · answered Oct 06 '20 at 14:33

3

The following is locating the indices where your desired column matches a specific value and then drops them. I think this is probably the more straightforward way of accomplishing this:

df.drop(df.loc[df['Your column name here'] == 'Match value'].index, inplace=True)

answered Oct 06 '20 at 14:33

grumpyTofu

826
7
5

this was helpful, i was able to use a variation of this in a list comprehension to cull a lot of unwanted columns quickly. `[df.drop(df.loc[df['col name'] == item].index, inplace=True) for item in some_list]` – Cory C Jul 04 '22 at 08:48

score 2 · Answer 3 · answered Jun 25 '19 at 13:14

2

Here's one possibility:

df = df.drop([col for col in df.columns if df[col].eq('Salty').any()], axis=1)

answered Jun 25 '19 at 13:14

bexi

1,186
5
9

Drop columns if rows contain a specific value in Pandas

3 Answers3

Linked

Related