39

I have something like this in my code:

df2 = df[df['A'].str.contains("Hello|World")]

However, I want all the rows that don't contain either of Hello or World. How do I most efficiently reverse this?

Xodarap777
  • 1,358
  • 4
  • 19
  • 42

2 Answers2

52

You can use the tilde ~ to flip the bool values:

>>> df = pd.DataFrame({"A": ["Hello", "this", "World", "apple"]})
>>> df.A.str.contains("Hello|World")
0     True
1    False
2     True
3    False
Name: A, dtype: bool
>>> ~df.A.str.contains("Hello|World")
0    False
1     True
2    False
3     True
Name: A, dtype: bool
>>> df[~df.A.str.contains("Hello|World")]
       A
1   this
3  apple

[2 rows x 1 columns]

Whether this is the most efficient way, I don't know; you'd have to time it against your other options. Sometimes using a regular expression is slower than things like df[~(df.A.str.contains("Hello") | (df.A.str.contains("World")))], but I'm bad at guessing where the crossovers are.

DSM
  • 342,061
  • 65
  • 592
  • 494
  • Much better than a convoluted negative lookaround test. No experience with Pandas myself, however, so I have no idea what would be the faster approach. – Martijn Pieters Jan 10 '14 at 21:58
  • 1
    The regex lookaround test took significantly longer (about 30s vs 20s), and the two methods apparently have slightly different results (3663K result vs 3504K - from ~3G original - haven't looked to see specifics). – Xodarap777 Jan 10 '14 at 22:10
  • @DSM I have seen this `~` symbol many times, specially in JavaScript. Haven't seen in python. What does it mean, exactly? – estebanpdl Oct 07 '16 at 19:14
9

The .contains() method uses regular expressions, so you can use a negative lookahead test to determine that a word is not contained:

df['A'].str.contains(r'^(?:(?!Hello|World).)*$')

This expression matches any string where the words Hello and World are not found anywhere in the string.

Demo:

>>> df = pd.DataFrame({"A": ["Hello", "this", "World", "apple"]})
>>> df['A'].str.contains(r'^(?:(?!Hello|World).)*$')
0    False
1     True
2    False
3     True
Name: A, dtype: bool
>>> df[df['A'].str.contains(r'^(?:(?!Hello|World).)*$')]
       A
1   this
3  apple
Community
  • 1
  • 1
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343