Check if pandas string column contains multiple words, in any order

Question

I am working on Twitter data and trying to find strings that contain more than one word. The following line works for one word and with the OR condition.

tweets_text[tweets_text.str.contains("break")] #Find strings with the word break

tweets_text[tweets_text.str.contains("break|social|media")] #Find strings with either break or social, or media

I am trying to find the strings that have these three words ("break & social & media")

Do you care about the order in which 'break','social','media' can occur? There are 3! = 6 possible orders, in theory. Could 'break' occur between 'social' and 'media'? — smci, Jun 17 '21 at 10:03
Related: [Select by partial string from a pandas DataFrame](https://stackoverflow.com/questions/11350770/select-by-partial-string-from-a-pandas-dataframe) — smci, Jun 17 '21 at 10:06

MAFiA303 · Accepted Answer · 2021-06-17T10:26:46.620

3

df = pd.Series(['break', 'break media social', 'break media'])

Series:

0                 break
1    break media social
2           break media

extraciton:

tweets_text[tweets_text.str.contains('(?=.*break)(?=.*social)(?=.*media)')]

output:

1    break media social

edited Jun 17 '21 at 10:26

answered Jun 17 '21 at 10:00

MAFiA303

1,157
11
10

Does this assume they occur in the strict order 'break', followed by 'social', 'media'? – smci Jun 17 '21 at 10:07
1

it finds them regardless of location, I added text to show an example. the regex `?=.*` will find it anywhere – MAFiA303 Jun 17 '21 at 10:09
Ah okay, great. – smci Jun 17 '21 at 10:12

score 1 · Answer 2 · answered Jun 17 '21 at 09:55

1

You can split them up like this:

tweets_text.loc[tweets_text.str.contains("break") & tweets_text.str.contains("social") & tweets_text.str.contains("media")]

answered Jun 17 '21 at 09:55

Rutger

593
5
11

score 0 · Answer 3 · answered Oct 18 '22 at 18:31

You can always add some additional parameters to ignore uppercase or lowercase letters, using flags. Using @Rutger 's code. Check the documentation for some additional parameters.

tweets_text.loc[tweets_text.str.contains("break", flags = re.IGNORECASE) & tweets_text.str.contains("social") & tweets_text.str.contains("media", flags = re.IGNORECASE)]

In addition to that you can do the same things by combining lambda function and all, as follows:

def find_words(data, list_of_words):
    function = lambda row: all(word.lower() in row.lower() 
                               for word in list_of_words)

    return data.loc[data[column_name].apply(function)]

Check if pandas string column contains multiple words, in any order

3 Answers3