Find if some strings are in a pandas data frame ( row wise and column wise operation )

Question

Hello stackoverflowers!!

I have the following data frame

import numpy as np 
import pandas as pd

dataframe = pd.DataFrame({'column1': ['a', 'alike', 'b', 'x', 'a'],
                          'column2': ['a', 'b', 'c', 'unlake', 'like']})

  column1 column2
0       a       a
1   alike       b
2       b       c
3       x  unlake
4       a    like

And I would like to create another column that has 1 if any of the following strings

check = ['like', 'lake', 'lik']

is in any of the two columns.

I started with this:

any([check1 in dataframe['column1'][1] for check1 in check]) # for one value this works

however when I want to do it for the whole column I do not get the expected result

dataframe[['column1']].apply(lambda row: any((check1 in row for check1 in check)), axis = 1) # this works but does not give the expected

I am propably missing something.

Thanks for the help

Answer 1

This should give you what you want

df['new_col'] = np.where(df.isin(check).any(axis=1),1,0) #this is for an exact match

Expanding to see if it contains any of the items

import numpy as np 
import pandas as pd

df = pd.DataFrame({'column1': ['a', 'alike', 'b', 'x', 'a'],
                      'column2': ['a', 'b', 'c', 'unlake', 'like']})
check = ['like', 'lake', 'lik']
pattern = '|'.join(check)
mask = np.column_stack([df[col].str.contains(pattern, na=False) for col in df])
df['check'] = np.where(mask.any(axis=1),1,0)

Answer 2

Combine np.logical_or with pd.Series.str.contains. This assumes partial matches are permitted.

import numpy as np 
import pandas as pd

df = pd.DataFrame({'column1': ['a', 'alike', 'b', 'x', 'a'],
                   'column2': ['a', 'b', 'c', 'unlake', 'like']})

test_str = '|'.join({'like', 'lake', 'lik'})

df['Test'] = np.logical_or(df['column1'].str.contains(test_str),
                           df['column2'].str.contains(test_str))


# output
# column1 column2 Test
# a a False 
# alike b True 
# b c False 
# x unlake True 
# a like True 

Answer 3

You can join your substrings together using the regex "or" operator | and then apply the pd.Series.str.contains method like so:

dataframe['substr_match'] = dataframe.apply(\
  lambda col: col.str.contains('|'.join(check))).any(axis=1)
#   column1 column2  substr_match
# 0       a       a         False
# 1   alike       b          True
# 2       b       c         False
# 3       x  unlake          True
# 4       a    like          True