Finding string over multiple columns in Pandas

Question

I am trying to find if a string exists across multiple columns. I would like to return a 1 if the string exists and 0 if it doesn't as a new series within the dataframe.

After searching the forums, I understand that str.contains could be used, but i'm searching over 100+ columns therefore it isn't efficient for me to work with individual series at a time.

There are some NAs within the columns if this is relevant.

Example simplified dataframe:

d = {'strings_1': ['AA', 'AB', 'AV'], 'strings_2': ['BB', 'BA', 'AG'], 
'strings_1': ['AE', 'AC', 'AI'], 'strings_3': ['AA', 'DD', 'PP'], 
'strings_4': ['AV', 'AB', 'BV']}
simple_df = pd.DataFrame(data=d)

If I am interested in finding 'AA' for example, I would like to return the following dataframe.

Example target dataframe:

d = {'strings_1': ['AA', 'AB', 'AV'], 'strings_2': ['BB', 'BA', 'AG'], 
'strings_1': ['AE', 'AC', 'AI'], 'strings_3': ['AA', 'DD', 'PP'], 
'strings_4': ['AV', 'AB', 'BV'], 'AA_TRUE': [1, 0, 0]}
target_df = pd.DataFrame(data=d)

Many thanks for help.

Do you have a list of strings ? You say ` 1 if the string exists` which string ? — Bharath M Shetty, Nov 08 '17 at 13:44
In my example it would be AA, however yes there would be a list i'd be interested in. I'll likely assign them individually as i'd like multiple labels. — shbfy, Nov 08 '17 at 13:47
Next time add that in the question too. How can we guess you have a list that holds ['AA'] ? — Bharath M Shetty, Nov 08 '17 at 13:47

jezrael · Accepted Answer · 2017-11-08T13:50:13.363

13

If need check mixed values - numeric with strings compare numpy array created by values, use DataFrame.any for check at least one True per row and last cast to int:

simple_df['new'] = (simple_df.values == 'AA').any(1).astype(int)
#or cast all values to string before comparing
#simple_df['new'] = (simple_df.astype(str)== 'AA').any(1).astype(int)
print (simple_df)
  strings_1 strings_2 strings_3 strings_4  new
0        AE        BB        AA        AV    1
1        AC        BA        DD        AB    0
2        AI        AG        PP        BV    0

Detail:

print ((simple_df.values == 'AA'))
[[False False  True False False]
 [False False False False False]
 [False False False False False]]

print ((simple_df.values == 'AA').any(1))
[ True False False]

If need check substring:

simple_df['new'] = simple_df.applymap(lambda x: 'G' in x).any(1).astype(int)
print (simple_df)
  strings_1 strings_2 strings_3 strings_4  new
0        AE        BB        AA        AV    0
1        AC        BA        DD        AB    0
2        AI        AG        PP        BV    1

edited Nov 08 '17 at 13:50

answered Nov 08 '17 at 13:45

jezrael

822,522
95
1,334
1,252

`how to use two substrings at a time in entire data. like here using G ,can use E and G at a time ?` – dondapati Aug 24 '18 at 13:16
`simple_df.applymap(lambda x: 'G|E' in x).any(1).astype(int)` i writing this way but it not gives the output – dondapati Aug 24 '18 at 13:18
1

@dondapati - need `simple_df.applymap(lambda x: any(y in x for y in ['G','E']))` – jezrael Aug 24 '18 at 13:20
`can we apply the above function with selected columns ? just apply the 1 and 2 columns` – dondapati Aug 27 '18 at 07:43
1

@dondapati - sure, use `simple_df.iloc[:, :1].applymap(lambda x: any(y in x for y in ['G','E']))` – jezrael Aug 27 '18 at 07:44

Finding string over multiple columns in Pandas

1 Answers1

Linked