How to apply regex to an element in a series using Python

Question

I have a dataframe and column named as 'Values'.

Please find the code below to create the dataframe

df = pd.DataFrame({'Person_id':[1,2,3,4,5],
 'Values':[np.nan,np.nan,'1.Yes','2.No', np.nan],
       'Ethnicity':['1.Chinese','2.Indian','3.Malay',np.nan,np.nan]})

The dataframe looks like as shown below after executing the above piece of code

I have given only a sample data and this is a part of the main program.

From the above dataframe, I would like to find whether a 'Values' column of a specific row contains 'Yes' or 'No' as values using regex

For example, I would like to know whether df['Values][2] contains 'Yes' keyword/term in its value

To that, I wrote the below piece of code but am not able to get the expected output

df['Values'] = df['Values'].astype(str) 
df['Values'][2].contains('Yes|No',regex=True)

Inspite of multiple tries/variations of above code and search through SO, I am not able to get/resolve this. I am getting the below error

AttributeError: 'str' object has no attribute 'contains'

How can I get whether the value 'Yes' or 'No' is present in specific cell of a column in dataframe.

Please note that this is part of a larger program where I use for loop and indices. Hence, I would like to perform the check at cell level and get the output. Using df.isin will not be of any help

`import re` and then `bool(re.search('Yes|No' , df['Values'][2]))`? — Wiktor Stribiżew, Jun 04 '19 at 10:54
alternatively: `df['Values'].str.contains('Yes|No',regex=True)[2]` — Simon Rogers, Jun 04 '19 at 10:55
You're forgetting to use the `.str` acccessor, eg: `df['Values'].str.contains(...)`... — Jon Clements, Jun 04 '19 at 10:55
Looks like you might even just want to create a new Series of booleans using `df['Values'].str.contains('Yes|No', na=False)` — Jon Clements, Jun 04 '19 at 10:56
Thank you for the response. I understand but when type(df['Values'][2] = Str, do I have to still use str? — The Great, Jun 04 '19 at 10:59
no... because you're then dealing directly with a `str` object which doesn't have a `contains` and its native membership testing doesn't have regex capabilities, you'll have to use `re.match` as @Wiktor said in the first comment here. — Jon Clements, Jun 04 '19 at 11:04

score 1 · Accepted Answer · answered Jun 04 '19 at 10:54

Use str.contains('Yes|No',regex=True)

Ex:

import pandas as pd
import numpy as np

df = pd.DataFrame({'Person_id':[1,2,3,4,5],
 'Values':[np.nan,np.nan,'1.Yes','2.No', np.nan],
       'Ethnicity':['1.Chinese','2.Indian','3.Malay',np.nan,np.nan]})

print(df["Values"].str.contains('Yes|No',regex=True))

Output:

0     NaN
1     NaN
2    True
3    True
4     NaN
Name: Values, dtype: object

How to apply regex to an element in a series using Python

1 Answers1