Remove pandas dataframe row if one column's element is non-numeric

Question

I have been searching for an answer for this simple thing for an hour now. I have a pandas dataframe that looks like this:

   title      |   price

"some text 1"     124.5
"some text 2"     543.2
"some text 3"     156.2
"some text 4"     "Price N/A"
"some text 5"     216.7

I want to remove the rows that don't contain an actual float price. I tried the suggestion from this answer:

raw_data[raw_data.price.apply(lambda x: x.isnumeric())]

and I get:

AttributeError: 'float' object has no attribute 'isnumeric'

Nothing seems to work for such a simple operation. I tried pretty much all the answers I found on stack and elsewhere.

The error might indicate that you are using Python 2. Are you using Python 3? — MEdwin, Mar 24 '21 at 13:29
raw_data[raw_data.price.apply(lambda x: isinstance(x, float))] — Urh, Mar 24 '21 at 13:32

aj7amigo · Accepted Answer · 2021-03-24T13:54:44.093

5

You can use the 'to_numeric' operation of pandas which will throw up an error when a string value is met. This error will be handled by 'coerce' which forces the content to null. We then use the 'notnull' operation to filter this particular row.

df[pd.to_numeric(df['price'], errors='coerce').notnull()]

edited Mar 24 '21 at 13:54

answered Mar 24 '21 at 13:34

aj7amigo

368
2
12

Always try to add some comment to your code and remember to use four spaces before the text. It will be displayed in a proper format then. – Arkadiusz Mar 24 '21 at 13:49
1

@Arkadiusz Thank you. I have edited it accordingly. – aj7amigo Mar 24 '21 at 13:55

score 1 · Answer 2 · answered Mar 24 '21 at 13:40

You get an error, since you are applying (lambda x: x.isnumeric()) on a float.

float.isnumeric() doesn't exist, but you can use isinstance(x, t), which returns True if x is of a type t.

raw_data[raw_data['price'].apply(lambda x : isinstance(x, float))]

Remove pandas dataframe row if one column's element is non-numeric

2 Answers2