replacing null values in a Pandas Dataframe using applymap

Question

I've got an "Age" column, but sometimes NaN values are displayed. I know I can use "fillna" for this purposes but I've tried to define my own function (and learning to do this way) and use applymap to dataframe

no success so far.

Age
69
49
NaN
54
NaN

I've tried

   def get_rid_of_nulls(value):
     if value == np.nan:
        return 'Is Null value'
     else:
        return value

with this not working either

 if value == None
   if value isnull
   if value == np.na
   if value ==''
   if value == NaN
   if value == 'NaN'

None of the comparisons seems to work. I'm wrong for sure but I'm stuck and I'm very stubborn to use fillna

thanks

Nothing is equal to `nan`, not even `nan` itself. You need to use a special function to check whether a value is `nan`, you can't use `==`. — Alex Riley, Jan 18 '16 at 17:59

mgc · Accepted Answer · 2016-01-18T20:48:38.403

12

As there is "replacing" in your title, and you mentioned fillna but not the replace() method, you can also obtain the same result doing something like that :

df.Age.replace(np.NaN, 'Is Null value', inplace=True)

# Or, depending on your needs:
df['Age'] = df.Age.replace(np.NaN, 'Is Null value')

# Or without `replace` :
df['Age'] = df.Age.apply(lambda x: x if not pd.isnull(x) else 'Is Null value')

edited Jan 18 '16 at 20:48

answered Jan 18 '16 at 20:39

mgc

5,223
1
24
37

1

No this won't work `NaN` intrinsically cannot be tested for value equality, only your last code snippet will work – EdChum Jan 18 '16 at 21:11
OK, my bad was passing string 'np.NaN' which is not the same – EdChum Jan 18 '16 at 21:15

score 3 · Answer 2 · answered Jan 18 '16 at 18:07

3

You can use pd.isnull():

In [4]:
def get_rid_of_nulls(value):
    if pd.isnull(value):
        return 'Is Null value'
    else:
        return value

df['Age'].apply(get_rid_of_nulls)

Out[4]:
0               69
1               49
2    Is Null value
3               54
4    Is Null value
Name: Age, dtype: object

Similarly you can use the property that NaN does not equal itself:

In [5]:
def get_rid_of_nulls(value):
    if value != value:
        return 'Is Null value'
    else:
        return value

df['Age'].apply(get_rid_of_nulls)

Out[5]:
0               69
1               49
2    Is Null value
3               54
4    Is Null value
Name: Age, dtype: object

answered Jan 18 '16 at 18:07

EdChum

376,765
198
813
562

this is the entire point of .fillna - using .apply this way (with an embedded isnull check is completely non performant) and much more code – Jeff Jan 18 '16 at 23:00
@jeff the op knows about fillna, and this approach is dumb I agree . I'm assuming the op just wanted to know why their approach didn't work as opposed to seeking the most performant method – EdChum Jan 18 '16 at 23:11
ok sure - always like to point out that apply is the last tool to go to – Jeff Jan 18 '16 at 23:13

replacing null values in a Pandas Dataframe using applymap

2 Answers2

Linked