Unable to test for NaT in Pandas dataframe using a function

Question

I am trying to write a function to compare a set of dates in each row of a dataframe after checking for NaT values in one of the columns. I get a "'Timestamp' object has no attribute" error on the isnull test.

import pandas as pd

a =['2018-01-01','2019-01-01','2020-01-01']
b =['2018-12-31','2019-12-31', 'NaT']

df = pd.DataFrame({"a":a,"b":b})
df = df.apply(pd.to_datetime)

#if date value in col b is later than col a return col b
def datefix(x):
    if x['b'].isnull() == True:
        result = x['a']

    elif x['b']> x['a']:
        result = x['b']
    return result

#isnull test in function fails with error 
df['c']=df.apply(datefix,axis=1)

What confuses me is when I use isnull over the same column in the dataframe direclty without a function the test works fine:

    #isnull test works over dataframe
    b = df[df['b'].isnull() == True]
    print(b)

           a   b
2 2020-01-01 NaT

Try without the True.. `if x['b'].isnull():` – XXavier Apr 19 '20 at 01:06 — XXavier, Apr 19 '20 at 01:06
How about is if `x['b'] is pd.NaT:` – XXavier Apr 19 '20 at 01:12 — XXavier, Apr 19 '20 at 01:12
Please provide the entire error message. – AMC Apr 19 '20 at 01:59 — AMC, Apr 19 '20 at 01:59
Thanks XXavier, `x['b'] is pd.NaT:` works. – Yael Apr 20 '20 at 08:48 — Yael, Apr 20 '20 at 08:48

score 1 · Answer 1 · answered Apr 19 '20 at 01:15

1

Found my answer here Issue checking for missing datetime value in series :

Use is pd.NaT instead of .isnull()

answered Apr 19 '20 at 01:15

Yael

21
1

This is the same as my answer in the comment right? – XXavier Apr 19 '20 at 01:19
Yes, thanks XXavier. Sorry I found myself before I saw you second comment – Yael Apr 20 '20 at 08:52

score 1 · Answer 2 · answered Apr 19 '20 at 01:28

1

Just to complemente the right answers, in python comparing to None, False, and True can not be done with == operator, but with is operator:

a = True
a == True # False
a is True # True

The same applies for np.nan and pd.NaT

a = pd.NaT
a == pd.NaT # False
a is pd.NaT # True

answered Apr 19 '20 at 01:28

jcaliz

3,891
2
9
13

Thanks jcaliz. That's helpful to know. – Yael Apr 20 '20 at 08:53

Unable to test for NaT in Pandas dataframe using a function

2 Answers2