89
nat = np.datetime64('NaT')
nat == nat
>> FutureWarning: In the future, 'NAT == x' and 'x == NAT' will always be False.

np.isnan(nat)
>> TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

How can I check if a datetime64 is NaT? I can't seem to dig anything out of the docs. I know Pandas can do it, but I'd rather not add a dependency for something so basic.

user65
  • 1,223
  • 1
  • 9
  • 12
  • 1
    You can compare with `np.datetime64('NaT')` actually: `nat == np.datetime64('NaT')` Output: `True`. – Vadim Shkaberda Jul 21 '16 at 19:17
  • 1
    Are you using 1.11.1? NaTs can no longer be compared: https://github.com/numpy/numpy/blob/master/doc/release/1.11.0-notes.rst – user65 Jul 21 '16 at 19:31
  • I apologize, it was my inattention. 1. Now you can compare `nat == nat` and it will return `True`. 2. As said in the github, in `numpy 1.12.0` you'll still be able to compare `NaT`: `nat != np.datetime64('NaT')` will return `True`, otherwise all comparisons involving NaT will return False. So, the final conclusion: first you'll need to check version of numpy, then choose how to compare `NaT`'s. – Vadim Shkaberda Jul 21 '16 at 19:46
  • Could you provide an example of how to check if a value is NaT? I'm still getting warnings regardless of what operator is used. – user65 Jul 21 '16 at 20:36
  • 3
    Could you unaccept Vadim's answer? The FutureWarning is saying that the NumPy developers plan to change things so that answer no longer works. – user2357112 Mar 22 '18 at 17:58

6 Answers6

107

can check for NaT with pandas.isnull:

>>> import numpy as np
>>> import pandas as pd
>>> pd.isnull(np.datetime64('NaT'))
True

If you don't want to use pandas you can also define your own function (parts are taken from the pandas source):

nat_as_integer = np.datetime64('NAT').view('i8')

def isnat(your_datetime):
    dtype_string = str(your_datetime.dtype)
    if 'datetime64' in dtype_string or 'timedelta64' in dtype_string:
        return your_datetime.view('i8') == nat_as_integer
    return False  # it can't be a NaT if it's not a dateime

This correctly identifies NaT values:

>>> isnat(np.datetime64('NAT'))
True

>>> isnat(np.timedelta64('NAT'))
True

And realizes if it's not a datetime or timedelta:

>>> isnat(np.timedelta64('NAT').view('i8'))
False

In the future there might be an isnat-function in the numpy code, at least they have a (currently open) pull request about it: Link to the PR (NumPy github)

MSeifert
  • 145,886
  • 38
  • 333
  • 352
  • Observation, which seems irrelevant: you use `NaT` and `NAT`. Again, it appears `pd.datetime64()` spits out the same for both, but it also does for `''` and `None`. Still, it seems like consistency within an answer will be less confusing to the reader. – Hendy Oct 04 '19 at 14:36
39

Since NumPy version 1.13 it contains an isnat function:

>>> import numpy as np
>>> np.isnat(np.datetime64('nat'))
True

It also works for arrays:

>>> np.isnat(np.array(['nat', 1, 2, 3, 4, 'nat', 5], dtype='datetime64[D]'))
array([ True, False, False, False, False,  True, False], dtype=bool)
MSeifert
  • 145,886
  • 38
  • 333
  • 352
29

Very simple and surprisingly fast: (without numpy or pandas)

    str( myDate ) == 'NaT'            # True if myDate is NaT

Ok, it's a little nasty, but given the ambiguity surrounding 'NaT' it does the job nicely.

It's also useful when comparing two dates either of which might be NaT as follows:

   str( date1 ) == str( date1 )       # True
   str( date1 ) == str( NaT )         # False
   str( NaT )   == str( date1 )       # False

wait for it...

   str( NaT )   == str( NaT )         # True    (hooray!)
CodeCabbie
  • 3,098
  • 2
  • 19
  • 30
  • 1
    That works for a single NaT but not in vectorized operations, for example it would give `False` for `np.array(['nat', 1, 2, 3, 4, 'nat', 5], dtype='datetime64[D]')` – MSeifert Oct 05 '17 at 20:37
  • Agreed. Just coming up with techniques that do no require dependencies since the OP mentioned that. (though it looks like user65 is already using numpy anyway so isnat would be the best way to go.) – CodeCabbie Oct 06 '17 at 17:15
  • Thanks, I don't know why but using np.isnat(np.datetime64('nat')) with my data raise ValueError: cannot convert float NaN to integer. I replaced it with your solution by: [dict[v] if str(v) != 'NaT' else None for v in list_of_values] – Rony Armon Feb 20 '23 at 08:31
27

INTRO: This answer was written in a time when Numpy was version 1.11 and behaviour of NAT comparison was supposed to change since version 1.12. Clearly that wasn't the case and the second part of answer became wrong. The first part of answer may be not applicable for new versions of numpy. Be sure you've checked MSeifert's answers below.

When you make a comparison at the first time, you always have a warning. But meanwhile returned result of comparison is correct: 
import numpy as np    
nat = np.datetime64('NaT')

def nat_check(nat):
    return nat == np.datetime64('NaT')    

nat_check(nat)
Out[4]: FutureWarning: In the future, 'NAT == x' and 'x == NAT' will always be False.
True

nat_check(nat)
Out[5]: True

If you want to suppress the warning you can use the catch_warnings context manager:

import numpy as np
import warnings

nat = np.datetime64('NaT')

def nat_check(nat):
    with warnings.catch_warnings():
        warnings.simplefilter("ignore")
        return nat == np.datetime64('NaT')    

nat_check(nat)
Out[5]: True

EDIT: For some reason behavior of NAT comparison in Numpy version 1.12 wasn't change, so the next code turned out to be inconsistent.

And finally you might check numpy version to handle changed behavior since version 1.12.0:

def nat_check(nat):
    if [int(x) for x in np.__version__.split('.')[:-1]] > [1, 11]:
        return nat != nat
    with warnings.catch_warnings():
        warnings.simplefilter("ignore")
        return nat == np.datetime64('NaT')

EDIT: As MSeifert mentioned, Numpy contains isnat function since version 1.13.
Vadim Shkaberda
  • 2,807
  • 19
  • 35
  • Did they just forget to extend isnull() to the datetime64 type or do they actually plan on making it impossible to check for NaT in 1.13? I understand why NaT comparisons have been depreciated, but I can't figure out why there is no alternative (other than suppressing warnings? really?) – user65 Jul 22 '16 at 18:08
  • @user65 I guess it's possible they'll make an analogue of `isnan()`, something like `isnat()`. And why do you mention 1.13? They can add it in version 1.12. And that means why you see a warning now: NaT comparisons **is not** deprecated, it **will be** deprecated(actually, its behavior will alter) in version 1.12. In version 1.11 it's still usable as earlier. – Vadim Shkaberda Jul 22 '16 at 18:31
  • NaN does not == NaN. Does NaT == NaT? Seems it does not, at least based on my fiddling around. So if we test a date value with this: *if thisDate == thisDate:*, and the result is false, then thisDate is NaT. Yes? – GDB Jan 04 '18 at 21:59
  • CodeCabbie's response is correct from my experience, and I'm not sure NaT ambiguity is an issue that could cause this not to be true. What if any ambiguity would cause this not to be the case? – GDB Jan 04 '18 at 23:17
  • @GDB When `thisDate = np.datetime64('NaT')` the result of comparison `if thisDate == thisDate ` is `True`. Al least for now. – Vadim Shkaberda Jan 05 '18 at 13:47
  • TypeError: Cannot convert NaTType to pandas._libs.tslibs.timestamps._Timestamp –  Nov 30 '20 at 13:50
  • 1
    @TTeaTie `pandas` is not used in the answer. – Vadim Shkaberda Dec 01 '20 at 09:01
1

This approach avoids the warnings while preserving the array-oriented evaluation. 

import numpy as np
def isnat(x):
    """ 
    datetime64 analog to isnan.
    doesn't yet exist in numpy - other ways give warnings
    and are likely to change.  
    """
    return x.astype('i8') == np.datetime64('NaT').astype('i8')
rusty
  • 21
  • 1
-2

Another way would be to catch the exeption:

def is_nat(npdatetime):
    try:
        npdatetime.strftime('%x')
        return False
    except:
        return True
Guest
  • 1