Comparing "float('nan')" and "math.nan"

Question

I have a float variable which may or may not be a number, and I want to check if that is the case. With x = float('nan'), I observed some behavior that surprised me:

    print(x is math.nan)

>>> False

This means that float('nan') and math.nan are different objects, which I didn't expect, but that's okay. However, the result is the same, when I check for equality with ==:

print(x == math.nan):

>>> False

I get the correct result for all kinds of not-a-number, if I use math.isnan(x). Still, why doesn't float('nan') == math.nan evaluate to True?.

Answer 1

"Not a number" is (in some sense) the absence of a value.

Traditionally, and per the IEEE floating-point specification, it does not equal itself.

That's because there is no meaningful value to compare.

In fact, some people use this fact to detect NaN, so you could try x != x as your condition instead (though the linked Q&A arguably has some better suggestions).

The expression math.nan is math.nan is true, though, because is does an object identity comparison rather than a value equivalence/equality comparison.

Answer 2

This is not special behaviour: is returns whether two object are actually referring to the same thing (essentially in memory) and == returns whether two objects have the same value.

To see if they refer to the same thing, we can use id().

>>> a = [1,2,3]
>>> b = a
>>> id(a)
140302781856200
>>> id(b)
140302781856200
>>> a == b
True
>>> a is b
True
>>> c = [1,2,3]
>>> id(c)
140302781864904
>>> a == c
True
>>> a is c
False

Here we see that by assigning b = a, they now refer to the same list: hence is and == are True. However when we define c to be a new variable with the same value as a and b, it is ==, but is returns False.

The same is true for NaNs.

Answer 3

That is because NaN is just a float value. Using is doesn't check for whether the variables have the same value, it checks whether they are the same object. If you create two floats with the same value, they are not the same object, they are two objects with the same value. Take this for example:

>>> a = float('nan')
>>> b = float('nan')
>>> a is b
False

So even if you create two NaN values the same way, they are not the same object. This is true even for more trivial floats. Try this:

>>> a = 1.
>>> b = 1.
>>> a is b
False

The default version of Python re-uses some values, so that any instance of that value is the same object. So take this for example (note the lack of decimal, these are integers not floats):

>>> a = 1
>>> b = 1
>>> a is b
True

But that is an implementation detail you should never rely on, it can change at any time and can vary between python implementations. But even with that, NaN is not one of the values the default Python interpreter does this for.

You can check whether two variables are the same object manually using the id function, which gives a unique number for each simultaneously-existing object (although the numbers can be re-used if a variable is deleted, even automatically).

>>> a=1.
>>> b=1.
>>> c=float('nan')
>>> d=float('nan')
>>> e=1
>>> f=1
>>> id(a)
139622774035752
>>> id(b)
139622774035872
>>> id(c)
139622774035824
>>> id(d)
139622774035800
>>> id(e)
139622781650528
>>> id(f)
139622781650528

As for why they aren't equal, that is just part of the definition of NaN as it is used on modern computers. By definition, NaN must never be equal to itself. It is part of an international standard on how floating-point numbers work, and this behavior is built into modern CPUs.

Answer 4

While they are not the same object (because they are from different modules where they were implemented separately) and they are not equal (by design NaN != NaN), there is the function math.isnan (and numpy.isnan if you want a vectorized version) exactly for this purpose:

import math
import numpy

math.isnan(math.nan)
# True
math.isnan(numpy.nan)
# True
math.isnan(float("nan"))
# True

Although they are all unequal to each other and not identical:

math.nan == numpy.nan or math.nan is numpy.nan
# False
math.nan == float("nan") or math.nan is float("nan")
# False
numpy.nan == float("nan") or numpy.nan is float("nan")
# False

Answer 5

You can use the "hex" function that is built into "float"

float('nan') == math.nan                   # FALSE

---

float('nan').hex() == math.nan.hex()       # TRUE

---

float('nan').hex() == float('nan').hex()   # TRUE

---

float('nan').hex() == numpy.nan.hex()   # TRUE

This is very helpful if you are using queries in pandas. I recently was trying to use:

df.eval('A == "NaN"')

Which should check if column A is NaN. But, pandas was automatically converting the string, "NaN", into a float. Most people would recommend using df['A'].isna(), but in our case, trying to pass an expression into a method, so it should handle any expression. The solution was to do:

df.applymap(lambda x: 'NaN' if x.hex() == float('NaN').hex() else x).eval('A == "NaN"')

Answer 6

You can convert the nan value to string for comparing. somthing like this:

x=float("nan")
s_nan = str(x)
if s_nan == "nan":
   # What you need to do...
   print('x is not a number')