When can Java produce a NaN?

Question

I know what Java Double.NaN is. I have some Java code that produces NaN.

// calculate errors
delta = m1 + m2 - M;
eta = f1 + f2 - F;
for (int i = 0; i < numChildren; i++) {
  epsilon[i] = p[i]*m1+(1-p[i])*m2+q[i]*f1+(1-q[i])*f2-C[i];
}

// use errors in gradient descent
// set aside differences for the p's and q's
float mDiff = m1 - m2;
float fDiff = f1 - f2;
// first update m's and f's
m1 -= rate*delta;
m2 -= rate*delta;
f1 -= rate*eta;
f2 -= rate*eta;
for (int i = 0; i < numChildren; i++) {
  m1 -= rate*epsilon[i]*p[i];
  m2 -= rate*epsilon[i]*(1-p[i]);
  f1 -= rate*epsilon[i]*q[i];
  f2 -= rate*epsilon[i]*(1-q[i]);
}
// now update the p's and q's
for (int i = 0; i < numChildren; i++) {
  p[i] -= rate*epsilon[i]*mDiff;
  q[i] -= rate*epsilon[i]*fDiff;  
}

Under what circumstances will Java produce a NaN value?

Answer 1

NaN is triggered by the following occurrences:

• results that are complex values
  • √x where x is negative
  • log(x) where x is negative
  • tan(x) where x mod 180 is 90
  • asin(x) or acos(x) where x is outside [-1..1]
• 0/0
• ∞/∞
• ∞/−∞
• −∞/∞
• −∞/−∞
• 0×∞
• 0×−∞
• 1^∞
• ∞ + (−∞)
• (−∞) + ∞

Sorry for such a general answer, but I hope that helped.

Answer 2

According to Wikipedia:

There are three kinds of operation which return NaN:

• Operations with a NaN as at least one operand
• Indeterminate forms
  • The divisions 0/0, ∞/∞, ∞/−∞, −∞/∞, and −∞/−∞
  • The multiplications 0×∞ and 0×−∞
  • The power 1^∞
  • The additions ∞ + (−∞), (−∞) + ∞ and equivalent subtractions.
• Real operations with complex results:
  • The square root of a negative number
  • The logarithm of a negative number
  • The tangent of an odd multiple of 90 degrees (or π/2 radians)
  • The inverse sine or cosine of a number which is less than −1 or greater than +1.

This Java snippet illustrates all of the above, except the tangent one (I suspect because of limited precision of double):

import java.util.*;
import static java.lang.Double.NaN;
import static java.lang.Double.POSITIVE_INFINITY;
import static java.lang.Double.NEGATIVE_INFINITY;

public class NaN {
    public static void main(String args[]) {
        double[] allNaNs = {
            0D/0D,
            POSITIVE_INFINITY / POSITIVE_INFINITY,
            POSITIVE_INFINITY / NEGATIVE_INFINITY,
            NEGATIVE_INFINITY / POSITIVE_INFINITY,
            NEGATIVE_INFINITY / NEGATIVE_INFINITY,
            0 * POSITIVE_INFINITY,
            0 * NEGATIVE_INFINITY,
            Math.pow(1, POSITIVE_INFINITY),
            POSITIVE_INFINITY + NEGATIVE_INFINITY,
            NEGATIVE_INFINITY + POSITIVE_INFINITY,
            POSITIVE_INFINITY - POSITIVE_INFINITY,
            NEGATIVE_INFINITY - NEGATIVE_INFINITY,
            Math.sqrt(-1),
            Math.log(-1),
            Math.asin(-2),
            Math.acos(+2),
        };
        System.out.println(Arrays.toString(allNaNs));
        // prints "[NaN, NaN...]"
        System.out.println(NaN == NaN); // prints "false"
        System.out.println(Double.isNaN(NaN)); // prints "true"
    }
}

References

• Wikipedia/NaN
• JLS 15.21.1 Numerical Equality Operators == and !=

  If either operand is NaN, then the result of == is false but the result of != is true. Indeed, the test x!=x is true if and only if the value of x is NaN. (The methods Float.isNaN and Double.isNaN may also be used to test whether a value is NaN.)

Answer 3

Given what I know about gradient descent, you are most probably jumping out to infinity because you do not have adaptive rate (i.e. your rate is too big).

Answer 4

Have you tried sprinkling your code with System.out.println statements to determine exactly where NaNs start occuring?