9

Using Java on a Windows 7 PC (not sure if that matters) and calling Math.cos() on values that should return 0 (like pi/2) instead returns small values, but small values that, unless I'm misunderstanding, are much greater than 1 ulp off from zero.

Math.cos(Math.PI/2) = 6.123233995736766E-17
Math.ulp(Math.cos(Math.PI/2)) = 1.232595164407831E-32

Is this in fact within 1 ulp and I'm simply confused? And would this be an acceptable wrapper method to resolve this minor inaccuracy?

public static double cos(double a){
    double temp = Math.abs(a % Math.PI);
    if(temp == Math.PI/2)
        return 0;
    return Math.cos(a);
}
dimo414
  • 47,227
  • 18
  • 148
  • 244
  • @dimo414: First a comment on your "not sure if that matters". The behavior of most Math operation are not precisely defined so the OS and CPU may matter. If you want math operation whose behavior is strictly defined (which arguably are easier to troubleshoot) you want to use StrictMath, not Math (of course StrictMath operations shall probably be slower for they cannot use the hardware accelerated operation available in the CPU). – SyntaxT3rr0r Feb 10 '10 at 09:41
  • Math.cos() is just a wrapper for StrictMath.cos(), which is itself a native function. – dimo414 Feb 10 '10 at 11:53

3 Answers3

11

Don't forget that Math.PI/2 is an approximation. It's not going to be exactly pi/2, so the result of cos(Math.PI/2) isn't going to be exactly 0. Math.cos may be returning a pretty accurate version of the cosine of the exact value returned by calculating Math.PI/2.

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • Very true, but that said, it seems intuitive to me that if Math.PI is the double closest to Pi, then Math.PI/2 ought to be the double closest to Pi/2. And since by the definition of cos() Pi/2 and 3Pi/2 are zero, it seems when the closest double approximation of those numbers is passed to Math.cos(), it should also be zero. Maybe this is just flat out bad, (which is the crux of my question) but that makes intuitive sense to me. – dimo414 Feb 10 '10 at 12:02
7

You should never use == with doubles. You must always do within en error margin. 10-17 is good precision if you ask me. Ulp figure of 10-32 is just precisson of double that is in 10-17 order of magnitude, as 2.220446049250313E-16 is the precision of the number in 100 magnitude.

Slartibartfast
  • 8,735
  • 6
  • 41
  • 45
  • @Slartibartfsat: +1, exactly. You beat me to it. – SyntaxT3rr0r Feb 10 '10 at 09:47
  • The error margin should be at least twice ulp(PI/2), because that is the imprecision of PI/2. cos has a derivative of -1 at that point, so the imprecision of PI/2 is reflected in the result, plus the imprecision of cos. – starblue Feb 10 '10 at 10:18
  • Normally I'd agree with you, but I want to limit my exception to as few cases as possible, so if the input is even /very/ slightly off Math.PI/2 then I'll let it calculate natively, but in the special case where it's exactly Math.PI/2 or one of its multiples, I'd like it to be exactly zero. Some limited testing has shown it works, at least on my computer. Is this reasoning ok, or should I still be checking a range? – dimo414 Feb 10 '10 at 12:07
2

This is a common error when you are starting out, this link has a very technical discussion of the reasons why. http://docs.sun.com/source/806-3568/ncg_goldberg.html

But in it's simplest form, in the same way that we can't exactly represent 1/3 in the decimal system, there are values that can't be represented exactly in the binary system

Goibniu
  • 2,212
  • 3
  • 34
  • 38