0

Possible Duplicate:
How is floating point stored? When does it matter?

Using the built-in calculator on my Win7 x64 I get the number -8.1648465955514287168521180122928e-39 when calculation sqrt(4)-2.

I would expect the result to be 0.

Community
  • 1
  • 1
Matthias Braun
  • 32,039
  • 22
  • 142
  • 171
  • What's to round with `sqrt(4)-2`? – Matthias Braun Apr 10 '12 at 22:29
  • How do you think the sqrt(4) gets evaluated? – JB King Apr 10 '12 at 22:31
  • 2
    @Mysticial: decidedly non-standard floating-point error; any *standard* floating-point system would produce 0. – Stephen Canon Apr 10 '12 at 22:31
  • @StephenCanon Whatever Windows is doing, it's not standard since that's a lot more than double-precision. Perhaps it's emulating quad-precision, but not perfectly. – Mysticial Apr 10 '12 at 22:33
  • My understanding is that Calculator guys implemented a high-precision floating point library so that they didn't get embarassing errors... like this :) – dlev Apr 10 '12 at 22:34
  • @dlev - indeed they did http://blogs.msdn.com/b/oldnewthing/archive/2004/05/25/141253.aspx – Martin Beckett Apr 10 '12 at 22:37
  • @dlev: it would seem they didn't do a very good job of it, since this is computed correctly in every standardized floating-point system =) – Stephen Canon Apr 10 '12 at 22:40
  • @StephenCanon Could you elaborate? I would have thought that the presence (or absence) of representation error would be based on the algorithm used to calculate the square-root. – dlev Apr 10 '12 at 22:48
  • @JB King I don't know how the windows calculator evaluates sqrt(4). Intuitively I thought it doesn't involve fractions that would introduce rounding error. – Matthias Braun Apr 10 '12 at 22:58
  • http://en.wikipedia.org/wiki/Methods_of_computing_square_roots#Approximations_that_depend_on_IEEE_representation has a few computations that do involve fractions. You may want to consider researching Numerical Analysis versus Symbolic Computation just as something to consider if you want to keep going down this rabbit hole. http://www.joelonsoftware.com/items/2007/09/26b.html would be Jel's explaining of an Excel bug that is a similar point. – JB King Apr 10 '12 at 23:03
  • @dlev: In every standard floating-point system, modest integers are represented exactly, and square root and subtraction are correctly-rounded operations, which leaves no possible source of error. – Stephen Canon Apr 10 '12 at 23:37
  • @StephenCanon Ah, I see. You're including how certain operations are performed in the definition. Makes sense, and agreed! – dlev Apr 10 '12 at 23:38

1 Answers1

1

There's some error with floating-point values, when you go to subtract them on occasion. You may get a representation that's 0 or really close to 0 (10^-39's pretty close).

For more information, check out Fractions in Binary on Wikipedia.

Makoto
  • 104,088
  • 27
  • 192
  • 230
  • Thanks. I didn't know how fractions were represented in binary. This [link](http://en.wikipedia.org/wiki/IEEE_754-1985) helped as well. – Matthias Braun Apr 10 '12 at 22:54