0

I have a fairly simple expression that I want to evaluate which is ZoomLevel = ZoomLevel - 0.1 where ZoomLevel is declared as a double.

I have called the routine multiple times:

  • 1.5 - 0.1 = 1.4
  • 1.4 - 0.1 = 1.3
  • 1.3 - 0.1 = 1.2

Then it is time to execute the same statement where ZoomLevel equals 1.2: enter image description here

Then I step over the statement:

enter image description here

Why does this statement evaluate to 1.09 and not 1.1?

(I guess it have something to do with lack of precision in the double implementation.)

EDIT: This is what I get when printing the content of ZoomLevel before and after execution of that statement.

?string.Format("{0:N20}", ZoomLevel);
"1,20000000000000000000"
?string.Format("{0:N20}", ZoomLevel);
"1,09000000000000000000"
Patrik B
  • 151
  • 1
  • 13
  • Good guess http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems – Steve Oct 17 '13 at 11:37
  • 5
    I picked one post here on SO at random when voting to close as duplicate. Do a search, it's been answered over and over. – Lasse V. Karlsen Oct 17 '13 at 11:41
  • Can you show the implementation of `ZoomLevel`? – Lasse V. Karlsen Oct 17 '13 at 11:42
  • But, why is the property rounded to 1.09? If it was a floating-point representation issue, then it would display something like 1.09999... The loss of precision is very big here. – Dalmas Oct 17 '13 at 11:47
  • There is something more in play here than just a couple of doubles. Most likely the property is rounding when storing, or something like it. We need to see the implementation of `ZoomLevel` to know what really happens here. – Lasse V. Karlsen Oct 17 '13 at 11:49
  • To be clear, there is no way `1.2 - 0.1 = 1.09` in .NET without some extra help. It would either output `1.1` or what athabaska posted as an answer, something like `1.099999(9....)`. – Lasse V. Karlsen Oct 17 '13 at 11:50
  • 2
    Gaah! Yes you are right. This is due to a rounding error in a lower part of the application because ZoomLevel is stored as an int. `(int)(ZoomLevel * 100);` where ZoomLevel is 1.0999999999999999 More minus points to me... – Patrik B Oct 17 '13 at 11:51

2 Answers2

2

It's because doubles are binary floating point numbers. Since we humans like to see numbers in decimal, they get converted to and from decimal, but that conversion is not precise for some values. So rounding errors can occur.

David Arno
  • 42,717
  • 16
  • 86
  • 131
0

I bet its actually 1.0999(9).

You got to read some about double in .NET.

athabaska
  • 455
  • 3
  • 22