0

I need some help here. I am trying to convert some numbers and running in to some weird things.

I declare a double to the value 264210.35. I then multiply it with 100, and in my opinion i get the wrong result. 

            double value = 264210.35;

            double sum = value.Value*100;
            //sum = 26421034.999999996

            var longSum = (long) sum;
            //longSum = 25421034

When i then multiply it with 100 i excpect to get the sum of 26421035, but i don't. I get 26421034.999999996. When i use a normal calculator and multiplies the value with 100 i get 25421035. How come i don't in this case?

And when i then cast it to a long, it ends up being 26421034. Which then makes it one didgit wrong.

Brewsli
  • 127
  • 1
  • 10
  • 2
    Not all fractional numbers can be exactly represented by a `double` maybe you should use `decimal` instead. Also casting just truncates, use `Math.Round` instead. – juharr Sep 26 '19 at 11:42
  • [What Every Computer Scientist Should Know About Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html) – Steve Sep 26 '19 at 11:44
  • `double` can represent exactly: small enough integer values (e.g. `1234`) or fractions of `2` (e.g. `0.125 = 1/8`, `0.5=1/2`, `0.375=3/8` etc.) or combinations of them, so if you have `264210.25` the result will be exact; if fraction part is `0.35` (as in `264210.35`) then we'll have no guarantee – Dmitry Bychenko Sep 26 '19 at 11:47
  • If you look closely at `value` you can find that it was actually 264210.349999999976716935634613037109375 (exactly) – harold Sep 26 '19 at 11:48
  • To add to what juharr said: `double` has a larger maximum and a smaller minimum value than `decimal`, but is only precise to about 17 digits, meanwhile `decimal` is precise to about 29 digits, so it looks like this is a rounding mistake – MindSwipe Sep 26 '19 at 11:49
  • `var longSum = (long) (sum > 0 ? sum + 0.5 : sum - 0.5);` – Dmitry Bychenko Sep 26 '19 at 11:52

0 Answers0