0

Using Visual Studio 2017, create a VB.net project, and run the following lines of code:

Dim a As Double = 0.52
Dim value As Double
value = a * 3              'value = 1.56
value = value + 1          'value = 2.56
value = value + 2          'value = 4.5600000000000005
Console.WriteLine(value)   '4.56

If you put a break point before addition of 2 to value, you can see value holds 4.5600000000000005 instead of 4.56.

Why does the value have the trailing 0.0000000000000005 after addition of 2 to it? And more importantly, how do I handle the trailing inaccuracies?

That inaccuracy can be magnified if I perform multiplication and will surely screw up my end result.

Ňɏssa Pøngjǣrdenlarp
  • 38,411
  • 12
  • 59
  • 178
Steven Chang
  • 94
  • 1
  • 3
  • 3
    Possible duplicate of [Is floating point math broken?](https://stackoverflow.com/questions/588004/is-floating-point-math-broken) – litelite Aug 31 '17 at 18:26
  • There are inherent inaccuracies with floating point math because of how approximations of their values are represented in memory. Most floating point stuff is not as visible in VB.net, but underneath everything there is constantly rounding and estimation going on, 1.1 isn't really equal to 1 + 0.1 to a computer, it's just close enough to call equal. What you've seen is simply an example of those imperfect representations bubbling up to the surface. Google floating point math if you want to learn more about the root cause, add C++ to the search to see more low-level explanations and examples – soohoonigan Aug 31 '17 at 18:52
  • Did you learn about significant figures in maths class at school? This is an example of where you can implement that. Any result you get from a calculation cannot have more significant figures than the inputs used. Also, the `Decimal` data type is far less prone to such issues but it is much slower to work with as a result. That's not an issue for a small volume of calculations but can be for large numbers. – jmcilhinney Sep 01 '17 at 02:36
  • @litelite - Thanks for pointing out this is a duplicate question I can't seem to find a way to tackle this rounding inaccuracy in a practical way, so here is what I end up doing. value = Math.Round(value, 6, MidpointRounding.AwayFromZero) 'value holds desired 4.56 – Steven Chang Sep 15 '17 at 17:42

0 Answers0