0

So this is something I was not expecting, this is very simple but not giving the results I expect.

I would expect doing a simple equation of 5.1 - 4.9 to return 0.2, but this does not appear to be the case. I can use the round function to work around it, but I just find it confusing that 5.1 - 4.9 = 0.19999999999999.

Could anyone advise why this happens, and the best way to work around it.

' returns 1.1
myvalue = 5.1 - 4
msgbox myvalue

' returns 0.6
myvalue = 5.1 - 4.5
msgbox myvalue

' returns 0.3
myvalue = 5.1 - 4.8
msgbox myvalue

' returns 0.19999999999999
myvalue = 5.1 - 4.9
msgbox myvalue

' returns 9.999999999999996E-02
myvalue = 5.1 - 5
msgbox myvalue

' returns 0.2
myvalue = Round(5.1 - 4.9,2)
msgbox myvalue
Jimmy
  • 31
  • 5
  • The reason is this: [Is floating point math broken?](http://stackoverflow.com/questions/588004/is-floating-point-math-broken) the fix is to round. – Alex K. Apr 13 '17 at 13:23
  • Jon Skeet is probably crying right now. – Ansgar Wiechers Apr 13 '17 at 13:28
  • Thank you very much, sorry for the duplicate question. Would you like me to delete this question? Kind regards. – Jimmy Apr 13 '17 at 13:34
  • 1
    For some purposes, you can use the `Currency` subtype if you want to avoid these issues. For example, `myvalue = CCur(5.1) - 4.9` evaluates to `0.2` as expected. The `CCur` causes the entire expression to be promoted to `Currency`. The main problem is that `Currency` only has 4 decimal places of accuracy (although it has a fairly large range). – John Coleman Apr 13 '17 at 15:21
  • @JohnColeman, your comment deserves to be an answer. – NoChance Dec 19 '20 at 20:47

0 Answers0