0

When I use irb, I got this:

2.3.1 :001 > 199 / (1.0 / 0.1)
 => 19.9
2.3.1 :002 > 199 * 0.1
 => 19.900000000000002

I am just curious about this. Can someone explain it to me?

ShallmentMo
  • 449
  • 1
  • 4
  • 15
  • Floating point error, it's a problem with all computer programs. Try the same calculation in calc.exe in Windows. – redcodefinal Aug 26 '16 at 22:24
  • @redcodefinal It's only a problem if you use floating points; many calculator programs don't exactly to avoid these precision problems; in most programming languages you can also use arbitrary-precision arithmetic when accuracy matters (e.g. when dealing with money). Floating point is typically used by default just because it's so much faster and because *usually* the accuracy is "good enough"... – Martin Tournoij Aug 27 '16 at 14:43
  • @Carpetsmoker Yeah that's what I'm saying. It has to do with the limit on the amount of decimal places. The reason we get a weird number is its the answer minus the floating point epsilon for that language. – redcodefinal Sep 01 '16 at 23:35
  • @redcodefinal Well, my previous comment was somewhat pedantic − I'm not sure why I even bothered posting it, I must of been drunk − but there *are* ways around this accuracy problem at the penalty of performance (so not "all computer programs" have it) ;-) – Martin Tournoij Sep 02 '16 at 00:04
  • @Carpetsmoker It's not really even a performance issue, it';s a memory issue. There is only so much space we can divvy out to a float, and therefore only so much accuracy. – redcodefinal Sep 02 '16 at 00:05

0 Answers0