1

I met strange problem. We have a script in our Ruby on Rails app, where the cut off point for some condition is 0.8.

What seems weird when I tried to divide 2.4 by 3 Ruby gives result 0.799999. For test purposes I divided 2.4 by 4 or 6 and the results were 0.6 and 0.4, respectively, so mathematically looks correct for those numbers. Do you have an idea why Ruby calculates 2.4 / 3 = 0.799999 ?

Lukas
  • 21
  • 3
  • 2
    Floating point rounding errors. If you're handling money try using cents instead of float values, or look at the money gem – Nick M Oct 08 '20 at 16:06
  • Hint: it's *not* decimal division, it's binary. – Jörg W Mittag Oct 08 '20 at 16:13
  • [What every computer scientist should know about Floating Point Arithmetic](https://www.itu.dk/~sestoft/bachelor/IEEE754_article.pdf) is the seminal article on this. A nicer, more user friendly summary is available at [https://floating-point-gui.de/](https://floating-point-gui.de/) – mcfinnigan Oct 09 '20 at 10:10

0 Answers0