-1

I was trying to solve a mathematical problem:

37.9 - 6.05

Ruby gives me:

37.9 - 6.05
#=> 31.849999999999998
37.90 - 6.05
#=> 31.849999999999998
37.90 + 6.05
#=> 43.949999999999996

Why am I getting this?

sawa
  • 165,429
  • 45
  • 277
  • 381
Fleep
  • 5
  • 2
  • 1
    Search SO for *is floating-point maths broken* and similar phrases. In a nutshell -- you need to learn a little more about how computers do arithmetic with fractions, the Wikipedia article on *floating-point arithmetic* is a good place to start. – High Performance Mark Apr 18 '18 at 10:06
  • 1
    This is about python, but it applies to Ruby floating point as well: https://docs.python.org/2/tutorial/floatingpoint.html – Marek Lipka Apr 18 '18 at 10:07

1 Answers1

0

In a nutshell, computers have a problem working with real numbers and use floating point representation to deal with them. Much in the same way you can only represent 256 numbers with 8 bits for natural numbers, you can only represent a fixed amount of numbers with 64 bits for real numbers. For more details on this, read this http://floating-point-gui.de/ or google for "floating point arithmetic".

How should i deal with that?

  • Never store currency values in floating point variables. Use BigDecimal or do your calculations in cents using only integer numbers.
  • use round to round your floats to a user friendly length. Rounding errors will occur, especially when adding up a lot of floats.
  • In SQL systems use decimal data type, or use integers and divide them by a constant factor in the UI (say you need 3 decimal digits, you could store 1234 as integer and display 1.234.
mikezter
  • 2,393
  • 1
  • 17
  • 24
  • ok, edited the answer to reflect that :) – mikezter Apr 18 '18 at 10:39
  • So short and painful, if i become the numbers from other source (which i get a string from), my only/best way to deal with this is transforming this number to cents, do the math, then convert again from cents to decimal, right? – Fleep Apr 18 '18 at 10:51
  • yes :) But if you get a string anyway, you can also use `BigDecimal.new("1.234")`. Calculations with this class will be correct. Read more here: https://ruby-doc.org/stdlib-2.1.1/libdoc/bigdecimal/rdoc/BigDecimal.html – mikezter Apr 18 '18 at 10:55