5

Does anybody has solution for this problem in ruby :

let say we have : a = 8.1999999

We wanted to round it by 2 decimal which is 8.20 and multiply it by 1,000,000 to become 8,200,000

We do it this way ;

(a.round(2) * 1000000).to_i

But what we got is 8199999, why?

The strage things is, we got the correct result if we multiply by 1000, 100000, or 10000000, but not 1000000. Any body know why?

We are using ruby 1.9.2 and try with 1.9.3 as well.

Thanks!

hudarsono
  • 389
  • 4
  • 19
  • 1
    possible duplicate of [ruby floating point errors](http://stackoverflow.com/questions/4055618/ruby-floating-point-errors) and hundreds of others. – mu is too short Jul 12 '12 at 04:47

3 Answers3

8

Whenever you get funky numbers in calculations use bigdecimal

require 'bigdecimal'
a = BigDecimal(8.1999999.to_s)
(a.round(2) * 1000000).to_i
victrnava
  • 1,134
  • 1
  • 9
  • 13
3

It becomes like that because a.round(2) returns a floating point number, thus calculations are not perfect.

For correct result, try following: (10*a).round.to_i * 100000

Johan Kotlinski
  • 25,185
  • 9
  • 78
  • 101
0

Your initial rounding is working,1 in a sense. The problem is that 8.2 doesn't have a precise internal representation. If you just type 8.2 into irb or display the results of the #round(2) method call, it looks like you have 8.2 but you don't. A number slightly smaller than 8.2 is actually stored.

You end up being defeated by the defaults of the output rounding logic. Once the internal slightly-less-than-8.2 bits are multipled, the error is shifted into the integer part of the number and this part won't be rounded unless you ask for it. You could do this: (a * 1000000).round

The problem is that we write the numbers in decimal but store them in binary. This works fine for integers; but it works poorly with fractions.

In fact, most of the decimal fractions we write cannot be represented exactly.

Every machine fraction is a rational number of the form x/2n. Now, the constants are decimal and every decimal constant is a rational number of the form x/(2n * 5m). The 5m numbers are odd, so there isn't a 2n factor for any of them. Only when m == 0 is there a finite representation in both the binary and decimal expansion of the fraction. So, 1.25 is exact because it's 5 / (22 * 50) but 0.1 is not because it's 1 / (20 * 51). In fact, in the series 1.01 .. 1.99 only 3 of the numbers are exactly representable: 1.25, 1.50, and 1.75.

Because 8.2 has no exact representation, it repeats in binary forever, never quite adding up to exactly 8.2. It goes on to infinity as 1100110011...


1. But note that you might have wanted a.round(1) instead of 2. The parameter to #round is the number of fraction digits you want, not the number of significant digits. In this case, the result was the same and it didn't matter.

DigitalRoss
  • 143,651
  • 25
  • 248
  • 329