1

Do you know how to fix the following issue with math precision?

p RUBY_VERSION # => "1.9.1"
p 0.1%1 # => 0.1
p 1.1%1 # => 0.1
p 90.0%1 # => 0.0
p 90.1%1 # => 0.0999999999999943
p 900.1%1 # => 0.100000000000023

p RUBY_VERSION # => "1.9.2"
p 0.1%1 # => 0.1
p 1.1%1 # => 0.10000000000000009
p 90.0%1 # => 0.0
p 90.1%1 # => 0.09999999999999432
p 900.1%1 # => 0.10000000000002274
Andrei
  • 10,918
  • 12
  • 76
  • 110

3 Answers3

5

Big Decimal


As the man said;

Squeezing infinitely many real numbers into a finite number of bits requires an approximate representation.


I have however had great success using the BigDecimal class. To quote its intro

Ruby provides built-in support for arbitrary precision integer arithmetic. For example:

42**13 -> 1265437718438866624512

BigDecimal provides similar support for very large or very accurate floating point numbers.


Taking one of your examples;

>> x = BigDecimal.new('900.1')
=> #<BigDecimal:101113be8,'0.9001E3',8(8)>
>> x % 1
=> #<BigDecimal:10110b498,'0.1E0',4(16)>
>> y = x % 1
=> #<BigDecimal:101104760,'0.1E0',4(16)>
>> y.to_s
=> "0.1E0"
>> y.to_f
=> 0.1


As you can see, ensuring decent precision is possible but it requires a little bit of effort.

mikej
  • 65,295
  • 17
  • 152
  • 131
Chris McCauley
  • 25,824
  • 8
  • 48
  • 65
  • 1
    Mind! You're not just adding precision: you're choosing a better representation for decimal numbers. 1/3 will still be rounded. – xtofl Sep 23 '10 at 11:24
  • @xtofl - that's a fair point but I hoped that the quote about 'approximate representation' touched on it. – Chris McCauley Sep 23 '10 at 13:58
2

This is true of all computer languages, not just Ruby. It's a feature of representing floating point numbers on binary computers:

What Every Computer Scientist Should Know About Floating Point Arithmetic

duffymo
  • 305,152
  • 44
  • 369
  • 561
1

Writing 0.1 into a floating point will always result in rounding errors. If you want 'precise' decimal representation, you should use the Decimal type.

xtofl
  • 40,723
  • 12
  • 105
  • 192
  • I haven't noticed such issue in `PHP`. Is there any good reason why Ruby doesn't have "'precise' decimal representation" by default? – Andrei Sep 23 '10 at 10:09
  • 1
    PHP has the same problem, because it's not about language: http://php.net/manual/en/language.types.float.php – duffymo Sep 23 '10 at 10:10