1

Doing some very basic decimal maths in Ruby (x = 1/b * c). 

a = BigDecimal.new("1"); 
b = BigDecimal.new("0.8903"); 
c = BigDecimal.new("0.8903"); 
x = a / b * c; 
puts x

Output: 0.9999999998379

To rule out problems I've simplified the test case to just one decimal (x = 1/b * b)

a = BigDecimal.new("1"); 
b = BigDecimal.new("0.8903"); 
x = a / b * b; 
puts x

Output: 0.9999999998379

However rearranging the original formula to x = c / b gives the correct answer.

a = BigDecimal.new("1"); 
b = BigDecimal.new("0.8903"); 
c = BigDecimal.new("0.8903"); 
x = c / b; 
puts x

Output: 1.0

Any ideas what's causing this or whether this is a BigDecimal bug?

Thanks for the comment telling me what's wrong with floating point maths, that's why I'm using BigDecimal not floats, unless BigDecimal also has the same issue?

Martin Steel
  • 572
  • 9
  • 19

1 Answers1

1

This has to do with the limit on the maximum digits for the new BigDecimal object.

If you use a limit of 4 the math will work out.

a = BigDecimal.new("1"); 
b = BigDecimal.new("0.8903"); 
c = BigDecimal.new("0.8903"); 
x = a.div(b, 4) * c; 
puts x

Output: 0.8903

You can find more about limit here

David G
  • 183
  • 7
  • It might be worth linking to some docs that aren't 11 years old: https://ruby-doc.org/stdlib-2.3.1/libdoc/bigdecimal/rdoc/BigDecimal.html – Jordan Running Nov 10 '16 at 22:40
  • I agree that the basics of what is going on here are solved in the other answer that was linked and the reference link was older the premise that you need to limit the number of decimal places in Ruby for the math to work out is still the same. If you knew a consistent number of places after the decimal, you most likely can get the math to calculate correctly with BigDecimal. – David G Nov 15 '16 at 06:32