How to prevent BigDecimal from truncating results?

Question

I want to calculate 1/1048576 and get the correct result, i.e. 0.00000095367431640625.

Using BigDecimal's / truncates the result:

require 'bigdecimal'
a = BigDecimal.new(1)
#=> #<BigDecimal:7fd8f18aaf80,'0.1E1',9(27)>
b = BigDecimal.new(2**20)
#=> #<BigDecimal:7fd8f189ed20,'0.1048576E7',9(27)>

n = a / b
#=> #<BigDecimal:7fd8f0898750,'0.9536743164 06E-6',18(36)>

n.to_s('F')
#=> "0.000000953674316406" <- should be ...625

This really surprised me, because I was under the impression that BigDecimal would just work.

To get the correct result, I have to use div with an explicit precision:

n = a.div(b, 100)
#=> #<BigDecimal:7fd8f29517a8,'0.9536743164 0625E-6',27(126)>

n.to_s('F')
#=> "0.00000095367431640625" <- correct

But I don't really understand that precision argument. Why do I have to specify it and what value do I have to use to get un-truncated results?

Does this even qualify as "arbitrary-precision floating point decimal arithmetic"?

Furthermore, if I calculate the above value via:

a = BigDecimal.new(5**20)
#=> #<BigDecimal:7fd8f20ab7e8,'0.9536743164 0625E14',18(27)>
b = BigDecimal.new(10**20)
#=> #<BigDecimal:7fd8f2925ab8,'0.1E21',9(36)>

n = a / b
#=> #<BigDecimal:7fd8f4866148,'0.9536743164 0625E-6',27(54)>

n.to_s('F')
#=> "0.00000095367431640625"

I do get the correct result. Why?

Analogous to `(1.0/3).is_a?(Float) #=> true`, `BigDecimal.new(1)/BigDecimal.new(3)`could be written, `BigDecimal.new(1)/3` (`(BigDecimal.new(1)/3).is_a?(BigDecimal) #=> true`), or even `BigDecimal.new(1)/Rational(3)`. — Cary Swoveland, Nov 26 '16 at 22:02
The fact that `n.to_s('F')` returns greater precision than `n` suggests that `BigDecimal` objects contain more than meets the eye. Do you feel like spending your weekend digging into `BigDecimal`? One place to start might be [BigDecimal#to_s](http://ruby-doc.org/stdlib-2.3.0/libdoc/bigdecimal/rdoc/BigDecimal.html#method-i-to_s). btw, can you explain `(BigDecimal.new(1)/3).finite? #=> true`? — Cary Swoveland, Nov 26 '16 at 22:16
@CarySwoveland _"`n.to_s('F')` returns greater precision than `n`"_ – what do you mean? — Stefan, Nov 27 '16 at 09:16
@CarySwoveland _"can you explain `(BigDecimal.new(1)/3).finite? #=> true`?"_ – according to the [docs](http://ruby-doc.org/stdlib-2.3.2/libdoc/bigdecimal/rdoc/BigDecimal.html#method-i-finite-3F), only `NAN` and `INFINITY` return `false`. — Stefan, Nov 27 '16 at 09:19
Re precision, I just meant `n=a/b` returns 10 significant digits whereas `n.to_s` returns 12. Please forget my reference to `finite`. Explaining what nonsense was going on in my brain would take too many words and would be an utter bore. — Cary Swoveland, Nov 27 '16 at 09:48
@CarySwoveland `inspect` also shows 12 digits, the last two are separated by a space, right before `E-6`, i.e.: `'0.9536743164 06E-6'` = 0.953674316406 × 10⁻⁶ — Stefan, Nov 27 '16 at 09:56
Interesting, I was not aware of this. If I understand correctly BigDecimal picks the precision based on deduced precision of the given parameters. Precision is greater for 5**20 than for 1 it seems. — Axe, Nov 27 '16 at 14:20

score 3 · Accepted Answer · edited Nov 27 '16 at 11:05

BigDecimal can perform arbitrary-precision floating point decimal arithmetic, however it cannot automatically determine the "correct" precision for a given calculation.

For example, consider

BigDecimal.new(1)/BigDecimal.new(3)
# <BigDecimal:1cfd748, '0.3333333333 33333333E0', 18(36)>

Arguably, there is no correct precision in this case; the right value to use depends on the accuracy required in your calculations. It's worth noting that in a mathematical sense†, almost all whole number divisions result in a number with an infinite decimal expansion, thus requiring rounding. A fraction only has a finite representation if, after reducing it to lowest terms, the denominator's only prime factors are 2 and 5.

So you have to specify the precision. Unfortunately the precision argument is a little weird, because it seems to be both the number of significant digits and the number of digits after the decimal point. Here's 1/1048576 for varying precision

1   0.000001
2   0.00000095
3   0.000000953
9   0.000000953
10  0.0000009536743164
11  0.00000095367431641
12  0.000000953674316406
18  0.000000953674316406
19  0.00000095367431640625

For any value less than 10, BigDecimal truncates the result to 9 digits which is why you get a sudden spike in accuracy at precision 10: at that point is switches to truncating to 18 digits (and then rounds to 10 significant digits).

† Depending on how comfortable you are comparing the sizes of countably infinite sets.

That explains, _why_ I may need to specify a precision for certain calculations (although my example has a finite result). But it doesn't really explain the _"what value do I have to use to get un-truncated results"_ part. For example, why do 12 up to 18 return the same results and 19 suddenly returns two more digits? — Stefan, Nov 27 '16 at 09:24
Furthermore, if I use 5^20/10^20 instead of 1/2^20 to calculate the result, i.e `BigDecimal.new(5**20) / BigDecimal.new(10**20)`, I do get the correct result. Can you explain that? — Stefan, Nov 27 '16 at 09:27
@Stefan I'll be honest, I don't entirely understand how the precision argument works. But for your specific example, dividing by any power of ten will always give you an accurate result because BigDecimals are stored as integers times powers of ten. — Max, Nov 28 '16 at 04:37

How to prevent BigDecimal from truncating results?

1 Answers1

Linked