-1

I faced the Floating-Point-Precision problem in JavaScript and to solve it i found libraries like Decimal.js or BigNumber.js. It was working fine for me until i tried the following.

I have a total of 60 and a share of 10. So the percentage of the share is 10/60*100 (16.666666666666666667%).

Now i want to know what 16.666666666666666667% of 360 are which is 60 (calculator) but for some reason i get 60.000000000000000001 with Decimal.js.

var percentage = (Decimal(10).div(60)).mul(100); 
console.log(percentage.toString()); // 16.666666666666666667

var share = (Decimal(360).mul(percentage)).div(100);
console.log(share.toString()); // 60.000000000000000001

I expected to get 60 but got this. Am i missing something or it is a problem of the library because im a bit confused that this is not working with a library that was created for such problems.

Djinii
  • 31
  • 1
  • 8

1 Answers1

2

Your calculator and your library are both rounding.

The exact result of multiplying 16.666666666666666667 by 60 is 1000.00000000000000002. If you multiply 166.66666666666666667 by 6, by hand, you begin with 6*7 = 2 with a carry-out of 4. After that, as you work to the left, you are multiplying 6 * 6 with a carry-in of 4. The result is 0 with a carry-out of 4. Finally 6*1+4 is 0 with a carry-out of 1.

What you are missing is that rounding error is not unique to binary floating point. Any radix X fraction representation, including decimal, can only represent exactly rationals whose denominator can be made a power of X. For decimal, X is ten and 1/6 cannot be represented exactly as A/10B for any pair of integers A and B.

Unless you go to a general rational number library you have to deal with rounding error. Even a library that can represent any rational number, subject only to memory limits, will have to round irrationals, such as most square roots.

Patricia Shanahan
  • 25,849
  • 4
  • 38
  • 75