1

Due to conversion between binary and decimal numbers we often see precision errors in JS.

eg.: 137.80 * 100 = 13780.000000000002

Although, is is technically possible that JS will return similar error but with negative precision?

eg.: 137.80 * 100 = 13779.999999999998

shep4rd
  • 86
  • 9
  • Does this answer your question? [How to deal with floating point number precision in JavaScript?](https://stackoverflow.com/questions/1458633/how-to-deal-with-floating-point-number-precision-in-javascript) – evolutionxbox Jan 08 '21 at 10:46
  • Thanks for the link. I know how to deal with it. I'm trying to understand boundaries of the problem. – shep4rd Jan 08 '21 at 11:32
  • I wonder if using the binary IEEE-754 versions of these number might help clear things up? – evolutionxbox Jan 08 '21 at 12:06

1 Answers1

0

Absolutely. For instance, 0.3 / 10000 is 0.000029999999999999997 because 0.3 can't quite be represented, and the closest representation is just slightly lower that the actual value 0.3. We can see that manifest in the output if we push the limits with the division.

T.J. Crowder
  • 1,031,962
  • 187
  • 1,923
  • 1,875
  • Is this also possible with multiplication / addition ? I guess yes. – shep4rd Jan 08 '21 at 11:34
  • @shep4rd - I don't know, I haven't gotten that far into IEEE-754. I note that `0.3 * .0001` (the mathematical equivalent of the `0.3 / 10000` above) results in an output of `0.00003`. But I wouldn't take that as a guarantee of anything. – T.J. Crowder Jan 08 '21 at 11:52