1

Why is this not a duplicate of these great SO articles?

While the two posts linked in the comments below are excellent I am specifically looking for information that helps me to address this issue in native JS. I know that JS shouldn't be the first choice for complex math, but given the limitation that this calculator is meant to run in the browser it is the tool that I have decided to work with.

Background

I'm trying to make a calculator with TypeScript without any libraries (like Big.js) and without using string concatenation in the inner logic of the calculator.

Examples

When a user wants to type the number 8.5:

  • The 8 key is pressed
  • The decimal key is pressed
  • The 5 key is pressed

Mathematically I create this number in the display with the following snippet:

8 + 5 * 0.1

This works but if I continue down the decimal places I encounter something unexpected:

8.5 + 5 * 0.01 // 8.55
8.55 + 5 * 0.001 // 8.555000000000001

Question

What is the best way to handle this without converting the number to a string? Is there an intelligent way to impose a limit on the precision of the calculator so that it only supports accuracy to so many decimal places?

Thanks for your help!

Mike Fleming
  • 2,593
  • 4
  • 14
  • 24
  • 2
    Possible duplicate of [Is floating point math broken?](https://stackoverflow.com/questions/588004/is-floating-point-math-broken) – ellipsis Sep 23 '19 at 18:54
  • 1
    Possible duplicate of [How to deal with floating point number precision in JavaScript?](https://stackoverflow.com/questions/1458633/how-to-deal-with-floating-point-number-precision-in-javascript) – FZs Sep 23 '19 at 19:03
  • JavaScript has some known issues with math. – Maiya Sep 23 '19 at 19:08
  • That first potential duplicate has a fascinating answer. I think they both give me a better high level understanding of the problem but neither of them give much guidance for me in JavaScript specifically. I'll edit my question to reflect that. – Mike Fleming Sep 23 '19 at 19:09
  • 1
    You should know that no calculation or algorithm can produce 8.55 or 8.555 in a JavaScript `Number` because those values are not representable in the format used for `Number`. The closest possible values are 8.550000000000000710542735760100185871124267578125 and 8.55499999999999971578290569595992565155029296875. – Eric Postpischil Sep 23 '19 at 21:27
  • For simple decimal numerals with a limited number of digits, you can get the closest possible result by accumulating the digits as an integer (such as 8555 for “8.555”) and then dividing by the appropriate power of 10 (1000). For numbers with more than 15 digits or scientific notation, you would need more sophisticated algorithms. I would expect JavaScript has some standard function that would convert a numeral string to a `Number`—you could accumulate key presses to build the string “8.555” and then call that function to convert “8.555” to a `Number`. – Eric Postpischil Sep 23 '19 at 21:28

2 Answers2

1

Use .toFixed(): https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/toFixed

or .toPrecision(): https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/toPrecision depending on your needs.

Note that you don't need to convert numbers all the time. The only place to convert - is for the final output to the view. In this case we can even leave it in string format.

That is an answer to the question "how to get manage with" (like in description). About "why do we get such result" - first comment provides great answer.

Alex Vovchuk
  • 2,828
  • 4
  • 19
  • 40
  • These conversions for display would successful conceal many errors, but they do not solve the problem of accurately converting the numbers from the input decimal to the internal `Number` format. The errors in that conversion may be important for various algorithms; merely concealing small errors during output may be inadequate. – Eric Postpischil Sep 23 '19 at 21:25
  • It is the way to deal with precision in result layout. If we need some advanced calculation logic which takes into the account some extremely high precision, we should use Tolerance Value for comparison and other similar staff. But it is not that case. – Alex Vovchuk Sep 23 '19 at 21:35
  • Comparing with a tolerance makes this worse by introducing false positives for the comparison. – Eric Postpischil Sep 23 '19 at 22:03
1

The easiest way to get the Number value that is closest to what the user enters is to build a numeral in a string from the user’s keypress and then convert them with String.toNumber().

Numbers such as 8.55 or 8.555 are not exactly representable in the Number format. The closest values are 8.550000000000000710542735760100185871124267578125 and 8.55499999999999971578290569595992565155029296875. Converting the strings “8.55” and “8.555” with .toNumber() should produce these values.

Because these are the closest representable values, no calculation or algorithm can produce any closer values in the Number format.

For simple additions, subtractions, and limited multiplications, you can mimic decimal arithmetic by rounding to a desired number of decimal digits after each Number operation. However, this is generally not a feasible approach because other operations and various sequences of operations will exceed the ability to mimic decimal arithmetic reasonably.

Eric Postpischil
  • 195,579
  • 13
  • 168
  • 312