3

From the discussion here and here we see that toFixed() can be used to help maintain precision when working with binary floats.

For various reasons, I do not want to use a third party library. However, for any fraction that can be expressed as a finite decimal within 64 bits, I would like to maintain maximum precision in the most generic way possible.

Assuming I use toFixed(), when do I need to use it? And is there a single "best" argument that I can pass to it to achieve the above goal?

For example, consider JavaScript's ability to represent the fraction 1/10 as a finite decimal after different operations:

10/100        = 0.1
0.09 + 0.01   = 0.09999999999999999
1.1 - 1       = 0.10000000000000009

The first requires no correction step while the last two do. Furthermore passing toFixed() a value of 15 works in this case (ex: Number((0.09 + 0.01).toFixed(15))):

10/100        = 0.1
0.09 + 0.01   = 0.1
1.1 - 1       = 0.1

but passing 16 does not:

10/100        = 0.1
0.09 + 0.01   = 0.1
1.1 - 1       = 0.1000000000000001

Try JSFIDDLE.

Will 15 as the argument for toFixed() always achieve the above objective? More importantly, when do I have to call toFixed()? In addition to add, subtract, multiply and divide, my math routines use Math.pow(), Math.exp() and Math.log().

Community
  • 1
  • 1
Karl
  • 1,814
  • 1
  • 25
  • 37
  • *"More importantly, when do I have to call `toFixed()`?"* Let's put that a different way: When might the result of an operation be imprecise? **At any time.** Even simple addition can result in an imprecise result, as in the famous `0.1` + `0.2` case. – T.J. Crowder Dec 23 '14 at 16:49

1 Answers1

0

Re when inaccuracies may occur:

Barring getting deep into the technical details of IEEE-754 double-precision floating point and having your code tailor itself to the specific characteristics of the two values it's operating on, you won't know when a result will be inaccurate; any operation, including simple addition, can result in an inaccurate result (as in the famous 0.1 + 0.2 = 0.30000000000000004 example).

Re what number to pass into toFixed:

You can use toFixed (and then conversion back to a number) for rounding (as you are in your question), but the number of places you should use is dictated by the precision you need in the result, and the range of values you're working with. Fundamentally, you need to decide how much precision you need, and then use rounding to achieve that precision with the best accuracy, because it's impossible to get perfect accuracy with IEEE-754 floating point across the full range of values (as you know). You get about 15 significant digits in total; you need to decide how to allocate them between the left and right-hand sides of the decimal.

But say your "normal" range of values is less than 10 billion with, say, four or five decimal places, you could use 5 as that gives you about 10 digits on the left and about five digits on the right.

Taking the famous example, for instance:

function roundToRange(num) {
  return +num.toFixed(5);
}

snippet.log(roundToRange(0.1 + 0.2)); // 0.3, rather than the usual 0.30000000000000004
<!-- Script provides the `snippet` object, see http://meta.stackexchange.com/a/242144/134069 -->
<script src="http://tjcrowder.github.io/simple-snippets-console/snippet.js"></script>

The rounded result may not be perfect, of course, because there's a whole class of base 10 fractional numbers that can't be represented accurately in base 2 (what IEEE-754 uses) and so must be approximated, but by working within your range, you can keep imprecisions from outside your range from adding (or multiplying!) up.

T.J. Crowder
  • 1,031,962
  • 187
  • 1,923
  • 1,875
  • Thank you. This is what made the light bulb come on: "You get about 15 significant digits in total; you need to decide how to allocate them between the left and right-hand sides of the decimal." Though I guess I should have realized it, the various documentation for `toFixed()` had not made it explicit that the value of the mantissa can also impact the result regardless of the argument passed. – Karl Dec 24 '14 at 14:15
  • @Karl: I'm glad that helped. *You* clearly got it, but I'm a bit nervous about having used the word "allocate" since of course we don't directly control it, it's a function of the magnitude of the numbers we're working with (if I'm going to have numbers up to 9,999,999,999 -- 10 digits to the left of the decimal -- then I know things will get pretty inaccurate as of roughly fifth digit to the right of the decimal). Hopefully others will get it like you did, though. :-) – T.J. Crowder Dec 24 '14 at 14:51