Javascript error in toPrecision() function

Question

I want to use toPrecision() to reduce the size of a number before I display it. However, I sometimes cannot multiply output of the function by another number without gaining a small rounding error. See the code sample below:

var x = 0.0197992182093305
alert (x.toPrecision(4)) //Correct: 0.01980
alert (Number(x.toPrecision(4))) //Correct: 0.0198
alert( Number(x.toPrecision(4)) * 100) //Incorrect: 1.9800000000000002

JSFiddle: http://jsfiddle.net/ueLsL460/4/ What's going on here?

It's not that function causing the problem. Have a look at [this question](http://stackoverflow.com/q/588004/615754) (amongst others). — nnnnnn, Aug 12 '15 at 03:21
[What Every Computer Scientist Should Know About Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html) None of the above numbers are exactly representable in binary floating-point, so don't expect it to be printed correctly without correct formatting — phuclv, Aug 12 '15 at 04:26
@EricBaldwin - I just made a slight modification to my answer to clarify that * 10 * 10 causes no problem - but only for this specific case. You could have a number where the loss of precision build up even with a * 10 — potatopeelings, Aug 12 '15 at 05:55

Nathan · Answer 1 · 2015-08-12T03:26:57.160

4

Based on what I understand, Number(x.toPrecision(4)) * 100 creates a new Number object which will not inherit the precision of the parent.

If you still want it to be precise after Math, you need to put it in precision again.

alert((x * 100).toPrecision(4));

Technically, it's not an error. It's just the way javascript is supposed to work.

edited Aug 12 '15 at 03:26

answered Aug 12 '15 at 03:22

Nathan

1,520
1
12
21

You don't need the `Number()` function here. `(x * 100).toPrecision(4)` will work. – nnnnnn Aug 12 '15 at 03:25
Thank you for the suggestions. Updated my answer. – Nathan Aug 12 '15 at 03:27

score 1 · Answer 2 · answered Aug 12 '15 at 03:26

The use of primitive constructors is not that ideal, unless you are trying to do something trivial. Can you please try to do the code on the following fiddle and see if this will do for you?

http://jsfiddle.net/ueLsL460/5/

 var x = 0.0197992182093305;
 alert((x * 100).toPrecision(4));

potatopeelings · Accepted Answer · 2015-08-12T05:54:12.283

Javascript doesn't have a decimal equivalent - so in your case floating points are used. What this means is that Number(x.toPrecision(4)) doesn't give you exactly 0.0198, but something the FP binary number closest to 0.0198. So any arithmetic you do will be subject to the loss of precision introduced in floating point arithmetic. You can see the same effect if you do

 alert(0.0198 * 100);

By the way

  alert(0.0198 * 10 * 10);

gives you no problem (the loss of precision does not build up enough to make it into the digits Javascript deems to display) - but that's for this particular number alone.

Javascript error in toPrecision() function

3 Answers3