2

Why is JavaScript's toPrecision method sometimes rounding up and sometimes down?

For example:

console.log("1.05.toPrecision(2) = " + (1.05).toPrecision(2)); // returns "1.1"
console.log("1.005.toPrecision(3) = " + (1.005).toPrecision(3)); // returns "1.00", shouldn't it be "1.01"?

Note: I noticed this behavior on all browsers except of IE (tested on IE 11).

Hopeful Llama
  • 728
  • 5
  • 26
Piotr Dabkowski
  • 5,661
  • 5
  • 38
  • 47
  • 4
    This seems to be a floating point error. I noticed it happens in Swift too, not just JavaScript. In Swift, 1.005 is represented as 1.0049999999999999, so I would assume it has something to do with the amount of digits reserved for the exponent. – rid Sep 05 '16 at 10:12
  • @rid good point, actually it happens in Python too, funny though that IE 11 is immune to this :) – Piotr Dabkowski Sep 05 '16 at 10:15
  • Check this too : http://stackoverflow.com/a/18358056/5830872 – Niklesh Raut Sep 05 '16 at 10:18

0 Answers0