1

Why is 10000000000000.126.toString() 1000000000000.127 and 100000000000.126.toString() not?

I think it must have something to do with the maximum value of a Number in Js (as per this SO question), but is that related to floating point operations and how?

I'm asking because I wrote this function to format a number using thousands separators and would like to prevent this.

function th(n,sep) {
    sep = sep || '.';
    var dec = n.toString().split(/[,.]/),
        nArr = dec[0].split(''),
        isDot = /\./.test(sep);
    return function tt(n) {
              return n.length > 3 ?
               tt(n.slice(0,n.length-3)).concat(n.slice(n.length-3).join('')) :
               [n.join('')]
            ;
        }(nArr)
        .join(sep)
        + (dec[1] ? (isDot?',':'.') + dec[1] : '');
}
sep1000(10000000000000.126); //=> 10.000.000.000.000,127
sep1000(1000000000000.126); //=> 1.000.000.000.000,126
Community
  • 1
  • 1
KooiInc
  • 119,216
  • 31
  • 141
  • 177
  • 3
    It's because of our good old friend float-point-accuracy. – deceze Jul 03 '11 at 10:15
  • 1
    duplicate of [Why does a C floating-point type modify the actual input of 125.1 to 125.099998 on output?](http://stackoverflow.com/questions/6532502/why-does-a-c-floating-point-type-modify-the-actual-input-of-125-1-to-125-099998-o) and **many others** – Alnitak Jul 03 '11 at 10:19
  • @Alnitak, I know, but additionally I'm searching for a way to prevent it. I am still searching related answers, really. – KooiInc Jul 03 '11 at 10:22
  • @KooiInc: I'm not at all sure you can, though you might be able to play games by changing the magnitude of the number the `Number` is trying to represent, which may give you access to more data. I've updated my answer with an example of that (which may well be flawed). – T.J. Crowder Jul 03 '11 at 10:32

2 Answers2

7

Because not all numbers can be exactly represented with floating point (JavaScript uses double-precision 64-bit format IEEE 754 numbers), rounding errors come in. For instance:

alert(0.1 + 0.2); // "0.30000000000000004"

All numbering systems with limited storage (e.g., all numbering systems) have this issue, but you and I are used to dealing with our decimal system (which can't accurately represent "one third") and so are surprised by some of the different values that the floating-point formats used by computers can't accurately represent. This sort of thing is why you're seeing more and more "decimal" style types out there (Java has BigDecimal, C# has decimal, etc.), which use our style of number representation (at a cost) and so are useful for applications where rounding needs to align with our expectations more closely (such as financial apps).

Update: I haven't tried, but you may be able to work around this by manipulating the values a bit before you grab their strings. For instance, this works with your specific example (live copy):

Code:

function display(msg) {
  var p = document.createElement('p');
  p.innerHTML = msg;
  document.body.appendChild(p);
}

function preciseToString(num) {
  var floored = Math.floor(num),
      fraction = num - floored,
      rv,
      fractionString,
      n;

  rv = String(floored);
  n = rv.indexOf(".");
  if (n >= 0) {
    rv = rv.substring(0, n);
  }
  fractionString = String(fraction);
  if (fractionString.substring(0, 2) !== "0.") {
     return String(num); // punt
  }
  rv += "." + fractionString.substring(2);
  return rv;
}

display(preciseToString(10000000000000.126));

Result:

10000000000000.126953125

...which then can, of course, be truncated as you see fit. Of course, it's important to note that 10000000000000.126953125 != 10000000000000.126. But I think that ship had already sailed (e.g., the Number already contained an imprecise value), given that you were seeing .127. I can't see any way for you to know that the original went to only three places, not with Number.

I'm not saying the above is in any way reliable, you'd have to really put it through paces to prove it's (which is to say, I'm) not doing something stoopid there. And again, since you don't know where the precision ended in the first place, I'm not sure how helpful it is.

T.J. Crowder
  • 1,031,962
  • 187
  • 1,923
  • 1,875
  • Thanks T.J., I'll go for your answer. Been tinkering around with your (and other code), it's surprisingly hard to show that `.126` part in the resulting string. Ah well, it's mostly not a real life problem I'd say, but it's an interesting exercise. – KooiInc Jul 04 '11 at 05:22
  • 1
    This isn't a binary/decimal issue. It's an issue of trying to put 17 significant digits in a type that only provides about 16. – dan04 Jul 11 '11 at 15:45
2

It's about the maximum number of significant decimal digits a float can store.

If you look at http://en.wikipedia.org/wiki/IEEE_754-2008 you can see that a double precision float (binary64) can store about 16 (15.95) decimal digits.

If your number contains more digits you effectively lose precision, which is the case in your sample.

VVS
  • 19,405
  • 5
  • 46
  • 65