0

Possible Duplicate:
Is JavaScript’s Math broken?

So I'm writing a program that calculates numbers to very precise values, and I learned the hard way that numerical values are never exactly as they seem. For example:

<script type="text/javascript">
var num=80.902
var difference=(num*10-Math.floor(num*10))/10;
document.write(difference);
</script>

This shows that there is an actual difference of 0.001999999999998181. Is there any way to go about making these numbers exact? I'm guessing it's some sort of memory thing, so is there a way I'm supposed to make it into a double or something? It's annoying having to compare data to a number smaller than 10^-13.

Community
  • 1
  • 1
user1804208
  • 165
  • 2
  • 4
  • 9

1 Answers1

0

This is an inherent limitation to javascript. Its not designed for fine precision float number math.

This question is where you want to look: How to deal with floating point number precision in JavaScript? and here Is floating point math broken?

... and I would guess that JavaScript is the worst language for float precision.

Community
  • 1
  • 1
Pimp Trizkit
  • 19,142
  • 5
  • 25
  • 39
  • Actually, that's not a restriction inherent to JavaScript, but a restriction inherent to floating point numbers. And JavaScript isn't significantly worse with regard to that than any other popular language. – Joachim Sauer Nov 07 '12 at 07:51
  • Well, its inherent to floating point math in programming, and since JavaScript is a programming language, its inherent to JavaScript as well... was my thought process. And in my experience, when using JavaScript, I couldnt get as high precision as other languages, I assume because in other languages I could use `double` – Pimp Trizkit Nov 07 '12 at 08:00
  • And dont forget Fortran, it was designed to be more precise. It may not be a "popular" language. But it is still the dominate language for a number of applications. The newest version was approved just over 2 years ago. – Pimp Trizkit Nov 07 '12 at 08:12
  • @PimpTrizkit: It's not a question of precision. Some numbers just can't be represented exactly in binary floating point format. Regardless weather you use float or double. It's the same problem as the value of 1/3 having no exact decimal representation. – slebetman Nov 07 '12 at 08:20
  • Yes, Im aware of this. But, you can hold more .333333s in a `double` than you can a `float` before the last few get distorted. Thus higher precision, IMHO. – Pimp Trizkit Nov 07 '12 at 08:23
  • Actually I found that using the toFixed() method solved my problem. Thanks for your help guys. – user1804208 Nov 07 '12 at 19:23