Javascript - slight division error

Question

Possible Duplicate:
Javascript Math Error: Inexact Floats

I did this

this.cues = []
for(var i, i = 1; i <= 8; i++){
    cue = this.length * (i/8.0)
   this.cues.push(cue)

where this.length is a double. I checked the division and found that I was off by .000002 up to about the fourth array cell.

Is there any javascript-specific cause for this anybody can think of? It may just be the environment, so if not no big deal
If so, can I fix it?

Many thanks in advance

There should be no reason for this to be off, since 8.0 is perfectly representable in IEEE-754, as should your reasonably-sized integers divided by it. — Ignacio Vazquez-Abrams, Jul 01 '12 at 19:29
375.386658 - interesting, a single voice of dissent but an extremely high-rep one :) — jamesson, Jul 01 '12 at 19:35
*That* would be why then; your original number *isn't* exactly precise. `>>> (375.386658).hex()` `'0x1.7762fc04c8bcap+8'` `>>> '%.25f' % 375.386658` `'375.3866580000000112704583444'` — Ignacio Vazquez-Abrams, Jul 01 '12 at 19:38

score 0 · Answer 1 · answered Jul 01 '12 at 19:29

This is expected. Floating point operations on computers have small accuracy errors like this because of the way they are stored internally in a computer. For more information, I recommend you read What Every Computer Scientist Should Know About Floating-Point Arithmetic.

Squeezing infinitely many real numbers into a finite number of bits requires an approximate representation. Although there are infinitely many integers, in most programs the result of integer computations can be stored in 32 bits. In contrast, given any fixed number of bits, most calculations with real numbers will produce quantities that cannot be exactly represented using that many bits. Therefore the result of a floating-point calculation must often be rounded in order to fit back into its finite representation. This rounding error is the characteristic feature of floating-point computation.

score 0 · Answer 2 · answered Jul 01 '12 at 19:30

0

You should read: "What Every Programmer Should Know About Floating-Point Arithmetic". This is simply how floating point numbers are represented on a computer.

answered Jul 01 '12 at 19:30

Simeon Visser

118,920
18
185
180

Javascript - slight division error

2 Answers2

Linked