0

If I do this equation in Firebase functions (in JavaScript):

var q = 0.1
var alpha = 0.1
var gamma = 0.99
var max5 = 0

q = (1 - alpha) * q + (alpha * (-1 + (gamma * max5)));

console.log(q)

the result is -0.009999999999999995

But if I do this equation on a calculator it is -0.01

I should note, I pasted a simplified version of the code above, but all the variables get the parseFloat operation, so I'm sure it has something to do with that. Or is it something else?

Juancki
  • 1,793
  • 1
  • 14
  • 21
Steven
  • 149
  • 1
  • 10
  • It is likely due to slight differences in internal precision - standard floating point has a _relative precision_. Search for "is floating point math broken" or "IEEE-754" float to see lots of details on the subject and how floating point numbers are stored. JavaScript uses IEEE-754 64-bit Binary (which can internally be processed as 80-bit in x86 FPUs..). That specific calculation? No clue.. it could also be that the calculator uses a different rounding method on display. – user2864740 May 16 '20 at 05:43

1 Answers1

1

This is caused by a weird way javascript handles floating point numbers. (You can read more here)

For example, if you do this in firebase or just the console in the browser

console.log(0.1 + 0.2);
// 0.30000000000000004

This is just a JavaScript wide thing. The way to get around it is to call .toFixed(2). This will round the number to 2 decimal places. 

console.log(+(0.1 + 0.2).toFixed(2));
// 0.3

(In case you are wondering why the + is in there before the numbers, it's because .toFixed() returns a string. The + turns it back into a number)

Mr. Simmons
  • 448
  • 3
  • 14