Why Lua works that way and how to fix floating numbers problem?

Question

That is my code:

a=20.4*100
print(a,math.floor(a),math.floor(2040))

Result in console:

2040    2039    2040

If a==2040 then why when I do math.floor with that number, its decreasing? How can I write mathematical functions with better precisions then that strange behavior?

Possible duplicate of [Is floating point math broken?](https://stackoverflow.com/questions/588004/is-floating-point-math-broken) — Sneftel, Sep 25 '19 at 21:28
here is how math.floor is defined: http://www.lua.org/source/5.3/lmathlib.c.html#math_floor — Nifim, Sep 25 '19 at 21:30
`a` doesn't equal `2040`. “When you have eliminated the impossible, whatever remains, however improbable, must be the truth.” — Tom Blodget, Sep 26 '19 at 21:32

score 1 · Answer 1 · answered Sep 26 '19 at 07:16

Put simply: computers don't have infinite memory, so they can only use so many bits to store a number. If the number has too many decimals, they will get cut off somewhere, and it just so happens that 0.4 (two fifth) in binary representation has more decimals than it cans save; so after cutting them off, you're left with 0.399999something, which print is smart enough to print as 0.4, but if you floor() it, it gets rounded down without mercy, leading to the results you see.

Also please note that you could have just googled this; it's one of the most common questions in programming ;)

As an example, try this:

string.format("%f",     20.4*100)
--> 2040.000000
string.format("%0.20f", 20.4*100)
--> 2039.99999999999977262632

Why Lua works that way and how to fix floating numbers problem?

1 Answers1