0

I'm fairly new to lua, but I read that it does not have integers, so all numbers are floats. I noticed in my log that some numbers get a slight inaccuracy added to it. For instance 0.14 is written as 0.14000000059605. In that function it doesn't really matter if the number is a little diferent, as it is a comparison with a random number. But I do a lot of equals comparisons with numbes like NumReg() == 2 where it would give a wrong result if the 2 gets unrounded. Do do I have to account for this by rounding them down, or are non-tiny numbers not affected?

Egor Skriptunoff
  • 23,359
  • 2
  • 34
  • 64
JohnBig
  • 97
  • 7
  • 1
    See https://0.30000000000000004.com/ for an explanation of why that happens. – Daniel Feb 08 '22 at 06:14
  • Does this answer your question? [How safe is comparing numbers in lua with equality operator?](https://stackoverflow.com/questions/12316769/how-safe-is-comparing-numbers-in-lua-with-equality-operator) – Daniel Feb 08 '22 at 06:15
  • 2
    If you're only doing integer arithmetic up to 2^52, there will most likely be no rounding errors (exceptions to this might be something like `sqrt` or `pow`, but the builtin arithmetic operators should otherwise be fine). You can usually safely use exact comparisons there. If you're doing computations on actual floating point numbers though, consider comparing with a small bias: `math.abs(x - expected) < 1e-6` for example. – Luatic Feb 08 '22 at 08:41
  • Lua 5.3+ does have integers, typically 64-bit. – lhf Feb 08 '22 at 10:04
  • Does this answer your question? [Is floating point math broken?](https://stackoverflow.com/questions/588004/is-floating-point-math-broken) – Colonel Thirty Two Feb 08 '22 at 13:40
  • Thanks for the answers. I figured this was something that has been asked before, but I couldn't find an answer because most of the terms are new to me. – JohnBig Feb 09 '22 at 12:42

1 Answers1

0

You can format a float for your needs.
Example to play with in an interactive Lua console...

> _VERSION
Lua 5.4
tonumber(string.format('%.3f', 0.14000000059605))
0.14
> print(tonumber(string.format('%.3f', 0.14000000059605)))
0.14
> type(tonumber(string.format('%.3f', 0.14000000059605)))
number
> math.type(tonumber(string.format('%.3f', 0.14000000059605)))
float
> tonumber(string.format('%.3f', 0.14000000059605)) == 0.140
true
koyaanisqatsi
  • 2,585
  • 2
  • 8
  • 15