2

As you know, float point has a precision problem, that is a value 1 will be 0.9999999. And lua use == in C to check whether two double numbers equal. So the problem is:

a = 5.6
b = 14 * 0.4
print(a==b) ; => false

But the worse thing is:

a = 1
...
if a < 1 then print("<1") end ; => sometimes a < 1

So how can i avoid this ? I check out lua source code, it seems i can modify luai_numeq/luai_numle) macros in luaconf.h, but is this necessary ?

update

The second example is not really correct. Actually my problem is, I pass a value 1 to c/c++ which use lua_tonumber to get the value, and i store this value in a double variable, and after sometime, I push the value (lua_pushnumber) to lua, and a < 1 happened:

in lua:
my_cfunction(1)
...
in c:
int my_cfunction(lua_State *L) {
    double val = lua_tonumber(L, 1);
    ...
...
lua_pushnumber(L, val);

in lua:
local a = my_cfunction2()
if a < 1 then ... end
kevin lynx
  • 785
  • 9
  • 17
  • 1
    This is a "problem" with all floating point math when you convert to a different radix (i.e. base2 to base10). Get used to it. Program around it. Definitely do NOT hack the language source. For one, you'll have non-portable code. Don't know what's up with example 2. What does ... represent? I think your case extremely unlikely... I wouldn't touch Lua with a bargepole if 1 < 1. – spender May 10 '12 at 02:11
  • 1
    Possible duplicate of http://stackoverflow.com/questions/6366954/why-is-lua-arithmetic-is-not-equal-to-itself – lhf May 10 '12 at 12:14
  • I suspect you did something wrong; arithmetic on integer values in Lua that doesn't overflow should always be precise (although Lua uses a C "double" type, IEEE floating-point is precise when operating on integer values that don't overflow). If you really have a test-case that you think shows otherwise, please include the _complete_ test-case; code snippets aren't sufficient. – snogglethorpe May 11 '12 at 02:24

2 Answers2

5

As you know, float point has a precision problem, that is a value 1 will be 0.9999999

I don't know that at all. Because it's not true. 1.0 is 1.0. This:

a = 1
if a < 1 then print("<1") end

Will never print "<1". Not unless you actually change a. Even this:

a = 2
a = a - 1
if a < 1 then print("<1") end

Will likewise never hit the print statement.

As long as you are performing integer arithmetic on Lua's numbers, you will end up with integers. No addition, subtraction, or multiplication of integer values will leave you with a non-integer number value.

Nicol Bolas
  • 449,505
  • 63
  • 781
  • 982
4

float point has a precision problem

It doesn't.

A 64 bit double can hold many more integer values precisely than a 32 bit integer.

Mud
  • 28,277
  • 11
  • 59
  • 92