Comparing numbers in Lua

Question

I'm checking for line-line intersection and need to figure out if the intersection point (x,y) is within the bounding box of a line segment l2 (consisting of points p1 and p2)

The following printout illustrates my problem:

the intersection point is (100,300)

print("x",x,">=",math.min(l2.p1.x,l2.p2.x),x >= math.min(l2.p1.x,l2.p2.x))
print("x",x,"<=",math.max(l2.p1.x,l2.p2.x),x <= math.max(l2.p1.x,l2.p2.x))
print("y",y,">=",math.min(l2.p1.y,l2.p2.y),y >= math.min(l2.p1.y,l2.p2.y))
print("y",y,"<=",math.max(l2.p1.y,l2.p2.y),y <= math.max(l2.p1.y,l2.p2.y))

which yeld:

x   100 >=  100 true
x   100 <=  100 false
y   300 >=  140 true
y   300 <=  300 false

What is going on and how can it be fixet?

(Lua version 5.2.3)

I couldn't reproduce your problem. Please provide a minimal, complete example that illustrates it. — Bartek Banachewicz, Sep 24 '15 at 09:24
I wasn't able to create a minimal example that produces the same error but the code is available here https://github.com/mhml92/lineSegmentVisibility Though you do need the love2d framework to run it and the test file (test.txt) The code can be run from its root folder `love . test.txt` — Mikkel Leivsen, Sep 24 '15 at 09:55
I suggest that you first test whether the two segments intersect and if so compute their intersection. See http://stackoverflow.com/a/3840235/107090. — lhf, Sep 24 '15 at 10:46

score 0 · Accepted Answer · answered Sep 25 '15 at 09:11

Say hello to float point arithmetic: http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

For example

> string.format("%.32f", 1/3)
0.33333333333333331482961625624739
> 1/3 == 0.3333333
false

So it depends on how your calculations of X and lp* looks like. You should use a tolerance while comparing float point numbers.

> math.abs(1/3 - 0.33) == 0
false
> math.abs(1/3 - 0.33333) < 1/10^6
false
> math.abs(1/3 - 0.33333) < 1/10^5
true

Comparing numbers in Lua

1 Answers1