You want to find the intersection point [P] of the two lines a and b:
[P] = [A] + [a]*r = [B] + [b] * s
That yields the linear equations:
[a]*r - [b]*s = [S] - [R]
or
| ax -bx | | r | |Sx - Rx|
| | * | | = | |
| ay -by | | s | |Sy - Ry|
Judging from your determinants, that's not the equation you are trying to solve. I think your function should look like this:
bool intersection(double start1[2], double end1[2],
double start2[2], double end2[2])
{
float ax = end1[0] - start1[0]; // direction of line a
float ay = end1[1] - start1[1]; // ax and ay as above
float bx = start2[0] - end2[0]; // direction of line b, reversed
float by = start2[1] - end2[1]; // really -by and -by as above
float dx = start2[0] - start1[0]; // right-hand side
float dy = start2[1] - start1[1];
float det = ax * by - ay * bx;
if (det == 0) return false;
float r = (dx * by - dy * bx) / det;
float s = (ax * dy - ay * dx) / det;
return !(r < 0 || r > 1 || s < 0 || s > 1);
}
Edit II: The algorithm as presented above is very coarse. It correctly calculates the intersection of two line segments that cross each other at a non-zero angle. The following issues are not resolved:
Collinearity. When the main determinant is zero, the direction vectors are parallel. To check whether the two line segments are collinear, test whether the cross product of one direction vector and the difference of the two starting points is zero:
ax * dy - ay * dx == 0
Collinearity doesn't necessarily mean that the line segments overlap. For that, the coefficient r for the staring and end points of the second curve must overlap with the interval [0, 1].
Zero-length segments. Here, the coefficient of the zero-length line segment is indeterminate and the main determinant is zero. The algoritm could check the length and determine whether the starting point of the degenerate segmant lies on the other.
Two zero-length segments. Test the points for equality or decide that points can coincide, but not intersect.
Precision. The code here just tests for zero, which usually isn't a good idea when working with floating-point coordinates. The test should probably made against a judiciously chosen epsilon:
if (fabs(det) < epsilon) ...
A robust line intersection algorithm should cater for these cases. This, as they say, is left as an exercise for the reader.
Edit: I've converted the logic to Javascript in order to visually check the function. (I've found two errors in the process, which I've fixed.) For you interactive intersection pleasure, copy the web page below to an html file and run it in a browser that can interpret the HTML5 canvas. A pair of lines are generated randomly and they should be red when the intersect and grey otherwise. Reload to reroll.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>Intersect!</title>
<script type="text/javascript">
function point(cx, a)
{
cx.fillRect(a[0] - 3, a[1] - 3, 7, 7);
}
function line(cx, a, b)
{
cx.beginPath();
cx.moveTo(a[0], a[1]);
cx.lineTo(b[0], b[1]);
cx.stroke();
}
function uni(x) {
return (Math.random() * x) | 0;
}
window.onload = function() {
var cv = document.getElementById("c");
var cx = cv.getContext("2d");
var a1 = [uni(500), uni(500)];
var a2 = [uni(500), uni(500)];
var b1 = [uni(500), uni(500)];
var b2 = [uni(500), uni(500)];
if (intersect(a1, a2, b1, b2)) {
cx.strokeStyle = "crimson";
cx.fillStyle = "crimson";
} else {
cx.strokeStyle = "slategray";
cx.fillStyle = "slategray";
}
point(cx, a1);
point(cx, a2);
point(cx, b1);
point(cx, b2);
line(cx, a1, a2);
line(cx, b1, b2);
}
</script>
<style type="text/css">
</style>
</head>
<body>
<canvas id="c" width="500" height="500">Kann nix.</canvas>
</body>
</html>
