Obtain the implicit equation of the line of support of your segment (as you say that "inside" is possible, your "line" must be a segment). It has the form
s(x, y) = a x + b y + c = 0
and S
is positive on one side of the line and negative on the other side.
Now take every polygon side in turn and plug the coordinates of the two vertices. If the S
changes sign between the vertices, the side intersects the line.
Now the trick is to use the parametric equation of the line from one endpoint to the other and find the intersection with the line in terms of the parameter t
along the line. Note that the segment will correspond to the t
interval [0, 1]
.
After scanning the whole polygon, you will have a list of t values. If the list is empty, no intersection. If all values in the list are <0
or >1
, no intersection. Otherwise, there is an intersection (one subsegment or more).