How to calculate Rayintersection 3D for Picking

Question

Im struggeling to implement a Function which basically tells me, if a Ray "gets close enough" to an object

Basically I implemented Implementing Ray Picking @nornagon soloution to create the Ray. My object on screen is centered around a Point. I assume that if the Ray is within a certain distance to this point, the object is selected.

I call those 3 Points(X,Y,Z) : _pickFrom, _pickTo and pO

For a start, this is my method to calculate the Ray depending on Mouseposition on screen:

    public static void Pick(int x, int y)
    {
        float nx = 2.0f * ((float)x) / ((float)_width) - 1.0f;

        float ny = 1.0f - 2.0f * ((float)y) / ((float)_height);

        Matrix4 unview = Matrix4.Invert(Matrix4.Mult(_projectionMatrix, _modelviewMatrix));

        Vector4 nearPoint = Vector4.Transform(new Vector4(nx, ny, 0, 1), unview);

        _pickTo = nearPoint - _pickFrom;
    }

_pickFrom is the position of the camera in the Scene. _pickTo is the direction of the pickray. _width and _height are the rendercontext sizes.

How would I now implement a function which gives me the distance of a point to the pickray?

Answer 1

I figured it out myself now:

public void SetMouse(int x, int y)
{
    float xpos = 2.0f * ((float)x / (float)_width) - 1.0f;
    float ypos = 2.0f* (1.0f - (float)y / (float)_height) - 1.0f;

    Vector4 startRay = new Vector4(xpos, ypos, -1, 1);
    Vector4 endRay = new Vector4(xpos, ypos, 1, 1);


    Matrix4 trans = _modelView.Data * _projection.Data;
    trans.Invert();
    startRay = Vector4.Transform(startRay, trans);
    endRay = Vector4.Transform(endRay, trans);

    _pickFrom = startRay.Xyz / startRay.W;
    _pickTo = endRay.Xyz / endRay.W;
}

....

public float Distance(Vector3 p)
{
    Vector3 x1 = _pickFrom;
    Vector3 x2 = _pickTo;

    return Vector3.Cross
    (Vector3.Subtract(p, x1), 
     Vector3.Subtract(p, x2)
    ).Length / Vector3.Subtract(x2, x1).Length;
}