Java OpenGL - Mouse position from window to world space

Question

I try to transform the window mouse coordinates (0/0 is the upper left corner) into world space coordinates. I just tried to solve it by this description. Here is my code:

public void showMousePosition(float mx, float my){
    Matrix4f projectionMatrix = camera.getProjectionMatrix();
    Matrix4f viewMatrix = camera.getViewMatrix();
    Matrix4f projMulView = projectionMatrix.mul(viewMatrix);
    projMulView.invert();
    float px = ((2*mx)/650)-1;
    float py = ((2*my)/650)-1;
    Vector4f vec4 = new Vector4f(px, py*(-1), 0.0f, 1.0f);
    vec4.mul(projMulView);
    vec4.w = 1.0f / vec4.w;
    vec4.x *= vec4.w;
    vec4.y *= vec4.w;
    vec4.z *= vec4.w;

    System.out.println(vec4.x + ", " + vec4.y);
}

But thats not 100% correct. I have an Object on 0/-11 on world space and when I move my mouse to this point, my function say 0/9,8. And when I go to the left side of my window the x value is 5,6 but it should be something like 28.

Someone know what is wrong on my code?

Orlando Aguilar · Answer 1 · 2015-12-30T07:09:54.490

First of all, your code says that your windows size is always width=650, height=650.

Then you are getting the position when z=0. But this z is in screen space and therefore it changes as you change the camera position and orientation. Normally, you get this information from the depth buffer, using glReadPixel. You should do it in this case.

However, there is another way to do this also. In the code I will share, I am looking for the intersection between a ray (generated from the mouse position) and the plane (0,0,0) with normal (0,1,0), I hope this helps.

/*Given the inverse PV (projection*view) matrix, the position of the mouse on screen and the size of the screen, transforms the screen coordinates to world coordinates*/
glm::vec3 Picking::OnWorld(glm::mat4 const& m_inv, glm::vec2 const & spos,size_t width, size_t height) {
float x = spos.x;
float y = spos.y;

y = height - y;

//InputOrigin, start of the ray for intersection with plane 
glm::vec4 inputO = glm::vec4(x / width*2.0f - 1.0f, y / height*2.0f - 1.0f, -1.0f, 1.0f);   //transforms screen position to the unit cube range
glm::vec4 resO = m_inv*inputO;      //transforms to world space
if (resO.w == 0.0f)  
    return glm::vec3(-1); //return an invalid value to show a problem during a calculation, normally this means that the m_inv matrix was incorrect

resO /= resO.w;     //homogeneous division

glm::vec4 inputE = inputO;      //inputEnd, the end of the ray
inputE.z = 1.0;

//End of ray to world space
glm::vec4 resE = m_inv*inputE;

//checks that the coordinates are correct
if (resE.w == 0.0f)
    return glm::vec3(-1); //return an invalid value to show a problem during a calculation, normally this means that the m_inv matrix was incorrect

resE /= resE.w;

//ray for intersection
glm::vec3 ray = glm::vec3(resE - resO);     //vector between z=-1 and z=1

glm::vec3 normalRay = glm::normalize(ray);
glm::vec3 normalPlane = glm::vec3(0, 1, 0);     //detects collision with plane 0, normal 1
float denominator = glm::dot(normalRay, normalPlane);       
if (denominator == 0)
    return glm::vec3(-1);   //return an invalid value to show a problem during a calculation, normally this means that the m_inv matrix was incorrect

float numerator = glm::dot(glm::vec3(resO), normalPlane);

//intersection between ray and plane
glm::vec3 result = glm::vec3(resO) - normalRay*(numerator / denominator);

return result;
}

The math for the intersection can be read from this link: https://www.cs.princeton.edu/courses/archive/fall00/cs426/lectures/raycast/sld017.htm

Java OpenGL - Mouse position from window to world space

1 Answers1