1

I am trying to follow this tutorial regarding mouse picking in OpenGL but I am failing to get it to work properly.

In my basic understanding to get world space coordinates from mouse coordinates I would need to multiply a bunch of inverse matrices. Then to get world space coordinates from local space I would multiply the local space coordinates with the model matrix. But the coordinates that I got don't line up with each other.

I tried to normalize the world coordinates for the model because the mouse coordinates is also normalized but I'm not sure if that is the case.

if (instance && button == GLFW_MOUSE_BUTTON_LEFT
    && action == GLFW_PRESS) {
    glfwGetCursorPos(window, &x, &y);

    pMat = instance->graphics->getPMat();

    cameraPos = instance->graphics->getCameraPos();

    //View matrix
    vMat = glm::translate(glm::mat4(1.0f), glm::vec3(cameraPos.x, cameraPos.y, -cameraPos.z));
    //vMat = glm::rotate(vMat, 0.6f, glm::vec3(1.0f, 0.0f, 0.0f));

    //normalized mouse coordinates
    x = (2.0f * x) / width - 1.0f;
    y = 1.0 - (2.0f * y) / height;

    glm::vec3 ray_nds = glm::vec3(x, y, 1.0f);

    glm::vec4 ray_clip = glm::vec4(ray_nds.x, ray_nds.y, -1.0, 1.0);

    glm::vec4 ray_eye = glm::inverse(pMat) * ray_clip;
    ray_eye = glm::vec4(ray_eye.x, ray_eye.y, -1.0, 0.0f);

    glm::vec3 ray_wor = glm::vec4(glm::inverse(vMat) * ray_eye);
    ray_wor = glm::normalize(ray_wor);

    Entity* e = instance->getCurrentScene().getEntities()[0];

    sMat = glm::scale(glm::mat4(1.0f), glm::vec3(e->getScaleX(), e->getScaleY(), e->getScaleZ()));
    tMat = glm::translate(glm::mat4(1.0f), glm::vec3(e->getX(), e->getY(), e->getZ()));

    rMat = glm::rotate(glm::mat4(1.0f), e->getRotY(), glm::vec3(0.0f, 1.0f, 0.0f));
    rMat = glm::rotate(rMat, e->getRotX(), glm::vec3(1.0f, 0.0f, 0.0f));
    rMat = glm::rotate(rMat, e->getRotZ(), glm::vec3(0.0f, 0.0f, 1.0f));

    mMat = tMat * rMat * sMat;
    mvMat = vMat * mMat;

    startCoords = mMat * glm::vec4(0.0f, 0.0f, 0.0f, 1.0f);
    startCoords = glm::normalize(startCoords);
    endCoords = mMat * glm::vec4(1.0f, -1.0f, 1.0f, 1.0f);
    endCoords = glm::normalize(endCoords);

    if (ray_wor.x >= startCoords.x
        && ray_wor.y <= startCoords.y
        && ray_wor.x <= endCoords.x
        && ray_wor.y >= endCoords.y) {
        std::cout << "\nFound!\n";
    }

    std::cout << "x: " << ray_wor.x << "\n"
        << "y: " << ray_wor.y << "\n"
        << "z: " << ray_wor.z << "\n\n";

    std::cout << "x: " << endCoords.x << "\n"
        << "y: " << endCoords.y << "\n"
        << "z: " << endCoords.z << "\n\n";

    std::cout << "x: " << startCoords.x << "\n"
        << "y: " << startCoords.y << "\n"
        << "z: " << startCoords.z << "\n\n";
}
genpfault
  • 51,148
  • 11
  • 85
  • 139
Jacky
  • 11
  • 1
  • 2
  • Last time I need to do this (a while ago, so the details are fuzzy), I ended up using the depth buffer to get an X,Y,Z in screen coordinates, which greatly simplified things. Just needed to transform screen to model with one matrix multiply afterwards. – Evan May 20 '19 at 20:00
  • Do you mind sharing some resources you have used to learn how to implement this? I looked on Google and they are not very clear and kind of confusing. Thanks. – Jacky May 20 '19 at 22:52
  • see [OpenGL 3D-raypicking with high poly meshes](https://stackoverflow.com/a/51764105/2521214) and compare with it what you are missing or doing differently... did you reverse the perspective division? that is not done by any matrix – Spektre May 21 '19 at 08:00
  • This link needs a free trial to continue, but the first three steps seem to exactly match my recollection of what I had to do. And it is very clearly written. https://subscription.packtpub.com/book/game_development/9781849695046/2/ch02lvl1sec21/implementing-object-picking-using-the-depth-buffer – Evan May 21 '19 at 14:50
  • And here's a related SO post: https://stackoverflow.com/questions/28032910/opengl-picking-fastest-way – Evan May 21 '19 at 14:53

0 Answers0