OpenGL Mouse position in 3D space

Question

I'm implementing successfully the picking ray sphere strategy, in order to get the clicked object. In figure, I selected the green cube, and the vertex bottom left is highlighted with a small red cube.

But I'm not able to move the cube on the plane z. In particular, my mouse position doesn't match the vertex position that I found moving the cursor. It works only with a Top view (XY plane).

I'm becoming crazy to understand where I'm wrong but I cannot understand.

I'm sure I didn't understand something, but what?

At a beginning, I store the mouse x,y coordinate (_init_drag_pos), then I make the difference between the previous coordinates and the new ones

dx = (mouse.X - _init_drag_pos.X); //pixel
dy = -1* (pmouse.Y - _init_drag_pos.Y); //pixel

//world position
vec3 res = glm.unProject(new vec3(mouse.Y, mouse.Y, 0), ModelViewMtx, ProjectionMtx, ViewPortVec);

I tried different approaches, this is the one that works best. Read many articles!

I want to see the cube small red cube that has the same position of the cursor!

Any suggestion?

see https://stackoverflow.com/a/51764105/2521214 , https://stackoverflow.com/a/50908533/2521214 and https://stackoverflow.com/a/52905600/2521214 for some inspiration. I do not use GLM nor unProject so I might be wrong but I am missing any depth/distance info passing to the transformation so how would the unProject know how to transform your mouse (unless it `glReadPixels` from Depth buffer on its own). Also how are you transforming dx,dy ? — Spektre, Aug 10 '21 at 07:25
Just remeber these transformation can not be done in relative terms so you can not transform `dx,dy` instead you have to transform actual and last mouse position and get `dx,dy,dz` from that and use that as translation vector for the movement... however beware the depth of previous mouse position must be also from previous frame ... so its better remembering already transformed mouse positions — Spektre, Aug 10 '21 at 12:20
I know, but I tried different approaches and this is the best one... at the moment. Another Idea is that the point I'm looking for lay on the ray vector. If I knew the plan z, I could calculate the exact position of that point using this formula: https://www.physicsforums.com/threads/equation-of-a-line-segment-in-3d.809114/ — Tostone, Aug 15 '21 at 15:48
You do not need to know the z plane, you have 2D screen position (mouse) and distance from focal point (depth buffer) you just convert it to 3D position by reversing the perspective and applying inverse transforms ... so you have 3D position of mouse targeted point from there you just apply translation vector (mouse drag) scaled by distance/perspective ... of coarse this works only if the anchor point is covered by object (not background pixel). — Spektre, Aug 15 '21 at 20:16
I'm a little confused! ... and distance from focal point (depth buffer)? what do you mean? could you provide me a sample? or a documentation where to study this? — Tostone, Aug 16 '21 at 14:31
I already did something like this: vec3 winCoords = glm.project(objVertex, ModelViewMtx, ProjectionMtx, ViewPortVec); pt_world = glm.unProject(new vec3(pt_screen.X, pt_screen.Y, winCoords[2]), ModelViewMtx, ProjectionMtx, ViewPortVec); but the drawned cube is on a different position than cursor. — Tostone, Aug 16 '21 at 15:28

score 0 · Answer 1 · answered Aug 30 '21 at 13:54

At the end I found the error ... it was in my formula.

The idea was simple: I get the ray vector and I intersect it with the plan. 3D Line Segment and Plane Intersection

vec3 v1 = glm.unProject(new vec3(winX, winY, 0f), mModelView, vp.ProjectionMtx, vp.GetVPVector()); 
    
vec3 v2 = glm.unProject(new vec3(winX, winY, 1f), mModelView, vp.ProjectionMtx, vp.GetVPVector()); 
    
//intersection in z=0
vec3 intersection = XglHelper.GetPointIn3DSegment(v1, v2, 0);



public vec3 GetPointIn3DSegment(vec3 p1, vec3 p2, float z)
{
    float t = (z - p1.z) / (p2.z - p1.z);

    float x = (1 - t) * p1.x + t * p2.x;
    float y = (1 - t) * p1.y + t * p2.y;

    return new vec3(x,y,z);
}

OpenGL Mouse position in 3D space

1 Answers1