How To Calculate a 3D Point from 2D Coordinates?

Question

I have recently run into a predicament involving DirectX11 and the calculation of a 3D coordinate. I wish to convert a mouse coordinate to this 3D coordinate. I wish to have it act similar to Maya or Unity when inserting a polygon model (default cube, etc.) or a mesh object into an empty space. What steps and maths do I require to calculate this coordinate?

I am not an expert in any of the sofware you have mentioned, but a cube is a 3-d object where as a computer mouse moves in 2-d - In a computer pad, the 3rd diminsion is always a constant value. I suggest you clarify your question. — NoChance, Jan 28 '15 at 01:03

score 0 · Accepted Answer · answered Jan 28 '15 at 06:58

This is the same math you use in ray tracing to convert a screen coordinate back to a ray in 3D space.

Here is the D3DXMath code from the legacy DirectX SDK sample "Pick10"

const D3DXMATRIX* pmatProj = g_Camera.GetProjMatrix();

POINT ptCursor;
GetCursorPos( &ptCursor );
ScreenToClient( DXUTGetHWND(), &ptCursor );

// Compute the vector of the pick ray in screen space
D3DXVECTOR3 v;
v.x = ( ( ( 2.0f * ptCursor.x ) / pd3dsdBackBuffer->Width ) - 1 ) / pmatProj->_11;
v.y = -( ( ( 2.0f * ptCursor.y ) / pd3dsdBackBuffer->Height ) - 1 ) / pmatProj->_22;
v.z = 1.0f;

// Get the inverse view matrix
const D3DXMATRIX matView = *g_Camera.GetViewMatrix();
const D3DXMATRIX matWorld = *g_Camera.GetWorldMatrix();
D3DXMATRIX mWorldView = matWorld * matView;
D3DXMATRIX m;
D3DXMatrixInverse( &m, NULL, &mWorldView );

// Transform the screen space pick ray into 3D space
vPickRayDir.x = v.x * m._11 + v.y * m._21 + v.z * m._31;
vPickRayDir.y = v.x * m._12 + v.y * m._22 + v.z * m._32;
vPickRayDir.z = v.x * m._13 + v.y * m._23 + v.z * m._33;
vPickRayOrig.x = m._41;
vPickRayOrig.y = m._42;
vPickRayOrig.z = m._43;

I've been meaning to convert it to DirectXMath and repost it to MSDN Code Gallery for some time, but it hasn't made it to the top of the stack yet. The code above assumes left-handed coordinates.

Remember that a 2D screen position plus the transformation matrix only provides 2 of the 3 degrees of freedom, so you have to make some assumption about the depth.

Also search for the terms "picking in 3D".

So from there I just do trigonometry to determine the final position, right? — Tyson May, Jan 28 '15 at 13:18
Yes, based on whatever assumptions you want to make about the 3D point along that ray that you think the user actually wants. — Chuck Walbourn, Jan 28 '15 at 16:14
Then I have one final question. What units is the direction vector in? — Tyson May, Jan 28 '15 at 23:28
It's a ray. It as a position in world space and direction vector. That gives you a line of points in 3 space that the user is 'over' with the mouse. For picking, you'd use a traditional ray-triangle (usually with a ray-bounding-volume early out) test to find out what you 'hit'. — Chuck Walbourn, Jan 29 '15 at 08:35

How To Calculate a 3D Point from 2D Coordinates?

1 Answers1