reconstructing position from depth using opencv

Question

I have a depth image, point cloud and I have 2D corner points that are computed using OpenCV feature, how would I compute the 3D positions of these corners using depth image and I have the camera internsics parameters ?

Answer 1

Your question seems related to this answer [Extracting 3D coordinates given 2D image points, depth map and camera calibration matrices

Here, you have a direct correspondence from 2D and 3D (actually 2.5D) values. By undistorting both, the depth and the 2D image, you can use the focal length and the measured depth to do the inverse mapping from 2D pixels to 3D world coordinates.