6

I have a single image and several known points and its projections to image plane. (each 3d point has projection to 2d). My goal is to find matrix 4x4, for easy calculation of projection of any 3d point to image plane. I tried to use Monte-Carlo method from this topic: How do I reverse-project 2D points into 3D? But result matrix is good for oXY plane, not for Z coordinate (points with non-zero Z are projected incorrectly). Also I used OpenCV.CalibrateCamera method. I got rotation matrix, camera matrix and translation vector, but it isn't good for non-zero Z points too. In addition, I am not interested in 3 matrices, I just want to get one matrix 4x4 for projecting and unprojecting points.

Look at this picture: Illustration

I'm pretty sure that my problem has solution, but doesn't know how.

Community
  • 1
  • 1
Inflight
  • 497
  • 4
  • 12
  • Probably because the transformation isn't reversible, for example the pixel in this picture that represents your point (0,0,0) also represents a vector of other points pointing straight into the camera like (guessing the numbers) (-1,-10,0.4) and (-2,-14,0.9). Also many cameras/lenses have distortion - straight lines in the physical view don't _always_ project to straight lines in the captured picture - for example pincushion distortion. – DisappointedByUnaccountableMod May 10 '17 at 11:29
  • Have you look at [`cv::solvePnP()`](http://docs.opencv.org/3.2.0/d9/d0c/group__calib3d.html#ga549c2075fac14829ff4a58bc931c033d)? You will get the camera pose from a set of known 3D world/object points and the corresponding set of 2D image points. From any 3D point in the world/object frame, you will be able to project it to the image plane. **The inverse is not true** (you cannot from a 2D image point get the 3D point in the camera frame as there is an infinite number of 3D points along the image ray). – Catree May 10 '17 at 12:38

1 Answers1

3

Finally, I found the solution. It is described in http://ece.ucsb.edu/~manj/ece181bS04/ECE_181b_HW3/camera_matrix_key.pdf The method has the name "Direct linear calibration" and requires at least 6 points in 3d and its projections to the image plane. The method returns camera matrix 4*3, which I did try to find.

Update: I even created npm package projection-3d-2d to help others with this problem. Package calculates 3x3 and 4x4 matrices for projection 2D to 2D and 3D to 3D accordingly. Read more

Inflight
  • 497
  • 4
  • 12