8

I have 3 viewpoints locations (x,y,z coordinates on 3d grid), directions (relative to viewpoints origin points x,y,z vector) in 3d space. They all look and each does see exactly three points (markers (x,y) with filtered out background) in space (say we have a red, blue, green dot on white images). We do not know any other viewpoints-camera properties except resolution and that this are same (extremely similar) cameras. How could we get our point position in space?

DuckQueen
  • 772
  • 10
  • 62
  • 134
  • You only need two point/viewport pairs, as each pair defines a line and the intersection of two lines forms a point. – François Andrieux May 11 '18 at 12:47
  • A viewport's resolution is not enough information for ray casting. You must be able to construct a viewport matrix. – François Andrieux May 11 '18 at 12:51
  • what if we had 3 points (red, green, blue) detected in each viewpoint? – DuckQueen May 12 '18 at 04:55
  • 2
    Sounds like [bundle adjustment](https://en.wikipedia.org/wiki/Bundle_adjustment). – Nico Schertler May 13 '18 at 18:21
  • What kind of markers are you using? If you use something like Aruco markers(https://docs.opencv.org/3.1.0/d5/dae/tutorial_aruco_detection.html) you could use pose estimation to try and figure out your location based on the known size and location of the markers. – Darth Coder May 14 '18 at 08:59
  • @Darth - no simple colored dots, imagine them as given (x,y, color) points on each viewpoint image – DuckQueen May 14 '18 at 14:08
  • This is a problem in Epipolar geometry. You essentially want to determine the projection, view, and calibration matrices of the camera, but you are missing the optical properties of the camera such as focal length. You already have the correlated points between multiple images. See this presentation https://cs.nyu.edu/~fergus/teaching/vision/9_10_Stereo.pdf for an explanation of stereoscopic 3D reconstruction. – Jacques Nel May 15 '18 at 08:10

2 Answers2

6

I assume you have your 3 cameras calibrated, otherwise I do not know if a solution exists to your problem.

If you have your cameras calibration parameters i.e. camera matrix (A), rotation vector (T), translation vector (R). From these you can get the projection matrix using P = A[R|T]. This means that you can project any 3d point in the real world X to image i using X.Pi = x which means you can get a system of linear equations for each view using xi' = xi.P3T - P1T and yi' = yi.P3T - P2T PiT is the transpose of the ith column in P. You can solve this system of equations using singular value decomposition.

References:

  1. http://www.robots.ox.ac.uk/~az/tutorials/tutoriala.pdf
  2. https://github.com/opencv/opencv_contrib/blob/master/modules/sfm/src/triangulation.cpp
  3. http://www.morethantechnical.com/2012/01/04/simple-triangulation-with-opencv-from-harley-zisserman-w-code/
Osama AlKoky
  • 214
  • 1
  • 5
3

You need one more piece of information: your camera's field of view:

https://blog.codinghorror.com/content/images/uploads/2007/08/6a0120a85dcdae970b0120a86d9495970b-pi.png

I'll refer to horizontal field of view as hFOV and vertical field of view as vFOV. You only need one of these numbers to solve your problem though, as hFOV/vFOV is equal to the aspect ratio (horizontal resolution / vertical resolution).

With that information, you can use the camera's position (Pc, a 3D vector), direction (Dc, a 3D vector), resolution (R, a 2D vector) and the point's position in view space (Pp, a 2D vector) to build a 3D vector that the point lies upon in space.

If the point lies exactly in the middle of the view (Pp / R = 0.5) then this vector is simply Pc + d * Dc, where d is any positive number, corresponding to the distance from the camera (this will become our variable that we need to solve for.

The x coordinate of Pp rotates Dc in x from -hFOV/2 to hFOV/2 about the vector perpendicular to both Dc and the x direction of the screen (you can find this with the cross product). The y coordinate does the same in y. We'll call this rotated vector D'c. Rotating a vector by θ about another vector is solved here: Rotating a Vector in 3D Space

So now we have the relationship:

Position of marker = Pc + d * D'c

Now, d is still an unknown (distance from camera). But if we do this for two cameras, we can solve the system of equations and find the absolute position of the marker. The third camera is useful for verification.

Note: This is more or less the same answer as @osama-alkoky, just the manual version. Learning about projection/transformation matrices is a good idea if you're going to be doing a lot of this.

Aaron Krajeski
  • 757
  • 5
  • 18