7

I'm trying to estimate the relative camera pose using OpenCV. Cameras in my case are calibrated (i know the intrinsic parameters of the camera).

Given the images captured at two positions, i need to find out the relative rotation and translation between two cameras. Typical translation is about 5 to 15 meters and yaw angle rotation between cameras range between 0 - 20 degrees.

For achieving this, following steps are adopted.

  • a. Finding point corresponding using SIFT/SURF
  • b. Fundamental Matrix Identification
  • c. Estimation of Essential Matrix by E = K'FK and modifying E for singularity constraint
  • d. Decomposition Essential Matrix to get the rotation, R = UWVt or R = UW'Vt (U and Vt are obtained SVD of E)
  • e. Obtaining the real rotation angles from rotation matrix

Experiment 1: Real Data

For real data experiment, I captured images by mounting a camera on a tripod. Images captured at Position 1, then moved to another aligned Position and changed yaw angles in steps of 5 degrees and captured images for Position 2.

Problems/Issues:

  1. Sign of the estimated yaw angles are not matching with ground truth yaw angles. Sometimes 5 deg is estimated as 5deg, but 10 deg as -10 deg and again 15 deg as 15 deg.
  2. In experiment only yaw angle is changed, however estimated Roll and Pitch angles are having nonzero values close to 180/-180 degrees.
  3. Precision is very poor in some cases the error in estimated and ground truth angles are around 2-5 degrees.
  4. How to find out the scale factor to get the translation in real world measurement units?

The behavior is same on simulated data also.

Have anybody experienced similar problems as me? Have any clue on how to resolve them. Any help from anybody would be highly appreciated.

(I know there are already so many posts on similar problems, going trough all of them has not saved me. Hence posting one more time.)

HYPEREGO
  • 7
  • 4
Aarambh
  • 133
  • 1
  • 6
  • I know this question is a few months old, but I've been working on something similar and I believe that I came across the same problem regarding the angles. After some research I found that the SVD decomposition yelds 4 different results. Can that be why you're getting those angles? By the way: Have you solved this problem? – zync Apr 13 '13 at 00:59

2 Answers2

2

In chapter 9.6 of Hartley and Zisserman, they point out that, for a particular essential matrix, if one camera is held in the canonical position/orientation, there are four possible solutions for the second camera matrix: [UWV' | u3], [UWV' | -u3], [UW'V' | u3], and [UW'V' | -u3].

The difference between the first and third (and second and fourth) solutions is that the orientation is rotated by 180 degrees about the line joining the two cameras, called a "twisted pair", which sounds like what you are describing.

The book says that in order to choose the correct combination of translation and orientation from the four options, you need to test a point in the scene and make sure that the point is in front of both cameras.

Diane
  • 51
  • 4
0

For problems 1 and 2, Look for "Euler angles" in wikipedia or any good math site like Wolfram Mathworld. You would find out the different possibilities of Euler angles. I am sure you can figure out why you are getting sign changes in your results based on literature reading.

For problem 3, It should mostly have to do with the accuracy of our individual camera calibration.

For problem 4, Not sure. How about, measuring a point from camera using a tape and comparing it with the translation norm to get the scale factor.

Possible reasons for bad accuracy:

1) There is a difference between getting reasonable and precise accuracy in camera calibration. See this thread. 2) The accuracy with which you are moving the tripod. How are you ensuring that there is no rotation of tripod around an axis perpendicular to surface during change in position.

I did not get your simulation concept. But, I would suggest the below test.

Take images without moving the camera or object. Now if you calculate relative camera pose, rotation should be identity matrix and translation should be null vector. Due to numerical inaccuracies and noise, you might see rotation deviation in arc minutes.

Community
  • 1
  • 1
satishffy
  • 153
  • 8
  • Thank you @CV Enthusiast for your reply. For rotation matrix to Euler angles conversion, I'm assuming the rotation matrix representation provided by wiki at http://en.wikipedia.org/wiki/Rotation_matrix (2.2 General rotations). For calibration, Zang's implementation provided by OpenCV is used. Undistorted image is looking good and reprojection error returned is 0.12. So not sure how to confirm accuracy of Calibration. For translation issue, I'm moving the tripod only in Horizontal direction without changing Height or vertical displacement. But trans vector has non-zero values in all directions. – Aarambh Nov 15 '12 at 01:53
  • Just to rule out accuracy issue of Calibration Matrix, I tried to simulate the data. By assuming calibration matrix as unit matrix, generated E = [R|t] with some 10 deg yaw angle. As calibration is Unity, essential matrix and Fundamental matrix would be same (E = F). Finally, using hypothesis x2' F x1 = 0; some point sets were generated. Such simulation data is also having similar behavior as of real data. Is this simulation method correct? Is there any other method where I can simulate points and verify my implementation? Thank you in advance. – Aarambh Nov 15 '12 at 02:19
  • I think I should have read your question correctly. If I am right, you are calculating relative pose between two cameras. And you are doing so by imaging a common target with both these cameras. Am I right? – satishffy Nov 15 '12 at 11:52
  • By the way, taking 2 images as inputs whose relative rotation and translation is close to 0 will not give you a 0-pose difference as the estimate degenerates as the viewpoint difference approaches 0. – oarfish May 05 '15 at 13:54