1

I'm doing a stereovision setup, with 2 cameras mounted above a wing. The left camera is tilted a few degree inwards while the right camera in parallel with the wing. All images available here

Then using (cut and pasted, but does not compile as is)

// performing stereocalibration given imagePoint_leftcamera and rightcamera
Flea3.reproj_error = stereoCalibrate(objectPoints,imagePoints_left,imagePoints_right,cameraMatrix_left,
        distCoeffs_left,cameraMatrix_right,distCoeffs_right,imageSize, Flea3.R, Flea3.T, Flea3.E, Flea3.F,
        TermCriteria(CV_TERMCRIT_ITER+CV_TERMCRIT_EPS, 100, 1e-5),
        CV_CALIB_FIX_INTRINSIC );

// Peform stereorectification
stereoRectify(cameraMatrix_left, distCoeffs_left, cameraMatrix_right, distCoeffs_right, imageSize, Flea3.R, Flea3.T, Flea3.R1, Flea3.R2, Flea3.P1, Flea3.P2, Flea3.Q, CALIB_ZERO_DISPARITY, -1, Size(), &Flea3.validRoi_left, &Flea3.validRoi_right);

//computes undistort and rectify maps
initUndistortRectifyMap(cameraMatrix_left, distCoeffs_left, R1, P1, imageSize, CV_16SC2, rmap[0][0], rmap[0][2]);
initUndistortRectifyMap(cameraMatrix_right, distCoeffs_right, R2, P2, imageSize, CV_16SC2, rmap[1][0], rmap[1][3]);

remap(src_left, img_left, rmap[0][0], rmap[0][4], CV_INTER_LINEAR);
remap(src_right, img_right, rmap[1][0], rmap[1][5], CV_INTER_LINEAR);

Shouldn't the black region be on the right in the right image? Since the right camera should be tilted right?

Also, the ROI (denoted by red box) is clearly wrong, though the stereorectification appears to work!

Is my calibration any good? BTW my re-projection error returned by cv::stereocalibrate is 0.6004

Ziyang
  • 45
  • 1
  • 7
  • See this other answer for tips on performing a camera calibration: http://stackoverflow.com/questions/12794876/how-to-verify-the-correctness-of-calibration-of-a-webcam/12821056#12821056 – Francesco Callari Mar 15 '14 at 17:28
  • I have already read that link before, but thanks! Also, it doesn't address the ROI question. Perhaps the problem is numerical in nature? – Ziyang Mar 17 '14 at 21:06
  • @user3417036 could you please provide me insentric parameters for flea3 camera. thank you. – VenushkaT Apr 19 '15 at 03:09
  • @VenushkaT That will not be useful since intrinsic parameters are dependent on lens used (the distortion coefficients). I recommend just doing the calibration on your own. If you don't want to code, this will do as well [link](http://www.vision.caltech.edu/bouguetj/calib_doc/) – Ziyang Aug 04 '15 at 16:14

1 Answers1

1

Please, look at the green matching lines at the bottom - there is no correspondence. In practice you have to use about 20-30 chess board pattern poses (calibration rigs) at different positions and orientation including in rotation in depth (slant, tilt), in-plane rotation and please cover the whole image at least in some of your calibration shots or cover it consistently part by part in different shots.

The reason for needing multiple calibration images is following. Think about the point at infinity (called ideal or vanishing point) in homogeneous coordinates. The ideal point along x direction is
Xinf = [1, 0, 0, 0]T
If you bring it back to Euclidean space you will get [1/0=Inf, 0/0, 0/0]T. If you multiply your projection matrix from the left with Xinf (in homog. coord) the result will have zeros everywhere except the first column. Conclusion: vanishing point in X direction gives you a first column of a projection matrix. Other columns come from other vanishing points. It is harder to prove the reverse - namely that to get correct P you need vanishing points, let's just assume it for now.

The problem with your calibration is that you don't have clear vanishing points in your rig since it front-faces camera. You have to slant it to make projected line converge (into a vanishing points at multiple directions). Another problem is that your rig occupies only a small portion of the image and that's where optimization happened at the expense of other areas. Recalibrate with multiple rig poses and you'll get better results.

Vlad
  • 4,425
  • 1
  • 30
  • 39
  • Hi, Thanks for your reply. Actually I used about 15 frames for calibration (although it is not in the box folder), then I used the uploaded image (which is in box) for verification. I understand the need for multiple calibration frames to estimate stereocalibration parameters. I think the calibration is in the right "ball park" since if you look at the checkerboard itself, the images looks frontal parallex. The checkerboard corners line up at least vertically. I will try to use more calibration frames as you suggested and see if it helps. – Ziyang Mar 17 '14 at 12:08
  • You will need more images since at the last image in your link stitched_cs.png green lines that connect supposedly same rows in the images fail at the bottom. This means your calibration is optimal only for the upper half unless you performed a better one by now. – Vlad Mar 17 '14 at 16:58
  • For the current application, the top half is fine since the wing bends upwards. However, I still have the problem of clearly invalid ROI. – Ziyang Mar 17 '14 at 21:03
  • invalid rectification at the bottom probably means that image rectification has failed due to erroneous pose estimation even if the top looks ok. – Vlad Mar 17 '14 at 21:10
  • Correct me if I'm wrong, in cv::stereocalibration, we are essentially computing the rotation (R) and translation (t) between the 2 camera coordinates system. Which means that the R,t has to be valid for the whole frame. Whether if it is any good, it is another question. Therefore, in cv::stereorectify if the "top" of the image lines up, the entire remap should be good (not necessary accurate). Am I wrong? – Ziyang Mar 20 '14 at 01:39
  • wrong, since for 6DOF problem there are many solutions. If you add intrinsic matrix and (God forbids) lens distortion coefficients the probability to get stuck in local minima is pretty high. Remember that you run non-linear Levenberg-Marquardt minimization of squared differences between your points on checker board at the expense of points elsewhere. Also, a final result is not R|T but a set of homographs that are applied to your images to make them virtually fronto-parallel. This would amplify your initial errors in R, T in the ways that is hard to predict. – Vlad Mar 20 '14 at 02:46