5

I'm trying to do calibration of Kinect camera and external camera, with Emgu/OpenCV. I'm stuck and I would really appreciate any help.

I've choose do this via fundamental matrix, i.e. epipolar geometry. But the result is not as I've expected. Result images are black, or have no sense at all. Mapx and mapy points are usually all equal to infinite or - infinite, or all equals to 0.00, and rarely have regular values.

This is how I tried to do rectification:

1.) Find image points get two arrays of image points (one for every camera) from set of images. I've done this with chessboard and FindChessboardCorners function.

2.) Find fundamental matrix 

 CvInvoke.cvFindFundamentalMat(points1Matrix, points2Matrix, 
_fundamentalMatrix.Ptr, CV_FM.CV_FM_RANSAC,1.0, 0.99, IntPtr.Zero);

Do I pass all collected points from whole set of images, or just from two images trying to rectify?

3.) Find homography matrices 

 CvInvoke.cvStereoRectifyUncalibrated(points11Matrix, points21Matrix, 
_fundamentalMatrix.Ptr, Size, h1.Ptr, h2.Ptr, threshold);

4.) Get mapx and mapy 

double scale = 0.02;
CvInvoke.cvInvert(_M1.Ptr, _iM.Ptr, SOLVE_METHOD.CV_LU);

CvInvoke.cvMul(_H1.Ptr, _M1.Ptr, _R1.Ptr,scale);
CvInvoke.cvMul(_iM.Ptr, _R1.Ptr, _R1.Ptr, scale);
CvInvoke.cvInvert(_M2.Ptr, _iM.Ptr, SOLVE_METHOD.CV_LU);
CvInvoke.cvMul(_H2.Ptr, _M2.Ptr, _R2.Ptr, scale);
CvInvoke.cvMul(_iM.Ptr, _R2.Ptr, _R2.Ptr, scale);

CvInvoke.cvInitUndistortRectifyMap(_M1.Ptr,_D1.Ptr, _R1.Ptr, _M1.Ptr, 
mapxLeft.Ptr, mapyLeft.Ptr) ;

I have a problem here...since I'm not using calibrated images, what is my camera matrix and distortion coefficients ? How can I get it from fundamental matrix or homography matrices?

5.) Remap 

CvInvoke.cvRemap(src.Ptr, destRight.Ptr, mapxRight, mapyRight, 
(int)INTER.CV_INTER_LINEAR, new MCvScalar(255));

And this doesn't returning good result. I would appreciate if someone would tell me what am I doing wrong.

I have set of 25 pairs of images, and chessboard pattern size 9x6.

BenMorel
  • 34,448
  • 50
  • 182
  • 322
user1326511
  • 81
  • 3

2 Answers2

0

The book "Learning OpenCV," from O'Reilly publishing, has two full chapters devoted to this specific topic. Both make heavy use of OpenCV's included routines cvCalibrateCamera2() and cvStereoCalibrate(); These routines are wrappers to code that is very similar to what you have written here, with the added benefit of having been more thoroughly debugged by the folks who maintain the OpenCV libraries. while they are convenient, both require quite a bit of preprocessing to achieve the necessary inputs to the routines. There may in fact be a sample program, somewhere deep in the samples directory of the OpenCV distribution, that uses these routines, with examples on how to go from chessboard image to calibration/intrinsics matrix. If you take an in depth look at any of these places, I am sure you will see how you can achieve your goal with advice from the experts.

gankoji
  • 853
  • 1
  • 10
  • 23
0

cv::findFundamentalMat cannot work if the intrinsic parameter of your image points is an identity matrix. In other words, it cannot work with unprojected image points.

Joel Bodenmann
  • 2,152
  • 2
  • 17
  • 44
Derza Arsad
  • 41
  • 1
  • 2