Image Rectification

Question

I need to implement image rectification .Problem is given image of object from four viewpoints(topL,topR,bottomL,bottomR)I need to do pairwise rectification .I tried some code in OpenCV but havent been able to make progress .Can someone tell me a good way(source code /tutorial) to execute the rectification? I need to use C/C++/OpenCV.

http://portal.acm.org/citation.cfm?id=1833349.1778777

Answer 1

This is an extremely broad question, here is a general outline of one way you could do it...

• Find strong features or corners on both images using cvGoodFeaturesToTrack()

• Pass the features you found into cvFindCornerSubPix() to get more precise floating point representations of the locations of strong features

• Calculate the optical flow between the two images using the sub-pixel features from the last step using cvCalcOpticalFlowPyrLK or another optical flow function. (I like cvCalcOpticalFlowPyrLK())

This is where it gets tricky... in order to rectify images it helps to have some knowledge about the intrinsic camera properites (field of view, focal length) especially if you are planning to do some kind of 3d reconstruction or compute disparity correspondence. Since you didn't mention anything about a calibrated camera set up I will continue on the uncalibrated algorithm.

• You will need to find the Fundamental Matrix which encodes all aspects of the scene used to compute a rectification matrix. Use cvFindFundamentalMatrix() to do this.

• Armed with the Fundamental Matrix you must now find the Homography Matrix, which will map both images to the same plane. Use cvStereoRectifyUncalibrated() to do this. Although the name would suggest that it is rectifying your images it is NOT, it just returns homography matrices that you can USE to rectify your images.

• Finally, using the Homography matrices from the last step you can rectify both images by calling cvInitUndistortRectifyMap() to get a remap matrix and then pass that into cvRemap() to do the actual remapping.

I must warn you that there are many parameters that go into each of these library calls and you will have to manipulate many matrices and many images, not to mention without the intrinsic camera calibration details you will have to make many assumptions that could drastically affect your results... It is no easy task.

---

I would recommend buying and/or reading Learning OpenCV if you want to learn more, it is far beyond the scope of a short paragraph on stackoverflow to expect to learn all of this :)

Answer 2

It has been long time ago that this question was published, however I think that nowadays the answer can be refreshed. Here are some tips.

According to the definition of image rectification which is a transformation process of two-or-more images into a common image plane. This can simplify the problem of finding matching points between images. Further it increases performances of many applications like a depth map extraction.

So to perform a rectification, you must know extrinsic camera parameters , intrinsic camera parameters and distortion parameters. First express the positions between cameras, others convert the pixel frame coordinates to camera frame coordinate, and last are responsible for remove radial distortion. Below you may find some tools to estimates the parameters:

1. Matlab -stereo-camera-calibration
2. MicMac
3. OpenCV
4. Kalibr

When you know these parameters finally you can do rectification between each camera separately:

1. the rectification process on openCV, check stereoRectify.
2. the same algorithm already implemented at ROS stereo pipeline.
3. in Computer Vision System Toolbox Matlab and several another tutorials like: Du Huynh

Answer 3

http://www.vision.caltech.edu/bouguetj/calib_doc/ maybe this will be helpful