Undistort a Fisheye Image using cv::fisheye::undistortImage

Question

I am trying to undistort a fisheye image using OpenCV. I obtained the camera matrix and the distortion coefficients from the internal memory of the camera. I'm assuming them to be accurate. As you can see in the below code, I'm using cv::fisheye::undistortImage. I found this GitHub Issue post claiming that it should work, however, the undistorted image frame doesn't look right! So, clearly something isn't working.

Here is the code:

#include <opencv2/highgui.hpp>
#include <opencv2/calib3d.hpp>
#include <opencv2/core/mat.hpp>

int main()
{
    cv::Mat cameraMatrix = cv::Mat(3,3, CV_64F, double(0));
    cv::Mat distortionCoeffs = cv::Mat(1,4, CV_64F, double(0));

    cameraMatrix.at<double>(0, 0) = 286.7037963867188;
    cameraMatrix.at<double>(0, 1) = 0;
    cameraMatrix.at<double>(0, 2) = 413.3463134765625;
    cameraMatrix.at<double>(1, 0) = 0;
    cameraMatrix.at<double>(1, 1) = 286.7817993164062;
    cameraMatrix.at<double>(1, 2) = 397.1785888671875;
    cameraMatrix.at<double>(2, 0) = 0;
    cameraMatrix.at<double>(2, 1) = 0;
    cameraMatrix.at<double>(2, 2) = 1;

    distortionCoeffs.at<double>(0,0) = -0.01078350003808737;
    distortionCoeffs.at<double>(0,1) = 0.04842806980013847;
    distortionCoeffs.at<double>(0,2) = -0.04542399942874908;
    distortionCoeffs.at<double>(0,3) = 0.008737384341657162;

    cv::Mat input_frame = cv::imread("fisheye_input.png");
    cv::Mat output_frame;

    cv::fisheye::undistortImage(input_frame,output_frame,cameraMatrix,distortionCoeffs, cv::noArray(), cv::Size(input_frame.cols,input_frame.rows));    

    cv::imshow("Input Image", input_frame);
    cv::imshow("Output Image", output_frame);
    cv::waitKey(-1);
    return 0;
}

Code output:

In case you want to try it yourself here is the original fisheye image:

[This](https://stackoverflow.com/questions/34316306/opencv-fisheye-calibration-cuts-too-much-of-the-resulting-image/53500300#53500300) one seems working — Yunus Temurlenk, Nov 19 '20 at 05:50

score 2 · Answer 1 · answered Nov 19 '20 at 06:20

I couldn't exactly find what the real problem is in your code. When I read the documentation, I have seen that:

The function is simply a combination of fisheye::initUndistortRectifyMap (with unity R ) and remap (with bilinear interpolation).

When I dig into about using these 2 function combination(initUndistortRectifyMap and remap) instead of using undistortImage, I have found this post which @Micka answered. I tried it and worked fine, here is the code and result:

#include <opencv2/highgui.hpp>
#include <opencv2/calib3d.hpp>
#include <opencv2/core/mat.hpp>

#include <opencv2/imgproc/imgproc.hpp>
int main()
{
    cv::Mat cameraMatrix = cv::Mat(3,3, cv::DataType<double>::type);
    cv::Mat distortionCoeffs = cv::Mat(4,1, cv::DataType<double>::type);

    cameraMatrix.at<double>(0, 0) = 286.7037963867188;
    cameraMatrix.at<double>(0, 1) = 0;
    cameraMatrix.at<double>(0, 2) = 413.3463134765625;
    cameraMatrix.at<double>(1, 0) = 0;
    cameraMatrix.at<double>(1, 1) = 286.7817993164062;
    cameraMatrix.at<double>(1, 2) = 397.1785888671875;
    cameraMatrix.at<double>(2, 0) = 0;
    cameraMatrix.at<double>(2, 1) = 0;
    cameraMatrix.at<double>(2, 2) = 1;

    distortionCoeffs.at<double>(0,0) = -0.01078350003808737;
    distortionCoeffs.at<double>(1,0) = 0.04842806980013847;
    distortionCoeffs.at<double>(2,0) = -0.04542399942874908;
    distortionCoeffs.at<double>(3,0) = 0.008737384341657162;

    cv::Mat E = cv::Mat::eye(3, 3, cv::DataType<double>::type);

    cv::Mat input_frame = cv::imread("fishEye.png");

    cv::Size size = { input_frame.cols, input_frame.rows };

    cv::Mat map1;
    cv::Mat map2;
    //cv::fisheye::undistortImage(input_frame,output_frame,cameraMatrix,distortionCoeffs, E, cv::Size(input_frame.cols,input_frame.rows));

    cv::fisheye::initUndistortRectifyMap(cameraMatrix, distortionCoeffs, E, cameraMatrix, size, CV_16SC2, map1, map2);

    cv::Mat undistort;

    cv::remap(input_frame, undistort, map1, map2, cv::INTER_LINEAR,
              CV_HAL_BORDER_CONSTANT);

    cv::imshow("Input Image", input_frame);
    cv::imshow("Output Image", undistort    );
    cv::waitKey(-1);
    return 0;
}

Output:

Why did you use R (Rectification transformation in the object space) = cv::Mat::eye(3, 3, cv::DataType::type), and P (New camera intrinsic matrix) = cameraMatrix ? My camera drivers reports them as {0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0} and {0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0}, respectively. I think that's why it wasn't working when I tried using these two functions originally. — csg, Nov 19 '20 at 16:40

Undistort a Fisheye Image using cv::fisheye::undistortImage

1 Answers1