OpenCV fisheye calibration C++ output differs from Python

Question

I’m currently working on a camera that have a fisheye (which is not much but still) and I’m trying to undistort it.

As starter : I’m not that familiar with opencv, I’m like always on the documentation while trying to do anything. (and/or here looking for answer to my problem(s)).

So I saw few example on the web, I tried the solutions and got some results on my part.

Here what I found :

DIM=(1094, 729)
K=np.array([
    [1307.2807020496643, 0.0, 530.3754311563506], 
    [0.0, 1318.342691460933, 354.98352268131123], 
    [0.0, 0.0, 1.0]
])
D=np.array([
    [-0.2994762856767568],
    [0.5036082961388784],
    [-4.231072729639434],
    [3.8646397788794578]
])
def undistort(img_path):    
    img = cv2.imread(img_path)
    h,w = img.shape[:2]
    print(K)
    print(D)
    print(np.eye(3))
    print(DIM)
    map1, map2 = cv2.fisheye.initUndistortRectifyMap(K, D, np.eye(3), K, DIM, cv2.CV_16SC2)
    
    undistorted_img = cv2.remap(img, map1, map2, interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT)
    cv2.imshow("undistorted", undistorted_img)
    cv2.imwrite("test.jpg", undistorted_img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()
    
if __name__ == '__main__':
    for p in sys.argv[1:]:
        undistort(p)

This is the python script I use to undistort image. The data at the top (K & D) are generated through another script. I didn’t mentioned it because I have a translation in C++ that show same result for the same base_picture

And for this function if I give a distorted image like : I got this result :

So I can see that the tiny distortion has been resolved.

But when I try to implement this same function in C++ as follow :

    src = cv::imread(“path/to/image.jpg");

    cv::Size size = {src.cols, src.rows};

    cv::Mat K(3, 3, cv::DataType<double>::type);
    K.at<double>(0, 0) = 1307.2807020496643;
    K.at<double>(0, 1) = 0.0;
    K.at<double>(0, 2) = 530.3754311563506;

    K.at<double>(1, 0) = 0.0;
    K.at<double>(1, 1) = 1318.342691460933;
    K.at<double>(1, 2) = 354.98352268131123;

    K.at<double>(2, 0) = 0.0;
    K.at<double>(2, 1) = 0.0;
    K.at<double>(2, 2) = 1.0;

    cv::Mat D(4, 1, cv::DataType<double>::type);
    D.at<double>(0, 0) = -0.2994762856767568;
    D.at<double>(1, 0) = 0.5036082961388784;
    D.at<double>(2, 0) = -4.231072729639434;
    D.at<double>(3, 0) = 3.8646397788794578;
    cv::Mat E = cv::Mat::eye(3, 3, cv::DataType<double>::type);

    cv::Mat map1;
    cv::Mat map2;

    std::cout << K << std::endl;
    std::cout << D << std::endl;
    std::cout << E << std::endl;
    std::cout << size << std::endl;

    cv::initUndistortRectifyMap(K, D, E, K, size, CV_16SC2, map1, map2);

    cv::Mat undistort;
    cv::remap(src, undistort, map1, map2, CV_INTER_LINEAR,
              CV_HAL_BORDER_CONSTANT);

The base image is still the same BUT I got this as result :

And as you can tell, it went worse…

I dumped K, D & E in my python script and c++ program (both gives the same result for the same base picture) (I mean when I compute the datas)

It goes wrong from the initUndistortRectifyMap up to the remap and finally imshow is totally not what I was expecting.

I try to dump map1 & map2 in python script & C++ program (wasn’t looking at ALL of the data) and I could notice that at the end of one of the map (didn’t look both) the results were differents.

Since I call the function using the same parameter (as far as I’m aware of) I expected for the map to be equal between both programs.

Is there something I’m doing wrong ? Like messing with the types or whatsoever ?

Does it change anything to compute the image from a cv::imread or from a frame acquisition from a camera ?

(Just to let you know, the frame (.jpg) with which computation is made, is a cv::imwrite from the frame acquisition of the camera, then I intend not to use images anymore but only work with cv::Mat filled with data from camera acquisition.)

(I know that at the end the enum aren't the same, but even using cv::BORDER_CONSTANT I still have the same issue.)

Why did you reassign K and D matrices elements by one by instead of giving directly a Mat? Can you try to replace ```cv::initUndistortRectifyMap(K, D, E, K, size, CV_16SC2, map1, map2);``` with ```cv::initUndistortRectifyMap(K, D, Mat(), Mat(), src.size(), CV_16SC2, map1, map2);``` — Yunus Temurlenk, Feb 07 '20 at 18:22
Well, I didn't know where I could have made a mistake so I decided to do it like that to ensure what I was doing (keeping same data w/ both programs). Well, nvm... I used your line of code `cv::initUndistortRectifyMap(K, D, Mat(), Mat(), src.size(), CV_16SC2, map1, map2);` but unfortunately I still have the same result. I also noticed something else : With a static camera, and a static chessboard in focus, when I compute K & D I got differents value for the mat.. And sometimes result aren't close at all ! — Nicolas Wadel, Feb 07 '20 at 20:42
Generally using only one image for calibration doesnt give good results, I'm also confused how you are getting a good result by using python code. The difference reason between the codes should be about K and D matrices otherwise i dont see any wrong — Yunus Temurlenk, Feb 07 '20 at 21:07
Well, I am definitely not using a single image for calibration. What I'm trying to do here, is to validate that for a given image at a given position I'm able to undistort this same image and only this same one. In the end, yeah sure I'll have a whole calibration process, assessing multiple picture of the chessboard at differents, angle & position (but my cam hasn't autofocus yet... so... single image for now) (And as said earlier K & D have same values (between programs), BUT didn't output same result) — Nicolas Wadel, Feb 07 '20 at 22:07
in C++ you used cv::initUndistortRectifyMap instead of Fisheye. Python was cv2.fisheye.initUndistortRectifyMap — Micka, Feb 08 '20 at 22:41

score 5 · Accepted Answer · answered Feb 09 '20 at 10:26

You've used the cv::initUndistortRectifyMap instead of cv::fisheye::initUndistortRectifyMap

using this code instead I get the proper result:

int main()
{
    cv::Mat src = cv::imread("C:/StackOverflow/Input/fisheyeCalib.jpg");

        cv::Size size = { src.cols, src.rows };

    cv::Mat K(3, 3, cv::DataType<double>::type);
    K.at<double>(0, 0) = 1307.2807020496643;
    K.at<double>(0, 1) = 0.0;
    K.at<double>(0, 2) = 530.3754311563506;

    K.at<double>(1, 0) = 0.0;
    K.at<double>(1, 1) = 1318.342691460933;
    K.at<double>(1, 2) = 354.98352268131123;

    K.at<double>(2, 0) = 0.0;
    K.at<double>(2, 1) = 0.0;
    K.at<double>(2, 2) = 1.0;

    cv::Mat D(4, 1, cv::DataType<double>::type);
    D.at<double>(0, 0) = -0.2994762856767568;
    D.at<double>(1, 0) = 0.5036082961388784;
    D.at<double>(2, 0) = -4.231072729639434;
    D.at<double>(3, 0) = 3.8646397788794578;
    cv::Mat E = cv::Mat::eye(3, 3, cv::DataType<double>::type);

    cv::Mat map1;
    cv::Mat map2;

    std::cout << K << std::endl;
    std::cout << D << std::endl;
    std::cout << E << std::endl;
    std::cout << size << std::endl;

    // Here's the error:
    //cv::initUndistortRectifyMap(K, D, E, K, size, CV_16SC2, map1, map2);
    cv::fisheye::initUndistortRectifyMap(K, D, E, K, size, CV_16SC2, map1, map2);

    cv::Mat undistort;
    cv::remap(src, undistort, map1, map2, CV_INTER_LINEAR,
        CV_HAL_BORDER_CONSTANT);

    cv::imwrite("C:/StackOverflow/Input/fisheyeCalib_output.jpg", undistort);
    cv::imshow("undist", undistort);
    cv::waitKey(0);
}

OpenCV fisheye calibration C++ output differs from Python

1 Answers1

Linked