2

After generating the map1 and map2 using the cv2.fisheyeUndistortRectifyMap(), while remapping using cv2.UndistortRectifyMap(), I am unable to fetch the full black region even by keeping the Balance as 1. The expected image shall have all corners visible including the missing bottom-left and upper-left and right portion, making the resulting image appear as a star.

#Camera Matrix
mtx = np.array([[965.,   0., 960.],
       [  0., 970., 540.],
       [  0.,   0.,   1.]])

#Distortion Matrix
dist = np.array([[-0.07234645], [-0.00707313],  [0.00762894], [-0.00343638]])

# Sample Image requiring the undistortion with black unwarp
img = cv2.imread("Sample.jpg")
img_shape = img.shape[:2][::-1]

# New camera matrix using fisheye
new_K = cv2.fisheye.estimateNewCameraMatrixForUndistortRectify(mtx, dist, img_shape, np.eye(3), balance=1)

# Overlaying the maps on a higher resolution to fetch the extensions
map1, map2 = cv2.fisheye.initUndistortRectifyMap(mtx, dist, np.eye(3), new_K, (int(1920*1.5), int(1080*1.5)), cv2.CV_32F)

# Using the remap function to overlay the resultant image
res_img = cv2.remap(img, map1, map2, interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT)

The sample used here is attached below along with the resultant image

Input to the code Sample.jpg: input to the code 327_Sample.jpg

Output of the code appears as so: output of the code appears as so

Jeru Luke
  • 20,118
  • 13
  • 80
  • 87
amar adak
  • 31
  • 4
  • use the `new_size` output parameter from `estimateNewCameraMatrixForUndistortRectify` for the undistorted image size – Micka Mar 02 '21 at 09:03
  • @Micka Thanks for your response. But if you look at the code, I am already overlaying the mapped pixels on a 1.5 * larger than the original image frame size. (Original size = 1920*1080 px) – amar adak Mar 02 '21 at 14:53
  • and your output image IS bigger: 2880x1620, but still not big enough. `estimateNewCameraMatrixForUndistortRectify` can return you the right size, why dont you use it? Can you try 2x larger than original just for testing? – Micka Mar 02 '21 at 15:23
  • @Micka Hi, I tried doing that too. But no luck :( – amar adak Mar 03 '21 at 10:00
  • can you post the `new_K` matrix values? Can you integrate and post the `new_size` vlaue? – Micka Mar 03 '21 at 10:03
  • Sure @Micka, The new_k = [[494.57725099 0. 960. ] [ 0. 497.13982742 540. ] [ 0. 0. 1. ]] and the new_size = 4k resolution = (3840, 2160) the image formed just shows the extension of the bottom right corner – amar adak Mar 03 '21 at 10:22
  • the new K should have the last column being the center of the new image dimensions. Can you SET the new size in estimateNewCameraMatrixForUndistortRectify with the `new_size` parameter? See https://docs.opencv.org/3.4/db/d58/group__calib3d__fisheye.html#ga384940fdf04c03e362e94b6eb9b673c9 – Micka Mar 03 '21 at 10:35
  • ok, just checked it in C++. I dont know why your new_K isn't correclty set. The new_size only has an influence to the resolution, not to the placing. If you find your result centered, but cropped, you'll have to adjust the `fov_scale = 1.0 ` parameter of `estimateNewCameraMatrixForUndistortRectify`. I get a good result for your image for fov_scale=1.8 – Micka Mar 03 '21 at 11:16
  • ok, double checked... the new_Size parameter HAS an effect, but only if it's different between estimateNewCameraMatrixForUndistortRectify and initUndistortRectifyMap. So set it to anything you like to in BOTH functions and adjust the fov_scale and your problem should be solved! – Micka Mar 03 '21 at 11:19
  • 1
    Hi @Micka, Thank you so much for your prompt responses. That did the trick. Really appreciate all your help in this. :) `initUndistortRectifyMap` did not have the `fov_scale`, but setting that on `estimateNewCameraMatrixForUndistortRectify` does the job for me. Thanks again :) – amar adak Mar 03 '21 at 12:05

0 Answers0