3

I try to calibrate camera with ultra wide angle lens > 180 degrees. I use OpenCV 3.2 and camera calibration code from opencv/samples/cpp/tutorial_code/calib3d/camera_calibration, as a result I get image similar to this one from this post. I found numerous movies on youtube.com on which people show how they calibrate fisheye cameras, for example this video.

I know that calibration of ultra wide lenses cameras could be tricky. I also found that there are another then standard model for fisheye camera calibrations in openCV contrib

Is it possible to use camera calibration model from standard OpenCV to calibrate such a wide angle camera or should I use another model e.g., from openCV contrib?

trojek
  • 3,078
  • 2
  • 29
  • 52

3 Answers3

7

I was working on the same problem a few weeks ago, what I found was that the standard model did not work with my fisheye images ( >180 FOV). Having said that, I did find a pull request on Github which addresses just this issue - https://github.com/opencv/opencv/pull/6801.

I eventually gave up on the standard model and just used the Omnidirectional model - http://docs.opencv.org/trunk/dd/d12/tutorial_omnidir_calib_main.html which worked like a charm.

Just as a side note, whilst I was researching the topic of distortion correction I came across a common misconception that fisheye images resulting from fisheye lenses are "distorted", indeed they might be distorted but the resulting images are simply a projection. A fisheye like other projections is one of many ways of mapping a 3D world onto a 2D plane. It doesn't help that majority of the related literature refer to it as "distortion". Fisheye lenses are deliberately designed to produce images with bent lines, when these lines would have been straight with standard (i.e. rectilinear) lenses. The word "distortion" is reserved solely for geometric aberrations that negatively affect standard (i.e. rectilinear) lenses.

In my case, even though I was looking into distortion correction I didn't end up correcting for any distortion, I just simply did a fisheye to perspective projection - you can find more on this on here http://paulbourke.net/dome/fish2/

Also, in regards to capturing checkerboard images follow these recommendations - How to verify the correctness of calibration of a webcam?

Jack Gold
  • 191
  • 1
  • 9
  • I change fisheye.cpp and fisheye.hpp as in the pull request, then recompile opencv/samples/cpp/tutorial_code/calib3d/camera_calibration/camera_calibration.cpp but the effect is the same. Do I have to build whole OpenCV before compiling camera_calibration.cpp? – trojek Jun 07 '17 at 15:52
  • @trojek I didn't try to merge the pull request myself, but as far as I am aware you would need to recompile OpenCV from source and not just the `camera_calibration.cpp` (correct me if I am wrong). If you check the 'Files changed' section - https://github.com/opencv/opencv/pull/6801/files you will see that the two files changed by the pull request are `fisheye.cpp` and `fisheye.hpp`. Let us know how you get on. – Jack Gold Jun 08 '17 at 08:49
  • I changed the two files from pull request and I didn't work. I think that I should build OpenCV. – trojek Jun 08 '17 at 17:23
  • Yep, try building OpenCV from source with the files that were changed in the pull request and see whether that works, I would also recommend building OpenCV with the contrib library. I'd be interested to hear whether this works, please keep us posted. – Jack Gold Jun 09 '17 at 08:47
  • I try to build OpenCV after changing files from pull request but I got an error. I decided to use omni_calibration module. I calibrate camera but I stuck with a undistortion image. I found the function 'omnidir::undistortImage(distorted, undistorted, K, D, _xi, int flags, Knew, imageSize)' but I cannot find an explanation of parameters in documentation. Do you have code which undistort image and save it? – trojek Jun 09 '17 at 09:40
  • I'm little bit confused. What do I need in order to calibrate image? My file with camera parameters has few hunderts of lines. Do I need all of them or I need only two matrixes: camera_matrix and distortion_coefficients? – trojek Jun 09 '17 at 11:10
  • @trojek You don't calibrate the image, you are calibrating the camera, distortion is caused by the lens causing geometric aberrations in the resulting image. However, as I pointed out in my answer - distortion correction and projection correction are two different things. It might very well be the case that you do not need to calibrate the camera and handle distortion, you might just be after the perspective projection. It would be helpful to see how your image looks like for starters and to see what your desired goal is. You are right, to correct for distortion you require both matrices. – Jack Gold Jun 09 '17 at 13:48
  • of course, that I calibrate the camera, not an image - it was a mistake. I understand the difference between distortion projection and projection correction. I want to reach effect as on the images: http://paulbourke.net/dome/fish2/spherical0.jpg, http://paulbourke.net/dome/fish2/spherical1.jpg – trojek Jun 09 '17 at 14:01
  • In that case, you do not need to correct for distortion, if you read Paul Bourke's page you will see that he does not apply any distortion correction parameters to achieve the resulting image, he simply does a projection from fisheye to spherical. The issue is that OpenCV does not provide a simple way of projecting between different spaces. I did try using the Fisheye namespace but it did not work well for me so that is the reason I moved to the Omnidirectional namespace, I would recommend you do the same unless you can compile OpenCV with the code from the pull request. – Jack Gold Jun 09 '17 at 14:26
  • In regards to OpenCV not making it easy to project from fisheye to sepherical I mean that it doesn't allow the projection between the two spaces without calibration so you will need to calibrate the camera (checkerboard images) to obtain the camera and distortion matrices and use those for the `fisheye::undistortImage()` or `omnidir::undistortImage` even though they are not needed in your case. To test this theory you can use [PTGui](https://www.ptgui.com/) to convert your fisheye image to a spherical projection. – Jack Gold Jun 09 '17 at 14:31
  • Unfortunately, PTGui works on Windows and Mac. I have only Linux installed. I calibrate and undistort image using 'calibrateCamera' and 'undistort' respectively in Python, but the output image is cropped (a lot of data from is's edges are removed). Could you send me your code with omnidir module in C++? – trojek Jun 09 '17 at 16:07
  • The crop is natural, you're always going to lose some data when you undistort an image. However, I know what you mean by it being cropped extremely, what you need to do is change the `newCamerMatrix` change the `fx` and `fy` values - that will change the resulting focal length and give you a larger FOV, i.e. it will give you control over how much is cropped. In regards to my code for the omnidirectional model, I literally just followed this tutorial - http://docs.opencv.org/trunk/dd/d12/tutorial_omnidir_calib_main.html – Jack Gold Jun 09 '17 at 16:22
1

I solved this problem for a 180 degrees FOV fisheye camera via cv::fisheye module from Open CV 3.4.0. (C++, MS Windows).

I used cv::fisheye::calibrate to make K and D (camera matrix and radial distortion coeffitients matrix). Then I used cv::fisheye::initUndistortRectifyMap to produce maps for X and Y coordinates. And finally I used cv::remap to undistort image from fisheye camera via maps from initUndistortRectifyMap. This solution has some limitations (OpenCV dewarps only a central part of fisheye image. Edges are moved outside) You'll get 140-150 FOV as a result. Detailed explanation of this limitation I placed HERE. If 140-150 degrees FOV is suitable for you, you can use this module without any modifications

VideoProcessingResearcher
  • 546
  • 3
  • 16
0

There are specific calibration functions for fisheye calibrations: see http://docs.opencv.org/trunk/db/d58/group__calib3d__fisheye.html

Make sure you get images in the borders of the image with various poses and distances to the camera for a good calibration. Adapt the calibration example to use the fisheye functions.

TFreitas
  • 290
  • 1
  • 12
  • I took 25 pictures from a different perspective and I still have problems with calibration. When I use a camera with 150 FOV I didn't have any problems. – trojek Jun 07 '17 at 08:58
  • Have you visited the borders of the images with the chessboard? Have you used fisheye calib functions? Even on low FOV images sometimes is hard to get a good calibration. Sometimes only one or two images increase highly the error of the calibration. – TFreitas Jun 07 '17 at 09:02
  • I used fisheye function and I have photos from different perspectives. If you are sure that it is possible to calibrate camera using pure OpenCV calibration model I can send you my images. – trojek Jun 07 '17 at 10:43
  • I think Jack Gold probably gave you a better answer. I guess the lens I've worked had a lower FOV and didn't had that problem. – TFreitas Jun 07 '17 at 10:53
  • I'm almost sure that Jack Gold give me better answer but I still don't have time to verify it. – trojek Jun 07 '17 at 10:56
  • I use a separate step where i plot the detected chessboard and make sure cv agrees with me (t's sometimes off.) Deleting images where the chessboaard detection is off significantly improves performance! – Jon Watte Sep 29 '17 at 04:46