33

Is it possible to use OpenCV's solvePNP with an equirectangular image? I have an equirectangular image and I have four points in this image (red dots) and their pixel coordinates, and then I have 4 corresponding world points e.g. [(0, 0, 0), (2, 0, 0), (2, 10, 0), (0, 10, 0)] how can I estimate the camera pose?

enter image description here

I tried using OpenCVs solvePnp but that expect intrinsics for a Brown camera model so didn't work. Can this be done for a spherical camera?

nickponline
  • 25,354
  • 32
  • 99
  • 167
  • 3
    Maybe try to convert the `equirectangular image` into multiple subimages with corresponding pinhole camera models? Quick search, found this: [Example Equirectangular To Pinhole](https://boofcv.org/index.php?title=Example_Equirectangular_To_Pinhole). Also look at OpenMVG, there should be some useful information: [sphere camera model and solve pnp #1557](https://github.com/openMVG/openMVG/issues/1557), [Spherical images to pinhole images conversion #621](https://github.com/openMVG/openMVG/issues/621). – Catree Feb 23 '20 at 13:39
  • Have you solved it already? can you please share your solution if you did? – Jenia Golbstein Aug 19 '21 at 21:04
  • Solving directly would need to account for the spherical image formation, not pinhole - so PnP directly won't work, unless you do as @Catree wrote. – Yuri Feldman Sep 11 '21 at 12:27
  • define an imaginary pinhole camera and use it as a stand-in. transform points into its frame. it's a hack, yes, because this only works as long as your points fit well within 180 degrees of view. – Christoph Rackwitz Jan 07 '22 at 12:17

2 Answers2

1

At first, you should compute the intrinsic parameters of the camera, that is the focal length, the optical center and the distortion parameters, so afterwards you can compute the pose of your camera. Since your image seems to be somehow wide lens, you should use cv.calibrateCamera() as described here. You should enable the CV_CALIB_RATIONAL_MODEL flag at the cv.calibrateCamera() method, which computes for distortion parameters since your image has big distortions.

Now the tricky part In order to obtain high-quality undistorted images, after you have calibrated your camera, you should use a chess pattern that covers your entire image, in order for the distortion parameters to be computed accurately, since the biggest radial distortions reside on the edge of the image. Also, in order to compute accurately the solvePnp parameters, you should pick your red dots all over your image.

Karantai
  • 73
  • 8
0

Use the solvePnPRansac and the OpenCV documentation for camera 3D reconstruction.

Cătălin George Feștilă
  • 1,364
  • 27
  • 48