2

I'm trying to realize a virtual Pan-Tilt-Zoom (PTZ) camera, based on data from physical fisheye camera (180 degrees FOV).

In my opinion I have to realize the next sequence.

  1. Get the coordinates of center of fisheye circle in coordinates of fisheye sensor matrix.
  2. Get radius of fisheye circle in the same coordinate system.
  3. Generate a sphere equation, which has the same center and radius as flat fisheye image on flat camera sensor.
  4. Project all colored points from flat image to upper hemisphere.
  5. Choose angles in X Y plane and X Z plane to describe the direction of view of virtual PTZ.
  6. Choose view angle and mark it with circle around virtual PTZ view vector which will be painted on the surface of hemisphere.
  7. Generate the plane equation which intersection with hemisphere will be a circle around the direction of view.
  8. Move all colored points from the circle to the plane of circle around the direction of view, using direction from hemisphere edge to hemisphere center for projection.
  9. Paint all unpainted points inside circle of projection, using interpolation (realized in cv::remap).

In my opinion the most important step is to rise colored points from flat image to 3-D hemisphere.

My question is: Will it be correct to just set Z-coordinate to all colored points of flat image in accordance with hemisphere equation, to rise points from image plane to hemisphere surface?

Initial image from fisheye camera

enter image description here

fisheye image after projection to hemisphere

Virtual PTZ with coordinates

Projection plane and view angle of virtual PTZ

Desirable result

VideoProcessingResearcher
  • 546
  • 3
  • 16
  • This sounds far too complex. I've looked at this question a few times, and I still don't get exactly what you are proposing. But the one part I do think you are planning is to copy some pixels exactly, and interpolate others. – MSalters Apr 12 '18 at 07:04
  • Thank you for reply. To make my question more clear, I added a picture with an example of desirable result. In top-left corner you can see image from fisheye camera with red, blue and green regions. This circle in top-left corner is a projection of hemisphere to plane of camera sensor. Red, green and blue regions are projections of rectangular regions, wrapped around of 3D hemisphere. In top-right, bottom-left, and bottom-right corners you can see this regions in dewarped form. This is my final task – VideoProcessingResearcher Apr 12 '18 at 07:58
  • The final result is pretty obvious for someone familiar with FishEye camera's. It's your proposed process that's not at all obvious. Did you google "OpenCV fisheye" ? – MSalters Apr 12 '18 at 08:25
  • Of course I did :) There are a lot of solutions for "Virtual PTZ via fisheye", but they all are commercial products :( – VideoProcessingResearcher Apr 12 '18 at 08:44
  • And yes, I used cv::fisheye module. Unfortunately cv::fisheye can't undistort 100% of initial fisheye image surface if camera has 180 degrees FOV or more. More detailed explanation I placed [HERE](https://stackoverflow.com/questions/48990136/opencv-undistorts-only-a-central-part-of-fisheye-image?noredirect=1&lq=1). As you can see on my last picture some commercial products make possible to undistort 100% of fisheye image surface. – VideoProcessingResearcher Apr 13 '18 at 07:08
  • Hi! Have you achieved some results for this time? – Pavlo Sharhan Jul 30 '21 at 12:44

0 Answers0