How do you redistort a point?

Question

I have a distorted image on which I use Cv2.Undistort() which straightens it out. After this I find some points in the image. I need to get the location of those points in the distorted image.

I have tried Cv2.ProjectPoints() but failed to get appropriate coordinates. They were outside the bounds of the image.

This is how I went about doing this:

List<float> arr = new List<float>();
foreach (var i in points)
{
    arr.Add(i.X);
    arr.Add(i.Y);
    arr.Add(0);
}
Mat pointArr = new Mat(3, points.Count, MatType.CV_32FC1, arr.ToArray());
float[] rotArr = {  1, 0, 0,
                    0, 1, 0,
                    0, 0, 1};

float[] transArr = { 0, 0, 0 };

Mat rot = new Mat(3, 1, MatType.CV_32F, rotArr);
Mat trans = new Mat(3, 1, MatType.CV_32F, transArr);


Cv2.ProjectPoints(pointArr, rot, trans, camMatrix, dist, outputPoints);
List<Point2f> point2Fs = new List<Point2f>();
var lngt = outputPoints.Rows * outputPoints.Cols * outputPoints.Channels();
var matarr = new List<float>();
for (int i = 0; i < outputPoints.Rows; i++)
{
    point2Fs.Add(new Point2f(outputPoints.ExtractChannel(0).At<float>(0, i), outputPoints.ExtractChannel(1).At<float>(0, i)));
}

points - the points in the undistorted image I want to find in the original

Any suggestions?

Thanks!

Does this answer your question? [Reverse of OpenCV projectPoints](https://stackoverflow.com/questions/34140150/reverse-of-opencv-projectpoints) — NetMage, Aug 13 '20 at 00:04
No. I do not have the real world coordinates of the points I am trying to find. I have pixel coordinates of the points in undistorted image. — Justas Puodžius, Aug 13 '20 at 05:10

score 2 · Answer 1 · edited Aug 14 '20 at 07:20

Unfortunately, OpenCV doesn't provide a function to do directly what you want, which is to apply the distortion coefficients to undistorted 2D points. You can:

Option 1

You are on the right track using projectPoints(). The trick to understand how this can work is that a point in the undistorted image "lives" in the image plane, which looks like this (picture from https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html):

Where the image plane is located at f (focal distance) from the origin and x and y coordinates are relative to c_x and c_y.

What you have to do is to build your 3D points (x,y,z) from the undistorted 2D points (x_u, y_u) like this:

x = (x_u - c_x) / f_x
y = (y_u - c_y) / f_y
z = 1

Instead of using (x_u, y_u, 0) as in your first foreach loop.

Then call projectPoints().

You are correctly using an identity rotation and 0 translation. f_x, f_y, c_x and c_y, are in cameraMatrix:

|f_x  0  c_x|
| 0  f_y c_y|
| 0   0   1 |

Option 2:

You can directly apply the equations for the distortion (https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html):

In the first step, you replace x' and y' by:

x' = (x_u - c_x) / f_x
y' = (y_u - c_y) / f_y

Then apply the rest of the equations. This is what projectPoints() does, without the rotation/translation part.

Thanks for answering! I have tried the first option you suggested but got unacceptable values such as outside of the image bounds or negative values. — Justas Puodžius, Aug 17 '20 at 09:31

score 1 · Accepted Answer · answered Aug 18 '20 at 05:51

I ended up using 2nd option proposed by Milo. I have rewritten the project point method without the rotation and translation part and starting the equation path from x' and y', where:

x' = (x_u - c_x) / f_x
y' = (y_u - c_y) / f_y

Then I applied the rest of the provided equations.

Here is the source code:

Point2f DistortPoint(Point2f point)
{
   var tempPt = new Point2f((point.X-intr.cx)/intr.fx, (point.Y-intr.cy)/intr.fy);
   var r2 = Math.Pow(tempPt.X, 2) + Math.Pow(tempPt.Y, 2);
   var xtmp = tempPt.X * ((1 + intr.k1 * r2 + intr.k2 * Math.Pow(r2, 2) + intr.k3 * Math.Pow(r2, 3)) /
                (1 + intr.k4 * r2 + intr.k5 * Math.Pow(r2, 2) + intr.k6 * Math.Pow(r2, 3))) + 2 * intr.p1 * tempPt.X * tempPt.Y + intr.p2 * (r2 + 2 * Math.Pow(tempPt.X, 2));
   var ytmp = tempPt.Y * ((1 + intr.k1 * r2 + intr.k2 * Math.Pow(r2, 2) + intr.k3 * Math.Pow(r2, 3)) /
                (1 + intr.k4 * r2 + intr.k5 * Math.Pow(r2, 2) + intr.k6 * Math.Pow(r2, 3))) + 2 * intr.p2 * tempPt.X * tempPt.Y + intr.p1 * (r2 + 2 * Math.Pow(tempPt.Y, 2));
   return new Point2f((float)(intr.fx * xtmp + intr.cx), (float)(intr.fy * ytmp + intr.cy));
}

score 0 · Answer 3 · answered Dec 27 '20 at 20:47

I did this by use indexing mapx and mapy returned from cv2.initUndistortRectifyMap().

 x_distored = mapx[y_rectified, x_rectified]
 y_distored = mapy[y_rectified, x_rectified]

The limitation of this method is that the indices must be within the ranges.

I aslo have problem to use Cv2.ProjectPoints() to project rectified locations to distorted, even after normalizing x_rectified and y_rectified. My problem is unable to convert R and T (I got them from cv2.stereoRectify(). They are not all zeros.) to rvec and tvec required by Cv2.ProjectPoints() as input. If anyone has successful experience please share it, thanks!

How do you redistort a point?

3 Answers3

Linked