The coordinates of the reconstructed 3D points are different after the virtual camera intrinsic K has also changed proportionally after image resize?

Question

As far as I know, after image resize, the corresponding intrinsic parameter K also changes proportionally, but why the coordinates of the 3D reconstruction of the same point are not the same?

The following python program is a simple experiment, the original image size is aaa , after resize it becomes aaa , the intrinsic parameter K1 corresponds to the original image, the intrinsic parameter K2 corresponds to the resize, RT1, RT2 are the extrinsic projection matrix of the camera (should remain unchanged?,[R,T], aaa size), without considering the effects of camera skew factor and distortions,why is there a difference in the reconstructed 3D points?

import cv2
import numpy as np

fx = 1040
fy = 1040
cx = 1920 / 2
cy = 1080 / 2

K1 = np.array([[fx, 0, cx],
               [0, fy, cy],
               [0, 0, 1]])

RT1 = np.array([[1, 0, 0, 4],
                [0, 1, 0, 5],
                [0, 0, 1, 6]])  # just random set

theta = np.pi / 6
RT2 = np.array([[np.cos(theta), -np.sin(theta), 0, 40],
                [np.sin(theta), np.cos(theta), 0, 50],
                [0, 0, 1, 60]])  # just random set

p1 = np.matmul(K1, RT1)  # extrinsic projection matrix
p2 = np.matmul(K1, RT2)  # extrinsic projection matrix

pt1 = np.array([100.0, 200.0])
pt2 = np.array([300.0, 400.0])

point3d1 = cv2.triangulatePoints(p1, p2, pt1, pt2)
# Remember to divide out the 4th row. Make it homogeneous
point3d1 = point3d1 / point3d1[3]
print(point3d1)

[[-260.07160113]
 [ -27.39546108]
 [ 273.95189881]
 [   1.        ]]

then resize image to test recontruct 3D point, see if it is numerical equal.

rx = 640.0 / 1920.0
ry = 480.0 / 1080.0
fx = fx * rx
fy = fy * ry
cx = cx * rx
cy = cy * ry
K2 = np.array([[fx, 0, cx],
               [0, fy, cy],
               [0, 0, 1]])
p1 = np.matmul(K2, RT1)
p2 = np.matmul(K2, RT2)
pt1 = np.array([pt1[0] * rx, pt1[1] * ry])
pt2 = np.array([pt2[0] * rx, pt2[1] * ry])
point3d2 = cv2.triangulatePoints(p1, p2, pt1, pt2)
# Remember to divide out the 4th row. Make it homogeneous
point3d2 = point3d2 / point3d2[3]
print(point3d2)

[[-193.03965985]
 [ -26.72133393]
 [ 189.12512305]
 [   1.        ]]

you see, point3d1 and point3d2 is not same,why?

xing cui · Accepted Answer · 2022-12-20T02:27:52.580

After careful consideration, I was lucky to get a more plausible explanation, which I now state as follows to help others. In a short conclusion:

Image scaling must specify a uniform (fx=fy) scaling factor in order to derive the correct intrinsic parameter K, otherwise inconsistencies in the x,y axis focal lengths with respect to the original image directly lead to deviations in the calculated 3D points!

Returning to the problem at the beginning, the given image size is 1080×1920, and its focal length is 1040 pixels, i.e. fx=fy=1040, because by definition fx=f/dx,fy=f/dy, where dx, dy are the number of pixels per unit length, and f is the actual physical size of the focal length; thus the a priori dx=dy can be introduced, which is constant This "convention" should also be followed for later image scaling. Imagine if the scaled image fx,fy were obtained in different proportions, dx,dy would not be the same, causing distortion of the image, and in addition, according to the external projection matrix P = K*[R,t], fx,fy in K would vary disproportionately leading to a deviation in the calculated P!

BTW, Similarly, I put the reference answer to the experiment done by matlab at this link.

The coordinates of the reconstructed 3D points are different after the virtual camera intrinsic K has also changed proportionally after image resize?

1 Answers1