2

Lets assume I have the following pinhole model: enter image description here

I know fx, fy, cx and cy from my camera matrix. And I know the pixels u and v on my plane and I also have the the extrinsics of my camera R and t. How can I calculate the ray of one pixel?

The initial idea would be to calculate the vextor from the camera center to the x,y coordinate on the image plane. But with this model i dont knwo the distance between image plane and camera center.

Gian Alessio
  • 95
  • 1
  • 9
  • have a look at: https://stackoverflow.com/questions/31265245/extracting-3d-coordinates-given-2d-image-points-depth-map-and-camera-calibratio/31266627#31266627 just let the "depth" value stay arbitrary – Micka Jul 05 '21 at 05:01
  • the distance between image plane and camera center is the focal length, btw. – Micka Jul 05 '21 at 05:04
  • yes I know. but in the concept of the camera matrix you have fx and fy, which are not the metrix focal lengths – Gian Alessio Jul 05 '21 at 11:00
  • if you need metruc information you h have to multiply by the size of the pixels (width/height) which you will get from reading the sensor/pixel size from the camera data sheet. – Micka Jul 05 '21 at 11:02
  • I do not have anything except for the camera matrix. Still struggling with the ray formulation. So in the thread they calculate 3D from pixel with: P3D.x = (x_d - cx_d) * depth(x_d,y_d) / fx_d P3D.y = (y_d - cy_d) * depth(x_d,y_d) / fy_d P3D.z = depth(x_d,y_d) In my case the depth parameter would be variable. Imagine I want to calculate the 3D-Coordinates for the intersection of the ray of a pixel with the ground plane so P3D.z=0. Looking at the equations for me it looks like the depth would be the same for every pixel of one image, i cannot see the link to the pixel. – Gian Alessio Jul 05 '21 at 13:24
  • the 3D coordinates after that formula are in camera coordinates, so 3D coordinates where the origin is the camera center. z=1 is the image plane. You will need extrinsics to go from camera coordinates to world coordinates, which is the first line of the second formula block (R and T applied). Make sure that the extrinsics use the same coordinate system as your ground plane. – Micka Jul 05 '21 at 13:40
  • Lets say the Extrinsic are given like this: Xw = R*Xc+t than: PXw = tx + lambda*(R*inv(K)*Xc+tx-tx) PYw = ty + lambda*(R*inv(K)*Xc+ty-ty) PZw = 0 = tz + lambda *R*(1+tz-tz) = tz+lambda*R -> lambda = -inv(R)*tz So lambda which would be my depth in camera coordinates is independent from the pixel location? – Gian Alessio Jul 05 '21 at 15:49
  • sorry, dont have the time to understand your formula. In the link, `x_d` and `y_d` are the pixels, so P3D depends on the pixel position and the depth. Let's say P3D = d*[pX,pY,1]. `R*P3D + T = d*[r11*pX + r12*pY + r13 + tX; r21*pX + r22*pY + r23 + tY; r31*pX + r32*pY + r33 + tZ] ` now you can ask for one single pixel, at what depth do you intersect with the global Z=0 plane. You obviously have a different ray for each pixel according to that formula. – Micka Jul 05 '21 at 16:01

1 Answers1

1

You can do this:

Image points: imgPts (2xN)
Camera matrix: M (3x3) = [fx 0 cx;0 fy cy;0 0 1];
Camera Center: camCenter = inv(Rt)*([0 0 0 1]');
Point at sensor: ptsCam = inv(M)*imgPts;
Points at world coordinates: pts = inv(Rt)*ptsCam; // Add one to the third coordinate, so ptsCam (3xN).
Direction vectors: v = pts - camCenter; v = v/norm(v); // normalize.

Output: pts and v.

Pedro77
  • 5,176
  • 7
  • 61
  • 91