1

This is a follow-up to another stack overflow question, here: 3D Correspondences from fundamental matrix

Just like in that question, I am trying to get a camera matrix from a fundamental matrix, the ultimate goal being 3d reconstruction from 2d points. The answer given there is good, and correct. I just don't understand it. It says, quote, "If you have access to Hartley and Zisserman's textbook, you can check section 9.5.3 where you will find what you need." He also provides a link to source code.

Now, here's what section 9.5.3 of the book, among other things, says:

Result 9.12. A non-zero matrix F is the fundamental matrix corresponding to a pair of camera matrices P and P if and only if PTFP is skew-symmetric.

That, to me, is gibberish. (I looked up skew-symmetric - it means the inverse is its negative. I have no idea how that is relevant to anything.) Now, here is the source code given (source):

[U,S,V] = svd(F);
e = U(:,3);
P = [-vgg_contreps(e)*F e];

This is also a mystery.

So what I want to know is, how does one explain the other? Getting that code from that statement seems like black magic. How would I, or anyone, figure out that "A non-zero matrix F is the fundamental matrix corresponding to a pair of camera matrices P and P if and only if PTFP is skew-symmetric." means what the code is telling you to do, which is basically 'Take the singular value decomposition. Take the first matrix. Take the third column of that. Perform some weird re-arrangment of its values. That's your answer.' How would I have come up with this code on my own?

Can someone explain to me the section 9.5.3 and this code in plain English?

john k
  • 6,268
  • 4
  • 55
  • 59

1 Answers1

2

Aha, that "PTFP" is actually something I have also wondered about and could not find the answer in literature. However, this is what I figured out:

The 4x4 skew-symmetric matrix you are mentioning is not just any matrix. It is actually the dual Plücker Matrix of the baseline (see also https://en.wikipedia.org/wiki/Pl%C3%BCcker_matrix). In other words, it only gives you the line on which the camera centers are located, which is not useful for reconstruction tasks as such.

The condition you mention is identical to the more popularized fact that the fundamental matrix for the view 1 & 0 is the negative transpose of the fundamental matrix for the views 0 & 1 (using MATLAB/Octave syntax here)

Consider first that the fundamental matrix maps a point x0 in one image to line l1 in the other

l1=F*x0

Next, consider that the transpose of the projection matrix back-projects a lines l1 in the image to a plane E in space

E=P1'*l1

(I find this beautifully simple and understated in most geometry / computer vision classes)

Now, I will use a geometric argument: Two lines are corresponding epipolar lines iff they correspond to the same epipolar plane i.e. the back-projection of either line gives the same epipolar plane. Algebraically:

E=P0'*l0
E=P1'*l1

thus (the important equation)

P0'*l0=P1'*l1

Now we are almost there. Let's assume we have a 3D point X and its two projections

x0=P0*X
x1=P1*X

and the epipolar lines

l1=F*x0
l0=-F'*x1

We can just put that into the important equation and we have for all X 

P0'*-F'*P1*X=P1'*F*P0*X

and finally

P0'*-F'*P1=P1'*F*P0

As you can see, the left-hand-side is the negative transpose of the right-hand-side. So this matrix is a skew symmetric 4x4 matrix.

I also published these thoughts in Section II B (towards the end of the paragraph) in the following paper. It should also explain why this matrix is a representation of the baseline.

Aichert, André, et al. "Epipolar consistency in transmission imaging."
IEEE transactions on medical imaging 34.11 (2015): 2205-2219.
https://www.ncbi.nlm.nih.gov/pubmed/25915956

Final note to @john ktejik : skew-symmetry means that a matrix is identical to its negative transpose (NOT inverse transpose)

André Aichert
  • 313
  • 2
  • 11