4

I am developing a pose estimation system from a planar marker (using Matlab). In order to do that, I detect a rectangle in an image captured with a webcam, get the coordinates of the 4 corner points and compute the homography between these corner points in homogeneous coordinates, e.g.

     58     46     75     90
 M = 67    108    133     89
      1      1      1      1

where the first row are the vertical and the second row are the horizontal coordinates.

I compute the homography with DLT (using several different homography calculation functions I've found on the web, as well as Matlab's cp2tform, which ALL give the same results) between these points and the reference points as I know that the marker is a square,

     1     1   100   100
 m = 1   100   100     1
     1     1     1     1

[Edit: They both are sorted in counter-clockwise order, so I make sure they match.]

I then plot the reprojection of these reference points

 m* = H*m

back into the webcam image in order to see how well the homography is fitting.

The results are fine AS LONG as I only rotate (i.e. holding it in front of the webcam and manually tilting it) the marker around the z-axis (=the norm vector of the marker); the reprojected points are projected almost exactly onto the earlier detected marker corner points and the decomposed z-axis angle is computed just fine.

However, if I rotate the marker around the x- and/or y-axis, the reprojected points are increasingly off by quite a bit. What I then realized is that the calculated homography matrix H is almost an affine one, e.g. H=

    0.2339   -0.0967   57.8362
H = 0.1339    0.4714   66.3639
   -0.0010    0.0005    1.0000

(elements h31 and h32 are almost zero no matter how much I tilt the marker) which can be confirmed by looking at the reprojected points which always look like the result of an affine transformation, not a projective one. Unsurprisingly the decomposed angles for x- and y-axis are almost zero/equal to zero.

Obviously the functions I found on the web cannot all get the homography calculation wrong, neither can Matlab's cp2tform, but unfortunately I do not see or understand what my mistake is. It must be in the usage of the pixel coordinates but as looking at many homography explanations and searching for 'homography estimation result affine' did not yield any results, I'd be very glad if someone could point me in the right direction.

Thanks.

Garp
  • 89
  • 6
  • Since I didn't see, here is the dummy question: are you taking care to match the corners you obtain with the corners you want to map them to ? – mmgp Jan 20 '13 at 22:06
  • I forgot to add that: yes, I do (and if I didn't, I guess the reprojection wouldn't work just fine for z-axis rotations). – Garp Jan 20 '13 at 22:13
  • Sorting in ccw order doesn't guarantee it, except if you are rotating by considering one point that you are sure that will be mapped to the correct one -- and that correspondence is always maintained. – mmgp Jan 20 '13 at 22:21
  • Once I have a working homography, I will perform an orthogonal transformation in order to look at the marker's content and resolve ambiguities, but that will only help to determine if the marker has been tilted by 0° or 90° etc., not with the other angles. Correct me if I misunderstood your point. – Garp Jan 21 '13 at 00:27
  • 'Orthogonal transformation' is misleading of course, I meant I'm doing a projective transformation with H_inv in order to get an orthogonal view at the marker. – Garp Jan 21 '13 at 00:40
  • can you post an example of 4 points for which you cannot find a correct transform? Post their coordinates and the transform you get returned. – Hammer Jan 21 '13 at 16:36
  • Well, the ones I posted in the original question are such coordinates along with their homography, but here's a more extreme example. Points(y,x): (101,47), (81, 85), (191,92), (187,51) => H = [0.7984, -0.3785, 100.3227 ; 0.0217, 0.1971, 46.6613 ; -0.0004, -0.0022, 1.000]. Sorry, formatting in a comment is not ideal. But as you can see, h31 and h32 are almost zero again while the marker was heavily tilted along the y-axis. – Garp Jan 22 '13 at 01:25
  • @Garp with those points here is the representation I get: http://i.imgur.com/cXkfwua.png (red rectangle is the one you provided as the perfect one in the question). Here is the result I get after performing a perspective transformation: http://i.imgur.com/2T9VPb6.png, which looks good enough to me. I've used a very simple method for finding the 8 coefficients, and the ones I got are `[ 1.97135227e-01 2.17285749e-02 4.66613256e+01 -3.78489925e-01 7.98373392e-01 1.00322651e+02 -2.18499636e-03 -3.64166060e-04]`, which are pretty much similar to yours but in some different order. – mmgp Jan 22 '13 at 03:10
  • Thank you very much for finally something correct to compare my values to. It turned out that for my vectors, I was still using the scheme [y,x,z] instead of [x,y,z] which - apparently - the DLT algorithms need. Quite strange, that it worked pretty correctly in spite of that. I still do not get any angles unequal to 0 for anything besides the z-axis, but the Homography is correct now, so I must be doing something wrong elsewhere. Thanks again. – Garp Jan 22 '13 at 15:14
  • Sorry, it seems I cannot even upvote your comment due to no reputation. – Garp Jan 22 '13 at 16:09
  • @Garp that is totally fine, enjoy your mappings :) – mmgp Jan 22 '13 at 18:40

1 Answers1

0

As I could see from the values @mmpg provided in the comments, my y- and x-axis were still switched to match Matlab's coordinate system for images (y first, x second) which turned out to be the problem. I didn't think of such a fundamental issue since the Homography I got was quite close to the correct one in a big range of angles.

Garp
  • 89
  • 6