4

Background:

Assuming there are two shots for the same scene from two different perspective. Applying a registration algorithm on them will result in Homography Matrix that represents the relation between them. By warping one of them using this Homography Matrix will (theoretically) result in two identical images (if the non-shared area is ignored).

Since no perfection is exist, the two images may not be absolutely identical, we may find some differences between them and this differences can be shown obviously while subtracting them.

Example:

Scene 1 Scene 2

Scene2 Warped to Scene 1

AbsDiff(Scene1,Scene2 Warped to Scene 1)

Furthermore, the lighting condition may results in huge difference while subtracting.

Problem:

I am looking for a metric that I can evaluate the accuracy of the registration process. This metric should be:

  1. Normalized: 0->1 measurement which does not relate to the image type (natural scene, text, human...). For example, if two totally different registration process on totally different pair of photos have the same confidence, let us say 0.5, this means that the same good (or bad) registeration happened. This should applied even one of the pair is for very details-reach photos and the other of white background with "Hello" in black written.

  2. Distinguishing between miss-registration accuracy and different lighting conditions: Although there is many way to eliminate this difference and make the two images look approximately the same, I am looking of measurement that does not count them rather than fixing them (performance issue).

One of the first thing that came in mind is to sum the absolute differences of the two images. However, this will result in a number that represent the error. This number has no meaning when you want to compare it to another registration process because another images with better registration but more details may give a bigger error rather than a smaller one.

Sorry for the long post. I am glad to provide any further information and collaborating in finding the solution.

P.S. Using OpenCV is acceptable and preferable.

Humam Helfawi
  • 19,566
  • 15
  • 85
  • 160

2 Answers2

2

You can always use invariant (lighting/scale/rotation) features in both images. For example SIFT features.

When you match these using typical ratio (between nearest and next nearest), you'll have a large set of matches. You can calculate the homography using your method, or using RANSAC on these matches. In any case, for any homography candidate, you can calculate the number of feature matches (out of all), which agree with the model. The number divided by the total matches number gives you a metric of 0-1 as to the quality of the model.

If you use RANSAC using the matches to calculate the homography, the quality metric is already built in.

Photon
  • 3,182
  • 1
  • 15
  • 16
  • Thanks. That is true but not normalized..and acutely that is what I am doing right now. although I am positive that it is between 0 and 1 but it much different between pair and another – Humam Helfawi Jan 08 '16 at 12:47
  • furthermore, you may got a perfect registration from just 4 points if they are 100% accurate. In same time this measurement will be so low if it is just 4 out of 1000 feature points for example – Humam Helfawi Jan 08 '16 at 12:48
  • It doesn't matter how many points are used to calculate the homography, you can still count how many points that are transformed using the homography produce the same coordinates as the matched feature. If the homography is correct, and given a scene that is not degenerate, you should get a high ratio. – Photon Jan 08 '16 at 12:57
  • yes I mean just 4 inliers with very strict threshold.. My points is you can't depend on the number of matching points to evaluate the homography matrix. – Humam Helfawi Jan 08 '16 at 13:03
  • Well, if your scene is such that only 4 point matches define the homography, and all other matches do not agree, you should probably not use a homography, since the depth differences are so great that it's not a good model. Homography works best when the scene is planar. – Photon Jan 08 '16 at 13:05
  • just a extreme case to make my point clear. it is planer in real – Humam Helfawi Jan 08 '16 at 13:07
1

This problem is given two images decide how misaligned they are.

Thats why we did the registration. The registration approach cannot answer itself how bad a job it did becasue if it knew it it would have done it.

Only in the absolute correct case do we know the result: 0

You want a deterministic answer? you add deterministic input.

a red square in a given fixed position which can be measured how rotated - translated-scaled it is. In the conditions of lab this can be achieved.

gpasch
  • 2,672
  • 3
  • 10
  • 12
  • I partily agree with you. But some times you now that registeration was not perfect but you have to stop due to perfotmance issue.. in this point how cab I tell in normlized way how much it was good or bad – Humam Helfawi Jan 09 '16 at 13:44