Compute similarity of 2 binary images

Question

I have 2 binary images (black/white). They have the same content but might slightly differ in rotation/translation and scale of the content (text).

How do I get a simple measure for the similarity of 2 images in OpenCV?

The operation needs to be as fast as possible (live).

Examples:

A:

B:

Try using SIFT or SURF to find same points in the image. Then compute an affine transform that adjusts the images to one another and compute the sum of the difference magnitudes — meetaig, Oct 13 '16 at 09:19
Yes, but most of the functionality is already implemented in opencv. If you have knowledge about he displacement of the images relative to each other you can use that. But when you have a set of coordinates you can use `getAffineTransform` to directly obtain the transform you need ;) — meetaig, Oct 13 '16 at 09:30
Get the `minAreaRect` of both texts, warp one onto the other, compute the `absdiff`, sum up the differences. That would be a measure of _dissimilarity_. This would be fairly fast — Miki, Oct 13 '16 at 09:33
@Miki sounds logical. Although I haven't had the best luck with `minAreaRect` (inaccurate). Could you provide an example for "wrapping one onto the other" and comparing with `absdiff`? — , Oct 13 '16 at 09:37
Have a look at [this](http://stackoverflow.com/a/33884654/5008845) for warping. The colored dots would be your minAreaRect corners. And [this](http://stackoverflow.com/a/31258083/5008845) for absdiff. Use `sum(...)` to get the sum of all pixels in the resulting image — Miki, Oct 13 '16 at 09:41
You are looking for the Procrustes transform, or Procrustes analysis. It translates, reflects, rotates and scales the text, you can then calculate a dissimiliarity measure to compare the images. It's implemented in Matlab, I couldn't find it in OpenCV with a quick googling. — Tapio, Oct 13 '16 at 09:43
@Tapio that seems promising, but really slow. Any thought on this? If you can post an answer with the matlab code showing the results on this image, porting from matlab to opencv should be easy — Miki, Oct 13 '16 at 09:47

score 2 · Answer 1 · answered Oct 13 '16 at 10:53

2

You can use LogPolarFFT registration algorithm to register the images, then compare them using similarity check (PSNR or SSIM).

answered Oct 13 '16 at 10:53

Andrey Smorodov

10,649
2
35
42

score 1 · Answer 2 · answered Oct 13 '16 at 09:28

You need to remove the scale and rotation.

To remove the rotation, take pca which gives you the primary axis, and rotate both images so the primary axis is along the x. (Use a shear rotate). To remove the scale, you then either simply take a bounding box or, if there's a bit of noise in there, take area and scale one until it is equal. Just sample pixel centres to scale ( a bit icky, but it hard to scale a binary image nicely).

I should put some support in the binary image library for this. You might find the material helpful

http://malcolmmclean.github.io/binaryimagelibrary/

score 0 · Answer 3 · answered Oct 13 '16 at 11:12

The one way I can think of is to find keypoint using SIFT/SURF and then calculate Homography between two images and warp them according to the calculated Homography ( so as to fix rotation and Translation). Then you can simply calculate similarity in terms of SAD.

score 0 · Answer 4 · edited May 23 '17 at 12:00

Need to use rotation and scale invariant approach. Also need a threshold based area segmentation before feature extraction is needed. I suggest to follow below steps:

1/ Binary threshold & scan line algorithm can be used to segment specific text line area.

2/ After segmentation you should adjust the rotation using warpAffine transformation. See this example

3/ On adjusted image you can apply SIFT or BRISK or SURF features to get features

4/ Use template matching approach to match or generate similarity or distance score.

see following link for more detail:

scale and rotation Template matching

Compute similarity of 2 binary images

4 Answers4