0

I have a set of images which all have 100 points arranged in the same pattern on each image. The points can be either circles or squares, and some images have a vertical and horizontal axis through the middle, or text around the outside.

I am trying to find a way to find this pattern on each image, then use the location of these to align the images so the points are always in the same location for some processing.

I've tried OpenCV's feature matching with a template (based off the test image). This seemed to be working at first when I was testing it on one image. I realised later it was picking up the slight variances in the shape of the points and matching these, not the location of the points. It doesn't work if I use any other images against the template.

So now I need some other method to detect the pattern in the images. Any ideas how I can do this? I'm thinking maybe some kind of contour detection, then processing to see if they match the pattern but not sure how do do this. Any help would be appreciated.

Edit to clarify: When the matching works I can use warpAffine to warp the image to match the template. It's matching up the points that I am struggling to do.

This is the template I made This is the template I made

Based off this image. Which the feature mapping works with Image that the template is found in

And other similar images look like this. Or various other formats, sometimes without the axis, or different text around them. Other images similar to this

These are the matches that the feature detector finds when using an image based on the template Matches against template-based image

These are the matches if another image is used. None of the points match to the correct locations Matches against a different image

ChrisHigs
  • 53
  • 1
  • 7
  • Can you give example images of the template and an image that it would belong inside? – alkasm Mar 14 '19 at 18:34
  • Hi. I've edit the original post to include some images. They're all eye test charts and I want to align them so the points are in the same location – ChrisHigs Mar 14 '19 at 20:19
  • Great, thanks for that. Now for the alignment, is there possible scale, rotation, skews from your template image? Or is there always just a translation between the template and the image? (specifically, I'm asking what possible deviations from the template are possible?) – alkasm Mar 14 '19 at 20:21
  • Yes, scales and slight rotations. Not sure about skew, I doubt it but possible – ChrisHigs Mar 14 '19 at 20:33
  • Part of the problem with your current approach is that the parts of the image that are empty are *also* features. "Big white area" is certainly a large feature of the image and the template but likely won't match up. You should look at masking the parts of the template that you don't want to match. A little bit of info and docs linked in my answer [here](https://stackoverflow.com/a/45460616/5087436). Alternatively, you can also try dense estimation with `findTransformECC()`; tutorial [here](https://www.learnopencv.com/image-alignment-ecc-in-opencv-c-python/). – alkasm Mar 14 '19 at 20:52
  • If you're not worried about skews, you can just find a Euclidean transform, which takes into account scale and rotations but not skews and perspective distortions. Once you get the transformation, you can map the image into the coordinates of your template; and you can use the coordinates of your template to find the things you're interested in on the image. You can do that warping with `warpAffine()` or manually with `remap()`. Conversely, you can use the transformation to map the areas of your template that you're interested in into the image coordinates and grab them from that. – alkasm Mar 14 '19 at 20:54
  • Thanks for the comments. I've added some clarification and more images to the post. I'm currently using feature detector/extractor to map the template. I'm then using warpAffine to warp the image and I have that working. But not when using a different image. – ChrisHigs Mar 14 '19 at 22:26
  • What would you suggest to find the Euclidian transform? Feature detector or something else? – ChrisHigs Mar 14 '19 at 22:27
  • The biggest problem you face with current methods is that basic feature matching doesn't really work since one small circle is similar to any of the other small circles. Really, you have a model of locations of these circles/squares, and you want to fit your image to that model via an image transformation. I'm not 100% what the easiest out-of-the-box way to do that with OpenCV is, but I might suggest the Lucas-Kanade implementation: https://docs.opencv.org/master/dc/d6b/group__video__track.html#ga473e4b886d0bcc6b65831eb88ed93323 – alkasm Mar 14 '19 at 22:34
  • This will allow you to roll your *own* feature detection algorithm---*you* can specify the feature points you're trying to track, and this algorithm will find where those feature points moved to. You can then find the transform that maps these points to each other, I believe. – alkasm Mar 14 '19 at 22:36
  • Thanks. I'll have a play around and see what I can work out – ChrisHigs Mar 15 '19 at 08:07

0 Answers0