1

I have 2 grid pattern images.

image1: reference image

image2: sensed image

I want to find the translation and rotation between these 2 images such that the overlap area is maximizing.

I believe this question is related to image registration and I followed the example from this website

https://www.sicara.ai/blog/2019-07-16-image-registration-deep-learning

but the result is not satisfactory

Here is my code

import numpy as np
import cv2

img1 = cv2.imread('image1.png', cv2.IMREAD_GRAYSCALE)
img2 = cv2.imread('image2.png', cv2.IMREAD_GRAYSCALE)

akaze = cv2.AKAZE_create()
kp1, des1 = akaze.detectAndCompute(img1, None)
kp2, des2 = akaze.detectAndCompute(img2, None)

bf = cv2.BFMatcher()
matches = bf.knnMatch(des1, des2, k=2)

good_matches = []
for m, n in matches:
    if m.distance < 0.75*n.distance:
        good_matches.append([m])

output = cv2.drawMatchesKnn(img1, kp1, img2, kp2, good_matches, None, flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)
cv2.imwrite('output.png', output)

Here is the output output.png

Any ideas or suggestions would be great, thank you!

Ken H
  • 11
  • 2
  • Are you sure that it is only 2d rotation and translation or do you need s full perspective homography? – Micka Dec 09 '21 at 07:22
  • 1
    I would use a RANSAC based approach and try to find the transformation with most inliers. Similar to https://stackoverflow.com/a/20975486/2393191 but can probably be optimized a lot to choosegopd matching candidates. – Micka Dec 09 '21 at 07:41
  • Quite challenging. I would start by processing the sensed image to find, for each dot, the four nearest neighbors in the N, W, S, O sectors. This will give you a grid-like graph, from which you could extract long paths that are approximately straight. Then you can put them in correspondence with rows of the ideal grid, and select the correspondence that gives the best matching. (Can be RANSAC as in the comment above, but performed with whole paths.) –  Dec 09 '21 at 07:47
  • You have to definitely think about a good feature detector and descriptor. I don't believe this will work with SIFT or AKAZE because of the repeated patterns with similar distances. I think there are methods that use the frequency domain. I would recommend these methods for images like that. [Example](https://stackoverflow.com/questions/32664481/matlab-template-matching-using-fft) – Grillteller Dec 09 '21 at 08:19
  • you should *expect* feature matching to fail if your data is hundreds of points that all look the same. they are indistinguishable, hence feature matching will pick the "best" match basically from nothing but sensor noise. RANSAC (for feature matching and homography) operates after the matching, so that doesn't help either. -- best you can hope to do is Iterative Closest Point (ICP) or other algorithms on **point clouds** (2d is still a cloud) -- even then, expect unresolvable ambiguity. your source pattern has no local structure like "structured light" would require. – Christoph Rackwitz Dec 09 '21 at 13:25

0 Answers0