python cv2 flann match more accurate results

Question

I am attempting to match Clash of Clans "buildings" from a screenshot.

Considering some of the buildings move around, I figured a straight matchTemplate wouldn't work (please correct me if I am wrong).

Consider the following image:

It won't work when used against the following:

As you can tell, the images are indeed there, but the Cannon is facing a different direction.

In another example, I am able to get a single match of the air defence which is identical in a few different spots:

Here is the code I am working with:

import numpy as np
import cv2
from matplotlib import pyplot as plt



imageToMatch=''
trainImage=''

MIN_MATCH_COUNT = 4

img1 = cv2.imread(imageToMatch)
img2 = cv2.imread(trainImage)  # trainImage

# Initiate SIFT detector
sift = cv2.xfeatures2d.SIFT_create()

# find the keypoints and descriptors with SIFT
kp1, des1 = sift.detectAndCompute(img1, None)
kp2, des2 = sift.detectAndCompute(img2, None)

FLANN_INDEX_KDTREE = 0
index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5)    
search_params = dict(checks=50)


flann = cv2.FlannBasedMatcher(index_params, search_params)

matches = flann.knnMatch(des1, des2, k=2)

# store all the good matches as per Lowe's ratio test.
good = []
#print(matches)
for m, n in matches:
    if m.distance < 0.7 * n.distance:
        #print(m.distance)
        #print(0.69 * n.distance)

        good.append(m)


if len(good) > MIN_MATCH_COUNT:
    src_pts = np.float32([kp1[m.queryIdx].pt for m in good]).reshape(-1, 1, 2)
    dst_pts = np.float32([kp2[m.trainIdx].pt for m in good]).reshape(-1, 1, 2)

    M, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC, 5.0)
    matchesMask = mask.ravel().tolist()

    h, w = img1.shape
    pts = np.float32([[0, 0], [0, h - 1], [w - 1, h - 1], [w - 1, 0]]).reshape(-1, 1, 2)
    dst = cv2.perspectiveTransform(pts, M)

    img2 = cv2.polylines(img2, [np.int32(dst)], True, 255, 3, cv2.LINE_AA)

else:
    print("Not enough matches are found - %d/%d" % (len(good), MIN_MATCH_COUNT))
    matchesMask = None

draw_params = dict(matchColor=(0, 255, 0),  # draw matches in green color
                   singlePointColor=None,
                   matchesMask=matchesMask,  # draw only inliers
                   flags=2)

img3 = cv2.drawMatches(img1, kp1, img2, kp2, good, None, **draw_params)

plt.imshow(img3, 'gray'), plt.show()

My Questions are:

Is there a way to match the Cannon Image to all of the Cannon images shown in the example Layout?
Is there a way to ensure that objects which do not rotate are also matched?

Any help or direction would be greatly appreciated!

My Env: Python 3.6, OpenCV 3.3.0, OSX.

How many possible orientations are there? If there's a fixed amount (like 4) then you can just use four different templates. If there's a higher amount though, you might want to try a few different options. Continuing with template matching, you could just use a fixed number of orientations (e.g. 4) and you'll likely get "close enough" for detection. But there may be icons which are similar to it and give you false detections. I'd try that first though as it's a simple solution. — alkasm, Oct 03 '17 at 06:30
You can create a threshold by matching the closest orientation you have in your template set to a canon which is oriented the furthest possible away from it (while still being more closely aligned with that template), and check the result from template matching. Then just grow it a bit and try it out around the map. — alkasm, Oct 03 '17 at 06:34

python cv2 flann match more accurate results

0 Answers0