Feature matching with flann in opencv

Question

I am working on an image search project for which i have defined/extracted the key point features using my own algorithm. Initially i extracted only single feature and tried to match using cv2.FlannBasedMatcher() and it worked fine which i have implemented as below:

Here vec is 2-d list of float values of shape (10, )
Ex:
[[0.80000000000000004, 0.69999999999999996, 0.59999999999999998, 0.44444444444444448, 0.25, 0.0, 0.5, 2.0, 0, 2.9999999999999996]

[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001]

[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001]

[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001]]

vec1 = extractFeature(img1)
vec2 = extractFeature(img2)

q1 = np.asarray(vec1, dtype=np.float32)
q2 = np.asarray(vec2, dtype=np.float32)

FLANN_INDEX_KDTREE = 0
index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)
search_params = dict(checks=50)   # or pass empty dictionary
flann = cv2.FlannBasedMatcher(index_params,search_params)
matches = flann.knnMatch(q1,q2,k=2)

But now i have one more feature descriptor for each key point along with previous one but of different length. So now my feature descriptor has shape like this:

[[[0.80000000000000004, 0.69999999999999996, 0.59999999999999998, 0.44444444444444448, 0.25, 0.0, 0.5, 2.0, 0, 2.9999999999999996],[2.06471330e-01,   1.59191645e-02,   9.17678759e-05, 1.32570314e-05,   4.58424252e-10,   1.66717250e-06,6.04810165e-11]

[[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001],[ 2.06471330e-01,   1.59191645e-02,   9.17678759e-05, 1.32570314e-05,   4.58424252e-10,   1.66717250e-06, 6.04810165e-11],

[[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001],[ 2.06471330e-01,   1.59191645e-02,   9.17678759e-05, 1.32570314e-05,   4.58424252e-10,   1.66717250e-06, 6.04810165e-11],

[[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001],[ 2.06471330e-01,   1.59191645e-02,   9.17678759e-05, 1.32570314e-05,  4.58424252e-10,   1.66717250e-06, 6.04810165e-11]]

Now since each point's feature descriptor is a list two lists(descriptors) with different length that is (10, 7, ) so in this case i am getting error:

setting an array element with a sequence.

while converting feature descriptor to numpy array of float datatype:

q1 = np.asarray(vec1, dtype=np.float32)

I understand the reason of this error is different length of lists, so i wonder What would be the right way to implement the same?

Catree · Accepted Answer · 2018-02-24T20:29:54.913

6

You should define a single descriptor of size 10+7=17.

This way, the space descriptor is now of 17 and you should be able to use cv2.FlannBasedMatcher.

Either create a global descriptor of the correct size desc_glob = np.zeros((nb_pts,17)) and fill it manually or find a Python way to do it. Maybe np.reshape((nb_pts,17))?

Edit:

To not favor one descriptor type over the other, you need to weight or normalize the descriptors. This is the same principle than computing a global descriptor distance from two descriptors:

dist(desc1, desc2) = dist(desc1a, desc2a) + lambda * dist(desc1b, desc2b)

edited Feb 24 '18 at 20:29

answered Feb 18 '18 at 14:36

Catree

2,477
1
17
24

would this be valid even if two features have different dimensions? – flamelite Feb 19 '18 at 05:04
No, the descriptor size for all features has to be the same. Otherwise, you cannot compute the distance between them. This is like computing the distance between a 2D point and a 3D point. This is not valid unless you convert the 2D point to a 3D point and assign the z-coordinate with a default value, e.g. 0. – Catree Feb 19 '18 at 10:59
sorry for the typo, i was asking for the case of vectors with different units or types, say vec1 is representing radius feature at a point and vec2 is representing area feature at key point. Do i have to assign certain weights to the vectors in order to normalize them? – flamelite Feb 19 '18 at 11:43
Yes, you probably need to normalize or weight the descriptor in order to not favor one descriptor type against the other. It is the same thing than with two descriptors, you have to find a way to compute a global distance: `dist(desc1a, desc2a) + lambda * dist(desc1b, desc2b)`. – Catree Feb 19 '18 at 14:22

Feature matching with flann in opencv

1 Answers1