10

I am using OpenCV 3.2

I am trying to use FLANN to match features descriptors in a faster way than brute force.

// Ratio to the second neighbor to consider a good match.
#define RATIO    0.75

void matchFeatures(const cv::Mat &query, const cv::Mat &target,
                   std::vector<cv::DMatch> &goodMatches) {
    std::vector<std::vector<cv::DMatch>> matches;
    cv::Ptr<cv::FlannBasedMatcher> matcher = cv::FlannBasedMatcher::create();
    // Find 2 best matches for each descriptor to make later the second neighbor test.
    matcher->knnMatch(query, target, matches, 2);
    // Second neighbor ratio test.
    for (unsigned int i = 0; i < matches.size(); ++i) {
        if (matches[i][0].distance < matches[i][1].distance * RATIO)
            goodMatches.push_back(matches[i][0]);
    }
}

This code is working me with SURF and SIFT descriptors, but not with ORB.

OpenCV Error: Unsupported format or combination of formats (type=0) in buildIndex

As it's said here, FLANN needs the descriptors to be of type CV_32F so we need to convert them.

if (query.type() != CV_32F) query.convertTo(query, CV_32F);
if (target.type() != CV_32F) target.convertTo(target, CV_32F);

However, this supposed fix is returning me another error in convertTo function.

OpenCV Error: Assertion failed (!fixedType() || ((Mat*)obj)->type() == mtype) in create

This assertion is in opencv/modules/core/src/matrix.cpp file, line 2277.

What's happening?

Code to replicate issue.

#include <opencv2/opencv.hpp>

int main(int argc, char **argv) {
    // Read both images.
    cv::Mat image1 = cv::imread(argv[1], cv::IMREAD_GRAYSCALE);
    if (image1.empty()) {
        std::cerr << "Couldn't read image in " << argv[1] << std::endl;
        return 1;
    }
    cv::Mat image2 = cv::imread(argv[2], cv::IMREAD_GRAYSCALE);
    if (image2.empty()) {
        std::cerr << "Couldn't read image in " << argv[2] << std::endl;
        return 1;
    }
    // Detect the keyPoints and compute its descriptors using ORB Detector.
    std::vector<cv::KeyPoint> keyPoints1, keyPoints2;
    cv::Mat descriptors1, descriptors2;
    cv::Ptr<cv::ORB> detector = cv::ORB::create();
    detector->detectAndCompute(image1, cv::Mat(), keyPoints1, descriptors1);
    detector->detectAndCompute(image2, cv::Mat(), keyPoints2, descriptors2);
    // Match features.
    std::vector<cv::DMatch> matches;
    matchFeatures(descriptors1, descriptors2, matches);
    // Draw matches.
    cv::Mat image_matches;
    cv::drawMatches(image1, keyPoints1, image2, keyPoints2, matches, image_matches);
    cv::imshow("Matches", image_matches);
}
Community
  • 1
  • 1
Santiago Gil
  • 1,292
  • 7
  • 21
  • 52

3 Answers3

14

Did you adjust the FLANN parameters?

Taken from http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature2d/py_matcher/py_matcher.html

While using ORB, you can pass the following. The commented values are recommended as per the docs, but it didn’t provide required results in some cases. Other values worked fine.:

index_params= dict(algorithm = FLANN_INDEX_LSH, table_number = 6, # 12 key_size = 12, # 20 multi_probe_level = 1) #2

Probably you can convert that to C++ api?

According to the comment, the C++ way is:

cv::FlannBasedMatcher matcher = cv::FlannBasedMatcher(cv::makePtr<cv::flann::LshIndexParams>(12, 20, 2));
Micka
  • 19,585
  • 4
  • 56
  • 74
  • 8
    Really worked. The call was `cv::FlannBasedMatcher matcher = cv::FlannBasedMatcher(cv::makePtr(12, 20, 2));` – Santiago Gil May 07 '17 at 20:52
  • 2
    As an addition, not all the matches have a correspondence using ORB and FLANN (maybe with other descriptors this also happens, but not for now). Then, in the second neighbor ratio test, I have added a security condition `if (matches[i].size() >= 2)`. – Santiago Gil May 07 '17 at 20:56
  • 4
    It is probably worth mentioning that `FLANN_INDEX_LSH = 6`. – scrutari Mar 07 '19 at 16:14
  • 1
    There is also a comment about `multi_probe_level` that it is "number of levels to use in multi-probe (0 for standard LSH)", so it might be better to use `multi_probe_level=0`. – scrutari Mar 07 '19 at 16:27
12

Binary-string descriptors: ORB, BRIEF, BRISK, FREAK, AKAZE, etc.

Floating-point descriptors: SIFT, SURF, GLOH, etc.

Feature matching of binary descriptors can be efficiently done by comparing their Hamming distance as opposed to Euclidean distance used for floating-point descriptors.

For comparing binary descriptors in OpenCV, use FLANN + LSH index or Brute Force + Hamming distance.

http://answers.opencv.org/question/59996/flann-error-in-opencv-3/

By default, FlannBasedMatcher works as KDTreeIndex with L2 norm. This is the reason why it works well with SIFT/SURF descriptors and throws an exception for ORB descriptor.

Binary features and Locality Sensitive Hashing (LSH)

Performance comparison between binary and floating-point descriptors

Nirmal
  • 1,285
  • 2
  • 13
  • 23
0

I believe there is a bug in the OpenCV3 version: FLANN error in OpenCV 3

You need to convert your descriptors to a 'CV_32F'.

Hayley
  • 29
  • 4