Object detection with OpenCV Feature Matching with a threshold/similarity score - Java/C++

Question

I am in the process of creating a small program which detects objects(small image) in the large image and I am using OpenCV java. As I have to consider rotation and scaling I have used FeatureDetector.BRISK and DescriptorExtractor.BRISK.

Following approach is used to filter the match results to get the best matches only.

I have two questions

1. Is there a way to find the below min_dist and max_dist with the loop I have used?
2. Most important question - Now the problem is I need to use these matches to determine whether the object(template) found or not. Would be great if some one help me here.

Thanks in advance.

    FeatureDetector  fd = FeatureDetector.create(FeatureDetector.BRISK); 
    final MatOfKeyPoint keyPointsLarge = new MatOfKeyPoint();
    final MatOfKeyPoint keyPointsSmall = new MatOfKeyPoint();

    fd.detect(largeImage, keyPointsLarge);
    fd.detect(smallImage, keyPointsSmall);

    System.out.println("keyPoints.size() : "+keyPointsLarge.size());
    System.out.println("keyPoints2.size() : "+keyPointsSmall.size());

    Mat descriptorsLarge = new Mat();
    Mat descriptorsSmall = new Mat();

    DescriptorExtractor extractor = DescriptorExtractor.create(DescriptorExtractor.BRISK);
    extractor.compute(largeImage, keyPointsLarge, descriptorsLarge);
    extractor.compute(smallImage, keyPointsSmall, descriptorsSmall);

    System.out.println("descriptorsA.size() : "+descriptorsLarge.size());
    System.out.println("descriptorsB.size() : "+descriptorsSmall.size());

    MatOfDMatch matches = new MatOfDMatch();

    DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMINGLUT);
    matcher.match(descriptorsLarge, descriptorsSmall, matches);

    System.out.println("matches.size() : "+matches.size());

    MatOfDMatch matchesFiltered = new MatOfDMatch();

    List<DMatch> matchesList = matches.toList();
    List<DMatch> bestMatches= new ArrayList<DMatch>();

    Double max_dist = 0.0;
    Double min_dist = 100.0;

    for (int i = 0; i < matchesList.size(); i++)
    {
        Double dist = (double) matchesList.get(i).distance;

        if (dist < min_dist && dist != 0)
        {
            min_dist = dist;
        }

        if (dist > max_dist)
        {
            max_dist = dist;
        }

    }

    System.out.println("max_dist : "+max_dist);
    System.out.println("min_dist : "+min_dist);

    double threshold = 3 * min_dist;
    double threshold2 = 2 * min_dist;

    if (threshold2 >= max_dist)
    {
        threshold = min_dist * 1.1;
    }
    else if (threshold >= max_dist)
    {
        threshold = threshold2 * 1.4;
    }

    System.out.println("Threshold : "+threshold);

    for (int i = 0; i < matchesList.size(); i++)
    {
        Double dist = (double) matchesList.get(i).distance;
        System.out.println(String.format(i + " match distance best : %s", dist));
        if (dist < threshold)
        {
            bestMatches.add(matches.toList().get(i));
            System.out.println(String.format(i + " best match added : %s", dist));
        }
    }


    matchesFiltered.fromList(bestMatches);

    System.out.println("matchesFiltered.size() : " + matchesFiltered.size());

---

Edit

Edited my code as follows.I know still it's not the best way to come to a conclusion whether the object found or not based on no of best matches. So please share your views.

    System.out.println("max_dist : "+max_dist);
    System.out.println("min_dist : "+min_dist);

    if(min_dist > 50 )
    {
        System.out.println("No match found");
        System.out.println("Just return ");
        return false;
    }

    double threshold = 3 * min_dist;
    double threshold2 = 2 * min_dist;

    if (threshold > 75)
    {
        threshold  = 75;
    }
    else if (threshold2 >= max_dist)
    {
        threshold = min_dist * 1.1;
    }
    else if (threshold >= max_dist)
    {
        threshold = threshold2 * 1.4;
    }

    System.out.println("Threshold : "+threshold);

    for (int i = 0; i < matchesList.size(); i++)
    {
        Double dist = (double) matchesList.get(i).distance;

        if (dist < threshold)
        {
            bestMatches.add(matches.toList().get(i));
            //System.out.println(String.format(i + " best match added : %s", dist));
        }
    }

    matchesFiltered.fromList(bestMatches);

    System.out.println("matchesFiltered.size() : " + matchesFiltered.size());


    if(matchesFiltered.rows() >= 1)
    {
        System.out.println("match found");
        return true;
    }
    else
    {
        return false;
    }

Answer 1

Your Edited code is working fine for me, and working perfectly,

Following are changes that i have done in your code for detecting objects(small image) in the large image :

1. using SURF method for feature detection as well as feature extraction.(SURF is available in opencv 4.1.1 for Android and earlier, after that it have been removed from that, so here i have used opencv 4.1.1)

2. change threshold of image matched or not from 1 to 4, in following line

   if(matchesFiltered.rows() >= 1)

to

if(matchesFiltered.rows() >= 4)

only this changes have worked perfectly for me, make sure that object/small image have rich texture(atleast should have keypoints that can be matched)

Answer 2

There are several approaches for detecting objects inside images. Just put some links here:

• Open CV 2 Computer Vision Application Programming Cookbook, Chapter 8/9
• http://docs.opencv.org/doc/tutorials/features2d/feature_homography/feature_homography.html
• http://robocv.blogspot.de/2012/02/real-time-object-detection-in-opencv.html

The last link shows a way to calculate the min and max value, should be nearly the same in Java. All links should hopefully show some ideas how to match objects.

I also recognized that there are a lot of magic numbers inside your code. Maybe you could put them in variables to reduce the possibility of error and have a better overview.