Comparing two images with OpenCV in Java

Question

I am trying to compare two images (determine whether they are similar or not) using the OpenCV library. I configured the java wrapper and found several tutorials (mostly in C/C++) that I am trying to rewrite into Java. I am using the feature detection approach.

The problem is that the algorithm that I currently have does not produce any reasonable results (it claims that two similar images have nothing in common and finds matches between other two images that are completely different). Could someone suggest how should I use the openCV matcher to produce some reasonable results?

This is my code for the image comparison

private static void compareImages(String path1, String path2) {
    System.out.println(path1 + "-" + path2);

    FeatureDetector detector = FeatureDetector.create(FeatureDetector.ORB);
    DescriptorExtractor descriptor = DescriptorExtractor.create(DescriptorExtractor.ORB);

    DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);

    // first image
    Mat img1 = Imgcodecs.imread(path1, Imgcodecs.CV_LOAD_IMAGE_GRAYSCALE);
    Mat descriptors1 = new Mat();
    MatOfKeyPoint keypoints1 = new MatOfKeyPoint();

    detector.detect(img1, keypoints1);
    descriptor.compute(img1, keypoints1, descriptors1);

    // second image
    Mat img2 = Imgcodecs.imread(path2, Imgcodecs.CV_LOAD_IMAGE_GRAYSCALE);
    Mat descriptors2 = new Mat();
    MatOfKeyPoint keypoints2 = new MatOfKeyPoint();

    detector.detect(img2, keypoints2);
    descriptor.compute(img2, keypoints2, descriptors2);

    // match these two keypoints sets
    MatOfDMatch matches = new MatOfDMatch();
    matcher.match(descriptors1, descriptors2, matches);

    for (DMatch m : matches.toArray()) {
      // how to use these values to detect the similarity? They seem to be way off
      // all of these values are in range 50-80 which seems wrong to me
      System.out.println(m.distance);
    }
  }

Unfortunately, algorithms like SURF and SIFT are not available in the java wrapper so I am using ORB. I have little to no experience with computer vision, I am just trying to get this simple comparison algorithm work to produce some reasonable outcome. I would be glad for any help!

EDIT: My use-case is running this algorithm against images taken from different angles. I updated my code to be better formatted.

Sample images to compare:

[might be useful](http://stackoverflow.com/questions/15544158/error-matching-with-orb-in-android) — GPPK, Jan 13 '16 at 09:33
The algorithm in the link seems to be exactly the same as what I have here... still producing very poor results. I would just like to load two images and produce some kind of value indicating their similarity — Smajl, Jan 13 '16 at 10:04
just a note: i tried to stitch two images with `cv::Stitcher` class which internally uses `xfeatures2d::SURF` and stitcing has failed. i think this means that it is hard to identify smilarity of the images by SURF — sturkmen, Jan 19 '16 at 19:32

Gilfoyle · Answer 1 · 2017-06-29T09:59:35.017

Just my two cents:

There is access to SURF and SIFT in java: openCV DescriptorExtractor Reference. I tried the FREAK implementation three years back and found out, that there is some change happening to the binary descriptor when openCV passes them over to Java. It may be that ORB is subject to the same problem. Did you compare the data of the descriptors from c or c++ to the ones on the java side?
The brute force matcher finds the best matching feature from the train image for EVERY feature in the query image. Even if it looks completely different. You have to sift through the matches and drop bad ones. There exist several strategies, an easy one would be to just take the best 20% of the matches (but this will not drop ALL outlier). Progressive Sample Consensus performed very well in my setup.
Using Features to compare image similarity has it's pitfalls. Feature count and quality are varying with picture size and content, which makes it difficult to compare images globally (in case you wanna know which of two images is more similar to a reference than the other). You could estimate a transform from one image to the other with Calib3d.findHomography(obj, scene, CV_RANSAC); and use a normalized pixel difference of the overlapping areas.

Thank you for the answer. As for the SURF and SIFT - they are listed in the enum of available algorithms but if you try to use them, you get "not supported" exception. I would have to use some older version of OpenCV to access them. I would be very glad if you could provide some sample Java code which works better than mine because every strategy for rating the image similarities that I tried produced very poor results. At this point, I am really stuck :/ — Smajl, Jan 18 '16 at 08:20

score 0 · Answer 2 · edited May 23 '17 at 12:34

As stated in this SO Question, the easiest and most straightforward way is to compare histograms. If your algorithm only needs to work for a specific dataset, try using different color channels to see where images in your set share most similarity.

The histogram approach might seem impractical, but given the color similarity of your images I believe this could be of some use.

After comparing your two images' histograms in Photoshop:

Comparing two images with OpenCV in Java

2 Answers2