OpenCV Android: How to draw matching key points over the compared images?

Question

Currently I am using OpenCV to compare two images to see if they are alike or not in Android. I'm using ORB Feature Detectors and Descriptor Extractors. Here is what I have thus far. I find all the feature key points in the first image and then find all the feature key points in the second image. I then find the descriptors for these key points and then do a match between the two images.

private void matchImages() {
    Mat refMat = new Mat();
    Mat srcMat = new Mat();

    Bitmap refBitmap = ((BitmapDrawable) mRefImg.getDrawable()).getBitmap();
    Bitmap srcBitmap = ((BitmapDrawable) mSrcImg.getDrawable()).getBitmap();

    Utils.bitmapToMat(refBitmap, refMat);
    Utils.bitmapToMat(srcBitmap, srcMat);

    MatOfDMatch matches = new MatOfDMatch();
    MatOfDMatch goodMatches = new MatOfDMatch();

    LinkedList<DMatch> listOfGoodMatches = new LinkedList<>();

    LinkedList<Point> refObjectList = new LinkedList<>();
    LinkedList<Point> srcObjectList = new LinkedList<>();

    MatOfKeyPoint refKeypoints = new MatOfKeyPoint();
    MatOfKeyPoint srcKeyPoints = new MatOfKeyPoint();

    Mat refDescriptors = new Mat();
    Mat srcDescriptors = new Mat();

    MatOfPoint2f reference = new MatOfPoint2f();
    MatOfPoint2f source = new MatOfPoint2f();

    FeatureDetector orbFeatureDetector = FeatureDetector.create(FeatureDetector.ORB);
    orbFeatureDetector.detect(refMat, refKeypoints);
    orbFeatureDetector.detect(srcMat, srcKeyPoints);

    DescriptorExtractor descriptorExtractor = DescriptorExtractor.create(DescriptorExtractor.ORB);
    descriptorExtractor.compute(refMat, refKeypoints, refDescriptors);
    descriptorExtractor.compute(srcMat, srcKeyPoints, srcDescriptors);

    DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
    matcher.match(refDescriptors, srcDescriptors, matches);

    double max_dist = 0;
    double min_dist = 100;
    List<DMatch> matchesList = matches.toList();

    for (int i = 0; i < refDescriptors.rows(); i++) {
        Double distance = (double) matchesList.get(i).distance;
        if (distance < min_dist) min_dist = distance;
        if (distance > max_dist) max_dist = distance;
    }

    for (int i = 0; i < refDescriptors.rows(); i++) {
        if (matchesList.get(i).distance < 3 * min_dist) {
            listOfGoodMatches.add(matchesList.get(i));
        }
    }

    goodMatches.fromList(listOfGoodMatches);

    List<KeyPoint> refObjectListKeypoints = refKeypoints.toList();
    List<KeyPoint> srcObjectListKeypoints = srcKeyPoints.toList();

    for (int i = 0; i < listOfGoodMatches.size(); i++) {
        refObjectList.addLast(refObjectListKeypoints.get(listOfGoodMatches.get(i).queryIdx).pt);
        srcObjectList.addLast(srcObjectListKeypoints.get(listOfGoodMatches.get(i).trainIdx).pt);
    }

    reference.fromList(refObjectList);
    source.fromList(srcObjectList);

    String result;
    if(listOfGoodMatches.size() > MIN_MATCH_THRESHOLD && listOfGoodMatches.size() < MAX_MATCH_THRESHOLD) {
        result = "They MATCH!";
    } else {
        result = "They DON'T match!";
    }

    AlertDialog alert = new AlertDialog.Builder(this)
            .setMessage(result)
            .setPositiveButton("OK", new DialogInterface.OnClickListener() {
                @Override
                public void onClick(DialogInterface dialog, int which) {
                    // close
                }
            }).create();
    alert.show();

    Mat outputImage = new Mat();
    Bitmap comboBmp = combineImages(refBitmap, srcBitmap);
    Utils.bitmapToMat(comboBmp, outputImage);

    Features2d.drawMatches(refMat, refKeypoints, srcMat, srcKeyPoints, goodMatches, outputImage);

    Bitmap bitmap = Bitmap.createBitmap(outputImage.cols(), outputImage.rows(), Bitmap.Config.ARGB_8888);

    Utils.matToBitmap(outputImage, bitmap);
    mRefImg.setImageBitmap(comboBmp);
    mRefImg.invalidate();
    mSrcImg.setImageBitmap(bitmap);
    mSrcImg.invalidate();
}

This is just a simple 'sandbox' application I created just to test out and play with this library. The result of the above code produces the following if I compare two images:

Result of comparing two images

As you can see, the background of the matches is black. How can I draw these matches over the image on the left? An example of what I would like my result to look like is here: https://stackoverflow.com/a/14909358/3779845

score 6 · Accepted Answer · answered Jun 17 '15 at 18:13

I'm not sure if this will still be of any help to you, but how I fixed the black background problem was instead of using a RGBA image i used

Imgproc.cvtColor(gabarito, gabaritoRgb, Imgproc.COLOR_RGBA2RGB, 1);
Imgproc.cvtColor(prova, provaRgb, Imgproc.COLOR_RGBA2RGB, 1);

to convert my images to RGB and then i used the new images in the drawMatched function!

Features2d.drawMatches(gabaritoRgb, keypointsGabarito, provaRgb, keypointsProva, matches, 
                    imagemDeSaida);

OpenCV Android: How to draw matching key points over the compared images?

1 Answers1