OpenCV image comparison in Android

Question

[EDIT] I have devised some code for image comparison. The matching part is still a bit flawed and I would love some assitance. The project can be found at - GitHub.

I have these two images Img1 and Img2:

When I use the following command in openCV

Mat img1 = Highgui.imread("mnt/sdcard/IMG-20121228.jpg");
Mat img2 = Highgui.imread("mnt/sdcard/IMG-20121228-1.jpg");

try{
    double l2_norm = Core.norm( img1, img2 );
    tv.setText(l2_norm+"");
} catch(Exception e) {
    //image is not a duplicate
}

I get a double value for l2_norm. This double value varies for duplicate image pairs. But if the images are different, then an exception is thrown. Is this how I identify duplicate images? Or is there a better method? I've Googled extensively and couldn't find a really convincing answer. I would like the code and explanation as to how I'd compare two images and get a boolean value of true or false depending upon the images.

EDIT

Scalar blah= Core.sumElems(img2);
    Scalar blah1=Core.sumElems(img1);

    if(blah.equals(blah1))
    {
        tv.setText("same image");
    }
    }

I've tried this, but the if condition is never satisfied. I'm assuming there are a few differences, but there is no compare function for Scalar. What do I do?

EDIT

try{
    Scalar blah= Core.sumElems(img2);
    Scalar blah1=Core.sumElems(img1);
    String b=blah.toString();
    String b1=blah1.toString();
    System.out.println(b+" "+b1);
    double comp=b.compareTo(b1);
    tv.setText(""+comp);
    }

This method is again flawed. Although it can be used to compare images with a decent accuracy, it fails when images are of different sizes.

When images are of different sizes and I print the scalar values I get this:

[9768383.0, 1.0052889E7, 1.0381814E7, 0.0] [1.5897384E7, 1.6322252E7, 1.690251E7, 0.0]

The variation between the second and third numbers although not much is quite large compared to when the images of same size are compared. The first number however suffers the most change.

What would be the best fastest way to compare the contents of two images?

[EDIT]

I'm using the code I found here.

What I'm not able to figure out is how to initialize the MatOfKeyPoint variables keypoints and logoKeypoints. Here's my code snippet:

           FeatureDetector detector = FeatureDetector.create(FeatureDetector.SURF);
        //FeatureDetector detector = FeatureDetector.create(FeatureDetector.FAST);
        //Imgproc.cvtColor(img1, img1, Imgproc.COLOR_RGBA2RGB);
        //Imgproc.cvtColor(img2, img2, Imgproc.COLOR_RGBA2RGB);

        DescriptorExtractor SurfExtractor = DescriptorExtractor
        .create(DescriptorExtractor.SURF);


        //extract keypoints
        MatOfKeyPoint keypoints, logoKeypoints;
        long time= System.currentTimeMillis();
        detector.detect(img1, keypoints);
        Log.d("LOG!", "number of query Keypoints= " + keypoints.size());
        detector.detect(img2, logoKeypoints);
        Log.d("LOG!", "number of logo Keypoints= " + logoKeypoints.size());
        Log.d("LOG!", "keypoint calculation time elapsed" + (System.currentTimeMillis() -time));

        //Descript keypoints
        long time2 = System.currentTimeMillis();
        Mat descriptors = new Mat();
        Mat logoDescriptors = new Mat();
        Log.d("LOG!", "logo type" + img2.type() + "  intype" + img1.type());
        SurfExtractor.compute(img1, keypoints, descriptors);
        SurfExtractor.compute(img2, logoKeypoints, logoDescriptors);
        Log.d("LOG!", "Description time elapsed" + (System.currentTimeMillis()- time2));

I obviously can't initialize the variables keypoints and logoKeypoints to null cuz I'll receive a null pointer exception then. How do I initialize them?

Answer 1

You should understand that this is not a simple question and you have different concepts you could follow. I will only point out two solution without source-code.

1. Histogram comparison: You could convert both images into grey-scale make a histogram in the range of [0,...,255]. Every pixel-value will be counted. Then use both histograms for comparison. If the distribution of pixel-intensities equals or is above some treshold (perhaps 90% of all pixels), you could consider this images as duplicates. BUT: This is one of the simplest solutions and it isn't stable if any picture has an equal distribution.
2. Interest-Point-Detectors/-Descriptors: Take a look at SIFT/SURF image-detectors and descriptors. A detector will try to determine unique keypoits of intensities in an image. A descriptor will be computed at this location I(x,y). A normal matcher with a bruteforce-approach and euclidean distance can match these images using their descriptors. If an image is a duplicate the rate of given matches should very high. This solution is good to implement and there could be enough tutorials regarding this topic.

I'll hope this helps. Please ask if you have questions.

[UPDATE-1] A C++-tutorial: http://morf.lv/modules.php?name=tutorials&lasit=2#.UR-ewKU3vCk

Some JavaCV-tutorials: http://code.google.com/p/javacv/w/list

[UPDATE-2] Here is an example with SIFT-Detector and SIFT-Descriptor using default parameters. RANSAC-Threshold for homography is 65, reprojection-error (epsilon) is 10, cross-validation enabled. You could try to count the matched. If the Inliner-Outlier-Ratio is too high you could see this pair as duplicates. For example: These images produce 180 keypoints in IMG1 and 198 in IMG2. The matched descriptors are 163 of which only 3 are outliers. So this gives a really good ratio which only could mean that these images could be duplicates.

[UPDATE-3] I don't understand why you can initialize the MatOfKeypoints. I've read the API and there's a public constructor. AND: You can use the Mat of the image you want to analyse. This is very nice. =)

MatOfKeyPoint reference = new MatOfKeyPoint(matOfReferenceImage);

For Matching use a BRUTEFORCE_SL2 Descriptor-Matcher cause you will need the euclidean distance for SURF or SIFT.

Answer 2

Use cv2.absDiff to compute the difference between the pictures and cv2.sumElems to get the sum of all pixels differences.

Then invent a threshold by which you judge wether two images are similar or not.

Answer 3

You can try the following code:

Mat img1 = Highgui.imread("mnt/sdcard/IMG-20121228.jpg");
Mat img2 = Highgui.imread("mnt/sdcard/IMG-20121228-1.jpg");
Mat result = new Mat();

Core.compare(img1,img2,result,Core.CMP_NE);

int val = Core.countNonZero(result);

if(val == 0) {
    //Duplicate Image
} else {
    //Different Image
}

Here in the code compare function will compare two images and then if there there is dis similarity between images then then particular matrix value will be 255 and all other values will be zero. Then you can count the number of non-zero values to determine if the images were equal. This would work only for Exactly same images.

If you want to compare images ignoring the light effects i suggest you to generate the edge image first (Using canny function of OpenCV) and then compare the images.

Hope this answer helps you!!