Using OpenCV to recognise similar (not completely identical) simple images?

Question

Say I have a very simple image or shape such as this stick man drawing:

I also have a library of other simple images which I want to compare the first image to and determine the closest match:

Notice that the two stick men are not completely identical but are reasonably similar.

I want to be able to compare the first image to each image in my library until a reasonably close match is found. If necessary, my image library could contain numerous variations of the same image in order to help decide which type of image I have. For example:

My question is whether this is something that OpenCV would be capable of? Has it been done before, and if so, can you point me in the direction of some examples? Many thanks for your help.

Edit: Through my searches I have found many examples of people who are comparing images, or even people that are comparing images which have been stretched or skewed such as this: Checking images for similarity with OpenCV . Unfortunately as you can see, my images are not just translated (Rotated/Skewed/Stretched) versions of one another - They actually different images although they are very similar.

Answer 1

You should be able to do it using feature template match function of OpenCV. You can use matchTemplate function to look for the feature and then, minMaxLoc to find its location. Check out the tutorial on OpenCV web site for matchTemplate.

Answer 2

seems you need feature points detections and matching. Check these docs from OpenCV:

http://docs.opencv.org/doc/tutorials/features2d/feature_detection/feature_detection.html http://docs.opencv.org/doc/tutorials/features2d/feature_flann_matcher/feature_flann_matcher.html

Answer 3

For your particular type of images, you might get good results by using moments/HuMoments for the connected components (which you can find with findContours).

Answer 4

since there is a rotation involved, I dont think template matching would work well. You probably need to use Feature point detection such as SIFT or SURF.

Answer 5

EDIT: This won't work with rotation. Same for matchTemplate. I am yet to try the findContours + moments as in bjoernz answer which sounds promising.

Failed Solution: I tried using ShapeContextDistanceExtractor(1) available in OpenCV 3.0 along with findContours on your sample images to get good results. The sample images were cropped to same size as original image(128*200). You can could as well use resize in OpenCV.

---

Code below compares images in images folder with 1.png as the base image.

#include "opencv2/shape.hpp"
#include "opencv2/opencv.hpp"
#include <iostream>
#include <string>

using namespace std;
using namespace cv;

const int MAX_SHAPES = 7;

vector<Point> findContours( const Mat& compareToImg )
{
    vector<vector<Point> > contour2D;
    findContours(compareToImg, contour2D, RETR_LIST, CHAIN_APPROX_NONE);

    //converting 2d vector contours to 1D vector for comparison
    vector <Point> contour1D;
    for (size_t border=0; border < contour2D.size(); border++) {
        for (size_t p=0; p < contour2D[border].size(); p++) {
            contour1D.push_back( contour2D[border][p] );
        }
    }

    //limiting contours size to reduce distance comparison time
    contour1D.resize( 300 );
    return contour1D;
}

int main()
{
    string path = "./images/";
    cv::Ptr <cv::ShapeContextDistanceExtractor> distanceExtractor = cv::createShapeContextDistanceExtractor();

    //base image
    Mat baseImage= imread( path + "1.png", IMREAD_GRAYSCALE);

    vector<Point> baseImageContours= findContours( baseImage );

    for ( int idx = 2; idx <= MAX_SHAPES; ++idx ) {

        stringstream imgName;
        imgName << path << idx << ".png";
        Mat compareToImg=imread( imgName.str(), IMREAD_GRAYSCALE ) ;

        vector<Point> contii = findContours( compareToImg );
        float distance = distanceExtractor->computeDistance( baseImageContours, contii );

        std::cout<<" distance to " << idx << " : " << distance << std::endl;
    }

    return 0;
}

Result

distance to 2 : 89.7951
distance to 3 : 14.6793
distance to 4 : 6.0063
distance to 5 : 4.79834
distance to 6 : 0.0963184
distance to 7 : 0.00212693

Answer 6

Do three things: 1. Forget about image comparison since you really comparing stroke symbols. 2. Download and play wth a Gesture Search app from google store; 3. Realize that for good performance you cannot recognize your strokes without using timestamp information about stroke drawing. Otherwice we would have a successful handwriting recognition. Then you can research Android stroke reco library to write your code properly.