3

How can I take two images and compare them to see how similar they are?

I'm not talking about comparing two exact images using MD5. The two images that I am comparing will be completely different, as well as likely different sizes at times.

Using Pokemon cards as an example: I'm going to have scanned HD images of each of the cards. I want the user to be able to take a picture of their Pokemon card with their phone and I want to be able to compare it against my scanned images and then determine which card it is that they took a picture of.

The processing does not have to be done directly on the phone, offloading to a web service is an option however note that my knowledge somewhat limited on the programming languages (limited to PHP/JAVA/Android pretty much). The server I'm using is my own Ubuntu server so I do have access to the exec command from php if this would help.

At first I figured someone would have done something like this before (comparing two images). I tried using php with imageik using an example I found that claimed to do what I was trying ( utilizing compareImages() ), but it didn't work at all. There doesn't seem to be much (if any) documentation on doing something like this which is why I'm so stuck. All I'm looking for is a push in the right direction.

My second thought was to try using OCR to pull just the title of the card and I would just compare that against a database of titles and display the images tied to that title. So far I've tried using phpocr first, which didnt work at all as it requires monochrome images to my understanding. Next I tried tesseract directly from the console on my server, and while it did WAY better than phpocr, more than 80% of the characters were either wrong or incorrect on a scanned image, so a lower quality image coming from a smart phone would really have troubles.

I also tried OpenCV for Android but couldnt get any of the samples working.

Has anyone done anything like this, or at least used something that can accomplish what Im looking for?

Badams
  • 569
  • 6
  • 25
  • http://www.google.com/#q=opencv+image+comparison you will have a lot of work to do, even with a ready to use library like OpenCV, there is no easy way for what you want to do and also the math for this can be very complex. – axis Oct 03 '12 at 19:13
  • This might be related: http://stackoverflow.com/questions/843972/image-comparison-fast-algorithm – Nir Alfasi Oct 03 '12 at 19:25

2 Answers2

1

There are two distinct tasks - identify area of interest ( which can be done with Haar cascades - same as face detection ) and recognition of identified image which can be
done with invariant moment techniques (like Hu moments - it was good enough to count soviet tanks on satellite images so it shall be good for pokemons). Nice property of invariant moments is soft degradation of results in case of low quality - you get a list of probability for symbols - like this is 80% pikachu and 30% something else.

We are developing OCR library based on invariant moments for use in android here:

https://sourceforge.net/projects/javaocr/

( pure java and reasonable speed , and there are android samples in demos subdirectory. And here is app based on javaocr, it will recognize black on white phone number and dial it: https://play.google.com/store/apps/details?id=de.pribluda.android.ocrcall&feature=search_result#?t=W251bGwsMSwyLDEsImRlLnByaWJsdWRhLmFuZHJvaWQub2NyY2FsbCJd )

You may also consider some aiming help so user positions symbol to be matched properly ( so first task will use real intellect )

Konstantin Pribluda
  • 12,329
  • 1
  • 30
  • 35
  • Can you point me in the direction of the android samples? After reading it sounds like this could potentially do exactly what I need. – Badams Oct 03 '12 at 21:28
  • added links in answer. it can be solution for you - but OCR is not trivial and there are not so much ready recipes which will works always. – Konstantin Pribluda Oct 04 '12 at 06:55
0

You should decide what kind of similarity comparison you need. There are geometric algorithms. They use edge detection and then try to match detected edges in both images. They are probably useful when dealing with different colours of objects with the same shape. And there are algorithms that are more based on colour similarity. They compare what colours are in the image and how they are distributed.

If you are looking for a concrete algorithm, you probably should have a look at the Hough Transform.

SpaceTrucker
  • 13,377
  • 6
  • 60
  • 99