0

Problem: I have a photo of an object (a manufactured part like the attached photo below), using my Andoird phone camera I want to verify if the object in camera preview matches to the template or not. (in other words, is it the same part as the template or not)

  • I can make the user to move the camera in order to have similar view of the template in camera preview as the template however there will be different noise level and/or lighting and maybe different background.

Question: What do you recommend me to use for solving this problem? I was thinking of Canny edge extraction and then matching the camera frames towards the canny edge extract from template? is this a good idea? if yes would you please tell me how can I implement this? any resources? samples? (I can do the Canny edge extraction but couldn't find a way to do the matching)

if Not a good idea then what do you recommend?

Things I have tried:

  1. Feature Extract and Matching: I used few different extractor and matcher implementations from OpenCV and my app is working and drawing the detected feature points and matches, etc. however being a beginner with image processing I cannot make sense of the result and also how to know what is a match. any idea, help, good resources?
  2. Template Matching: I used OpenCV template matching however the performance was horrible and I decided that this cannot be the solution.

sample template photo

Asha
  • 3,871
  • 7
  • 44
  • 57
  • Could you provide some sample images of your problem? Input images and the desired output. Thus, people can try some approaches in order to help you. – Gabriel Archanjo Aug 18 '14 at 19:56
  • @GabrielArchanjo there is one sample image attached to this question. the input image will also be very similar (as I said ideally I will be processing the camera input frames) – Asha Aug 18 '14 at 20:11
  • What exactly is the part to be analysed? You have a lot of grey in there, so it's very confusing. As for resources regarding Canny Edge, google helps – user3791372 Aug 18 '14 at 20:50
  • There are basic shapes like circles and squares, at least in the image you have provided, so you may be able to make use of their features such as shape, size and their relative positions. Canny is a good choice I think, and if there's no huge perspective deformation in the captured image, hough-circles will also help. – dhanushka Aug 19 '14 at 01:51
  • can you provide the "template" for that object, too? Otherwise it's hard to say what kind of properties the matching mechanic will have to fulfill... I think the template will be some kind of CAD model?!? – Micka Aug 19 '14 at 12:09
  • @Micka I was thinking of using a taken photo (similar to the above photo maybe with less background noise) as template. however CAD model is the other option, Do you recommend using a CAD model? is that even possible? if yes would you please explain more. thanks – Asha Aug 19 '14 at 12:15
  • using a photo you might be able to use Darshan's technique (SIFT/SURF/ORB feature matching), if illumination and material doesn't change too much. If you have to use a CAD model/draft or something more abstract as a template, keypoint matching probably won't work. You might want to use some shape matching techniques there instead. – Micka Aug 19 '14 at 12:19
  • @Micka of course it will be the best (and I guess more accurate) if I could use the CAD model as template, however I Have no idea how is that possible and what methods/algorithms I should use for this? is it Template matching? Will it work with Camera input frames? Would you please explain more (maybe as answer so I can potentially accept it as the accepted answer if it works) – Asha Aug 19 '14 at 12:43
  • 1
    didnt try it yet and didnt work with it yet, so you will have to do some research on your own (shape matching). Easy things that come in my mind are: contour extraction with canny; `Moments` for shape matching: http://docs.opencv.org/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html#moments ; ICP (iterative closest points) for accurate fine placement ; maybe `chamfer matching` if you can normalize scale and orientation with other methods. – Micka Aug 19 '14 at 12:54
  • maybe this one helps: http://stackoverflow.com/questions/8785664/finding-shapes-in-an-image-using-opencv/8786545#8786545 – Micka Aug 19 '14 at 13:00

2 Answers2

2

I tried object recognition with my phone on your test image and the results were positive.

Detector used :ORB(Binary Detector).

Descriptor used :ORB.

Matching Technique : Brute-force matching .

Image Size 640x480.

I was able to detect around 500 feature points (number of keypoints is around sufficient but it might produce false matches when you have more images with similar looking objects.you need to refine your matching to avoid false matches). enter image description here

Result of object recognition on two different scales. enter image description here enter image description here

Regarding you finding difficulties in understanding object recognition. What exactly did you not understand(Specific topic). I recommend you to go thru the these two books

  1. Learning OpenCV by By Adrian Kaehler, Gary Bradski
  2. OpenCV 2 Computer Vision Application Programming Cookbook by by Robert Laganière(chapter 8 & 9).

Cheers!

mrid
  • 5,782
  • 5
  • 28
  • 71
Darshan
  • 1,018
  • 10
  • 19
  • Thanks a lot for the answer, is it possible to share the code snippet with me? I'm specially interested to know how you're extracting the good matches and what is the criteria to say Object found (or matched) or not (How do you draw the green square around the object?) many thanks – Asha Aug 19 '14 at 09:27
  • another question: Do you think I will get a better (more accurate) feature extract if I send the result of edge extraction (i.e Canny edge extraction result) to the feature extraction method? – Asha Aug 19 '14 at 09:31
  • How much would you pay me if I share the code snippet ? :-). Just kidding!. Unfortunately I cant share the code since my research institute holds the copyright for my algorithm. You could easily understand the procedure of object recognition and matching result analysis if you read the two chapters from the book I have mentioned above. You can obtain that green square using a technique called Homography.Please follow [this](http://docs.opencv.org/doc/tutorials/features2d/feature_homography/feature_homography.html) tutorial to know how to implement object recognition . – Darshan Aug 19 '14 at 11:22
  • no problem I understand :) thanks a lot for the help. What do you think about my second question? will it create a more precise result? because I'm only interested in the outline of the part and main features like nuts, holes, etc. – Asha Aug 19 '14 at 11:25
  • 1
    Supplying result of canny to feature extraction would not help you in any way since feature extraction is based on much better technique called corner detection which excludes many ambiguous points in the image and keep only highly distinct and unique points for matching. – Darshan Aug 19 '14 at 11:26
  • After your research if you find my answer to be correct, please do the favour of accepting the answer(the green tick mark :-)) – Darshan Aug 19 '14 at 11:32
0

from what I understand canny edge detection might not be an optimal solution. according to me after some basic pre-processing of the test image find its sift features and compare it with the sift features of the template. sift being really versatile should work here too.

you can also try opensurf feature they are faster than sift but i havent had an opportunity to work alot with them to be able to comment on its accuracy

akshay
  • 13
  • 3