34

I found many ways to detect different shapes. But hard luck when I am going for a physical object. From what I read we should have a black border around images to make a pattern file. If I follow this concept and generate a pattern then my application detects images on printout. But in the real world a physical object not necessarily has a black border square shape around it.

Update

Although I accept an answer, my question remains unsolved. As there is still no solution for detecting physical object.

Any further research and links are welcome!

naXa stands with Ukraine
  • 35,493
  • 19
  • 190
  • 259
Tofeeq Ahmad
  • 11,935
  • 4
  • 61
  • 87
  • What libraries are you trying to use, and can you elaborate on your use case a little: Are you trying to detect and recognize a single physical object like a real teapot, or detect and distinguish between multiple physical objects like a teapot, a toy car, and a can of coke. Are you working at desktop scale, or outdoors? Or are you trying to recognize a picture of a real object like a picture of a teapot on a fiducial marker? – dabhaid Jan 18 '12 at 11:54
  • 1
    Thnx for you responce..I am trying to detect one object only like Teapot or bulb.And I am using AndAR library.So how make pattern file for physical object. – Tofeeq Ahmad Jan 18 '12 at 12:02
  • 3
    Hi..Sameer!! finally after 3 years do you got any feasible solution to track any real object (Real car as marker) using Vuforia or any SDK? – Sanket Prabhu Apr 02 '16 at 12:12

4 Answers4

17

The bad news is, you can't use AndAR to detect physical objects. AndAR is based on a fiducial marker approach, where the marker is made of two components: a solid border and an interior pattern. The pattern encodes a value that can be used to address a particular model to render on the marker, and the border makes it easy to determine the relative orientation of the marker to the device. Clearly this is just planar image recognition.

To do object recognition on a 3D object is a more complicated problem, and I don't know of any Android libraries that provide a turn-key solution, but recognizing just one object is probably feasible on a mobile device.

One possibility might be to investigate the available Android AR toolkits (Layar, Junaio, Qualcomm AR SDK) which all now support some image recognition. It may be that by taking images of your teapot at various rotations and using those as the images you want your app to match against that you might get this solution working, but keep in mind they are only designed to do planar matching on images, not real 3D objects, so the performance might not be great. (Well, the Metaio Mobile SDK Pro does 3D recognition and tracking, but it's very expensive).

While object recognition is perhaps best done by comparing camera frames with images of the object you wish to recognize (or by comparing image features from the camera frames with pre-computed image features etc), tracking is a different matter. If you want to accurately track your 3D object in 3D space you'll really need to have or build a 3D model of it, and for each frame determine point correspondences between the camera image and the 3D object for tracking. True unassisted (i.e. no depth-camera) 3D tracking is hard.

I hope this gives you some background you can use to evaluate your next steps.

Update: Qualcomm's Vuforia SDK allows you to track "multi targets", which are objects with a set of planar tracking surfaces with a fixed spatial relationship. If you made a "cube" different photos of the 6 sides of your object (teapot) that might work somewhat. https://ar.qualcomm.at/qdevnet/developer_guide/Trackables

End of 2013 Update:

I have no experience with these, but:

Metaio now offer 3D tracking of CAD models: https://dev.metaio.com/sdk/tutorials/3d-tracking-based-on-cad-data/

ARLab.com's LinkAR promises object matching. http://www.arlab.com/objecttracking

I would note the use of the word "matching" - I think the use case here is you know the object you want to overlay (a toy-box, and engine etc). Differentiating between multiple 3D objects may be entirely out of scope.

dabhaid
  • 3,849
  • 22
  • 30
  • 1
    Thank you..i am very disappointed that except open source nature, there is no library available for physical object detection.Can you have sample to comparing two images using camera frame so that i can go for this.But i really appreciate your work. +1 – Tofeeq Ahmad Jan 19 '12 at 06:34
  • Would you like an example of doing image matching from scratch, or how to set up e.g. Junaio Glue to do image matching? – dabhaid Jan 19 '12 at 11:56
  • Yes give the example link for Matching image with live camera stream – Tofeeq Ahmad Jan 19 '12 at 12:16
  • 1
    This is a very broad question :) Perhaps the best approach for mobile would be the histogram matching method discussed here: http://stackoverflow.com/a/844113/312007 - it's fast to computer, it's very easy to match against the frame, but it's not very robust, and you won't know exactly where in the image the object is located. – dabhaid Jan 19 '12 at 12:23
  • Thanks again..Is not any other way to match image with camera preview? – Tofeeq Ahmad Jan 19 '12 at 12:26
  • 1
    There are a great many approaches to image matching. If you describe the entire use case, maybe another approach might appear better, but histogram matching would be the most straightforward for recognizing a 3D object on a mobile device in real time. If you need the position (for augmenting), you could try to follow the recognition step with template matching. http://en.wikipedia.org/wiki/Template_matching There are many more solutions, but they are more sophisticated. – dabhaid Jan 19 '12 at 12:37
  • Thanks for sharing lots of knowledge with me.I will wait for another response from community.Then i will decide about bounty :).But really thank you for your efforts – Tofeeq Ahmad Jan 19 '12 at 12:49
  • Do you have time to discuss in chat? – Tofeeq Ahmad Jan 19 '12 at 12:50
  • Not any more today, and tomorrow is a busy day, but maybe over the weekend? – dabhaid Jan 19 '12 at 13:20
  • i appreciate your help by awarding you bounty.I need a favor if you donot mind.can you upvote my question so that it will show in popular category.as i need its all possible solution – Tofeeq Ahmad Jan 25 '12 at 09:18
  • Sure, visual 3D object detection is still an area of active research though, so I would be surprised if you got a "cookbook" solution from SO. – dabhaid Jan 25 '12 at 10:55
  • nice to talk you..i will continue my work.What you do dabhaid? – Tofeeq Ahmad Jan 25 '12 at 11:03
  • Good luck! I'm a mixed and augmented reality researcher. – dabhaid Jan 25 '12 at 11:25
  • can i have your personal email to talk you later – Tofeeq Ahmad Jan 25 '12 at 11:27
  • 2
    The Vuforia link no longer works, maybe this was it: https://developer.vuforia.com/resources/dev-guide/trackables – Rui Marques Feb 15 '13 at 20:11
  • is there any other sdk that provides tracking of real objects ? I also need this kind of functionality. – Amit Hooda Nov 27 '13 at 04:16
  • I've updated the answer a bit, the landscape is quickly moving in the direction of 3D tracking. – dabhaid Dec 02 '13 at 18:55
  • Hi Sir, Finally there is any solution now? – Sanket Prabhu Apr 02 '16 at 12:14
3

I recently read about research being doing on hierarchical shape vocabularies used for object representation. Of course, there is no library available for download, but in case you are interested in the general approach here you can find some papers.

Also you might be interested in this paper. It describes an algorithm for detecting objects based on a set of contours.

Pedro
  • 4,100
  • 10
  • 58
  • 96
0

Have you considered OpenCV for detecting and tracking object? Take a look at Find Objects with a Webcam tutorial (C++ / Qt). OpenCV is available for Android - OpenCV 4 Android.

Enjoy Detection!

naXa stands with Ukraine
  • 35,493
  • 19
  • 190
  • 259
0

Since 2018 you can detect objects in Android apps & ARCore apps.

In May 2018 Google announced a mobile SDK – ML Kit, that's a machine learning API for Android and iOS developers that you can use along with ARCore SDK (what now has Augmented Images API and Augmented Faces API). At the moment it's still in beta stage but it's got the following working features:

enter image description here

Google ML Kit engineers say about it the following:

ML Kit brings Google's machine learning expertise to Android and iOS apps in a powerful yet easy-to-use package. Whether you're new or experienced in machine learning, you can implement the functionality you need in just a few lines of code. There's no need to have deep knowledge of neural networks or model optimization to get started. On the other hand, if you are an experienced ML developer, ML Kit provides convenient APIs that help you use your custom TensorFlow Lite models in your mobile apps.

Community
  • 1
  • 1
Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
  • But, it does not allow custom object detection or does it? –  Jan 14 '20 at 22:05
  • What do you mean exactly saying `custom object detection`? – Andy Jazz Jan 15 '20 at 05:19
  • custom objects in the sense anything apart from faces for instance toys, sign board etc.., –  Jan 15 '20 at 09:54
  • Yes, you can definitely do it – https://developers.google.com/ml-kit/vision – Andy Jazz Jan 15 '20 at 10:05
  • 1
    thank you! wonderful.. and will the model be available for offline use? –  Jan 15 '20 at 14:21
  • 1
    According to documentation: `Add machine learning to your app with out-of-the-box solutions (base APIs) running on device or in the Cloud, or custom models, depending on your specific needs`. https://developers.google.com/ml-kit – Andy Jazz Jan 15 '20 at 14:49