11

After discovering the power of OpenCV, I decided to use that library to develop the natural marker tracking engine that I am working on now. But my problem is I have no idea of a proper approach to the implementation of such tracker.

I have devised the following plan:

  1. Use one of the object tracking algorithm (e.g. SIFT, SURF etc.) to describe and extract keypoints from a live camera feed.
  2. Based on the extracted keypoints, convert them to histogram and compare the histogram with histograms of stored markers.
  3. Once a match is found, convert those position information and pass it to the engine responsible for rendering the 3d objects.

I tried the SIFT and SURF algorithm in describing and extracting key points and the end result is super low fps for both algorithm (i.e. less than 0 fps). I notice that SIFT and SURF are quite computationally expensive and will it be suitable for such tracking on a live camera feed?

Thanks.

aaronljx
  • 151
  • 1
  • 6
  • I haven't worked with SURF, but SIFT, well implemented, is awesome not for mobile devices i guess, but with actual avg hardware it works perfectly for live tracking. Sear online articles from Leonardo Chang. Hi is very experienced on that. – jsan Apr 07 '11 at 17:56
  • 1
    The purpose of SURF was to be faster than SIFT... Anyway realtime performance for augmented reality is an issue by itself. i.e. something difficult that depends on your data, your hardware, and the level of accuracy you want. – log0 Apr 10 '11 at 00:12
  • 1
    If interested, you can participate in this new StackExchange proposal: http://area51.stackexchange.com/proposals/30436/augmented-reality – Stéphane Péchard May 02 '11 at 15:07
  • I use FAST, modified for being pyramidalFAST. We achieve more than 20fps – Jav_Rock Jun 29 '12 at 14:03

3 Answers3

4

SIFT is a nice algorithm but you cannot get the best spped from it. There are methods that use FAST for detection and then build a reduced SIFT descriptor of the points detected (instead of 128 values they use, for example, 32). Also pyramidal approaches for FAST were developed (you have ORB, but its descriptors are not good enough).

Now OpenCV has just released FREAK and they promise it is the fastest and robust, so I will try it soon. You can have a look to this kind of tutorial for augmented reality on OpenCV.

Community
  • 1
  • 1
Jav_Rock
  • 22,059
  • 20
  • 123
  • 164
4

Developing such a markers requires you to have a deep knowledge of Image Processing, 3D Imaging, Tracking, etc. Is not like developing a simple application.

Is better to use developed ones ;)

FERNS is a much efficient and simpler than SIFT. You can use it. It was developed by researches at EPFL. If you read AR/Tracking papers you will see these guys are the leaders of the industry/field. It is also implemented in later Versions of OpenCV (I think in 2.1 or 2.2?)

Otherwise you can always get the source code for that algorithm from here: Ferns: Planar Object Detection

EDIT:

Basically algorithms like FERNS will tell you the position/rotation,etc (this is changes are represented by a matrix called Homography) a certain surface will take with reference to another frame. This Homography is everything you need for 3D rendering ;)

Using OpenGL or alike 3D libraries you draw the object using the calculated Homography. If you repeat this process for each frame you will have a simple AR Application.

Theory Books on: Image Processing and 3D Imaging

For understanding AR read: ARToolKit paper

More on FERNS: oezuysal'site

nacho4d
  • 43,720
  • 45
  • 157
  • 240
  • 1
    oh thanks a lot. but i still don't get the whole process of augmented reality. do you have any in depth articles or papers on implementing augmented reality? – aaronljx Apr 10 '11 at 09:20
  • thanks a lot for the information. i will look through them (: – aaronljx Apr 11 '11 at 02:32
1

SIFT and SURF are successful visual features and are probably the right approach (though features faster to compute exist).
SIFT can be computed efficiently on GPU. See siftGPU.

log0
  • 10,489
  • 4
  • 28
  • 62