13

I'm working on an augmented reality app for iPhone that involves a very processor-intensive object recognition algorithm (pushing the CPU at 100% it can get through maybe 5 frames per second), and in an effort to both save battery power and make the whole thing less "jittery" I'm trying to come up with a way to only run that object recognizer when the user is actually moving the camera around.

My first thought was to simply use the iPhone's accelerometers / gyroscope, but in testing I found that very often people would move the iPhone at a consistent enough attitude and velocity that there wouldn't be any way to tell that it was still in motion.

So that left the option of analyzing the actual video feed and detecting movement in that. I got OpenCV working and tried running their pyramidal Lucas-Kanade optical flow algorithm, which works well but seems to be almost as processor-intensive as my object recognizer - I can get it to an acceptable framerate if I lower the depth levels / downsample the image / track fewer points, but then accuracy suffers and it starts to miss some large movements and trigger on small hand-shaking-y ones.

So my question is, is there another optical flow algorithm that's faster than Lucas-Kanade if I just want to detect the overall magnitude of camera movement? I don't need to track individual objects, I don't even need to know which direction the camera is moving, all I really need is a way to feed something two frames of video and have it tell me how far apart they are.

Ertebolle
  • 2,322
  • 2
  • 18
  • 22

1 Answers1

3

Why not use the combo of the accelerometer/gyro motion sensing, and a very low res image tracker? Each method seems to be confused by completely different user motions.

hotpaw2
  • 70,107
  • 14
  • 90
  • 153
  • Interesting idea... the question is whether the ranges of motions that they can detect will actually overlap or whether they'll be some types of motion that neither one can pick up reliably without getting a lot of hand-shaking interference. I'll try this out and report back shortly, thanks! – Ertebolle Dec 09 '10 at 01:14
  • This turned out to be the winning strategy, though it took some extra refining of OpenCV's L-K code and a very lightweight additional image check to make it work well - thanks! – Ertebolle Dec 11 '10 at 14:29
  • 1
    @Ertebolle do you have any sample code for your stuff to start on same? thanks in advance. – Sandeep Singh Jun 23 '16 at 11:32