9

I am curious about the logic behind KLT in openCV.

From what I have known so far, the images sent to find optical flow in OpenCV is firstly converted to grayscale.

What I am curious is that, when running the algorithm, we need set of features for computation. What are the features used in finding optical flow method in openCV?

Thank you :)

ra bes
  • 143
  • 1
  • 1
  • 5
  • 1
    most probably harris corners or good-features-to-track. – Abid Rahman K Sep 18 '13 at 06:12
  • There is a brilliant video on KLT here in [this](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=6&cad=rja&uact=8&ved=0ahUKEwj79JuPzIzcAhVLqI8KHRHKB_sQwqsBCHcwBQ&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DtzO245uWQxA&usg=AOvVaw0DaMXtk7H9dfdTO3zNs2o) video. You must check this out – Trect Jul 07 '18 at 08:43

1 Answers1

20

There are 2 types of optical flow. Dense and sparse. Dense finds flow for all the pixels while sparse finds flow for the selected points.

The selected points may be user specified, or calculated automatically using any of the feature detectors available in OpenCV. Most common feature detectors include GoodFeaturesToTrack which finds corners using cornerHarris or cornerMinEigenVal

The feature list is then passed to the KLT Tracker calcOpticalFlowPyrLK.

Feature can be any point in the image. Most common features are corners and edges.

sgarizvi
  • 16,623
  • 9
  • 64
  • 98
  • @e_phi... What do you mean by dynamic background? – sgarizvi Jul 02 '14 at 12:28
  • A background that is changing (i.e. a moving camera). – e_phi Jul 02 '14 at 12:29
  • 1
    Yes that will work, but will fail if there is a sudden drastic change in the background, i.e. the background becomes a completely different scene. Infact different advanced algorithms work utilizing the optical flow. e.g. it can be used for video stabilization as implemented in the OpenCV video stabilization module. – sgarizvi Jul 02 '14 at 12:57
  • I may be getting a little off topic I suppose but besides the median flow tracker which is used in the TLD library, what is the best solution in your opinion for tracking an object in a changing background then? – e_phi Jul 02 '14 at 13:03
  • @e_phi... Well it depends on type of object and the scenario. Different trackers work better in certain conditions. Generally, feature based object tracking using a combination of SIFT/SURF feature descriptors and Flann based feature matcher would do the job. But again, they require parameter tweaking depending on scenario. – sgarizvi Jul 02 '14 at 13:22