1

I have a video of simple moving dots (that sometimes overlap) that is saved as a sequence of images. At each image I detect all the dots and save their coordinates:

Image 1 Image 2

(snapshot 1 -> snapshot 2)

I would like to infer the trajectory of each dot. The dots move smoothly and not too fast from one frame to the other, but if for each point of the first image I just find their closest point of the next image it often fails to reconstruct the trajectory.

I tried on opencv the multitrackers but the trackers very quickly lose their target by jumping on a different dot when the dots tend to overlap. The detection works very nicely though.

The video and the objects to track are simple. I do not want to believe that I need to implement something more technical to accurately track these dots. Which is why I decided to ask here, I am out of ideas. Any tip or advice is appreciated... Thanks.

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
Matt
  • 123
  • 4
  • please research tracking. this is known stuff, not a new problem. please review [ask]. -- you need "state". you need *objects* (things with an identity), with a position and a velocity, that you update based on observations. if an observation doesn't explain an existing object, it becomes a new object. if an object is not observed for a while, delete it. associate an observation to the most likely object, based on their previous position and velocity, predicting current position. – Christoph Rackwitz Sep 02 '22 at 09:20
  • 1
    forget opencv's "multitrackers". they live in contrib, they are abandoned gsoc projects, nobody's maintaining them, and they're for actual objects with some appearance, not for mere dots. -- overlap is always difficult, and in your case, with dots having no distinguishing features, next to impossible to get right. – Christoph Rackwitz Sep 02 '22 at 09:22
  • Thank you for your input, but it seems you didn't read my post too attentively: I did research tracking, and also implemented some of the know algorithm for tracking provided by opencv. They failed to track accurately my video. So instead of diver deeper in implementing a robust tracking method, I am asking if I could find an easy solution using detection in images. I believe my question is appropriate as asking the community for a smart solution to a know problem is relevant. – Matt Sep 02 '22 at 09:23
  • I understand, reading your comment I develop the intuition that my problem is not so simple after all. If mapping the coordinates from detection fails, I will turn again towards tracking and work on a more robust solution. Thanks – Matt Sep 02 '22 at 09:25
  • 1
    you didn't show your own effort, so I had to assume there wasn't one. I don't see what you did, so I must assume that my recommendations are new to you (especially the part about velocity, other per-object state... and about your points lacking any way to distinguish them). please review [mre]. – Christoph Rackwitz Sep 02 '22 at 09:25
  • 1
    if you want to be *recommended* an easy solution for tracking near-featureless points... this is a common thing in some of the sciences (flow of air, water, even tracking small insects) so there are programs out there that have been optimized for some tasks... however, recommendations are off-topic. if I had reason to know any of these programs, I'd share recommendations anyway. -- I hope you have heard of Kalman filters... that's just a fancy formulation of the general idea of keeping state, predicting into the future, (associating observations,) updating the state – Christoph Rackwitz Sep 02 '22 at 09:27
  • @Matt "*for each point of the first image I just find their closest point of the next image it often fails to reconstruct the trajectory*", the closest distance between every two consecutive frames should work, if not, then either you might consider a **higher frame rate** to get better trajectory accuracy, or you might provide an **explanation why this approach didn't work**! – Bilal Sep 02 '22 at 10:52
  • "explanation" meaning data for a counterexample... but I can imagine. nearest neighbor association is a greedy matching strategy, sensitive to iteration order, and not optimal, considering almost any cost function based on distances to true matches. when matching gets ambiguous, it may pay to let the detections "coast" through the intersection, only trusting detections after they're obvious again. – Christoph Rackwitz Sep 02 '22 at 10:55

0 Answers0