-3

I am currently using a CNN based object detection module which gives me objects which I then use as input for tracking using OpenCV. The object detection module produced rectangles until now but I want to shift to a segmentation module like Mask-RCNN which outputs masks along with rectangles for each object. Masks are a more accurate representation of an object. All the trackers in OpenCV take rectangles as input. Is there any way to use the masks for tracking an object rather than the boxes. I can convert the masks to contours if that will help me track the object.

Community
  • 1
  • 1
Anand C U
  • 885
  • 9
  • 29
  • afaik there is no built-in way to use masks during tracking in OpenCV, but the code is open source, so you might be able to adapt it. – Micka Mar 27 '18 at 07:25
  • @Micka But most of those methods take rectangles as inputs and just give out x,y and w,h as outputs. Could you explain what parts I will need to change? – Anand C U Mar 28 '18 at 08:37
  • 1
    no, you'll have to understand the code yourself, sorry, and it depends strongly on the specific tracking method. Just as an example: In one of the tracking mechanisms, a cascade classifier is trained on-the-fly. You could probably use the mask to separate the object from the background while creating the training data and introduce additional random/different background to make the training better and the tracking more robust. – Micka Mar 28 '18 at 09:09

2 Answers2

1

Sorry, there is no built-in out-of-box solution in OpenCV for active contour models.

This segmentation model is widely used on computer vision problems (was proposed by Kass on 1988 and is the starting point for other segmentation model based on energies like level sets models, geodesic active contours or fuzzy-snake model.

So, trying to perform the active contour segmentation on OpenCV, there are several solution, but I think you must understand the mathematical model in order to be able to set the parameters properly according to the context of application.

There is a nice implementation (a bit obfuscated) by Eric Yuan

And others implementation from SO, that could help you to link between theory and implementation:

My advice:

  1. Read the original paper to understand the parameters.
  2. Test some examples on Matlab to play a bit with parameters and results.
  3. Test some of the implementation using OpenCV that are linked here.
  4. Determine the best parameters for you problem context and test them.

Think about contributing to OpenCV with you results.

amalbala
  • 118
  • 1
  • 12
0

active contours can track using contours as input. https://www.ee.iitb.ac.in/uma/~krishnan/research.html

So you initialize the first frame using contour from cnn model and in subsequent frames, you don't need to call the expensive forward but able to update the contour to a new one based on this model.

Zaw Lin
  • 5,629
  • 1
  • 23
  • 41