Detecting number crossing line in computer vision

Question

I have a college assignment which considers openCV that I have to solve but we had close to none lectures on computer vision.

Namely we are given around 10 videos in which numbers are crossing 2 lines if they cross one line then they are added to the sum, if they cross second line they will be subtracted from the sum.

Videos: https://drive.google.com/drive/folders/0B1ZJXQY32LBUU3FiTS14a3NZd1U

What I know at the moment is that I would need to use Hough transformation to find lines. And that I will need somehow to detect the numbers.

What my questions are how will I

detect that number has crossed the line?
which number has crossed it?
detect overlap between numbers crossing one line?

I *think* the idea behind such an assignment is that have you solve it. — Jongware, Jan 04 '18 at 15:50
@usr2564301 I would have solved it we had learned python + openCV more than 3 hours. I am asking for directions not for solution. — Milorad, Jan 04 '18 at 16:06
How exactly is a convolutional neural network factoring into this? — bendl, Jan 04 '18 at 16:14
The problem seems to be your school, then. If your questions are purely theoretical, [cs.se] might be a better place to ask. — Jongware, Jan 04 '18 at 16:15

score 2 · Accepted Answer · answered Jan 04 '18 at 16:45

I find it really hard to believe that you had zero computer vision knowledge in this course just to be assigned a fully loaded computer vision assignment. Regardless, since you are simply looking for directions, then these are my recommendations:

For starters, your video feed has random dots which act as noise. Read up morphological operations to get rid of them first. Why? A clean video = higher accuracy.
You are right that hough line can be used for detection. But the next stage is differentiating between the green and blue one. This blog is a good starter on how to go about doing it.

At this point, we have a clean feed with the lines detected respectively. The next task is character recognition where this answer post has a couple recommendations you can explore. You can peek into this and this as well. The second post uses scikit and the standard MNIST dataset. I would recommend you use the second one because the digits in your video feed seem like they're from MNIST.

With the digits detected, you need to find the intersection between the digit contour and a line segment. Consider implementing this suggestion.

Two Cents:

You seriously do not need to train a neural network for this. Why call the fireman to put a match out?
After digit detection, you might want to consider tracking it. Tracking is always less expensive than detection. Ideally, you run detection at the initialization stage, then followed by tracking. Afterwards you simply track then run detection after say 10-20 frames (obviously depending on the application).
If you TRULY haven't been taught ANY computer vision, BUCKLE UP for this.

Cheers :)

Detecting number crossing line in computer vision

1 Answers1