7

We are making an autonomous robot (in college event) that follows certain signs and directions and goes through a directed route. The robot will have a camera mounted at head. It will follow the signs drawn on the road ahead of it, or the walls and accordingly make its decision. The signs will be GREEN ARROWS (For GO Signal), or RED T's as a sign for halt. The robot will scan these symbols in Real Time and perform the necessary action. These signs can be on wall directly in front or drawn on the path ahead.

I have tried looking for necessary Image transform algorithms or method, but we are quite new into this field. I seek you help on how could this problem be tackled and necessary code that may help us (Assuming us to be starters).

I have looked into following threads but I'm confused : - OpenCV Object Detection - Center Point - How to recognize rectangles in this image? - http://www.chrisevansdev.com/computer-vision-opensurf.html (I'm not able to use it)

One of the hints given for the event was that we can model the arrows as a rectangle and triangle put together as find whether the center of triangle is to the right of that rectangle (which means to go right) or otherwise. Similarly for the T's.

Thanks! :)

Community
  • 1
  • 1
Vivek Rai
  • 890
  • 1
  • 9
  • 22
  • If you can guarantee that the red and green colours are relatively unique in your environment the problem becomes much simpler, you can find the signs by just looking for those colours... – Hammer Dec 17 '12 at 17:13
  • 1
    You need to better specify the scope for this event. Is the environment free of other marks ? Can you show examples for the signs and sample input from your camera ? Do you want this robot to go as fast as possible or is hitting the halt symbol all you want ? If the former, the algorithms that can be applied are reduced. – mmgp Dec 17 '12 at 22:11
  • Yes, the environment is supposed to be free of other distracting colors or images. Just a simple plane colored probably White wall with these figures.. The path could however have other color, but that too assumed to be of same pattern. – Vivek Rai Dec 19 '12 at 16:12

2 Answers2

8

If the signs are previously known you can use the method "recognize objects by feature detection".

The idea is that you have a picture of the sign (the arrow or the T) and you perform the following training steps, offline:

1 - Feature Detection (using, SURF, FAST, ...)

2 - Descriptor Extraction (from the features) using SIFT, FREAK, etc...

Then it comes the real time part. For every frame you have to perform feature detection and descriptor extraction, but then you need to do the matching to the training images to see which object you got. An example that will work real time:

cv::FAST detector;
cv::FREAK descriptor;
BFMatcher matcher = BFMatcher(NORM_HAMMING,false);

detector.detect(frame,keypoints_frame);
descriptor.compute(frame, keypoints_frame,descriptors_frame);
matcher.match(descriptors_trainning, descriptors_frame);

That would be a first approach for the matching, then you need to refine and remove outliers. Some techniques are

  • Ratio Test

  • Cross Check

  • RANSAC + homography

Here you got a complete example.

Jav_Rock
  • 22,059
  • 20
  • 123
  • 164
3

I assume you can get the signs before the event : take the arrow sign, and get "sift descriptors" from him and store them in your robot.

Then in each frame that the robot aquires look for the color of the sign, when you see something that resembles to a sign, take sift descriptors and try to register between the stored descriptors and the new ones. if you succeed try to calculate the rotation and translation matrices between the original stored sign, and the sign you found on the image.

To read about sift, i would recommend on this site : http://aishack.in/tutorials/sift-scale-invariant-feature-transform-introduction/ After you will understand the basics of sift, i recommend to download some implementation instead of implementing it yourself, it's the very tedious job and has many pitfalls

BTW Although sift is "Scale Invariant Feature Transform" i'm pretty sure that it will work in your case also even though you have "perspective transform" also.

Hope it helped

Utkarsh Sinha
  • 3,295
  • 4
  • 30
  • 46
OopsUser
  • 4,642
  • 7
  • 46
  • 71
  • 3
    The idea is good as SIFT descriptors are very robust, the problem is to apply them real time. If the robot has a good processor then there is no problem, but for a college event maybe they are using a tablet, for example, with its integrated cam. Then SIFT won't do it for real time. – Jav_Rock Dec 19 '12 at 08:50
  • And if SIFT is too slow, there's SURF, ORB, BRIEF, etc. –  Dec 24 '12 at 17:24