5

I am trying to detect white rectangles in a grayscale image using different approaches: contour detection and Hough transform. Unfortunately there are some limitations of the image I am processing, i.e.

  1. There are many features in the image and rectangle is not the only features
  2. The rectangle could be merged to other features (e.g. one of the rectangle edges could be overlapped with a long straight line)
  3. The rectangle could contain some other features (e.g. letter, numbers or some logo inside the rectangle)
  4. There are some features look like rectangle (e.g. character 'D' looks like a rectangle with small arc on the top right and bottom right; another example is trapezoid instead of parallelogram)
  5. The rectangle could be rotated from 0 to 15 degrees both clockwise and anti-clockwise
  6. It is possible that the lines are broken into several lines in different lighting condition (e.g. 1 pixel gap) so the minimum line length to filter the lines has to be small (e.g. in Hough Transform)
  7. When the minimum line length is set to small value, it is more often to see duplicate lines for the same line in different orientations (i.e. need to combine several lines)

For the contonours approach, the contours of some images are broken. In addition, the image could contain features like rectangle (e.g. character 'D'). I am not sure if this is a good approach.

I have seen many articles/forum suggesting to use Hough transform to detect rectangle like the following post. Unfortunately I have to set the small value of minimum line length and have seen duplicate lines. I have no idea how to deal with the points mentioned above (e.g. combine all the duplicate lines and pick only one line for each edge, how to differentiate the features with most part are lines but with small arcs like 'D', and how to isolate the square with one edge merged with a long straight line, etc).

Hough transformation vs Contour detection for Rectangle recognition with perspective projection

Any suggestions are welcomed!

EDIT: Add some pictures

Character D

Character D

Rect edge merged with long straight line

Rectangle with logo and the edges are merged with long straight line

enter image description here

Trapezoid (with shadow on the top forming trapezoid in the bottom)

Jeru Luke
  • 20,118
  • 13
  • 80
  • 87
chesschi
  • 666
  • 1
  • 8
  • 36

1 Answers1

1

I would suggest that you try using a binary threshold (adaptive or otherwise) on each image, this will give some clear lines for your contour detection. You can also erode/dilate the images to remove noise (such as the thin lines in your second image)

Then use contour detection, and count the contours, finding the largest object in the image with four sides (this will probably be your object).

Make a copy of the image before you use the binary/erode, so that once you have the region of interest from contour detection, you can crop the copy image to that area.

Apologies the example links are written in python, but i'm sure once you get the idea, porting it to C++ will be easy.

Hope this helps.

EDIT

Just tried the above method myself by thresholding each image, contour detection, and then drawing a bounding box around the largest set of contours.

Below see results:

enter image description here

Bounding box around largest set of contours

enter image description here

The same, drawn over the original images

enter image description here

Aphire
  • 1,621
  • 25
  • 55
  • I have done similar thing as you suggested in the past. However I found that the contours of the rectangle could be broken into 2-3 halves in different lighting condition after binary thresholding. It looks to me Hough transform gave a better line. I understand we need to find the intersection points to get the rectangle. However, if there are more than one line for each edge with different orientation, do you know how can I combine the lines into one and find the intersection? For the contours approach, I think this will be more difficult to combine the contours than the Hough lines. – chesschi May 08 '15 at 07:02
  • If you use an adaptive threshold (as seen in the link) you should find the object is not broken up, as the threshold is calculated in several local areas, instead of the entire image, this will ensure lighting conditions will have a far smaller effect on the binary you get at the end. If you find hough lines is giving you better results, maybe stick with that, its always easier I find, to adapt something you are comfortable with to suit your purpose. If you are able to get the XY or each end of the lines you might be able to choose the correct lines, based on those that surround it – Aphire May 08 '15 at 07:38
  • I remembered I have also tried adaptive threshold but it gave worse results. For the hough transform, is there a common practice to "choose" the correct line, or combine the lines into one for each edge? – chesschi May 08 '15 at 11:01
  • I would suggest you revisit theresholding and contours, I just had a try at it and it was quite successful, I used a global threshold to obtain a binary image, then detected contours, and drew a bounding box around the largest set of contours, I will edit my post with images for you – Aphire May 08 '15 at 11:36
  • Many thanks for your effort... but the main objective is to identify only rectangle (e.g. bicycle) and filter out all other rectangular-ish objects like character 'D' and trapezoid. I am sorry if I did not make it clear in the question. – chesschi May 08 '15 at 15:56
  • Ah right, sorry I think I misunderstood. In which case, I would still suggest using the binary threshold before you try your houghlines, as it makes a much cleaner image, hopefully this article might help you : http://opencv-code.com/tutorials/automatic-perspective-correction-for-quadrilateral-objects/ Good luck! – Aphire May 08 '15 at 16:27
  • You might be able to use the above link to find an "average corner" by going through all your houghlines, running the example to find corner based on intersecting lines, and then find the areas where the most lines intersect, if you have too many houghlines, this might help pinpoint the right corners – Aphire May 11 '15 at 07:30
  • Interesting point! I did have a look on this post before but have no idea to combine the Hough lines. So are you suggesting not to merge the lines and then find the intersections but you would suggest the other way round. Is that correct? I am not familiar with Hough transform. Is "find the areas where the most lines intersects" a common practice to find the rectangle when there are many hough lines? Many thanks! – chesschi May 11 '15 at 16:37
  • I have to admit, I don't know if its common practise, but it's what I would do :P I think if you use the binary threshold and then canny the image, that should get rid of most of the noise, allowing for your houghlines to be more precise, if you are still getting lots of lines, then you could see how accurate the "average corner" thing works out :) – Aphire May 12 '15 at 07:30