opencv contour detection seems to fail when a line crosses

Question

I'm trying to detect a rectangle and it works perfectly when the background is a flat distinctive color. But when it has drawings or lines that cross... things get weird. How can I detect this rectangle?

Here is what I did: I get the image from the camera, convert it to gray and blur it. Here is the result:

As you can see, my paper is on a very different background in a sofa.

Now I pass it a canny filter:

Imgproc.Canny(mGrayMat, mGrayMat, 75, 200);

Perfect. I would expect this to be the easiest image for OpenCV to find contour and get that rectangle. But it doesn't... that line crossing seems to be the problem because if I move it out of there to another place in the sofa, it works just fine.

Am I missing something? why this rectangle is not detected?

Here is my code for contour detection:

Imgproc.FindContours(inputMat, mContourList, mHierarchy, Imgproc.RetrExternal, Imgproc.ChainApproxSimple);
findQuadrilateral(findGreatestContourByArea(mContourList.ToList(), 10, 0));

static List<MatOfPoint> findGreatestContourByArea(List<MatOfPoint> points, int minRows, int minCols)
        {
            if (points == null || points.Count == 0)
            {
                return null;
            }

            MatOfPoint pointsResult = null;
            MatOfPoint matOfPoint = null;
            double area = 0;

            for (int i=0; i<points.Count; i++)
            {
                matOfPoint = points[i];
                if (matOfPoint == null)
                    continue;

                if (matOfPoint.Rows() < minRows)
                {
                    continue;
                }

                if (matOfPoint.Cols() < minCols)
                {
                    continue;
                }

                double areaTmp = Imgproc.ContourArea(matOfPoint);
                if (areaTmp > area)
                {
                    pointsResult = matOfPoint;
                    area = areaTmp;
                }
            }
            if (pointsResult == null)
                return null;

            List<MatOfPoint> result = new List<MatOfPoint>();
            result.Add(pointsResult);
            return result;
        }
        
        private static Quadrilateral findQuadrilateral(IList<MatOfPoint> mContourList)
        {
            MatOfPoint2f c2f = new MatOfPoint2f();
            for (int i = 0; i < mContourList.Count && i < 2; i++)
            {
                c2f.FromList(mContourList[i].ToList()); 
                // double peri = Imgproc.ArcLength(c2f, true);
                MatOfPoint2f approx = new MatOfPoint2f();

                // l.Count * 0.05
                // 0.02 * Imgproc.ArcLength(c2f, true)
                Imgproc.ApproxPolyDP(c2f, approx, 0.02 * Imgproc.ArcLength(c2f, true), true);

                // select biggest 4 angles polygon
                if (approx.Rows() == 4)
                {

                    return new Quadrilateral(approx);
                }
            }
            return null;
        }

and it returns nothing... any ideas?

For reference I leave here another image with exactly the same problem. Gray And the canny

Opencv doesn't find a rectangle ... It's your Algorithm. So take some time to understand how Findcontour works and . To detect a rectangle in different situations where the output contour isn't a quadrilateral shape , you need to write your algorithm. Using line fitting or hough line detection... etc — Ziri, Dec 23 '19 at 21:32
@ziri In my question there is code there to detect rectangles. And it works perfectly if given a rectangular approximation shape. Do you have code that detects it better? — Joaquin Grech, Dec 23 '19 at 21:59
your code works only in situations where the contour is perfect and has 4 corners otherwise it will fail . So check this answer here : https://stackoverflow.com/questions/10533233/opencv-c-obj-c-advanced-square-detection — Ziri, Dec 24 '19 at 00:10
Here's an approach: Load image, convert to grayscale, Otsu's threshold for binary image, find contours and fill contours using `drawContours` with `-1` for thickness parameter. From here perform morphological closing by creating a rectangular kernel with `getStructuringElement`. This should remove all lines and only leave you with the rectangle — nathancy, Dec 30 '19 at 21:31

score 2 · Answer 1 · answered Dec 24 '19 at 01:39

The easiest way to modify your code to work for those images you posted is to Auto-threshold images before using FindContour (using Otsu for example).

 Imgproc.threshold(inputGray, outputBinary, threshold, 255, Imgproc.THRESH_OTSU);

you'll get a result similar to this :

But for more robustness, you'll need to use some segment detection method and search rectangles based on some criteria.

opencv contour detection seems to fail when a line crosses

1 Answers1