2

I am working on a program where I am trying to extract the colored squares from a puzzle. I take a frame from a video capture then find the all contours. I then remove contours that aren't in the shape of a square (This works alright but looking for a better method). The main problem I am facing is that there are overlapping contours. I use RETR_TREE to get all contours but when using RETR_EXTERNAL The contours become harder to detect. Is there a way I can improve the detection of squares? Or a way that I can remove the overlapping contours in the image.

Here is an image of where there are overlapping contours: There were 11 contours found in this image but I want only 9.(I draw the rects to see the overlapping a little easier) Overlapping image

. How can I remove the inner contours? Here is my code:

public Mat captureFrame(Mat capturedFrame){

    Mat newFrame = new Mat();
    capturedFrame.copyTo(newFrame); 

    //Gray
    Mat gray = new Mat();
    Imgproc.cvtColor(capturedFrame, gray, Imgproc.COLOR_RGB2GRAY);

    //Blur
    Mat blur = new Mat();
    Imgproc.blur(gray, blur, new Size(3,3));
    //Canny image
    Mat canny = new Mat();
    Imgproc.Canny(blur, canny, 20, 40, 3, true);

    //Dilate image to increase size of lines
    Mat kernel = Imgproc.getStructuringElement(1, new Size(3,3));
    Mat dilated = new Mat();
    Imgproc.dilate(canny,dilated, kernel);


    List<MatOfPoint> contours = new ArrayList<>();
    List<MatOfPoint> squareContours = new ArrayList<>();

    //find contours
    Imgproc.findContours(dilated, contours, new Mat(), Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_NONE);


    //Remove contours that aren't close to a square shape.
    //Wondering if there is a way I can improve this?
    for(int i = 0; i < contours.size(); i++){

        double area = Imgproc.contourArea( contours.get(i)); 
        MatOfPoint2f contour2f = new MatOfPoint2f(contours.get(i).toArray());

        double perimeter = Imgproc.arcLength(contour2f, true);
        //Found squareness equation on wiki... 
        //https://en.wikipedia.org/wiki/Shape_factor_(image_analysis_and_microscopy)
        double squareness = 4 * Math.PI * area / Math.pow(perimeter, 2);

        //add contour to new List if it has a square shape.
        if(squareness >= 0.7 && squareness <= 0.9 && area >= 3000){
           squareContours.add(contours.get(i));
        }
    }

    MatOfPoint2f approxCurve = new MatOfPoint2f();
    for(int n = 0; n < squareContours.size(); n++){

        //Convert contours(n) from MatOfPoint to MatOfPoint2f
        MatOfPoint2f contour2f = new MatOfPoint2f( squareContours.get(n).toArray());
        //Processing on mMOP2f1 which is in type MatOfPoint2f
        double approxDistance = Imgproc.arcLength(contour2f, true)*0.02;
        Imgproc.approxPolyDP(contour2f, approxCurve, approxDistance, true);

        //Convert back to MatOfPoint
        MatOfPoint points = new MatOfPoint( approxCurve.toArray());

        // Get bounding rect of contour
        Rect rect = Imgproc.boundingRect(points);
        //length and width should be about the same
        if(rect.height - rect.width < Math.abs(10)){
            System.out.printf("%s , %s \n", rect.height, rect.width);
        }

         // draw enclosing rectangle (all same color, but you could use variable i to make them unique)
        Imgproc.rectangle(newFrame, new Point(rect.x,rect.y), new Point(rect.x+rect.width,rect.y+rect.height),new Scalar (255, 0, 0, 255), 3); 

        }  

    return newFrame;
}
Jeru Luke
  • 20,118
  • 13
  • 80
  • 87
ProgrammingCuber
  • 377
  • 4
  • 14
  • Hey.. We meet again from OpenCV ;). So if I understood you correctly, you want to remove the contours in red? – eshirima Aug 13 '17 at 01:14
  • Yes we meet again :). When I detect the contours I find some that overlap each other, I draw the rects to see the overlapping contours easier. I want to remove the overlapping contours. As you can see in the image I have 2 rectangles on top of each other because there were 2 contours that were overlapping. @eshirima – ProgrammingCuber Aug 13 '17 at 01:18
  • Why not filter them out by their area? I'm pretty sure there'll be some contours whose [area](http://docs.opencv.org/trunk/d3/dc0/group__imgproc__shape.html#ga2c759ed9f497d4a618048a2f56dc97f1) is too small. You'd've to mess around with different values to see which one makes sense – eshirima Aug 13 '17 at 01:34
  • I do filter out them by area already. If the contour area is less than 3000 it doesn't get added. It's that sometimes it detects the black outside as a separate contour then the sticker as another contour inside the black outline. @eshirima – ProgrammingCuber Aug 13 '17 at 01:38
  • My bad...I'm really sorry.. Its getting pretty late were I am. A different approach. Instead of removing them, why not try to group them together? One is by using their [bounding boxes](https://stackoverflow.com/questions/19079619/efficient-way-to-combine-intersecting-bounding-rectangles). [Here's](https://stackoverflow.com/questions/8973017/opencv-c-obj-c-connect-nearby-contours) another way. – eshirima Aug 13 '17 at 01:47

1 Answers1

5

Fortunately, cv::findContours also provides us with hierarchy matrix, which you have ignored in your snippet, the hierarchy is very useful for all modes other than RETR_EXTERNAL, You may find the detailed doc of hierarchy matrix here.

Hierarchy matrix contains the data in the format [Next, Previous, First_Child, Parent] for each contour. Now you may filter contours, which a logic such as select only those contours where parent == -1, this will eliminate the sub contours inside the parent contour.

To make use of the hierarchy matrix you need to call cv::findContours as:

cv::Mat hierarchy;
Imgproc.findContours(dilated, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_NONE);
ZdaR
  • 22,343
  • 7
  • 66
  • 87