image rectangle extraction with cv2

Question

I work on a image from which I extracted edges and where I did morphological closing on vertical and horizontal lines.

I want to keep only the rectangle on the middle from this image :

I am a little bit out of ideas on how to do that. Can you help me ?

Answer 1

You may close the gaps in rectangle using dilate, then look for the largest contour that also has contours inside.

The rectangle is the largest contour, but finding the largest contour seams too specific.
For generalizing the solution, I suggest looking for the largest contour that has another contour inside it.
The assumption is that the rectangle is distinguished by it's "child contours".

Looking for the grandchild:
The Contours Hierarchy is a bit difficult to understand...
We need to handle the fact that an empty contour is composed of two contours - the outer contour and the inner contour (the inner contour is a child of the outer contour).
Instead of looking for contour that has a child, we need to look for contour that has a grandchild.

You may use the following stages:

• Use "dilate" morphological operation for closing the gaps between the lines of the rectangle.
• Find contours and hierarchy, use RETR_TREE for creating a tree of contours within contours.
• Sort contours (and hierarchy) by area - sort in descending order so the largest comes first.
• Search the contour with largest area that has a grandchild contour.

---

Here is a code sample:

import numpy as np
import cv2

img = cv2.imread('rect_img.png', cv2.IMREAD_GRAYSCALE)  # Read input image as Grayscale
img = img[10:-10, 10:-10]  # Crop the margins
thresh = cv2.threshold(img, 0, 255, cv2.THRESH_OTSU)[1]  # Convert to binary (the input image is not binary).

# Use "dilate" morphological operation for closing the gaps between the lines of the rectangle
thresh = cv2.dilate(thresh, np.ones((3,3)))

# Find contours and hierarchy, use RETR_TREE for creating a tree of contours within contours
cnts, hiers = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)[-2:]  # [-2:] indexing takes return value before last (due to OpenCV compatibility issues).

hiers = hiers[0]  # Remove redundant dimensions of hierarchy

# Merge two lists to list of pairs (before sorting), cnts_hiers is [(c,h), (c,h), (c,h), ...]
# https://stackoverflow.com/questions/2407398/how-to-merge-lists-into-a-list-of-tuples
cnts_hiers = zip(cnts, hiers)

# Sort contours (and hierarchy) by area - the largest comes first.
cnts_hiers = sorted(cnts_hiers, key=lambda c: cv2.contourArea(c[0]), reverse=True)

# Iterate the sorted contours (and hierarchy):
# Look for contour with largest area that have a grandchild contour
# Note: we are looking for a grandchild because each empty contour has an inner contour and an outer contour.
for c_h in cnts_hiers:
    c = c_h[0]  # The first index of c_h is the contour
    h = c_h[1]  # The second index of c_h is the hierarchy of the contour

    # https://docs.opencv.org/master/d9/d8b/tutorial_py_contours_hierarchy.html
    # OpenCV represents hierarchy as an array of four values : [Next, Previous, First_Child, Parent]
    # If there is no child or parent, that field is taken as -1

    # Check if contour has at least one child contour, and than check if the child has a child (look for grandchild)
    if h[2] >= 0:
        child_idx = h[2]  # Get the index of the child (the child is the "inner contour").
        child_hier = hiers[child_idx]  # Get the hierarchy that applies the index of the child.

        # Check if the child has a child (look for grandchild)
        if child_hier[2] >= 0:
            best_cnt = c  # The best candidate contour was found.
            break  # Break the loop


img = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)  # Convert to BGR before drawing the contour.

# Mark best_cnt with green line
cv2.drawContours(img, [best_cnt], 0, (0, 255, 0), 2)

cv2.imshow('img', img)  # Show result for testing
cv2.waitKey()
cv2.destroyAllWindows()

---

Result:

---

Cropping the rectangle:

rect = cv2.minAreaRect(best_cnt)  # Find rectangle with minimum area
box = cv2.boxPoints(rect)
box = np.int0(box) # convert all coordinates floating point values to int
(topy, topx) = (np.min(box[:,1]), np.min(box[:,0]))  # https://stackoverflow.com/questions/28759253/how-to-crop-the-internal-area-of-a-contour
(bottomy, bottomx) = (np.max(box[:,1]), np.max(box[:,0]))
rect = img[topy:bottomy+1, topx:bottomx+1, :]

---

Result: