Python & OpenCV: How to crop half-formed bounding boxes

Question

I have a script that creates gridlines for a gridless table:

Before script:

After script:

Is there a simple way, using OpenCV, to crop the "after script" image so that it includes only the four-sided bounding boxes? Sample output:

EDIT:

I'm currently working on a solution that finds the first/last all-black pixel line going vertically/horizontally. It'll work, but was wondering whether there's something more elegant.

Answer 1

Here is one way to do that in Python/OpenCV by getting the min and max x and y from all the contours except the largest.

Input:

import cv2
import numpy as np

# read image
img = cv2.imread('test_table.png')
hh, ww = img.shape[:2]

# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# threshold
thresh = cv2.threshold(gray, 128, 255, cv2.THRESH_BINARY)[1]

# crop 1 pixel and add 1 pixel white border to ensure outer white regions not considered small contours
thresh = thresh[1:hh-1, 1:ww-1]
thresh = cv2.copyMakeBorder(thresh, 1,1,1,1, borderType=cv2.BORDER_CONSTANT, value=(255,255,255))

# get contours
contours = cv2.findContours(thresh, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)

# get min and max x and y from all bounding boxes larger than half the image size
area_thresh = hh * ww / 2
xmin = ww
ymin = hh
xmax = 0
ymax = 0

for cntr in contours:
    area = cv2.contourArea(cntr)
    if area < area_thresh:
        x,y,w,h = cv2.boundingRect(cntr)
        xmin = x if (x < xmin) else xmin
        ymin = y if (y < ymin) else ymin
        xmax = x+w-1 if (x+w-1 > xmax ) else xmax
        ymax = y+h-1 if (y+h-1 > ymax) else ymax


# draw bounding box     
bbox = img.copy()
cv2.rectangle(bbox, (xmin, ymin), (xmax, ymax), (0, 0, 255), 2)

# crop img at bounding box, but add 2 all around to keep the black lines
result = img[ymin-3:ymax+3, xmin-3:xmax+3]

# save results
cv2.imwrite('test_table_bbox.png',bbox)
cv2.imwrite('test_table_trimmed.png',result)

# show results
cv2.imshow("thresh", thresh)
cv2.imshow("bbox", bbox)
cv2.imshow("result", result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Bounding Box of all the bounding boxes on input:

Trimmed Image:

Answer 2

Note: I know that there's already an accepted answer, but I want to provide a more simple version.

Basically, first find the contours of every shape in the image (every cell) that has an area greater than a chosen number that will filter out any noise.

Loop through the contours, and find the smallest and greatest x and y coordinates. With the 4 points, we can save the pixels from the image within the four coordinates into a separate array, fill in the original image with white, and draw the table back onto the image.

The code:

import cv2

img = cv2.imread("table.png")
h, w, _ = img.shape

x1, y1 = w, h
x2, y2 = 0, 0

contours, _ = cv2.findContours(cv2.Canny(img, 0, 0), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

for cnt in contours:
    if cv2.contourArea(cnt) > 1000:
        x1 = min(cnt[..., 0].min(), x1)
        y1 = min(cnt[..., 1].min(), y1)
        x2 = max(cnt[..., 0].max(), x2)
        y2 = max(cnt[..., 1].max(), y2)

pad = 2
x1 -= pad
y1 -= pad
x2 += pad * 2
y2 += pad * 2

table = img[y1:y2, x1:x2].copy()

img.fill(255)
img[y1:y2, x1:x2] = table
cv2.imshow("lined_table.png", img)
cv2.waitKey(0)

Output:

Explanation:

1. Import the opencv module and read in the image. Get the dimensions of the image and define temporary coordinates for the first corner of the table and the last corner of the table:

import cv2

img = cv2.imread("table.png")
h, w, _ = img.shape

x1, y1 = w, h
x2, y2 = 0, 0

2. Get the contours of the image, and loop through each contour, filtering out the contours with an area of less than 1000:

contours, _ = cv2.findContours(cv2.Canny(img, 0, 0), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

for cnt in contours:
    if cv2.contourArea(cnt) > 999:

3. Update the values of the coordinates of the first corner and last corner of the table:

        x1 = min(cnt[..., 0].min(), x1)
        y1 = min(cnt[..., 1].min(), y1)
        x2 = max(cnt[..., 0].max(), x2)
        y2 = max(cnt[..., 1].max(), y2)

4. Apply a pad around each coordinate, depending on the width of the lines:

pad = 2
x1 -= pad
y1 -= pad
x2 += pad * 2
y2 += pad * 2

5. Copy part of the image into a variable according to the found x and y coordinates, empty the image, and redraw the table onto the image. Finally, show the image:

table = img[y_1:y_2, x_1:x_2].copy()

img.fill(255)
img[y_1:y_2, x_1:x_2] = table
cv2.imwrite("lined_table.png", img)
cv2.waitKey(0)