Detecting the edge of a colorful picture OpenCV

Question

I am new to CV and I just learned how to detect the edge of a paper. I want to try something more complicated. So I make a screenshot from a movie website and want to detect the poster from the website. It works well if the background color is different from the poster. But when they are similar in color, I can't find the edge of the picture by cv2.findContours() The original Picture is: Poster

And what I do is:

img = cv2.imread('pic5.jpg')
orig = img.copy()
image = orig
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
binary = cv2.medianBlur(gray,3)
# blur = cv2.GaussianBlur(binary, (5, 5), 0)
# ret, binary = cv2.threshold(blur,127,255,cv2.THRESH_TRUNC)
edged = cv2.Canny(binary, 3, 30)
show(edged)

# detect edge
contours, hierarchy = cv2.findContours(edged.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cnts = sorted(contours, key=cv2.contourArea, reverse=True)[:5]

#
for c in cnts:
    # approx
    peri = cv2.arcLength(c, True)
    eps = 0.02
    approx = cv2.approxPolyDP(c, eps*peri, True)

    # detect square (4 points)
    if len(approx) == 4:
        screenCnt = approx
        break

res = cv2.drawContours(image, [screenCnt], -1, (0, 255, 0), 2)
show(orig)

And the result is: after preprocess What I detect

I don't know if this method works. Is it possible to detect the square part based on the background color (regardless of the poster's color)?

Hi! You may try to do closing operation after you detect edges, so you can fill the gaps in the poster and get something smoother, closer to a rectangle. https://docs.opencv.org/master/d9/d61/tutorial_py_morphological_ops.html — Filip Kubicz, Jun 16 '21 at 21:14
But detecting edges doesn't sound like a great approach here, because it very much depends on the content and color of the poster. You could try locating some fixed elements of the website and later cutting the poster from a known x,y offset. — Filip Kubicz, Jun 16 '21 at 21:15
Another approach would be to traverse the image, and find the bounding box where you have some colors different from background and different from text color. But this could fail if you have a poster which is the same color as the website background. — Filip Kubicz, Jun 16 '21 at 21:18
If this is a real task, then you maybe have to combine several approaches, check how big is the poster that each algorithm finds, and algorithm fails, fallback to another one. If it's only for education, you can move on to some well-defined problem, or follow the great tutorials here: https://docs.opencv.org/master/d6/d00/tutorial_py_root.html — Filip Kubicz, Jun 16 '21 at 21:22
Thanks for your reply! I thought about the situation that the poster has the same color as the website background. But I suppose that only a very small part of the poster has the same color. In that case, even though some parts of the edges are not found, we can find the smallest rectangle that contains the remaining part. — Zakkkk, Jun 16 '21 at 22:44
But I don't know how to implement my idea with python. So I found some pictures on the website and try to understand how those functions on CV2 work in this problem. But it seems too difficult for me now. — Zakkkk, Jun 16 '21 at 22:49
And thanks for the suggestion to start from the well-defined problem. Trying to solve this problem just drives me crazzzzy. — Zakkkk, Jun 16 '21 at 23:24

score 6 · Accepted Answer · answered Jun 16 '21 at 21:40

You may continue with the edged result, and use closing morphological operation for closing small gaps.

Instead of searching for a rectangle using approxPolyDP, I suggest you to find the bounding rectangle of the largest connected component (or largest contour).

In my code sample, I replaced findContours with connectedComponentsWithStats due to the external boundary line.
You may use opening morphological operation to get rid of the external line (and use continue using findContours).

You may also use approxPolyDP for refining the result.

Here is the code sample:

import numpy as np
import cv2

img = cv2.imread('pic5.png')
orig = img.copy()
image = orig
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
binary = cv2.medianBlur(gray, 3)
edged = cv2.Canny(binary, 3, 30)

edged = cv2.morphologyEx(edged, cv2.MORPH_CLOSE, np.ones((5,5)))  # Close small gaps

#contours, hierarchy = cv2.findContours(edged, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
#c = max(contours, key=cv2.contourArea) # Get the largest contour
#x, y, w, h = cv2.boundingRect(c)  # Find bounding rectangle.

nb_components, output, stats, centroids = cv2.connectedComponentsWithStats(edged, 8)  # finding components

# https://stackoverflow.com/a/61662694/4926757
# Find the largest non background component.
# Note: range() starts from 1 since 0 is the background label.
max_label, max_size = max([(i, stats[i, cv2.CC_STAT_AREA]) for i in range(1, nb_components)], key=lambda x: x[1])

# Find bounding rectangle of largest connected component.
x = stats[max_label, cv2.CC_STAT_LEFT]
y = stats[max_label, cv2.CC_STAT_TOP]
w = stats[max_label, cv2.CC_STAT_WIDTH]
h = stats[max_label, cv2.CC_STAT_HEIGHT]

res = image.copy()
cv2.rectangle(res, (x, y), (x+w, y+h), (0, 255, 0), 2)  # Draw a rectangle

cv2.imshow('edged', edged)
cv2.imshow('res', res)
cv2.waitKey()
cv2.destroyAllWindows()

Results:

edged:

res:

It is a great idea to find the largest connected component! And I just thought `cv2.erode(), cv2.dilate(), cv2.morphologyEx()` might be helpful but I am not sure how to use them. Thank you so much for your reply! — Zakkkk, Jun 16 '21 at 23:02

Detecting the edge of a colorful picture OpenCV

1 Answers1