Predict Rectangle in image using opencv python

Question

I want to predict rectangle in the image and draw the rectangle in box shape only using opencv python. i used below code for predict and draw rectangle but its not working properly.

    import numpy as np
    import cv2
    from PIL import Image
    import sys
    
    Path='D:\Artificial intelligence\Phyton'
    filename='Test.png'
    
    
    # Load image, grayscale, Gaussian blur, and Otsu's threshold
    image = cv2.imread('D:\Artificial intelligence\Phyton\Img21122020113231AM.Jpg')
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    thresh = cv2.threshold(gray, 190, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
    
    # Find contours and sort using contour area
    cnts = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    cnts = cnts[0] if len(cnts) == 2 else cnts[1]
    cnts = sorted(cnts, key=cv2.contourArea, reverse=True)
    for c in cnts:
        # Highlight largest contour
        cv2.drawContours(image, [c], -1, (36,255,12), 3)
        break
    
    
    cv2.imwrite(filename+'_Processingimage_color.jpg', image)

My Input Image :

Input Image

My Result :

My Result image

Answer 1

For the shape-detection there is a great-tutorial called opencv-shape-detection. However the pre-processing in the tutorial won't help you to find the big-box in the image. You need to apply adaptiveThreshold instead threshold. Here are the steps:

• 1. Resize the image and calculate the ratio
• 2. Smooth the image
• 3. Apply adaptive-threshold
• 4. Find and grab the contours.
• 5. Calculate perimeter and the approximate length
• 6. If length equals to 4 (means either square or rectangle), draw the contour.

---

• Step-1

  • We resize the image to make the computation and detection easier. However, we also need to calculate the ratio so we don't lose the center of each contour.

• Step-2

  • 

  • We applied gaussian-blur to smooth the image. Most of the artifacts in the image was removed.

  • blr = cv2.GaussianBlur(gry, (5, 5), 0)
    

• Step-3

  • Simple-thresholding was not producing satisfactory results with different parameters. Therefore I used adaptiveThreshold to get the result:

  • 

  • thr = cv2.adaptiveThreshold(blr, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 41, 21)
    

• Step-4

  • Finding the contour in the same way you did with a different hierarchy parameter. See Contours-hierarchy

• Step-5

  • For each contour, perimeter and approximation parameters were calculated. See Contour-features

• Step-6

  • If the approximation length is equals to 4, draw the contour. Result will be:

  • 

---

Code:

---

import cv2
import imutils

# Load the image
img = cv2.imread("zE2lg.jpg")

# Resize the image
rsz = imutils.resize(img, width=300)

# Calculate the ratio
ratio = img.shape[0] / float(rsz.shape[0])

# Convert to gray-scale
gry = cv2.cvtColor(rsz, cv2.COLOR_BGR2GRAY)

# Apply Gaussian-blur
blr = cv2.GaussianBlur(gry, (5, 5), 0)

# Apply threshold
thr = cv2.adaptiveThreshold(blr, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 41, 21)

# Find and grab contours
cnt = cv2.findContours(thr.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnt = imutils.grab_contours(cnt)

# Loop over contours
for c in cnt:
    mmn = cv2.moments(c)

    if mmn["m00"] != 0:
        x = int((mmn["m10"] / mmn["m00"]) * ratio)
        y = int((mmn["m01"] / mmn["m00"]) * ratio)

        prm = cv2.arcLength(c, True)  # perimeter
        apx = cv2.approxPolyDP(c, 0.09 * prm, True)  # approximation

        if len(apx) == 4:
            c = c.astype("float")
            c *= ratio
            c = c.astype("int")
            cv2.drawContours(img, [c], -1, (0, 0, 255), thickness=5)

Answer 2

You should be able to detect your cardboard box in this particular image by following the below steps. You should be able to do all of this with numpy and OpenCV. It will be a lot of work though, so I haven't done it. If someone else wants to go through with it and provide the source code, feel free to mark their answer as the correct one and not this one.

1. Convert a copy of the image to HSV colorspace.
2. Detect your cardboard color, which is like a dark orange or dark yellow. If the HSV ranges are 0-360, 0-255, 0-255, then the color range you would want to detect would be around 20-60, 20-255, 20-100.
3. Convert the result of that to a 2d black and white image.
4. Perform morphological erosion with a small kernel size to get rid of noisy specks. Maybe a kernel size of around 3x3.
5. Perform morphological dilation with a large kernel size in order to connect any disconnected regions.　Maybe a kernel size of around 20x20.
6. Find the largest contour object.
7. Convert the contour to a best fit quadrilateral in order to get rid of the smaller cardboard box on the right. I've sometimes found it helpful to first convert the contour to a convex hull before converting it into a quadrilateral.
8. Draw your quadrilateral on your original image. Or crop it out. Or whatever you want to do with it.

This stackoverflow post will help with step 7: How to force approxPolyDP() to return only the best 4 corners? - Opencv 2.4.2

Another alternative potential solution to the method I outlined above (after detecting the brown cardboard color) can be found here: Extracting the dimensions of a rectangle