How to perform automated detection and cropping of region of interest in an image in python?

Question

I want to perform operation on the region of my interest..that is central rectangular table which you can see in the image. I am able to give the co-ordinates of my region of interest manually and crop that part img = cv2.imread('test12.jpg',0) box = img[753:1915,460:1315]

but i want to crop that part automatically without giving the pixels or coordinates manually.Can anyone please help me with this?

http://picpaste.com/test12_-_Copy-BXqHMAnd.jpg this is my original image.

http://picpaste.com/boxdemo-zHz57dBM.jpg this is my cropped image. for doing this I entered the coordinates of the desired region and cropped. But , now i have to deal with many similar images where the coordinates of my region of interest will slightly vary. I want a method which will detect the table(my region of interest) and crop it. Currently I'm using this img = cv2.imread('test12.jpg',0) box = img[753:1915,460:1315] to crop my image.

Answer 1

You could try using the openCV Template Matching to find the coordinates of your rectangular table within the image. Template Matching

The following is a test program to find the coordinates for images I am trying to find.

from __future__ import print_function
import cv2
import numpy as np
from matplotlib import pyplot as plt

try:
    img = cv2.imread(r'new_webcam_image.jpg',0)
    template = cv2.imread(r'table_template.jpg',0)
except IOError as e:
    print("({})".format(e))
else:
    img2 = img.copy()
    w, h = template.shape[::-1]

# All the 6 methods for comparison in a list
 methods = ['cv2.TM_CCOEFF', 'cv2.TM_CCOEFF_NORMED', 'cv2.TM_CCORR',
             'cv2.TM_CCORR_NORMED', 'cv2.TM_SQDIFF', 'cv2.TM_SQDIFF_NORMED']


for meth in methods:
    img = img2.copy()
    method = eval(meth)

    # Apply template Matching
    res = cv2.matchTemplate(img,template,method)
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)
    print("Method: %s" , meth)
    print("min_val: " , min_val)
    print("max_val: " , max_val)
    print("min_loc: " , min_loc)
    print("max_loc: " , max_loc)
    print(" ")
    # If the method is TM_SQDIFF or TM_SQDIFF_NORMED, take minimum
    if method in [cv2.TM_SQDIFF, cv2.TM_SQDIFF_NORMED]:
        top_left = min_loc
    else:
        top_left = max_loc
    bottom_right = (top_left[0] + w, top_left[1] + h)

    cv2.rectangle(img,top_left, bottom_right, 255, 2)

    plt.subplot(121),plt.imshow(res,cmap = 'gray')
    plt.title('Matching Result'), plt.xticks([]), plt.yticks([])
    plt.subplot(122),plt.imshow(img,cmap = 'gray')
    plt.title('Detected Point'), plt.xticks([]), plt.yticks([])
    plt.suptitle(meth) #; plt.legend([min_val, max_val, min_loc, max_loc], ["min_val", "max_val", "min_loc", "max_loc"])

    plt.show()
    box = img[top_left[1]:top_left[1]+h,0:bottom_right[1]+w]
    cv2.imshow("cropped", box)
    cv2.waitKey(0)

Answer 2

I don't have a full solution for you. The code shown was based on some code I was using to fix output from a scanner. The template solution to me sounds like a better approach, but the following should give you something else to work with.

import cv2

imageSrc = cv2.imread("test12.jpg")

# First cut the source down slightly
h = imageSrc.shape[0]
w = imageSrc.shape[1]
cropInitial = 50
imageSrc = imageSrc[100:50+(h-cropInitial*2), 50:50+(w-cropInitial*2)]

# Threshold the image and find edges (to reduce the amount of pixels to count)
ret, imageDest = cv2.threshold(imageSrc, 220, 255, cv2.THRESH_BINARY_INV)
imageDest = cv2.Canny(imageDest, 100, 100, 3)

# Create a list of remaining pixels
points = cv2.findNonZero(imageDest)

# Calculate a bounding rectangle for these points
hull = cv2.convexHull(points)
x,y,w,h = cv2.boundingRect(hull)

# Crop the original image to the bounding rectangle
imageResult = imageSrc[y:y+h,x:x+w]
cv2.imwrite("test12 cropped.jpg", imageResult)

The output does not crop as much as you need. Playing with the various threshold parameters should improve your results.

I suggest using imshow at various points on imageThresh and imageDest so you can see what is happening at each stage in the code. Hopefully this helps you progress.