1

I am working on a problem that extracts rectangular boxes from image and sort those rectangular boxes sequentially. Code that I tried is:

import cv2
import matplotlib.pyplot as plt
# Load image, grayscale, adaptive threshold
image = cv2.imread('3.jpg')
result = image.copy()
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.adaptiveThreshold(gray,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV,51,9)

# Fill rectangular contours
cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    cv2.drawContours(thresh, [c], -1, (255,255,255), -1)

# Morph open
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9,9))
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel, iterations=4)
plt.imshow(thresh)

# Draw rectangles
cnts = cv2.findContours(opening, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
cnts = sorted(cnts, key=lambda x: [cv2.boundingRect(x)[1], cv2.boundingRect(x)[0]])
# cnts = sorted(cnts, key=lambda ctr: cv2.boundingRect(ctr)[0] + cv2.boundingRect(ctr)[1] * image.shape[1], reverse=True)
i = 0
for c in cnts:
    area = cv2.contourArea(c)
#     print("Area:", area)
    x,y,w,h = cv2.boundingRect(c)
    
#     cv2.rectangle(image, (x+8, y+8), (x + w-8, y + h-8), (36,255,12), 1)
    roi = image[y:y+h, x:x+w]
    cv2.imwrite('{}.jpg'.format(i), roi)
    i += 1


cv2.imwrite('output_image.jpg', image)
cv2.imshow('thresh', thresh)
cv2.imshow('opening', opening)
cv2.imshow('image', image)
cv2.waitKey()
cv2.destroyAllWindows()

Image used to detect and extract contours: here. I want to extract the boxes containing numbers in order 1-2-3-4-5-6-7-----32. The above code extracts boxes in random order, sometimes from left-to-right and sometimes from right-to-left.

[Edited] Images that yield contour in order 4-3-2-1 8-7-6-5... 1 2
Images that yield contour in order 1-2-3-4 5-6-7-8... 3 5

Prabin Nepal
  • 41
  • 1
  • 4
  • 2
    Does this answer your question? [How to sort a list of x-y coordinates](https://stackoverflow.com/questions/37111798/how-to-sort-a-list-of-x-y-coordinates) – HansHirse Jan 26 '21 at 07:45
  • It worked partially. I tested on 6 images and 4 images returned contours sequentially but contours of 2 images are obtained in the order 4-3-2-1..8-7-6-5..12-11-10-9........ – Prabin Nepal Jan 26 '21 at 08:54
  • 1
    Please add two of the successful and both erroneous images to your question by [editing](https://stackoverflow.com/posts/65897395/edit) it. – HansHirse Jan 26 '21 at 08:57
  • I have edited the question. – Prabin Nepal Jan 26 '21 at 09:15
  • See https://www.pyimagesearch.com/2016/03/21/ordering-coordinates-clockwise-with-python-and-opencv/ – fmw42 Jan 26 '21 at 20:18
  • Hi! Does this solve your problem? [Python: Sorting items from top left to bottom right with OpenCV](https://stackoverflow.com/q/66946804/13552470) – Red Oct 29 '21 at 12:14
  • It worked partially on my case. – Prabin Nepal Oct 30 '21 at 13:09

1 Answers1

2

It's the orientation – or: the rotation – of the paper, that decides, whether simply checking the (x, y) coordinates of the contours' bounding boxes is sufficient or not.

So, I'd simply stick to some good old comparison method, e.g. this earlier answer from me on the same issue implemented in C++. Unfortunately, using the old way using the cmp parameter in sorted was removed entirely in Python 3. Nevertheless, we can use cmp_to_key from the functools module to get that functionality mapped to a proper key function.

Now, first, let's create the comparison method:

def contour_sort(a, b):

    br_a = cv2.boundingRect(a)
    br_b = cv2.boundingRect(b)

    if abs(br_a[1] - br_b[1]) <= 15:
        return br_a[0] - br_b[0]

    return br_a[1] - br_b[1]

The abs(...) <= 15 (or any other proper threshold) is needed for the cases, where the right boxes are higher than the left boxes (i.e. the main problem).

Then, add this line to your imports:

from functools import cmp_to_key

Finally, replace the key function in your sorted call to that:

cnts = sorted(cnts, key=cmp_to_key(contour_sort))

Doing so, I can extract the correct order for all the given images.

If someone can provide a simple key function without the need of a comparison method, I'd be very interested in that! I couldn't find one myself.

----------------------------------------
System information
----------------------------------------
Platform:      Windows-10-10.0.16299-SP0
Python:        3.8.5
OpenCV:        4.5.1
----------------------------------------
HansHirse
  • 18,010
  • 10
  • 38
  • 67