sort contours left to right in one line (row), then going down and sort left to right in second line

Question

I am trying to sort contours for OCR to detect them, the problem is that I have tried to sort the contours from left to right and top to bottom but that does not help. I have text on two lines where I want the contours in first line to be sorted left to right and then the same for the second line. Have tried several ways to figure this out but I keep failing at it. Below is a code snippet which shows what I have tried to implement

def get_precedence(contour, cols):
  tolerance_factor = 10
  origin = cv2.boundingRect(contour)
  return ((origin[1] // tolerance_factor) * tolerance_factor) * cols + origin[0]

image = cv2.imread(file)
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)


edged = cv2.Canny(blurred, 30, 150)
cnts, h= cv2.findContours(edged.copy(), cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
cnts.sort(key=lambda x:get_precedence(x, edged.shape[1]))

The resulting order I get is still messed up. The image I am working on is this

Check out [this](https://stackoverflow.com/questions/66946804/python-sorting-items-from-top-left-to-bottom-right-with-opencv/67008153) post. — stateMachine, May 19 '21 at 07:26
Although this does solve the problem of sorting, the code in the link sorts and draw the contour lines on the image. I need to be able to store those contours as you would using these lines : cnts = cv2.findContours(edged.copy(),v2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE) cnts = imutils.grab_contours(cnts) Then I want to be able to iterate over them to input to the OCR model I have trained. — Muaz Shahid, May 19 '21 at 11:11
Separate the words first using a horizontal morphology open kernel to connect the letters. Then loop over the words and separate the letters. — fmw42, May 19 '21 at 16:03

Norbert Tiborcz · Accepted Answer · 2021-05-20T00:19:12.013

1st step: I reduced the data with cv2.boundingRect() from contours.

2nd step: I filtered every segments what touched the edge of the image.

3rd step: I sorted the remaining data from top ('y' values).

4th step: I sliced data to lines by the difference of every following 'y's where it's greater than the median of the character's heights.

5th step: I sorted each lines from left.

code:

import cv2
import numpy as np


def edge_filter(sgmts):
    x, y, w, h = sgmts[:, 0], sgmts[:, 1], sgmts[:, 2], sgmts[:, 3]
    res_y, res_x = image_shape
    return sgmts[((res_x - w - x) * (res_y - h - y) * x * y) != 0]


image = cv2.imread('sample.png')
image_shape = image.shape[:2]
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
edged = cv2.Canny(blurred, 30, 150)
cnts = cv2.findContours(edged, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)[0]

sgmts = np.array([cv2.boundingRect(i) for i in cnts])
filtered = edge_filter(sgmts)
sorted_top = filtered[np.argsort(filtered[:, 1])]

median_height = np.median(sorted_top[:, -1])
diff_y = np.diff(sorted_top[:,1])
new_line = np.where(diff_y > median_height)[0] + 1
lines = np.array_split(sorted_top, new_line)

sorted_left = [line[np.argsort(line[:, 0])] for line in lines]

for line, num in zip(sorted_left, range(len(sorted_left))):
    print(f'line {num + 1}:\n {line}\n')

here is my output:

sort contours left to right in one line (row), then going down and sort left to right in second line

1 Answers1