Detect handwritten characters in boxes from a filled form using Fourier transforms

Question

I am trying to extract handwritten characters from boxes. The scanning of the forms is not consistent, so the width and height of the boxes are also not constants.

Here is a part of the form.

My current approach:
1. Extract horizontal lines
2. Extract vertical lines
3. Combine both the above images
4. Find contours ( used opencv)

This approach gives me most of the boxes. But, when the box is filled with characters like "L" or "I", the vertical line in the character is also getting extracted as a part of vertical lines extraction. Hence the contours also get messed up.
Since the boxes are arranged periodically, is there a way to extract the boxes using Fast Fourier transforms?

You can try use cv2.InRange() in HSV colorspace for blue ink select. — Alex Alex, Jun 19 '20 at 14:08

score 1 · Accepted Answer · answered Jun 07 '20 at 21:59

I recently came up with a python package that deals with this exact problem.
I called it BoxDetect and after installing it through:

pip install boxdetect

It may look somewhat like this (you need to adjust parameters for different forms:

from boxdetect import config

config.min_w, config.max_w = (20,50)
config.min_h, config.max_h = (20,50)
config.scaling_factors = [0.4]
config.dilation_iterations = 0
config.wh_ratio_range = (0.5, 2.0)
config.group_size_range = (1, 100)
config.horizontal_max_distance_multiplier = 2


from boxdetect.pipelines import get_boxes

image_path = "dumpster/m1nda.jpg"
rects, grouped_rects, org_image, output_image = get_boxes(image_path, config, plot=False)

You might want to check below thread for more info:
How to detect all boxes for inputting letters in forms for a particular field?

score 0 · Answer 2 · answered Mar 10 '20 at 13:52

The Fourier transform is the last thing I would think of.

I'd rather try with a Hough line detector to get long lines or as you did, with edge detection, but I would reconstruct the grids explicitly, finding their pitch and the exact locations of the rows/columns, hence every individual cell.

score 0 · Answer 3 · answered Jun 19 '20 at 17:16

You can try select handwritten characters by color. example:

import cv2
import numpy as np
img=cv2.imread('YdUqv .jpg')
#convert to hsv
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
#color definition
color_lower = np.array([105,80,60])
color_upper = np.array([140,255,255])
# select  color objects
mask = cv2.inRange(hsv, color_lower, color_upper)
cv2.imwrite('hand.png', mask)

Result:

Detect handwritten characters in boxes from a filled form using Fourier transforms

3 Answers3