Digit recognizing, using opencv

Question

I have 4 small images.

There are digits 6, 16 , 9 and 9. I compare pics with digits with my templates, there are only 30 variants [0-30]. Pics - screenshots are. Digits appear at different place at squares (9 in left corner and 9 in right corner on examples).

I use two ways: calculate quanity of pixels with white colour.

original = cv2.imread('im/16.png')
sought = [254,254,254]
result = np.count_nonzero(np.all(original==sought,axis=2))

This way work always, except 6 and 9. Quanity of white pixels are equal in this case.

Second way: obtain location of all pixels on image and compare data with other image:

# tit - title of image
# same list for equal images
difference = cv2.subtract(original,image_to_compare)
b,g,r = cv2.split(difference)
cv2.countNonZero(b) == 0:
    same.append(tit)
if len(same) > 1:
    res = same
    print(res)

This method help me distinguish 6 from 9, but! In two images with 9 in different corners it also recognise a difference.

I want my code recognise every digit and not see a difference between one digit in right or left sides of images.

Possible duplicate of [How to classify blurry numbers with openCV](https://stackoverflow.com/questions/37645576/how-to-classify-blurry-numbers-with-opencv) — en_Knight, Aug 29 '18 at 21:25

ralf htp · Answer 1 · 2018-08-29T18:42:19.950

In opencv you can train a classifier on both 6 and 9 like Haar Feature-based Cascade Classifier for Object Detection ( https://docs.opencv.org/3.4/d5/d54/group__objdetect.html , https://docs.opencv.org/3.4/dc/d88/tutorial_traincascade.html )

Example code is in i.e. https://coding-robin.de/2013/07/22/train-your-own-opencv-haar-classifier.html

i do not know if this is an assignment and if you are fixed to opencv if not you can use neural networks, cf chapter Tunnel Vision of https://medium.com/@ageitgey/machine-learning-is-fun-part-3-deep-learning-and-convolutional-neural-networks-f40359318721 or https://towardsdatascience.com/convnets-series-spatial-transformer-networks-cff47565ae81 however spatial transformer networks are a bit overcomplicated for this problem

score 3 · Accepted Answer · answered Oct 04 '18 at 11:11

You can find many papers and software about OCR, because it is widely used in many applications. I want to present quite simple solution for your problem, using numpy and opencv, that will do the job.

What we will do:

Import numpy and opencv
Load images you have provided
Treshold them
Make function, that will return array of digits in given image
Compare digit from image 1 and image 2
Make our "bank of digits" so we know how number 9 looks like
Compare digits we found in image 3 with our "bank of digits"

Code:

import cv2
import numpy as np

treshold = 70

#Treshold every image, so "0" in image means no digit and "1" is digit 
image1 = (cv2.imread("number_1.png",0) > treshold).astype(np.uint8)
image2 = (cv2.imread("number_2.png",0) > treshold).astype(np.uint8)
image3 = (cv2.imread("number_3.png",0) > treshold).astype(np.uint8)
image4 = (cv2.imread("number_4.png",0) > treshold).astype(np.uint8)

Function, that will return array of digits in given image:

def get_images_of_digits(image):
    components = cv2.connectedComponentsWithStats(image, 8, cv2.CV_16U) #Separate digits
    #Get position of every components
    #For details how this works take a look at 
    #https://stackoverflow.com/questions/35854197/how-to-use-opencvs-connected-components-with-stats-in-python
    position_of_digits = components[2] 
    number_of_digits = len(position_of_digits) - 1 #number of digits found in image
    digits = [] #Array with every digit in image
    for i in range(number_of_digits):    
        w = position_of_digits[i+1,0] #Left corner of digit
        h = position_of_digits[i+1,1] #Top corner of digit
        digit = image[h:h+height_of_digit,w:w+width_of_digit] #Cut this digit out of image

        #Count how many white pixels there are
        px_count = np.count_nonzero(digit)
        #Divide every pixel by square root of count of pixels in digit. 
        #Why? If we make convolution with the same digit it will give us sweet "1", which means these digits are identical
        digit = digit / np.sqrt(px_count)  
    digits.append(digit)

return digits #Return all digits

Get digits

d_1 = get_images_of_digits(image1)[0] #Digit "9" from first image
d_2 = get_images_of_digits(image2)[0] #Digit "9" from second image
d_3 = get_images_of_digits(image4)[0] #Digit "6" from last image

print(cv2.filter2D(d_1,-1,d_2).max()) #Digit "9" on image 1 and 2 match perfectly (result of convolution is 1).
#Filter2D does convolution (correlation to be precise, but they are the same for our purpose)

Put number "9" from first image and number "6" from last image into digit bank. Then go trough every number we find in image 3 and compare it with our digit bank. If score is below 0.9, it is not match.

bank_of_digits = {"9":d_1, "6":d_3} 
for digit in get_images_of_digits(image3):
    #print(digit)
    best_restult = 0.9 #If score is above 0.9, we say it is match
    #Maybe tweak this higher for separating chars "8" and "9" and "0"
    matching_digit = "?" #Default char, when there is no match
    for number in bank_of_digits:
        score = cv2.filter2D(digit,-1,bank_of_digits[number]).max() #Returns 0-1 . 1 Means perfect match       
        print("Score for number " + number +" is: "+ str(np.round(score,2)) )
        if score > best_restult: #If we find better match
            best_restult = score #Set highest score yet
            matching_digit = number #Set best match number
    print("Best match: " + matching_digit)

Final result then will be "?" for first digit in image 3, because there is no number "1" in our bank, and second result will be "6" with score of 0.97.

TLDR: I made algorithm that separates digits from your images, and compares these digits. Best matches are printed.

Digit recognizing, using opencv

2 Answers2

Linked