How to detect gas station prices with python and OpenCV

Question

I am trying to create a python script to extract gas price information from gas station totems. I'm thinking about using number recognition that is built in opencv-python, but firstly I need to somehow extract the totem from the picture itself.

For example, from this image: Benzina gas tation totem with red prices img

The output would be something like:

• Diesel - 32.20
• Gas 95 - 32.20
• LPG - 13.90

It's fine to ignore the extra offers, like the blue "Verva Diesel" and such. I'm thinking that the best way to know what the offer is, is to find the square next to the price and average it's color. Green being gas, black being diesel and yellow being LPG. Those colors are the de facto standard in my country.

I have tried using cv2.findContours and cv2.Canny to preprocess the image. I still haven't figured out how to extract the rectangle containing the whole totem or the rectangles that contain the prices themselves.

Here's what I've figured out until now:

import cv2
import numpy as np

src = cv2.imread("media/totem1.jpeg")
src2 = src.copy()
cv2.imshow("original", src)

gray = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(gray, 100 , 250 ,0)

cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    cv2.drawContours(src, [c], -1, (0, 255, 0), 1)

cv2.imshow("findContours", src)

edges = cv2.Canny(gray ,100,200)

cv2.imshow("Canny", edges)

output img

I've also tried using HoughLines, but I wasn't able to get any decet results even with tweaking the settings. Here's the output of that:

Hough lines test img

Answer 1

You can use tesseract to recognize the numbers, but you need a good black & white picture:

import subprocess
import cv2
import pytesseract

# Image manipulation
# Commands https://imagemagick.org/script/convert.php
mag_img = r'D:\Programme\ImageMagic\magick.exe'
con_bw = r"D:\Programme\ImageMagic\convert.exe" 

in_file = r'Benzin_cut1.png'
out_file = r'Benzin_cut_bw.png'

# Play with black and white and contrast for better results
process = subprocess.run([con_bw, in_file, "-resize", "180%","-threshold","38%","-brightness-contrast","-20x30", out_file])

# Text processing
pytesseract.pytesseract.tesseract_cmd=r'C:\Program Files\Tesseract-OCR\tesseract.exe'
img = cv2.imread(out_file)

# Parameters see tesseract doc 
custom_config = r'--psm 6 --oem 3 -c tessedit_char_whitelist=01234567890' 

tex = pytesseract.image_to_string(img, config=custom_config)
print(tex)

with open("cartootn.txt", 'w') as f:
    f.writelines(tex)

cv2.imshow('image',img)
cv2.waitKey(0)
cv2.destroyAllWindows()

Output - not exact correct, but good: