OpenCV MSER detect text areas - Python

Question

I have an invoice image, and I want to detect the text on it. So I plan to use 2 steps: first is to identify the text areas, and then using OCR to recognize the text.

I am using OpenCV 3.0 in python for that. I am able to identify the text(including some non text areas) but I further want to identify text boxes from the image(also excluding the non-text areas).

My input image is:

Original

And the output is:

Processed

I am using the below code for this:

img = cv2.imread('/home/mis/Text_Recognition/bill.jpg')
mser = cv2.MSER_create()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) #Converting to GrayScale
gray_img = img.copy()

regions = mser.detectRegions(gray, None)
hulls = [cv2.convexHull(p.reshape(-1, 1, 2)) for p in regions]
cv2.polylines(gray_img, hulls, 1, (0, 0, 255), 2)
cv2.imwrite('/home/mis/Text_Recognition/amit.jpg', gray_img) #Saving

Now, I want to identify the text boxes, and remove/unidentify any non-text areas on the invoice. I am new to OpenCV and am a beginner in Python. I am able to find some examples in MATAB example and C++ example, but If I convert them to python, it will take a lot of time for me.

Is there any example with python using OpenCV, or can anyone help me with this?

Hey were you able to figure it out? – Oer Oct 30 '17 at 17:43 — Oer, Oct 30 '17 at 17:43

score 26 · Answer 1 · edited Sep 12 '19 at 10:14

26

Below is the code

# Import packages 
import cv2
import numpy as np

#Create MSER object
mser = cv2.MSER_create()

#Your image path i-e receipt path
img = cv2.imread('/home/rafiullah/PycharmProjects/python-ocr-master/receipts/73.jpg')

#Convert to gray scale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

vis = img.copy()

#detect regions in gray scale image
regions, _ = mser.detectRegions(gray)

hulls = [cv2.convexHull(p.reshape(-1, 1, 2)) for p in regions]

cv2.polylines(vis, hulls, 1, (0, 255, 0))

cv2.imshow('img', vis)

cv2.waitKey(0)

mask = np.zeros((img.shape[0], img.shape[1], 1), dtype=np.uint8)

for contour in hulls:

    cv2.drawContours(mask, [contour], -1, (255, 255, 255), -1)

#this is used to find only text regions, remaining are ignored
text_only = cv2.bitwise_and(img, img, mask=mask)

cv2.imshow("text only", text_only)

cv2.waitKey(0)

edited Sep 12 '19 at 10:14

answered Aug 22 '17 at 07:28

RAFI AFRIDI

411
6
3

1

This answer was very helpful to me. But I am bothered by the question why the `reshape(-1, 1, 2)` call is used. It returns a 3-dimensional array seemingly containing 2-dimensional data: `[[[38, 30]], [[39, 30]], [[40, 30]], ...]`. As far as I understand omitting that call leads to the same output of `convexHull`. Thus my question: What is it for? – ideaboxer Jul 25 '18 at 22:02
@ideaboxer did you get what the `.reshape` here does ? – Shreyas Moolya Aug 27 '19 at 10:29
Is there any way to convert this answer to detect textboxes,button,radiobutton,checkboxes,combobox etc ? This would be really helpful for me. – Ashish Johnson Jul 31 '20 at 06:24

score 4 · Answer 2 · answered Sep 16 '19 at 16:17

This is an old post, yet I'd like to contribute that if you are trying to extract all the texts out of an image, here is the code to get that text in an array.

import cv2
import numpy as np
import re
import pytesseract
from pytesseract import image_to_string
pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"
from PIL import Image

image_obj = Image.open("screenshot.png")

rgb = cv2.imread('screenshot.png')
small = cv2.cvtColor(rgb, cv2.COLOR_BGR2GRAY)

#threshold the image
_, bw = cv2.threshold(small, 0.0, 255.0, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)

# get horizontal mask of large size since text are horizontal components
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (20, 1))
connected = cv2.morphologyEx(bw, cv2.MORPH_CLOSE, kernel)

# find all the contours
contours, hierarchy,=cv2.findContours(connected.copy(),cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
#Segment the text lines
counter=0
array_of_texts=[]
for idx in range(len(contours)):
    x, y, w, h = cv2.boundingRect(contours[idx])
    cropped_image = image_obj.crop((x-10, y, x+w+10, y+h ))
    str_store = re.sub(r'([^\s\w]|_)+', '', image_to_string(cropped_image))
    array_of_texts.append(str_store)
    counter+=1

print(array_of_texts)

OpenCV MSER detect text areas - Python

2 Answers2

Linked