3

I have the following code which, given an image, it extract ROI from it.

Warning: as is, the code will save all extracted ROIs (45 files) on Desktop.

import cv2

extr_path = ('C:\\Users\\Bob\\Desktop\\')

# Read the input image
im = cv2.imread(extr_path + 'extracted.jpg')

# Convert to grayscale and apply Gaussian filtering
im_gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
im_gray = cv2.GaussianBlur(im_gray, (5, 5), 0)

# Threshold the image
ret, im_th = cv2.threshold(im_gray, 90, 255, cv2.THRESH_BINARY_INV)

# Find contours in the image
image, ctrs, hier = cv2.findContours(im_th.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

# Sort the bounding boxes
sorted_ctrs = sorted(ctrs, key=lambda ctr: cv2.boundingRect(ctr)[0])

# Extract ROI

i = 0

for i, ctr in enumerate(sorted_ctrs):
    # Get bounding box
    x, y, w, h = cv2.boundingRect(ctr)

    # Getting ROI
    roi = im[y:y+h, x:x+w]

    cv2.imwrite('C:\\Users\\Bob\\Desktop\\' + str(i) + '.jpg', roi)

This is image used to run the code:

extracted.jpg

This is the result:

Extracted ROI

Green is ok, like other images.

As you can (barely...) see, there is something wrong with the number five (29.jpg).

Detailed ROI problem

There is a part above the char which miss in the extracted ROI.

I think is a problem with the cv2.boundingRect() function. To be clear, this is an exception. If I give more numbers, let's say it recognize and extract 7 digits on 10 near perfectly. Others have problems like the one above...

Also, I saw that ROI rectangles give themselves automatic dimensions (width and height) to match the digit they found.

I tried to add other image pre-processing like adaptive threshold or medianBlur but it doesn't change. Maybe because the image already satisfy some conditions (black writing on white paper for example...)

I also found a link to another question but I don't understand if the problem is the same of me..

Why this happen ? What I have to modify ? I mean, it is even possible to "give instructions" to the area which bounding rectangle have to crop ?

If someone could help, it would be greatly appreciated.

Thank you

UPDATE 1: tried also with cannyEdge filter but is the same. Dilation and Erosion as well has been tried but the result is the same.. Now I really think that the problem could be the boundingRect function inside OpenCV core.

lucians
  • 2,239
  • 5
  • 36
  • 64
  • Please, include a [mcve] (emphasis on the Complete -- I should be able to run it without having to add anything, such as imports) as well as an input image that can be used to reproduce the problem. The pictures you've provided so far are good for illustration, but we also need something to run the code with. – Dan Mašek Jul 26 '17 at 18:48
  • Thank you. I added source image and made the code runnable as is. – lucians Jul 27 '17 at 07:56
  • @DanMašek I saw one of your answers, the license plate one (which I used to learn a bit more of openCv in the early stage of the project). The important thing could be this: you used some computer generated chars to train the program. Can't it be done the same here but with, for example, MNIST dataset ? I think about first recognize the digits and after extract the ROI...what do you say ? – lucians Jul 27 '17 at 08:43
  • IHMO you need to do segmentation before recognition. Problem now is that the blur+threshold causes that last symbol to break into several disconnected pieces, so you get multiple contours: http://i.imgur.com/AdJoQKz.png | Try some alternative approaches (e.g. vertical projection), perhaps combine several. You might want some heuristic to combine adjacent bounding boxes when appropriate. – Dan Mašek Jul 27 '17 at 14:27
  • It seems pretty complex to do but I will try first the segmentation. Blur+threshold did output of your image or was together segmentation ? I'm new to OpenCV world so I am trying various aproaches but it seems no one work well... – lucians Jul 27 '17 at 14:32
  • That image shows the contents of `im_th` from your script, cropped and zoomed in 4x, so that it's easier to see. – Dan Mašek Jul 27 '17 at 14:39
  • Applied some MSER to the image and this is the result: http://imgur.com/tEaJADo.png Can I do something with it ? Can I implement in the other code or it's just a waste of time ? – lucians Jul 27 '17 at 14:52

0 Answers0