3

I am using python, PIL, opencv and numpy to detect single color texts (i.e one is red, one is green). I want to detect these colorful text up to 6 meters away during live stream. I have used color detection methods but they did not work after 30-50 cm. Camera should be close to colors. As a second method to detect these texts, I used ctpn method. Although it detects texts, It does not provide the coordinate of these texts since I need coordinate points of texts also. I also tried OCR method in Matlab to automatically detect text in natural image but it failed since it finds another small objects as text. I am so stuck about what to do.

Let say for example, there are two different texts in an image that is captured 6 meters away. One text is green, the other one is red. The width of these texts are approximately 40-50 cm. In addition, they are only two different words, not long texts. How can I detect them and specify their location as (x1,y1) and (x2,y2)? Is that possible ? needy for any succesfull hint ?

A sampe image captured from nearly 2-2.5 meters and width of texts are nearly 20 cm.

Output of detected green pixels.

import numpy as np
from PIL import Image

# Open image and make RGB and HSV versions
RGBim = Image.open("AdjustedNewMaze3.jpg").convert('RGB')
HSVim = RGBim.convert('HSV')

# Make numpy versions
RGBna = np.array(RGBim)
HSVna = np.array(HSVim)

# Extract Hue
H = HSVna[:,:,0]

# Find all green pixels, i.e. where 100 < Hue < 140
lo,hi = 100,140
# Rescale to 0-255, rather than 0-360 because we are using uint8
lo = int((lo * 255) / 360)
hi = int((hi * 255) / 360)
green = np.where((H>lo) & (H<hi))

# Make all green pixels black in original image
RGBna[green] = [0,0,0]

def find_nearest(array, value):
    array = np.asarray(array)
    idx = (np.abs(array - value)).argmin()
    return array[idx]

value = 120 & 125

green = find_nearest(RGBna, value)
print(green)

count = green[0].size
print("Pixels matched: {}".format(count))
Image.fromarray(green).save('resultgreen.png')
Ender Ayhan
  • 308
  • 3
  • 14
  • 2
    The focal length of the lens will play a major role in the size of the text in the images. Please provide sample images. – Mark Setchell Sep 08 '18 at 08:28
  • @MarkSetchell focal length of my camera is: [ 1171,02428928540 1170,24715742738 ]. I attached in the post a scaled prototype image of my environment. – Ender Ayhan Sep 08 '18 at 08:38
  • 1
    Have a look at my answer here for detecting pixels of a certain colour... https://stackoverflow.com/a/52183666/2836621 Note however that if you use **OpenCV**, hues are scaled on range 0-180 rather than 0-255 that **PIL** uses. – Mark Setchell Sep 08 '18 at 08:41
  • I tried your code, It just found a letter in the word. That's why the distance is the main problem. @MarkSetchell – Ender Ayhan Sep 08 '18 at 08:52
  • 1
    You will see the original image was circular (because there is an alpha channel making the corners transparent) but this was ignored in the first piece of code and the result came out square because I ignored it.In the second example, I saved the alpha channel with `savedAlpha = im.getchannel('A')` and restored it at the end with `result.putalpha(savedAlpha)`. Real cameras don't normally create alpha channels - it's only graphics people that do that :-) – Mark Setchell Sep 08 '18 at 08:57
  • @MarkSetchell I got you, thank you for the hint and information. Although your code does not provide the exact solution, It is the best source for me until now. thank you! One more thing, what can you recommend me to search for that I can get more pixel values of red and green from far away ? – Ender Ayhan Sep 08 '18 at 09:09
  • You can only really use a longer lens, but that will give you a smaller field of view and you would have to "pan" it around to capture the whole scene. Not sure what you can do really. – Mark Setchell Sep 08 '18 at 09:15
  • thank you anyway. Last but not the least, I have increased the contrast value by using clahe on my original image. Now It finds more green pixels. However, it also finds green pixels on red text and around somewhere. Can I do a math operation on green pixel coordinates to get the real green area. I mean, is it possible to use numpy to find the closest and the biggest area of green pixels ? so I can specify the real green area. I attached output image in the post. @MarkSetchell – Ender Ayhan Sep 08 '18 at 09:26
  • More light, bolder + brighter text would help alot too. Maybe you can shine two lights specifically on the signs if you have control over that? – devdob Sep 08 '18 at 09:36
  • @devdob In real situation, texts are bolder and brighter. But light condition is independent from me. Even if I can do what you said, color detection code will still some pixels on different places. Is it possible to apply numpy operation for the area has most density of green and red ? – Ender Ayhan Sep 08 '18 at 09:40
  • Im no expert with Numpy, but maybe its `unique()` function could help you find your values better – devdob Sep 08 '18 at 09:48
  • I attached the code in the post. I found the nearest value to a value between green pixels. I want to plot this point on the original image. How can I do that ? @devdob – Ender Ayhan Sep 08 '18 at 10:42
  • 1
    Using PIL, you can use `Image.open()` and `ImageDraw.Draw` to manipulate your image as you wish. Check this small example here http://effbot.org/imagingbook/imagedraw.htm – devdob Sep 08 '18 at 11:30

0 Answers0