Text Detection with YOLO on Challenging Images

Question

I have images that look as follows:

My goal is to detect and recognize the number 31197394. I have already fine-tuned a deep neural network on text recognition. It can successfully identify the correct number, if it is provided it in the following format:

The only task that remains is the detection of the corresponding bounding box. For this purpose, I have tried darknet. Unfortunately, it's not recognizing anything. Does anyone have an idea of a network that performs better on these kind of images? I know, that amazon recognition is able to solve this task. But I need a solution that works offline. So my hopes are still high that there exist pre-trained networks that work. Thank's a lot for your help!

97% is very impressive for a homebrew solution :) May I ask what you used for text recognition? Another single shot detector or an RNN-based model? — Ari, Jan 10 '22 at 14:57

ibra ndiaye · Answer 1 · 2022-02-23T15:56:35.457

Don't say darknet doesn't work. It depends on how you labeled your dataset. It is true that the numbers you want to recognize are too small so if you don't make any changes to the image during the pre-processing phase, it would be complicated for a neural network to recognize them well. So what you can do that will surely work is:
1---> Before labeling, increase the size of all images by 2 times its current size (like 1000*1000)
2---> used this size (1000 * 1000) for the darket trainer instead of the default size proposed by darknet which is 416 * 416. You would then have to change the configuration file
3---> use the latest darknet version (yolo v4)
4---> On the configuration file, always keep a number of subdivisions at 1.
I also specify that this method is too greedy in memory, it is therefore necessary to provide a machine with RAM > 16 GB. The advantage is that it works...

score 1 · Accepted Answer · answered May 05 '21 at 09:19

1

Thanks for your answers guys! You were right, I had to finetune yolo to make it work. So I created a dataset and fine-tuned yolov5. I am surprised how good the results are. Despite only having about 300 images in total, I get an accuracy of 97% to predict the correct number. This is mainly due to strong augmentations. And indeed the memory requirements are large, but I could train on a 32 GM RAM machine. I can really encourage anyone who faces similar problems to give yolo a shot!!

answered May 05 '21 at 09:19

spadel

998
2
16
40

Mind in sharing what was the preprocessing used? – Diogo Santiago Aug 23 '21 at 14:15
Data augmentation you mean? basically almost anything you yolo offers that is prebuilt, except for mirroring and strong rotations (which makes no sense in this case). – spadel Aug 24 '21 at 10:58

score 0 · Answer 3 · answered Apr 27 '21 at 14:40

0

Maybe use an R-CNN to identify the region where the number is and then pass that region to your fine-tuned neural network for the digit classification

answered Apr 27 '21 at 14:40

MD Mushfirat Mohaimin

1,966
3
10
22

Text Detection with YOLO on Challenging Images

3 Answers3