1

Background:

I am working on my final year undergraduate college project and the topic I am involved on is the Paper Note Detection by Optical Character Recognition. I have already started working on basic image processing techniques, and since I am new on Image Processing with Java, progress is a bit slower.

I have a basic idea of image processing since I took a paper on previous semester.

Basically, I am working on the Nepali Paper Notes and the idea is to extract the key information from it. The notes I am using are the Nepali Currency Notes of rupees 100, 500 and 1000.

enter image description here

The image above is the Nepalese Currency of Rupees 500. The idea is to extract the information from the image and identify which currency the image belongs to.

enter image description here

The primary goal of my project is to determine the currency type, which is basically done with the recognition of the bottom right area. The bottom right area of the image defines the value of the currency.

The secondary goal is to fetch the number of the currency (unique) and store it in the database.

Question:

Well, my question is, how fairly this problem could be solved? What are the necessary prerequisites before entering into this project? How do I select the region of interest from the image?

The other two paper notes on which my project should recognize are listed below:

enter image description here

Nepalese Paper Note: Rs. 1000

enter image description here

Nepalese Paper Note: Rs. 100

Since I am new to Image Processing with Java, I need a fair suggestion on how to achieve my problem to success.

Aashish K
  • 103
  • 2
  • 12
  • if your goal is just the value of the number, you might be able to get away with just preprocessing + OCR – qwr Jun 18 '18 at 09:27
  • sorry, but I didn't get you? – Aashish K Jun 18 '18 at 09:28
  • https://en.wikipedia.org/wiki/Optical_character_recognition – qwr Jun 18 '18 at 09:29
  • the goal is to get the value of numbers, but the question is how do I get to the value, because the image has lots and lots of details in it. – Aashish K Jun 18 '18 at 09:33
  • 1
    Assuming that the notes have already been localized and perspective corrected (as shown in your samples), you can try a template based approach as shown [here](https://stackoverflow.com/a/24464968/2571705) to extract information (When i say template, don't confuse this with template-matching. This is entirely different). You might also be able to use aspect ratio of width and height to differentiate between the notes and apply the correct template. After extracting the information you can apply pre-processing/OCR. – dhanushka Jun 19 '18 at 10:12
  • @dhanushka it makes sense in some ways but how do I actually extract the information from the left region of interest. the idea is to use OCR by the template matching procedure but can you elaborate some other techniques to obtain the result? – Aashish K Jun 21 '18 at 09:55

1 Answers1

0

I'm going to try and answer this step by step and since these are sequential, your accuracy will depend on the how well you do each and every step.

  1. Determining and Extracting ROI: Considering you're working on currency notes, it is safe to assume that your input test/train data will be aligned the way it is in the images given above. Try using contouring to extract a region of interest around the numbers. Another thing you can do is create a mask which will filter out the remaining area of the image and leave you with only the area that you require. The second approach is more of a hardcode and will fail incase the image is not aligned.

  2. Pre-processing: Oncec you have your ROI, you will need to go through some preprocessing techniques before you feed the data to an OCR. Most OCR show better accuracy with binary images, sometimes with grayscale too. This step is essential for getting good results from your OCR.

  3. Applying OCR: You can always use the Tesseract OCR or others, but since the types of currency notes are limited, I would also suggest you have a look at object detection models. Many of them are readily available online and you can train them on your own by providing images of currency and manually labeling them the corresponding value. OCRs don't always return the best results and in your use case, I would suggest you try out other alternatives such as image matching or making a model.

Vaibhav Mehrotra
  • 416
  • 2
  • 7