4

What method is suitable to capture (detect) MRZ from a photo of a document? I'm thinking about cascade classifier (e.g. Viola-Jones), but it seems a bit weird to use it for this problem.

lizarisk
  • 7,562
  • 10
  • 46
  • 70

5 Answers5

4

If you know that you will look for text in a passport, why not try to find passport model points on it first. Match template of a passport to it by using ASM/AAM (Active shape model, Active Appearance Model) techniques. Once you have passport position information you can cut out the regions that you are interested in. This will take some time to implement though.

ivan_a
  • 613
  • 5
  • 12
1

Consider this approach as a great starting point:

  • Black top-hat followed by a horisontal derivative highlights long rows of characters.
  • Morphological closing operation(s) merge the nearby characters and character rows together into a single large blob.
  • Optional erosion operation(s) remove the small blobs.
  • Otsu thresholding followed by contour detection and filtering away the contours which are apparently too small, too round, or located in the wrong place will get you a small number of possible locations for the MRZ
  • Finally, compute bounding boxes for the locations you found and see whether you can OCR them successfully.

It may not be the most efficient way to solve the problem, but it is surprisingly robust.

KT.
  • 10,815
  • 4
  • 47
  • 71
0

A better approach would be the use of projection profile methods. A projection profile method is based on the following idea:

Create an array A with an entry for every row in your b/w input document. Now set A[i] to the number of black pixels in the i-th row of your original image.

(You can also create a vertical projection profile by considering columns in the original image instead of rows.)

Now the array A is the projected row/column histogram of your document and the problem of detecting MRZs can be approached by examining the valleys in the A histogram.

This problem, however, is not completely solved, so there are many variations and improvements. Here's some additional documentation:

Daniel Martín
  • 7,815
  • 1
  • 29
  • 34
  • Projection profile will give similar results for any line of text, I guess, and I need to capture only the MRZ. – lizarisk Dec 28 '12 at 08:58
0

Viola & Jones' Haar-like features generate many (many (many)) features to try to describe an object and are a bit more robust to scale and the like. Their approach was a unique approach to a difficult problem.

Here, however, you have plenty of constraint on the problem and anything like that seems a bit overkill. Rather than 'optimizing early', I'd say evaluate the standard OCR tools off the shelf and see where they get you. I believe you'll be pleasantly surprised.

PS:
You'll want to preprocess the image to isolate the characters on a white background. This can be done quite easily and will help the OCR algorithms significantly.

Sean Connolly
  • 5,692
  • 7
  • 37
  • 74
  • I'm using tesseract, but I doubt that it has tools that can perform good segmentation and capture the MRZ, thats why I consider using an additional algorithm for preprocessing. – lizarisk Dec 28 '12 at 07:13
  • The problem with simple methods is that MRZ can be mixed up with other fields of passport that contain text, so I need some segmentation algorithm, which can be tricky. – lizarisk Dec 28 '12 at 08:56
0

You might want to consider using stroke width transform.
You can follow these tips to implement it.

Shai
  • 111,146
  • 38
  • 238
  • 371