59

Background

For my final project at university, I'm developing a vehicle license plate detection application. I consider myself an intermediate programmer, however my mathematics knowledge lacks anything above secondary school, which makes producing the right formulas harder than it probably should be.

I've spend a good amount of time looking up academic papers such as:

When it comes to the math, I'm lost. Due to this testing various graphic images proved productive, for example:

alt text

to

alt text

However this approach only worked to that particular image, and if the techniques were applied to different images, I'm sure a poorer conversion would occur. I've read about a formula called the "bottom hat morphology transform", which does the following:

Basically, the trans- formation keeps all the dark details of the picture, and eliminates everything else (including bigger dark regions and light regions).

I can't find much information on this, however the image within the documentation near the end of the report shows its effectiveness.

Other constraints

  • Developing in C#
  • Confining the project to UK registration plates only
  • I can choose the images to convert as a demonstration

Question

I need advice on what transformation techniques I should focus on developing, and what algorithms can help me.

EDIT: New information present on Continued - Vehicle License Plate Detection

Community
  • 1
  • 1
Ash
  • 3,494
  • 12
  • 35
  • 42
  • 2
    OpenCv is great place to start/extend. I've found Emgu in C# to be very good port of OpenCV. http://www.emgu.com/wiki/index.php/License_Plate_Recognition_in_CSharp – kenny Jan 16 '11 at 19:54
  • You may want to check out www.openalpr.com as a starting point. It uses a number of techniques to find and analyze the license plate. – Derrick Johnson Jan 17 '14 at 22:45
  • you can use rectangle detection in opencv or train haar like features to detect plates which also present in opencv – mrgloom Aug 10 '12 at 09:32
  • This is clearly a computer vision type of problem. Take a look at [OpenCV](http://opencv.willowgarage.com/wiki/FullOpenCVWiki). It's in C++, but probably you'll be able to [interface with it somehow](http://www.emgu.com/wiki/index.php/Main_Page). – 9000 Jan 16 '11 at 19:54
  • You can take a look at one of emgucv example that show you a real world working examples of vehicle plate detection using OCR http://www.emgu.com/wiki/index.php/License_Plate_Recognition_in_CSharp – Luca Del Tongo Jan 17 '11 at 00:20

8 Answers8

15

There are a number of approaches you can take but the first strategy that pops into mind is to:

  • Discovery/research: Identify the set of colors and fonts that you may need to identify. If your sample picture is representative of most British plates then your job is made easier. E.g. Simple, singular font and black lettering on a white background
  • Code: Attempt to identify a rectangular region of an image where the colors are predominantly white and black. This is not a terribly math-heavy problem and it should give you the license plate region to concentrate on.
  • Code: Do some clean up on your subregion such conversion to pure black and white (monochrome) and perhaps scaling/shifting into a nice, tight rectangle.
  • Use API: Next employ an existing OCR (optical character recognition) algorithm on your sub-selected image region so see if you can read the text.

Like I said, this is one strategy of many but it comes to mind as one requiring the least amount of heavy math... that is if you can find an OCR implementation that will work for you.

Paul Sasik
  • 79,492
  • 20
  • 149
  • 189
  • Thanks for the breakdown, think I'll focus on something similar to your description, detecting a shape which resembles a rectangle containing pixels of a certain colour, or colour bracket. – Ash Jan 16 '11 at 20:11
  • @Paul sasik "Attempt to identify a rectangular region of an image where the colors are predominantly white and black" - can you suggest some good methods to do this; I am trying to localize license plates images... – Deepak Jul 18 '14 at 17:38
3

Here's how I suggest you should do this task. Read my detailed answer here.

  1. Convert to Grayscale.
  2. Gaussian Blur with 3x3 or 5x5 filter.

  3. Apply Sobel Filter to find vertical edges.

    Sobel(gray, dst, -1, 1, 0)

  4. Threshold the resultant image to get a binary image.
  5. Apply a morphological close operation using suitable structuring element.
  6. Find contours of the resulting image.
  7. Find minAreaRect of each contour. Select rectangles based on aspect ratio and minimum and maximum area.
  8. For each selected contour, find edge density. Set a threshold for edge density and choose the rectangles breaching that threshold as possible plate regions.
  9. Few rectangles will remain after this. You can filter them based on orientation or any criteria you deem suitable.
  10. Clip these detected rectangular portions from an image after adaptiveThreshold of the original (grayscale) image and apply OCR.
Community
  • 1
  • 1
Abdul Fatir
  • 6,159
  • 5
  • 31
  • 58
3

I did a similar project a few years ago in Java, first I applied the Sobel operator and then masked all the image with an image of a plate (with the Sobel operator applied too). The region of maximum coincidence is where the plate is. Then apply an OCR to the selected region to get the number.

rodrigoap
  • 7,405
  • 35
  • 46
  • Thanks for the reply, I actually saw something about the Sobel operator in a paper I lost. I'll look into it :) – Ash Jan 16 '11 at 20:12
  • Actually your idea is great! Only just occurred to me heh. Any idea on available pseudo code to describe the Sobel operator? – Ash Jan 16 '11 at 22:00
1

You can also refer to Automatic License Plate Recognition library & this query. This will also give you some idea about how to approach things,and how existing solutions are.

But as answered by paul, you should first try to find the rectangular number plate from the complete image and then binarize it and then use the OCR libraries available (Tesseract would be recommended)

You can refer to this link which will help you find the rectangular plate. You need to use openCV libraries, so you will not need alot of math but yes a basic understanding of what is happening behind the scenes can help you solve the problem in better way.

Community
  • 1
  • 1
Mayank
  • 353
  • 6
  • 20
1

It tells you exactly how to compute the bottom hat transform(sort of looks like an inverted graduated threshold transform to me).

First thing to do is implement the two morphology functions dilation and erosion.

To do this you need your f and b then you compute the function over a small region of the image at a point keeping the largest value found.

(f ⊕ b)(s, t) = max{f (s − x, t − y) + b(x, y)
|(s − x), (t − y) ∈ Df ; (x, y)∈Db}

What this says is, take the maximum of the expression over all points in the domain region(such as a small rectangle centered at your point (s,t).

simple pseudo code would be

max = -infinity // for the point (s,t) on the image, must compute this for all points
for(x = -5 to 5)
for(y = -5 to 5)
max = Max(max, f(s - x, t - y) + b(x,y))

effectively we now have a new image of the max values.

It's actually quite simple so don't make it harder than it is(we are simply adding b(x,y) to each point in the region and finding out which one gives the maximum value).

you do the same for the erosion(very similar to above)

Now the opening and closing is the composition of the two

You can think of it first as performing a dilation and then an erosion for an opening.

It says finally subtract the closing from the original image and you should have your transform.

Femaref
  • 60,705
  • 7
  • 138
  • 176
AbstractDissonance
  • 1
  • 2
  • 16
  • 31
1

UK already has a system that does that. I recall seeing a TV show in which they demonstrated they can find a car within London within 10 minutes (assuming they know the number and the car is driving around) Just reading Wikipedia gives you the pointers you need to start thinking about the issue: http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

Mircea
  • 11
  • 1
  • The images describing different effects were really helpful. Gaussian Blur > Grayscale > Constrast + 60 worked great on all images, just need to work on detection algorithm :) – Ash Jan 16 '11 at 22:25
1

If you're interested in the problem of detecting the presence of a license plate (as opposed to recognizing it), you should probably look at the text detection in images as it is related to what you're doing.

This question is related to yours: Algorithm to detect presence of text on image

Community
  • 1
  • 1
carlosdc
  • 12,022
  • 4
  • 45
  • 62
1

I would suggest using a service or third party for this. Open ALPR, provides an open source package that is very accurate for this service.

Open ALPR - https://www.openalpr.com/

Demo of Video Open ALPR

https://www.youtube.com/watch?v=E-U_H9EbW60

Or you could use an API -

Macgyver Computer Vision API

https://askmacgyver.com/explore/program/license-plate-recognition/3X5D3d2k

In this API you would simply make a post request to -

https://macgyver.services

Example Payload

{
id: "3X5D3d2k",
key: "free",
data: {
    "image_url": "https://storage.googleapis.com/marketing-files/program-markdown-assets/license-detection/license-plate.jpg",
    "country": "us",
    "numberCandidates": 2
  }
}

enter image description here

The image above would return the following - 

"plate": "284FH8"
"confidence": 90.601013
Timothy Moody
  • 479
  • 1
  • 7
  • 9