License plate recognition

Project Report on

License Plate Recognition System

By:

Harshit Saxena

Indian Institute of Technology Roorkee

2011

LICENSE PLATE RECOGNITION 2011

1 | P a g e

Abstract:

This project report aims to develop a license plate recognition system. This report presents an algorithm

for localization of yellow colored license plates using morphological operations, character segmentation

using histogram and intensity projections and Optical Character recognition using Template matching.

Keywords: License plate, Character segmentation, Optical Character Recognition, Morphological

Operations Histogram.

Introduction

The application of autonomous systems has increased widespread. Significant research has been carried out in the area of transportation to make it intelligent and efficient. A License Plate Recognition(LPR) system involves detection and identification of a license plate.

It plays a major role in automatic highway electronic toll collection, vehicle tracking etc. It has been used for speed enforcement in countries like Austria, Italy and The Netherlands [1]. Thus by using such an automated system it becomes easy to monitor and control the large number of vehicles on the road. The most vital part of an LPR system is detection and extraction of the License Plate. A number of factors like the angle, noise, uneven illumination contribute to the overall system accuracy.

A robust License Plate Recognition System consists mainly of three modules:

[1] License plate detection, [2] character segmentation and [3] Optical Character Recognition (OCR).

Methodology


2 | P a g e

Assumptions:

Input is an image of a stationary Car.

Only the most common type of license plates (single line) will be dealt with. The license Plate has a yellow background with text written in Black.

License Plate Detection:

(I) Preprocessing:

Before applying any operation on the test image some image enhancement techniques are used for improving the overall result. In this process, we use methods that include adjusting the intensity of the image and reducing the contrast in the image. The technique used for intensity adjustment is known as histogram equalization. The contrast in the image can be reduced by several methods that are normally used for contrast enhancement. Since we have assumed that the license plate has a yellow background , we now identify the regions in the image that contain the intensity of RGB corresponding to the color yellow. (a< R< b) && (p< G< q) && (x< B< y) where, R is the intensity of the color Red, G of Green and B that of Blue. Based on this condition we obtain a Binary Image, we change yellow to white and non-yellow to black.


3 | P a g e

Here: a=100, b=250, p=50 , q=180, x=0 , y=100

The image obtained contains noise and many undesirable regions. In order to remove these unwanted regions we apply morphological operations. (II) Morphological Operations: Mathematical morphology commonly refers to a broad set of image processing operations that process images based on shapes. These are Non-linear filters, with the function of restraining noises, extracting features and segmenting images etc

The following morphological operations have been used Fill (MATLAB function – imfill): fills holes and regions in the binary image. A hole is a set of

background pixels that cannot be reached by filling in the background from the edge of the image.

Open (MATLAB function – imopen): performs morphological opening on the grayscale or binary image with the pre-defined structuring element. The morphological open operation is erosion followed by dilation, using the same structuring element for both operations.


4 | P a g e

Dilate (MATLAB function – imdilate): dilates the grayscale, binary, or packed binary image returning the dilated image. The argument se is a structuring element object, or array of structuring element objects, returned by the strel function.

On applying these morphological operations on the original image, we obtain:

bw = imfill(I,'holes'); se = strel('rectangle',[3,3]); bw = imopen(bw,se); bw = imdilate(bw,se);


5 | P a g e

(III) Horizontal and Vertical Segmentation Once the Preprocessing is over, the next step is to segment the license plate candidates from the image. This technique is used to extract the probable rectangular regions of the license plates. We first do a horizontal segmentation of the image using the histogram method. Steps:-

Calculate the horizontal and vertical projections of intensity. Then we find the mean of the local minimas of horizontal projection plot. Based on the threshold calculated from the above local minimas, we find x locations of the

segmented regions.

The above plot shows the horizontal segmentation of intensity. These values indicate the number of pixels that are white in a particular column. Thus we can observe that there are no white pixels in the immediate neighborhood of the region AB and CD. Hence the regions AB and CD can be the probable regions containing the license plate. Thus by finding the points A, B, C, D we can get the probable x coordinates of the license plate.


6 | P a g e

Another important point to note here is that the length of AB is very less and does not qualify to be the location of a license plate. We have also used a padding of 0s of 1 column and 1 row wide in the beginning. This enables us to detect the points using 0 to 1 transition or vice versa. Thus by this method the start point of the x coordinate of the plate will always be odd. We can get the width of the probable regions by getting the distance between the points A (x1) and B(x2) say.

Now we compare this width to a threshold width value.

fT (binh)= 1 if fT (x1,x2) >=T 0 if fT (x1,x2)<=T

Thus if the width falls between the threshold, then the points between x1 and x2 are made 1 otherwise they are made 0.The output after these conditions is:

Hence AB and CD are the two possible regions. Note that A/C is the left and B/D is the right edge of the license plate respectively. Similarly we perform a vertical segmentation. The result obtained is as follows:


7 | P a g e

Thus P/R are the top edges and Q/S are the bottom edges of the license plate respectively.

(V) Identifying the License Plate: Out of the many candidate regions that we have obtained from the so far mentioned procedure, our task now is to identify the region containing the license plate. In this experiment, two features are defined and extracted in order to decide if a candidate region contains a license plate or not, these features are 1. Aspect ratio 2. Edge Density Even though these features are not scale-invariant, luminance-invariant, rotation-invariant, but they are insensitive to many environment changes. Aspect ratio The aspect ratio is defined as the ratio of the width to the height of the region. Aspect Ratio = width/height Edge density Applying the above feature to filter the segmented regions, a lot of non-license plate regions can be removed. However, there are still many candidate regions left which take similar rectangularity and aspect ratio features as the license plate regions do, such as often the head lights. Considering that the license plate regions generally take higher local variance in its pixels‟ values due to the presence of characters, an important feature to describe license plate region is local variance, which is quantized using the edge density.

Thus after Step IV we get the following two regions:

Notice that the plate can be differentiated from the first figure by the property of edge density. Thus the extraction and detection of license plate is completed.


8 | P a g e

Character segmentation:

This is the second major part of the License Plate detection algorithm. The Steps involved in character Segmentation are: -

(I) Preprocessing Preprocessing is very important for the good performance of character segmentation. Our preprocessing consists of conversion to grayscale and binarization using an object enhancement technique. Steps:

Conversion to Grayscale Binarization: Compared with the usual methods of image binarization, this algorithm uses the

information of intensity and avoids the abruption and conglutination of characters that are the drawbacks of usual image binarization techniques.

Horizontal Segmentation:

For this we calculate the horizontal and vertical projections of intensity. Then we find the local minima for horizontal projection. Based on the threshold calculated from the above local minimas, we find x locations of the

segmented regions. In order to locate the right and left edges of license plate from candidate region, the vertical projection after mathematical morphology deal is changed into binary image.


9 | P a g e

Optical Character Recognition (OCR)

OCR is the mechanical or electronic translation of images of handwritten or typewritten text (usually captured by a scanner) into machine-editable text.

The procedure consists of two important steps, training and recognition. Training: The program is first trained with a set of sample images for each of the characters to extract the important features based on which the recognition operation would be performed. Our program is trained on a set of 10 characters with 10 samples of each. The training algorithm involves the following steps 1. Preprocessing: Before preparing the template for each of the characters for further use , we need to do some processing on the images . The following are the operations that are performed : - Binarization. - Inversion of intensity of the characters. - Finding the connected component that represents the character - Finding the smallest rectangle enclosing this connected component - Normalization of the image to size 15 X 15. - Storing the intensity values using the below mentioned algorithm for each of the characters. 2. Creating the template In order to create the template for each character we do the following operation. For every white pixel we insert the value 1 and for every black pixel 0. We do this for all the 10 training samples for each character and calculate the weights to get the template. Recognition: 1. Preprocessing: The image obtained after segmentation is Grayscale. Follow the preprocessing steps used for the training of the characters. 2. Calculate the score for each of the characters: We calculate the matching score of the segmented character from the templates of the character stored by the following algorithm. We compare the pixel values of the matrix of segmented character and the template matrix, and for every match we add 1 to the matching score and for every mis-match we decrement 1. This is done for all 225 pixels. The match score is generated for every template and the one which gives the highest score is taken to be the recognized character.


10 | P a g e

Results:

License Plate detection and extraction algorithm worked on 90% of the images.

The technique of horizontal segmentation did not produce desired results for plates at an angle.

This limited the accuracy of the algorithm.

For horizontal license plates around 65% of the letters were able to be recognized in cases where

character segmentation was achieved.

Future Work:

Inclusion of license plates with different colored backgrounds.

Skew correction and efficient character segmentation algorithm need to be worked upon which

can further enhance the overall system efficiency.


11 | P a g e

Some Sample Results:

Example 1:


12 | P a g e

Example 2:


13 | P a g e

Example 3:

Example 4:


14 | P a g e

Example 5:

Conclusion:

The license plates were detected successfully in nearly all the images. Also an improved

character segmentation technique would have increased the system accuracy greatly. The

aim of the project was to familiarize oneself with the basics of image processing and gain

experience in programming in Matlab which was achieved successfully.


15 | P a g e

References:

[1] Automatic license plate recognition-Wikipedia

(http://en.wikipedia.org/wiki/Automatic_number_plate_recognition)

[2] Vehicle License Plate location Based on Histogramming and Mathematical Morphology - Feng Yang, Zheng Ma - 2005

[3] Building an Automatic Vehicle License-Plate Recognition System - Tran Duc Duan, Tran

Le Hong Du, Tran Vinh Phuoc, Nguyen Viet Hoang – Febraury 2005

[4] A New Algorithm for Character Segmentation of License Plate - Yungang Zhang ,Changshui Zhang - June 2003

[5] Mean Shift for Accurate Number Plate Detection - Wenjing Jia, Huaifeng Zhang, and Xiangjian

He - July 2005

[6] Humayun K. Sulehria, Ye Zhang, Danish Irfan, “Mathematical Morphology Methodology for Extraction of Vehicle Number Plates” in International Journal of Computers; Issue 3, Volume 1,

2007.

[7] Ondrej Martinsky, B.Sc. Thesis on “Algorithmic and Mathematical Principles of Automatic

Number Plate Recognition Systems” in 2007.

[8] Digital Image Processing using Matlab by Gonzalez and Woods

Date post:	09-Feb-2017
Category:	Engineering
Upload:	rahul-bhambri
View:	181 times
Download:	5 times

License plate recognition

Engineering