LICENSE PLATE RECOGNITION OF MOVING VEHICLES

Siti Rahimah Binti Abd Rahim

Bachelor of Engineering with Honors

(Electronics & Computer Engineering)

2009/2010

UNIVERSITI MALAYSIA SARAWAK

R13a

BORANG PENGESAHAN STATUS TESIS

Judul: LICENSE PLATE RECOGNITION OF MOVING VEHICLES

SESI PENGAJIAN: 2009/2010

Saya SITI RAHIMAH BINTI ABD RAHIM

(HURUF BESAR)

mengaku membenarkan tesis * ini disimpan di Pusat Khidmat Maklumat Akademik, Universiti Malaysia Sarawak

dengan syarat-syarat kegunaan seperti berikut:

1. Tesis adalah hakmilik Universiti Malaysia Sarawak.

2. Pusat Khidmat Maklumat Akademik, Universiti Malaysia Sarawak dibenarkan membuat salinan untuk

tujuan pengajian sahaja.

3. Membuat pendigitan untuk membangunkan Pangkalan Data Kandungan Tempatan.

4. Pusat Khidmat Maklumat Akademik, Universiti Malaysia Sarawak dibenarkan membuat salinan tesis ini

sebagai bahan pertukaran antara institusi pengajian tinggi.

5. ** Sila tandakan ( ) di kotak yang berkenaan

SULIT (Mengandungi maklumat yang berdarjah keselamatan atau kepentingan

Malaysia seperti yang termaktub di dalam AKTA RAHSIA RASMI 1972).

TERHAD (Mengandungi maklumat TERHAD yang telah ditentukan oleh organisasi/

badan di mana penyelidikan dijalankan).

TIDAK TERHAD

Disahkan oleh

(TANDATANGAN PENULIS) (TANDATANGAN PENYELIA)

Alamat tetap: NO. 1, JALAN 1, TAMAN

BATU 30,

44300 BATANG KALI, DR. MOHD SAUFEE BIN MUHAMMAD

Nama Penyelia

SELANGOR DARUL EHSAN.

Tarikh: Tarikh:

CATATAN * Tesis dimaksudkan sebagai tesis bagi Ijazah Doktor Falsafah, Sarjana dan Sarjana Muda.

** Jika tesis ini SULIT atau TERHAD, sila lampirkan surat daripada pihak berkuasa/organisasi

berkenaan dengan menyatakan sekali sebab dan tempoh tesis ini perlu dikelaskan sebagai

SULIT dan TERHAD.

Final Year Project attached here:

Title: License Plate Recognition of Moving Vehicles

Author Name: Siti Rahimah Binti Abd Rahim

Metric Number: 17312

Is hereby read and approved by:

__________________________ _________________________

Dr. Mohd Saufee Bin Muhammad Date

Project Supervisor

LICENSE PLATE RECOGNITION OF MOVING VEHICLES

SITI RAHIMAH BINTI ABD RAHIM

Thesis is submitted to

Faculty of Engineering, University Malaysia Sarawak

in Partial Fulfillment of the Requirements

for Degree of Bachelor of Engineering with Honors

(Electronics & Computer) 2009/2010

ii

Dedicate this dissertation to my lovely parents, Abd Rahim Bin Mohd Ali and

SitiSakniahBintiSarman and my siblings AbdRahiman Bin Abd Rahim and

SitiRaihaniahBintiAbd Rahim for their love and being supportive family for me.

iii

ACKNOWLEDGEMENTS

My sincerest appreciation extended to my supervisor, Dr. Mohd Saufee bin

Muhammad for guidance, support, advices, comments and suggestions throughout

the process of completing this project.

I would like to thank the Faculty of Engineering lecturers for the guidance

given for pursuing engineering knowledge and skills during my years here. I express

my thank to the Final Year Project coordinator, Madam Ade Syahida Wani for the

information’s and guidance’s to complete the project report as the format that has

been explained during Semester 1.

A special thank goes to my family and friends for the supports, guidance’s,

comments, suggestions and encouragements to complete this project. Special thanks

go those who are that upload their project on the internet or blog because these

information’s help a lot for the completion of the project.

iv

ABSTRAK

Pengecaman corak telah diketahui oleh ramai penganalisis sebagai satu

aplikasi dalam rangkaian neural. Ramai penyelidik memilih tajuk ini sebagai bahan

penyelidikan samada pengecaman huruf dan lain-lain corak. Pengecaman huruf

adalah satu aplikasi yang paling terkenal dalam pengecaman corak samada huruf

tulisan tangan atau lain-lain seperti huruf Arab and China. Projek ini melibatkan

pengecaman huruf dari pendaftaran kenderaan sewaktu kenderaan sedang bergerak di

atas jalan raya. Pendaftaran kenderaan mempunyai dua jenis huruf iaitu abjad dan

nombor. Huruf dari pendaftaran kenderaan dapat dikecam dengan mengunakan

teknik pemprocessan gambar dan aplikasi rangkaian neural yang terdapat dalam

perisian MATLAB. Projek ini mempunyai dua bahagian, dimana gambar kenderaan

akan diproses mengunakan teknik pemprosesan gambar dan kemudian akan dikecam

mengunakan rangkaian neural. Pengecaman huruf dari pendaftaran kenderaan akan

dikecam mengikut sasaran yang telah ditentukan. Seterusnya, perbandingan diantara

50 dan 100 neurons lapisan tersembunyi dilaksanakan untuk mengenalpasti

pengecaman huruf yang terbaik. Pada peringkat akhir projek, pengenalpastian huruf

akan dibentang dan dibincangkan.

v

ABSTRACT

Pattern recognition has been identified by researchers as one of the neural

network applications. There are many researches on this topic whether it character

recognition or other pattern. The famous application in pattern recognition is the

character recognition whether it handwritten recognition or others such as Arabic and

Chinese character. In this project, the character recognition is for moving vehicles

where character from license plate of moving vehicles will be recognized. License

plate character consists of alphabet and number. Incorporated with image processing

and neural network toolbox, this simulation will be design using the MATLAB

toolbox. This project consists of two parts where the image will be process in image

processing part while the character will be recognized using the backpropagation

neural network. The character recognition will be recognizing according to the target

output. In addition, performing recognition simulations compare between 50 and 100

neuron of hidden layer for the best character recognition. At the end of this project

the recognized character from the license plate will be presented.

vi

TABLE OF CONTENTS

Pages

Dedication ii

Acknowledgement iii

Abstrak iv

Abstract v

Table of Contents

vi

List of Tables x

List of Figures xi

List of Abbreviations xiii

Chapter 1

INTRODUCTION

1.1 Background

1.2 Project Objectives

1.3 Statement of Expected Problems

1.4 Proposed Solutions

1.5 Expected Outcomes

1.6 Report Outlines

1

2

3

3

4

5

vii

Chapter 2 LITERATURE REVIEW

2.1 License Plate Recognition System

2.2 Image Preprocessing

2.3 Image Enhancement

2.3.1 Contrast Manipulation

2.3.2 Histogram Manipulation

2.4 Image Enhancement using Filters

2.4.1 Sharpening Filter in Spatial Domain

2.4.2 Sharpening Filter in Frequency Domain

2.5 Image Segmentation

2.6 Feature Extraction

2.7 Neural Network

2.8 Artificial Neural Network

2.8.1 Feed-Forward Neural Network

2.8.2 Recurrent Neural Network

2.9 Back-propagation Neural Network

2.10 Training Algorithm

2.10.1 Supervised Training

2.10.2 Unsupervised Training

2.11 Introduction to MATLAB

2.11.1 What is MATLAB?

2.11.2 The MATLAB System

7

8

9

10

13

16

16

18

20

21

22

23

24

26

26

27

28

28

29

29

30

viii

Chapter 3 METHODOLOGY

3.1 Character Recognition Tools

3.2 Flow Chart of Character Recognition

3.3 Image Acquisition

3.4 Image Preprocessing

3.4.1 Image Cropping

3.4.2 Grayscale Conversion

3.4.3 Image Enhancement

3.5 Neural Network in Character Recognition

3.5.1 Network Creation

3.5.2 Network Initialization

3.5.3 Network Training

3.5.4 Network Simulation

3.5.5 Network Performance

3.6 Training Characters

3.7 Gradient Descent with Momentum BP (traingdm)

3.8 Training Object

3.9 Network Properties

3.10 Testing Set Images

3.11 Implementation of Graphical User Interface (GUI)

33

34

35

35

36

37

38

39

40

41

42

43

44

44

46

46

46

47

48

Chapter 4 RESULTS AND DISCUSSIONS

4.1 Image Processing Process

4.1.1 Step On Image Processing

4.2 GUI Results

49

50

51

ix

4.3 Result Discussions

4.4 Number of Neuron in Hidden Layer

4.5 Number of Epochs

4.6 Learning Rate

4.7 Project Discussions

55

57

58

58

58

Chapter 5

CONCLUSSIONS AND RECOMMENDATIONS

5.1 Project Achievements

5.2 Recommendation for Future Works

59

60

REFERENCES

61

APPENDIX

APPENDIX A

APPENDIX B

APPENDIX C

APPENDIX D

67

68

69

74

x

LIST OF TABLES

Tables

Pages

3.1 Training Character with the Targets 45

3.2 Network Properties 47

4.1 Characters Result with Respective Compet Function

Answer

56

xi

LIST OF FIGURES

Figures

Pages

1.1 Standard Configuration of Malaysia License Plates 2

2.1 Gray Level Transformation for Contrast

Enhancement

10

2.2 Gray Level Slicing Transformation 11

2.3 Result Image of using Technique in Figure 2.2 (a) 11

2.4 The Eight Bit Planes of the Eight Bit Image 12

2.5

High and Bright Intensity Image with Histogram 14

2.6 Low-contrast and High-contrast Image with

Histogram

14

2.7 Result Of Histogram Equalization And Their

Histogram

15

2.8 Result Images of Ideal Highpass Filter 19

2.9 Extracting the Character from License Plate 20

2.10 License Plate Character 21

2.11 Feature Extracted using Kirsch Edge Detection 22

2.12 Multilayer Net Architecture 23

2.13 Feed-Forward Neural Network (Single Hidden

Layer)

25

2.14 Multilayer Feed-Forward Neural Network 25

2.15 Basic Feedback Structure 26

xii

2.16 Back-propagation Net

27

3.1 Flow Chart of Character Recognition 34

3.2 Image Preprocessing Process 36

3.3 Converting RGB to Grayscale Image 37

3.4 Neural Network Design 39

3.5 Neural Network Training Windows 43

3.6 License Plate Recognition GUI 48

4.1 Image of Moving Vehicles

50

4.2 Load Image into GUI 51

4.3 License Plate Image after Extraction 52

4.4 Image of Seven License Plate Character

52

4.5 Convert into Binary Image 53

4.6 Load Image for Neural Network Test 53

4.7 Recognized Character 54

4.8 Uncompleted Recognize Characters 54

4.9 License Plate with Segmentation Problem 57

xiii

LIST OF ABBREVIATIONS

ANN Artificial Neural Network

BP Back-propagation

DSLR Digital Single-lens Reflex Camera

FYP Final Year Project

MATLAB Matrix Laboratory

NN Neural Network

RGB Red Green Blue

UNIMAS University Malaysia Sarawak

GUI Graphical User Interface

1

CHAPTER 1

INTRODUCTION

1.1 Background

Vehicles license plate recognition is one of the important techniques that can

be used for identification of vehicles around the world. It useful in many applications

such as entrance admission, security, parking control, road traffic control, speed

control and so on [1]. This project entitled “License Plate Recognition of Moving

Vehicles” is a system that will be developed to recognize the license plate characters

in various speeds and conditions of moving vehicles. This project consists of

simulation program to recognize license plate characters where a captured image of

moving vehicles will be the input. The image will then be processed and analyzed

using image processing and neural network techniques. Based on network

performance error calculated from neural network output and target output will

determined whether the neural network recognize the input as the target. Figure 1.1

shows the standard layout configurations of license plates for Malaysian private

vehicles [1].

2

Figure 1.1: Standard Configuration of Malaysia License Plates

1.2 Project Objectives

The project objectives of designing the software for license plate recognition

as has been discussed with the supervisor have been identified, as follows:-.

1. Develop the coding to loading image and neural network training to

recognizing the character from license plate of moving vehicles.

2. Develop the coding that can extract the character from single line

pattern with seven character of license plate in Malaysia particularly

Sarawak state.

3. Develop the coding for image processing process and neural network

training.

3

1.3 Statement of Expected Problems

The main expected problem that will be encountered in this project is blurred

image that produce when capturing moving vehicles license plate image using

normal camera. This problem occurs due to slow camera speed as compared to the

moving vehicles. The blurred image capture also occurs when capturing image

during the different conditions weather such as raining, sunshine, night and cloudy.

Blurred image problem is basically causes by the hardware part of this project.

The similarity of some alphabet and number patterns are also foreseen to the

problems occur in this project. The alphabet and number that might be similar are

misinterpreted by developed software are “1” with “7”, “2” with “Z” and “8” with

“B”. This may causes error where incorrect results are displayed by the simulation

software developed

1.4 Proposed Solutions

The solution on blurred image can be solved by using the appropriate camera

that suitable for capture the moving image such as high speed camera. Although this

camera speed can capture the freeze moving image, it is still incapable of solving the

blurred image. The images will still certain same blurred edges. Therefore, the image

must undergo some image processing technique using MATLAB software to remove

the blur edges. After this process, the image obtain will be used to undergo the next

part for character recognition.

4

The neural network approach will recognize each character that has been

extract from image, where it will solve the misinterpreted character problem. The

solution on recognizing the similar character will be also solved using neural

network approach where the best chosen learning pattern identified will be used for

this problem. There are a lot of neural network types can be chosen for better

accuracy on recognizing the character. Large amount of neural network training in

MATLAB software toolbox will give more understanding on this approach and

increases the ability of this project to recognize the character correctly.

1.5 Expected Outcomes

This FYP project is based on simulation program develop using MATLAB.

However, this project still employs hardware system on the image acquisition part.

This project consist the hardware used to capture the moving vehicles image and the

simulation program used to do the image preprocessing and recognize the license

plate character based on target output given in neural network training process. This

simulation program will be developed using MATLAB software. Image

preprocessing consist the process of image enhancement, filtering the noise and

extracts the character from license plate where neural network part consists of

process to recognize each license plate character based on error calculation between

network output and target output. Video camera is used to capture the moving image

and convert it into image frame.

5

1.6 Report Outlines

The report outline contains the undergoing chapter of the final year project

report. Chapter 1 starts with introduction of the project, benefits and also the aims

and objective of the project. This chapter also gives explanation on the statement of

problem together with the proposed solution on each problem and project outline that

has been followed while undergoing the final year project.

Chapter 2 is the literature review where it summarizes the recent research and

scholarly sources relevant on the particular issue and theory in this project. This

chapter also summarizes the particular of theory on simulation approach that

connected with this project. In this context, the research on license plate recognition

using another approach and the explanation on simulation approach such as image

processing part will be discusses.

Chapter 3 is methodology which summarize about the method that will be

used in this project to obtain the result. In this project the method that will be used in

recognizing the license plate character is image processing approach and neural

network simulation using MATLAB.

Chapter 4 explains the result and discussion from this project. The result

represent in from of network performance graph, the network simulation result from

the network training and the discussion on recognition result.

6

Chapter 5 concludes the report summary of the finding obtain though out the

whole FYP project. The conclusion on work experience and work effort done to meet

the requirement on this project development has been told here. The future work on

improve this project and recommendation on new title research that similar with the

project also been suggest here.

7

CHAPTER 2

LITERATURE REVIEW

2.1 License Plate Recognition System

License Plate Recognition of Moving Vehicles is based on image processing

and neural network where image processing techniques such as edge detection,

thresholding and re-sampling has been used to locate and isolate the license plate and

the characters. The neural network was used for successful recognition the license

plate number [2]. There are many researches on this project title where they using a

different method for license plate character recognition [1, 2, 3].

Among the thesis on license plate recognition system titles Vehicles license

plate character recognition by neural network by M. Khalid et. al. [1], Smart License

Plate Recognition System based on Image Processing using neural network by

V. Koval et. al. [2] and car license plate recognition with neural network and fuzzy

logic by J.A.G Nijhuis et. al. [3]. Some of the method that they used is almost same

especially on the image preprocessing, image segmentation and also use the

grayscale image.

8

2.2 Image Preprocessing

Image preprocessing is an important process which it used to manipulate the

images for character recognition operation. The image preprocessing applies some

standard image processing technique such as contrast stretching and noise filtering to

enhance the quality of the image [3]. Capturing image of moving object will produce

the blurred image, using the computer algorithm; image will be preprocessing to

improve the quality to allow the character in the images to be recognized. In image

preprocessing, color image (RGB) acquired by a digital camera is converted to gray-

scale image based on the RGB to gray-scale conversion technique. The basic idea of

this conversion is performed by eliminating the hue and saturation information while

retaining the luminance. Equation (2.1) shows an optimal method for RGB to gray-

scale conversion [4].

(2.1)

where

Lu is luminance

R refers to red components

G refers to green components

B refers to blue components