Research ArticleColor Scale and Rotation Independent Multiple LicensePlates Detection in Videos and Still Images
Narasimha Reddy Soora and Parag S Deshpande
Department of Computer Science amp Engineering Visvesvaraya National Institute of Technology Nagpur 440010 India
Correspondence should be addressed to Narasimha Reddy Soora snreddy75yahoocouk
Received 4 March 2016 Revised 7 June 2016 Accepted 16 June 2016
Academic Editor Daniel Zaldivar
Copyright copy 2016 N R Soora and P S Deshpande This is an open access article distributed under the Creative CommonsAttribution License which permits unrestricted use distribution and reproduction in any medium provided the original work isproperly cited
Most of the existing license plate (LP) detection systems have shown significant development in the processing of the imageswith restrictions related to environmental conditions and plate variations With increased mobility and internationalization thereis a need to develop a universal LP detection system which can handle multiple LPs of many countries and any vehicle inan open environment and all weather conditions having different plate variations This paper presents a novel LP detectionmethod using different clustering techniques based on geometrical properties of the LP characters and proposed a new characterextraction method for noisymissed character components of the LP due to the presence of noise between LP characters and LPborder The proposed method detects multiple LPs from an input image or video having different plate variations under differentenvironmental and weather conditions because of the geometrical properties of the set of characters in the LP The proposedmethod is tested using standard media-lab and Application Oriented License Plate (AOLP) benchmark LP recognition databasesand achieved the success rates of 973 and 937 respectively Results clearly indicate that the proposed approach is comparableto the previously published papers which evaluated their performance on publicly available benchmark LP databases
1 Introduction
License plate recognition (LPR) system plays a key role inintelligent transportation systems such as traffic controlparking lot access control electronic toll collection andinformation management Typical LPR system contains fourprocessing steps The first step is to get the image or videofrom the camera The second step is LP detection fromthe input image The third step is to extract the charactersfrom the LP and the final step is to recognize the extractedcharacters using different classifiers These four steps can beachieved by the combination of different techniques of imageprocessing and pattern recognition Out of these four stepsthe LP detection and character recognition steps are verycrucial for the success of LPR systems
LP detection systems have shown significant develop-ment for more than a decade with good performance reportsbut most of these systemsrsquo evaluation is carried on propri-etary data sets having controlled conditions on environmentand plate variations To assess the performance of the LP
detection methods there is a need for a common publiclyavailable benchmark LPdata set which should contain videosand images taken in an open environment and with differentplate variations A common publicly available benchmark LPdata set for performance evaluation of LPR systems which isinitiated by Anagnostopoulos et al in paper [1] and contains741 still images of Greek LPs with several open environmentalconditions and different plate variations is present at [2]For evaluation of the proposed approach we have used 741still images of media-lab Greek LP database 159 Indian andIsraeli LPs from videos and still images As media-lab GreekLP database missed motorcycles vehicles with rotated LPsthe combination of different types of vehicles and more thanonemotorcycle in a single image an appropriate care is takenwhile selecting Indian and Israeli LP images to achieve all thecombinations of plate variations which are missed by media-lab Greek LP database
In this paper we have proposed a new approach forfinding the LPLPs in an image using various clustering tech-niques on geometrical properties of the LP characters and
Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2016 Article ID 9306282 14 pageshttpdxdoiorg10115520169306282
2 Mathematical Problems in Engineering
a new approach for finding and extracting the noisymissedcharacters of the LPLPs due to the presence of noise suchas dirt or screw or stamp between LP characters and LPborder The clustering techniques proposed in this paper usegeometrical properties of the components of LP characterssuch as the distance between the components the anglebetween the components and the height of the componentsto find the probable LPLPsThis is the first time that differentclustering techniques are applied on geometrical propertiesof the components of an input image for finding the probableLPLPsThe proposed geometry-based clusteringmethod forfinding the vehicle LPLPs is scale and rotational invariantand is suitable for many countries LP detection for any typeof vehicles and motorcycles having different plate variations
The performance of the proposed LP detection methodis more prominent when compared with other competitiveLP detection methods from the literature by taking intoconsideration publicly available benchmark LP databases Itis inappropriate to declare which methods are better becauseinmost of the previously publishedmethods the performanceevaluations were carried on proprietary data sets havingrestricted conditions and were not revealed to the public toassess their performance In this paper we are proposing newmethods for LP detection noisymissed character extractionand LP characters rotation correction New findings in thispaper are as follows
(i) Proposing a new method for LP detection usingdistance-based line-based and height-based cluster-ing techniques on geometrical properties of the LPcomponents
(ii) Proposing a new method to remove unwanted clus-tered components using the thinning and resizingtechnique
(iii) Proposing a new method for correcting LP rotationof the probable LP cluster components using theaverage angle amongst successive probable LP clustercomponentsrsquo left-top coordinates and 119909-axis
(iv) Proposing a new method to extract the noisymissedcharacters of the probable LP because of the presenceof noise between LP characters and LP border
The remaining sections of this paper are planned as followsSection 2 exhibits the existing similar research Section 3describes the proposed approach for multiple LPs detectionSection 4 elaborates on the proposed methodology formultiple LPs detection Section 5 describes the extraction ofnoisymissed characters due to the presence of noise betweenLP characters and LP border This section also describes amethod for probable LP characters rotation if the vehicle LPis rotated Experimental results are discussed in Section 6 andConclusion in Section 7
2 Existing Similar Research
Many LPR algorithms have been proposed in the literaturefor the past ten years and even today LP detection remainsthe challenging area due to different environmental condi-tions and plate variations In the literature there is no LP
detection method which will work for many countries forall types of vehicles and motorcycles without any constraintLP detection is challenging and crucial in LPR systemswhich influences the recognition rate Most of the existingLP detection papers from the literature are based on edgeinformation morphological operations template matchingand color information of the LP
Lee et al in paper [3] proposed a color image processing(CIP) method to extract the LP of Korean carrsquos based on LPbackground and LP characters color using the color histo-gram A Neural Network (NN) classifier is used to classifya color This paper used the aspect ratio of the LP regionto select the most probable LP and reported 9125 successrate for LP detection over 80 car images The drawback ofpaper [3] is that the LP detection will not work properlyif the vehicle color matches either background color of theLP or color of the LP characters A hybrid LP localizationscheme is presented by Bai and Liu in paper [4] based onthe edge statistics and morphology (ESM) The proposedapproach had four sections Section 1 handles the verticaledge detection Section 2 takes care of the edge statisticalanalysis Section 3 finds the hierarchical-based LP locationand Section 4 finds the morphology-based LP extractionThis paper reported 996 overall success rate for detectingthe LP out of 9825 images The drawback of paper [4] is thatit uses edge information and morphology-based approachesto detect the LP Some LPs are not so easy to detect usingedge information andmorphology-based approaches have todefine Structuring Element (SE) to perform morphologicaloperations to find the probable LPLPs from an input imageDefining a particular SE to perform morphology-basedoperations to detect the probable LPLPs is a nongenericapproach and will fail to detect the LPLPs from the inputimages under various characteristics of LPLPs in the images
Yang et al in paper [5] proposed a new method based onfixed color collocation (FCC) to locate the LP This methodused the color collocation of the platersquos background andcharacters to recognize the LPs This paper reported 95success rate for LP detection The drawback of paper [5] isalso similar to the drawback of paper [3] Anagnostopoulos etal in paper [6] proposed a new image segmentation methodcalled sliding concentric windows (SCW) for LP detectionThe SCW method works based on local irregularities in theimage The method uses statistics such as standard deviationand mean value for possible LP location SCW uses twoconcentric windows A and B with different sizes to scanthe image from left to right and top to bottom to find themean and standard deviation of the regions of the concentricwindows If the ratio of the statistical measurements exceedsa threshold value set by the user then the central pixel of thetwo concentric windows is considered to be the part of LPThis paper reported a success rate of 965 for LP detectionusing media-lab proprietary LP data set The limitation ofpaper [6] is that the statistical measurement threshold valueset by the user has to be decided according to the applicationafter a trial-and-error procedure which is not a genericsolution
Faradji et al in paper [7] proposed a real-time and robust(RTR) method to find LP location Finding LP location has
Mathematical Problems in Engineering 3
several stages with the combination of Sobelmask histogramanalysis and morphological operations The overall successrate for detecting the LP by this paper was 835The limita-tion of paper [7] is the same as that in paper [4] Huang et alin paper [8] proposed LPR strategy formotorcycles for check-ing annual inspection status The LP data set considered bythis paper contains onlymotorcycles having the LP charactersfalling in only one line The method proposed in this paperfinds the LP using search window with the help of horizontaland vertical projections (SWHVP) and reported an averageLP detection rate of 9755The drawback of paper [8] is thatit is not mentioned how to get the initial size of the searchwindow to perform horizontal and vertical projections Italso used morphology-based dilation operation which isnot a generic solution to detect the LPLPs as explained inpaper [4] drawback Wen et al in paper [9] proposed twomethods to find the LP from an input image (two pass)These two methods are based on Connected ComponentAnalysis (CCA) model Before applying these methods theinput image is binarized using improved Bernsen algorithmto remove shadows and uneven illumination Method 1 isused to find the candidate regions based on prior knowledgeof the LP The frame is detected using CCA methodologyIf the frame is broken the LP cannot be detected correctlyWhen the LP is not detected using Method 1 then Method 2is adopted Method 2 extracts the LP using large numeralextraction technique This paper reported a success rate of9716 The drawback of paper [9] is that the proposedMethod 1 fails to detect the LPLPs from the input image ifthe LP frame is brokenThe drawback of theMethod 2 is thatit will fail to detect the LPLPs from the input image if theLPLPs are not in horizontal position
Haneda and Hanaizumi in paper [10] proposed RELIPalgorithm which performs a global search for the probableLP using multiple templates 3D cross-correlation functionand Principal Component ExpansionThis paper uses cornerdetection to remove deformation of LPs RELIP reported 97LP detection success rate The drawback of paper [10] is thatit uses spatial similarity with an LPLPs template to detect theLP which is not a generic solution in the real world contextas the LPs having various deformations such as tilt rotationand pan from various viewpoints Zhou et al in paper [11]proposed a new approach for LP detection based on PrincipalVisual Word (PVW) discovering and visual word matchingIn visual word matching it will compare the extracted SIFTfeatures of the test image with all discovered PVW and locatethe LP based on matching results This method published932 success rate on the proprietary data set and 848success rate on Caltech dataset The drawback of paper [11]is that it works based on the prior knowledge of the LP
Al-Ghaili et al in paper [12] proposed a new approachin which a color image is converted to grayscale and thenthe adaptive threshold is applied on the grayscale image toconvert it into a binary image ULEA method is appliedto the grayscale image to enhance the quality by removingthe noise Next VEDA is applied to detect the LP from theinput image In order to detect the true LP some statisticaland logical operations are applied The success rate reportedby this paper was 9165 for LP detection The drawback
of paper [12] is that it extracts the LPLPs from the inputimage based on extracting vertical edges which is the sameas the drawback of paper [4] Hsu et al in paper [13] pro-posed a new approach (AOLP) for detecting LP candidatesusing Expectation-Maximization clustering method on ver-tical edges of grayscale images This paper reported 9333success rate on AOLP proprietary benchmark LP data set andreported 921 success rate onmedia-lab benchmark LP dataset for LP detection It ismentioned in the paper that the LPRsolution is designed primarily based on LPs of Taiwan and isnot optimal for other countriesThe drawback of paper [13] isthe same as that in paper [4] because its LP detection is basedon extracting vertical edges
Abo Samra and Khalefah in paper [14] proposed a newLP localization algorithm using dynamic image processingtechniques and genetic algorithms (GA) (DIP-GA) CCAtechnique is used to detect the candidate objects of the inputimage and improved the CCA technique with the help ofmodifiedGAThe system ismade adaptable to any country byintroducing a scale-invariant geometric relationship matrixto model the LP symbols The speed of the LP detection isimproved by introducing two new crossover operators Thesystem reported 9761 success rate using publicly availablemedia-lab benchmark LP database by considering only 335images out of 741 images This paper also reported 9875success rate using proprietary data set having 800 imagesamples and reported 9841 overall accuracyThe drawbackof paper [14] is that it is not able to detect multiple LPs in animage
The LP detection methods which use edge informationand morphological operations mainly focus on finding thecomponents which are rectangular in shape with specificaspect ratio Such type of LP extraction methods will failto identify the LP if the LP does not follow a rectangularshape with proper aspect ratio The problem with templatematching LP detection methods is that it will not work forall types of plate variations The color based LP extractionmethods use the background color of the LP to identify theprobable LP candidate region because some countries use aparticular background color in their LPs Such type of LPRsystems will fail to detect the LP properly if the body color ofthe vehicle matches with the LP background colorThe abovecategorized LP detection methods used the features of LP inan image to extract the LP location from an input imageThese categories of LP detection methods have limitations inextracting LPLPs from an image because of the features theyadopted to extract LPLPs
In this paper we are proposing for the first time anew LP detection method which will work for the LPdetection of many countries having any shape which usesdifferent clustering techniques on geometrical propertiesof the character components of the LPLPs in the inputimageThe advantage of using different clustering techniqueson geometrical properties of the character components ofthe LP is that they are independent of scale rotation tiltand orientation There are very few techniquespublicationsin the literature which talk about LP detection methodsunder various environmental conditions and plate variations
4 Mathematical Problems in Engineering
mentioned which will work for any type of vehicle andmotorcycle having any LP shape
3 Proposed Approach for Multiple LicensePlates Detection
This paper proposes a new method for LPLPs detectionand noisy characters extraction for any type of vehicle andmotorcycle having different plate variations under differentenvironmental and weather conditions The environmentalconditions include different illumination weather and back-ground conditions The plate variations include location ofthe plate anywhere on the vehicle many plates in singleimage different combination of vehicles with different plateorientations different sizes of plates background color ofplates plates with dirt rotated plates LPs having two linesof characters and each line of characters are of different sizeand tilted LPs The proposed method can be articulated asa generalized method for identifying the LPLPs because itis independent of plate variations under different environ-mental andweather conditions and can be applicable tomanycountries and for any vehicle having multiple lines in the LPIn the proposed approach we have not used any type of edgedetection templatematchingmorphological operations andcolor information of the LPs which are extensively used bypreviously published papers to detect the LP
The proposed method uses CCA to label the componentsand applies different clustering techniques on geometricalproperties of the labelled components such as the location ofthe components the angle between the components and theheight of the components to extract the probable LPLPs Inmost of the countries the LP characters are near to each otherare positioned along one or multiple lines and are similar inheight Based on these properties of LP characters clusteringtechniques can be applied on geometrical properties of LPcharacters to identify the LPLPs from an input image Mostof these properties are followed by many nations whiledesigning their LPs That is why the proposed approach canbe applicable to detect the LP of many nations which followthe properties of the LP characters mentioned in this paperwhile designing their LPsThe proposedmethod contains thefollowing steps
(i) Apply preprocessing steps on the input image If it isa video convert the video into different frames andthen apply preprocessing steps After completing thepreprocessing steps label the components of the inputimage to find the number of components 119873
119868and to
extract the geometrical properties of the each of 119873119868
components such as left-top coordinates width andheight
(ii) Apply newly proposed distance-based clusteringalgorithm on each of119873
119868componentsrsquo left-top coordi-
nates This algorithm divides the image componentsinto various distance-based clusters which are closeto each other Let119873
119889be the number of clusters formed
after distance-based clustering from 119873119868components
of the preprocessed input image
(iii) Recluster each 119873119889distance-based cluster into dif-
ferent line-based clusters If the components ofthe distance-based clusters subtend a similar anglebetween the lines joining left-top coordinates of thecomponents with the 119909-axis then construct line-based clusters from such components Line-basedclustering algorithm reclusters 119873
119889distance-based
cluster components into119873119897line-based clusters which
are in line and close to each other(iv) Recluster each 119873
119897line-based cluster based on the
cluster components height This is height-based clus-tering which reclusters119873
119897line-based clusters into119873
ℎ
height-based clusters(v) After the distance line and height-based clustering
techniques the resultant clustered components havethe properties such as near to each other positionedalong a line and being similar in height Theseproperties belong to the characters of LP of anyvehicle and motorcycle in the world
(vi) In the resultant 119873ℎclusters there may be a few
non-LP clusters in which all the components followthe properties of LP characters In order to removesuch non-LP clusters apply the thinning and resizingtechnique on119873
ℎheight-based clusters Let119873tr be the
number of probable LP clusters in the image afterapplying thinning and resizing technique
(vii) Refine 119873tr clusters further by finding the border ofthe cluster components If the border percentage foreach cluster is less than predefined threshold removesuch clusters from the list of probable LP cluster (119873tr)After this step let 119873 be the number of probable LPclusters
(viii) Now apply the newly proposed character extractionmethod to extract noisymissed characters of the LPon each of ldquo119873rdquo probable LPs due to the presenceof noise such as screw or dirt or stamps between LPcharacters and border of the LP
(ix) After extracting the noisymissed characters rotatethe probable LP characters using the average anglebetween the lines joining adjacent componentsrsquo left-top coordinates and the119909-axis so that all the probableLP characters will be horizontal to the 119909-axis
The above mentioned outlined procedures are explained indetail in the following sections
4 Detailed Description of the ProposedApproach for Multiple License PlatesDetection from Videos and Still Images
This section describes in detail the proposed approach to findthe probable LPLPs in an image with the help of variousclustering thinning and resizing techniques This sectionalso explains the method and the need to further refine theprobable LPLPs by finding the border of the LPLPs
Mathematical Problems in Engineering 5
Input image 1
(a)
Complemented binary image
(b)
Binarized image after removingsmall components
(c)
Figure 1 Results of preprocessing stage
41 Preprocessing Stage
Steps(1) Due to the effect of illumination in an open envi-
ronment and the presence of shadows it is verydifficult to process an input image with the help oftraditional threshold binarization methods and willnot give satisfactory results In this paper we are usingBernsen algorithm to overcome the illumination andshadowproblems in an image Let119891(119909 119910) denote grayvalue at a point (119909 119910) of an image Let (119909 119910) be thecentre of a block of size (2119908+1)times(2119908+1) in the imagewhere 119908 is the number of pixels Threshold 119879(119909 119910) atthe point (119909 119910) can be computed using
(2) Convert the input image 1 (shown in Figure 1) intograyscale If it is video convert it into frames and thenconvert each frame into grayscale Apply the Bernsenalgorithm to overcome from uneven illuminationsor shadows present in the grayscale image Comple-ment the binarized image whose output is shown inFigure 1 second column Remove the componentswhose height is less than three pixels from the com-plemented binary image because no LP character isless than three pixels in height The image in Figure 1third column shows the output after removing thecomponents that are less than three pixels in height
(3) To implement the rest of the operations on individualcomponents such distance-based clustering line-based clustering and height-based clustering extractthe geometrical properties of individual componentssuch as left-top coordinates width and height
(4) Geometrical properties of the individual componentsdescribed in Step 3 can be extracted using the follow-ing procedure
(a) To find the number of components present inan image use CCA method to apply labellingto preprocessed image Let119873
119868be the number of
components present in the input image at thisstage
(b) After labelling find left-top coordinates widthand height of119873
119868components
(c) Crop each individual component from thepreprocessed image using left-top coordinateswidth and height
(d) Save the cropped components of the inputimage as separate image components
42 Clustering Stage The purpose of the clustering stage isto prepare the probable LP clusters from 119873
119868components of
the preprocessed input image In this stage the proposedsystem performs three types of clustering techniques oneafter the other The first clustering technique is the distance-based clustering whose purpose is to divide119873
119868components
of the preprocessed input image into groups which arenear to each other The second clustering technique is theline-based clustering whose significance is to divide eachdistance-based cluster into an array of line-based clustersThecomponents of the line-based clusters are in line and closeto each other from the viewpoint of left-top coordinates ofthe componentsThe third clustering technique is the height-based clustering whose significance is to regroup the line-based cluster components which are similar in their heightAfter all these clustering techniques the resultant clustercomponents in each cluster are close to each other positionedin a line and alike in their heights which are the probableLPLPs of the input image
Context dependent variables are cluster size and maxdistance The first context dependent variable cluster sizeindicates the minimum number of characters in a row of theLP and can be tuned to satisfy country specific LP constraintsIn our experiments we have considered cluster size as 4The next context dependent variable max distance is usedduring various clustering stages which explain the maximumdistance that is allowed between the successive componentsof the LP clusters The value of the variable max distance iscomputed as one-third of the columns of the input imageTheproposed geometry-based clustering method has followingsteps
Steps
(1) After preprocessing stage cluster all the compo-nents of the image based on the distance betweenleft-top coordinates of each individual component
6 Mathematical Problems in Engineering
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 2 An example image showing the components after dis-tance-based clustering
using distance-based clustering algorithm which isexplained in detail in the following steps
(2) Distance-based clustering algorithm prepares a ma-trix (distance matrix) of size [119873
119868 119873119868] where119873
119868indi-
cates the number of components of the input imageafter the preprocessing stage In the distance matrixthe 1st row indicates the distance between the 1stlabel component and rest of the componentsrsquo left-topcoordinates 2nd row indicates the distance between2nd label component and rest of the componentsrsquo left-top coordinates and so on Distance-based clusteringalgorithm prepares maximum 119873
119868clusters one for
each component with the help of distance matrix
(3) Remove those distance-based clusters whose size isless than cluster size andwhich are the subset of otherdistance-based clusters Retain those distance-basedclusters with a large number of components in it whenperforming the subset removal operation At thisstage the 119873
119868components of the image are clustered
into groups based on the distance between the left-top coordinates of each individual component
(4) Figure 2 shows an example image with the com-ponents C1 sdot sdot sdotC13 which are clustered into twodistance-based clusters highlighted with rectangularborder (in red color) The components C1 sdot sdot sdotC9formed as first distance-based cluster and C10 sdot sdot sdotC13as second distance-based cluster
(5) Figure 3 shows the resultant image after distance-based clustering Let 119873
119889be the number of distance-
based clusters at this stage which are shown as ellipsesin Figure 3 As most of the components of the inputimage are very small in size it is not possible toobservewith the naked eye the components formed asdistance-based clusters from the input image Hencethe same is explained by taking an example as shownin Figure 2
Figure 3 Resultant image after distance-based clustering
(6) Now apply line-based clustering technique on each of119873119889distance-based clusters to find those components
which are in line with each other(7) In the line-based clustering consider individual
distance-based cluster and take each component fromthe cluster and draw a line from left-top coordinates(1199091 1199101) of one component to the left-top coordinates
(1199092 1199102) of the next component in the current cluster
Find the angle 120579 between 119909-axis and the line thatis drawn between two left-top coordinates of com-ponents as shown in (2) In the same way find theangle between rest of the components and 119909-axisas a matrix (angle matrix) of size [119873119873] where 119873indicates the number of individual components ineach distance-based cluster
120579 = (
180
120587
) lowast tanminus1 ((1199102minus 1199101)
(1199092minus 1199091)
) (2)
(8) Cluster those components as line-based clusterswhich subtend similar angle with the 119909-axis andwhich are close to each other (based on max dis-tance) for each row of the angle matrix Nowremove those line-based clusters which are less thancluster size and which are subsets to other line-basedclusters This is the line-based clustering technique
(9) Line-based clustering is used to recluster the distance-based cluster components based on the propertyof having similar angle and closeness of the clustercomponents After this stage the components ofan input image are clustered using distance-basedclustering and line-based clustering techniques
(10) Figure 4 shows an example image in which the dis-tance-based clusters (shown in the red border) aredivided into line-based clusters (shown in the greenborder) In Figure 4 the first distance-based clusterwith the components C1 sdot sdot sdotC9 is divided into twoline-based clusters with the components C1 sdot sdot sdotC5and C6 sdot sdot sdotC9 indicated with rectangular (green incolor) boxes The second distance-based cluster withthe components C10 sdot sdot sdotC13 resulted into a line-basedcluster with the components C10 C11 and C13 andthe component C12 is removed from the resultant listbecause C12 is not in line with other componentsTheresultant image after line-based clustering is shown in
Mathematical Problems in Engineering 7
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 4 An example image showing how the distance-basedclusters are divided into the line-based clusters
Figure 5 Resultant image after line-based clustering
Figure 5 in which the line-based clusters are markedin rectangular boxes
(11) There is no difference between Figures 3 and 5 basedon the number of components and positions of thecomponents are concerned The resultant distance-based clusters are shown in the ellipse shape withvarious colors in Figure 3 The resultant line-basedclusters are shown in the rectangular shape withvarious colors in Figure 5
(12) After line-based clustering apply height-based clus-tering technique to remove few unwantedjunk com-ponents which are very close to and in line with thecomponents but shows much difference in height ascompared with other components in each line-basedclusterNow remove those clusterswhich are less thancluster size and which are subsets to other clustersThis is height-based clustering
(13) Figure 6 shows an example image in which thecomponent C3 (indicated with the arrow) is showingmuch difference in height as compared to the rest ofthe components it will be removed from the line-based cluster and will result in height-based clusterwith the components C1 C2 C4 and C5 Figure 7shows the image after height-based clustering and theresultant height-based clusters are shown in ellipses
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 6 An example image showing the height-based clustering
Figure 7 Resultant image after height-based clustering
At this stage let 119873ℎbe the number of height-based
clusters(14) After height-based clustering few of the clusters
from the final cluster list may contain all unwantedcomponents which obey all the characteristics basedon distance line and height-based clustering Suchjunk component clusters can be removed by thinningand resizing technique
(15) Apply infinite thinning and resizing technique oneach 119873
ℎheight-based cluster of the image Thinning
is a morphological operation used for skeletonizationof the binary image components When we applythinning and resizing operations junk componentswill retain its shape but the LP characters will fadeaway completely Remove those clusters from thecluster list which retain its shape after thinning andresizing operations This technique removes all junkcluster components from the final cluster list Let 119873
1
be the number of clusters after removing junk clustersusing the thinning and resizing technique Figure 8shows image after thinning and resizing techniqueand the resultant clusters are shown in ellipses shape
43 Finding Border of the License Plate Contrasting toa typical LPR system the proposed system first findsthe probable LP characters and then finds the border of
8 Mathematical Problems in Engineering
Figure 8 Resultant image after thinning and resizing technique
the components The reason for finding the border of LP isthat there can be a group of non-LP characters in the imagewith similar properties of LP characters without the borderIn order to avoid such type of characters the system proposesto find the border of the LP
Predefined Border Percentage Beta Beta is a user definedvariable which decides the border percentage that an LPcan have After rigorous experimentations with the help ofmany datasets from multiple countries we have come to aconclusion to decide the value of Beta as 70
Steps
(1) Let 1198731be the number of probable LP clusters at
this stage Take each individual component fromthe cluster and traverse from left-top coordinatesof each individual component towards the upwarddirection till the traversal reaches the border pointor three times the height of each individual charactercomponent
(2) Apply the same procedure towards the downwarddirection to find the bottom border point for eachindividual component
(3) Save the top and bottom border points for all thecomponents of each cluster
(4) Find the border percentage for each cluster usingthe top and bottom border points Retain a clusteronly when the percentage is greater than or equalto the predefined border percentage ldquoBetardquo Retainedclusters indicate the LPLPs of an image and itscomponents indicate the individual characters of theLP
(5) Remove those clusters which fall below predefinedborder percentage ldquoBetardquo This is another way tofurther refine the cluster list to get the required LPregion Figure 9 shows the image after LP borderidentification
5 Noisy Characters Extraction and LicensePlate Characters Rotation
Noisymissed characters extraction stage is to extract few ofthe LP characters which may be missed during the previous
Figure 9 Resultant image after license plate border identification
stages because of the presence of noise such as screw ordirt or stamps between the LP border and LP charactersThis section proposes a new approach to extract such typeof noisymissed charactersThe proposed noisymissed char-acters extraction algorithm can be applied after Section 43
Steps
(1) The proposed algorithm uses (3) to find the noisy LPcharactersrsquo left-top coordinates (119909
2 1199102) at a distance
119889 from the leftmost cluster componentrsquos left-topcoordinates (119909
1 1199101) and the average slope119898 amongst
the cluster components
1199092= 1199091plusmn (
119889
radic (1 + 1198982)
)
1199102= 1199101plusmn (
119898 lowast 119889
radic (1 + 1198982)
)
(3)
(2) At this stage we have few probable LP clusters Takeeach probable LP cluster and find the average heightand average slope (119898) amongst the subsequent clustercomponentsrsquo left-top coordinates and the 119909-axis
(3) From each probable LP cluster take left-top coordi-nates of the first component (119909
1 1199101) and move in the
downward direction to (14)th of the average heightof probable LP cluster Now traverse the image rightside one pixel at a time using (3) mentioned aboveto find any noisymissed LP character components
(4) If any noisymissed LP character component is foundwhich is not part of the probable LP cluster com-ponent and is not a background pixel then findthe left-top coordinates (119909
2 1199102) and the width of
the noisymissed component To find the left-topcoordinates (119909
2 1199102) and thewidth of the noisymissed
component we have to perform three traversals asdescribed as follows
(i) The first traversal is towards the top side ofthe noisymissed character component to com-pensate the left-top coordinatersquos slope with theaverage slopeThe second traversal is to traversetowards the left side of themissed component inthe direction of the average slope till it reaches
Mathematical Problems in Engineering 9
leftmost pixel of the noisymissed componentwithin the average height using (3) After reach-ing the left most side of the noisymissed com-ponent fix it as left-top coordinates (119909
2 1199102) of
the noisymissed component(ii) Take the newly found left-top coordinates of
the noisymissed LP character component andmove towards the right side (third traversal)in the direction of the average slope till itreaches the right most point of the missedcomponent within the average height using (3)The difference between the 119909-coordinates of thenewly found left-top coordinate point (119909
1 1199101)
and the right most point is the width of thenoisymissed component
(iii) Crop the noisymissed component using left-top coordinates width and the average heightof the cluster components
(5) Repeat the same procedure till the traversal reachesthe last component of the probable LP cluster com-ponent Repeat the same procedure for all probableLP clusters
(6) Rotate each individual component of the probableLP cluster by using average angle 120579 which can becalculated from the average slope (119898) amongst theprobable LP cluster components using (4) Con-sider each component of the binary image as 119865 =119865(119894 119895) 119894 = 1 2 119868 119895 = 1 2 119869 and can bedefined as shown in (5) One has
120579 = (
180
120587
) lowast tanminus1 (119898) (4)
119865 (119894 119895) =
1 white pixel
0 black pixel(5)
(7) Let 119865(119909 119910) be the image component before rota-tion and let 119865(1199091015840 1199101015840) be the image component afterrotation Use average angle 120579 to rotate 119865(119909 119910) Theequation for each individual pixel of 119865(1199091015840 1199101015840) can beobtained by using
1199091015840= 119909 lowast cos (120579) + 119910 lowast sin (120579)
1199101015840= minus119909 lowast sin (120579) + 119910 lowast cos (120579)
(6)
(8) In a rotated image if the average angle of the probablecluster components is above a certain threshold thenthere is a chance that the other part of the charactercomponent will be present in the target componentIn such a case retain the bigger component fromthe target component and remove the rest of thecomponents
6 Experimental Results
The above described concepts are implemented using MAT-LAB on Intel core i3 processor machine having 4GB RAM
The performance of the proposed LP detection methodis compared with some of the competitive LP detectionmethods by taking into consideration publicly availablemedia-lab benchmark LP database Israeli LP images fromthe web and proprietary Indian LPs having different platevariations and weather conditions in an open environmentTotal images from all these data sets are 900 To furtherassess the performance of the proposed LP detectionmethodwe have considered AOLP benchmark LP database having2049 imageswith three subsets For experimentation we haveconsidered Indian Israeli and media-lab Greek benchmarkLPs as single data set having different characteristics of theLPs in images as described in Table 1
In an open environment there are many possible waysby which we can capture an image from cameras Theproposed geometry-based clustering techniques in this paperare invariant to size tilt pan and rotation That is why theproposed approach works properly with extreme observationviews There is no restriction on the size of LP characters todetect the LP which can be observed from Figures 10(a23)and 10(b23) There are very few methods in the literaturewhich talks about the LP detection of motorcycles where LPcharacters fall in two lines and each line of characters areof different size The proposed method works for any typeof vehicles motorcycles vans and trucks having multiplelines of LP characters and also each line of characters havingdifferent sizes The proposed approach will fail to detect theLPLPs from an input image if the LP characters touch theborder of the LP or there are less than cluster size charactersin the LP or there are no characters present in LP at all Table 1shows the summarization of the various characteristics ofLPs in the images described in Figure 10 For exampleFigure 10(a1) (S number 1 in Table 1) indicates an LP inan input image from Greek in an open environment In allimages we have assumed neither fixed number of charactersin the LP nor the number of lines We assumed that the sizesof the LP characters may be different due to different viewconditions
Figure 10 shows the sample results for all categories ofthe LPs which include images of different plate variationsenvironmental and weather conditions from media-labIsraeli and proprietary Indian LP databases whose summaryis given in Table 1 The odd column of Figure 10 shows theactual image and the even column shows the binarized imageRed border in the binary image of Figure 10 indicates theidentification of the LP and without a red border belongs to afailed case Most of the vehicles in many countries includingmotorcycles will have only a single line of characters in theirLPs but there may be a chance that few countries like Indiawill have LP characters that will fall into more than one lineand there may be a chance that the characters in each line ofLP may vary in size as shown in Figures 10(a8) 10(a18) and10(a19) From such type of the LPs most of the existing LPRsystems detect only the line of characters which are bigger insize Hence such type of LP detection systems will not satisfythe real-time requirementsThe proposed approachwill workin all such conditions
Figures 10(a21) 10(b21) 10(a22) and 10(b22) show imageswith failed LP detection due to overlapping of LP characters
10 Mathematical Problems in Engineering
Table 1 Summarization of the characteristics of LPs in the images described in Figure 10
S number Figure number Country Characteristics of the imageRecognized by theproposed approach
(yesno)1 Figure 10(a1) Greek Image in open environment Yes2 Figure 10(a2) Greek Image with blur Yes3 Figure 10(a3) Greek Image with shadow Yes4 Figure 10(a4) Greek Image with reflectance and illumination Yes5 Figure 10(a5) Greek Image with dirt and screw in the LP Yes6 Figure 10(a6) Greek Image with distorted LP Yes7 Figure 10(a7) Greek Image with LP at a different place Yes8 Figure 10(a8) Greek Image taken at night Yes9 Figure 10(a9) Greek Image with dirt and shadows taken on difficult tracks Yes10 Figure 10(a10) Israel Image with different tilt Yes11 Figure 10(a11) Israel Image with different tilt Yes12 Figure 10(a12) Greek Image with an extreme pan Yes13 Figure 10(a13) Indian Image with multiple LPs (two cars) Yes14 Figure 10(a14) Indian Image with multiple LPs (three motorcycles) Yes15 Figure 10(a15) Indian Image with multiple LPs and different combination of vehicles Yes
16 Figure 10(a16) Indian Image with rotation (an image containing car with leftdiagonally rotated LP) Yes
17 Figure 10(a17) Indian Image with rotation (an image containing car with rightdiagonally rotated LP) Yes
18 Figure 10(a18) Indian Image with rotation (an image containing motorcycle with leftdiagonally rotated LP) Yes
19 Figure 10(a19) Indian Image with rotation (an image containing motorcycle with rightdiagonally rotated LP) Yes
20 Figure 10(a20) Urdu lingual country Image from different country Yes21 Figure 10(a21) Greek Image with lot of dirt in the LP No22 Figure 10(a22) Greek Image with lot of dirt in the LP No23 Figure 10(a23) Israel Image with close view of LP Yes24 Figure 10(a24) Greek Image with few noisy characters in the LP Yes
with LP border because of the presence of lot of dirt betweenLP characters and LP border Figures 10(a24) and 10(b24)show images with LPs 2nd (character ldquoKrdquo) and 6th (characterldquo7rdquo) characters touching the border of the LP due to thepresence of noise such as screw or dirt or stamp betweenLP characters and border of the LP These characters areextracted successfully using the proposed noisymissed char-acters extraction algorithm from Section 5 Figures 10(a1)ndash10(a9) 10(a12) 10(a21) 10(a22) and 10(a24) show Greek LPsFigures 10(a13)ndash10(a19) show Indian LPs Figures 10(a10)10(a11) and 10(a23) show Israeli LPs and Figure 10(a20)shows an LP from Urdu lingual country
From Table 1 and Figure 10 we can conclude that theimages are considered from four different countries andwe have also considered AOLP benchmark database forperformance evaluation which contains images from Taiwancountry With these results we can claim that the proposedapproach will work successfully to identify LPLPs from animage whose individual character properties are near to eachother in line with each other and similar in height in aparticular line There is no restriction on the number of lines
present in the LP of the vehicle and the characters in each lineof the LP can be of different size
The performance comparison amongst few of the promi-nent LP detection methods and the proposed approach isshown in Table 2 The method proposed by Bai and Liuin paper [4] is superior to the proposed approach witha remarkable LP detection success rate of 996 whichsupersedes all other methods The methods proposed byHuang et al in paper [8] Wen et al in paper [9] Yang etal in paper [5] Haneda and Hanaizumi in paper [10] andAnagnostopoulos et al in paper [6] reported 9755 9716953 97 and 965 LP detection rates respectively whichare less than the proposed methodrsquos success rate of 9874Lee et al in paper [3] and Al-Ghaili et al in paper [12]reported 9125 and 9165 success rates but their data setcontains only cars Faradji et al in paper [7] reported lowersuccess rate as compared to others Zhou et al in paper [11]reported 932 success rate on the proprietary data set and848 on Caltech data set Abo Samra and Khalefah in paper[14] reported 9875 success rate which is equivalent to theproposed methods success rate of 9874 The success rates
Mathematical Problems in Engineering 11
(a1) (b1) (a2) (b2)
(a3) (b3) (a4) (b4)
(a5) (b5) (a6) (b6)
(a7) (b7) (a8) (b8)
(a9) (b9) (a10) (b10)
(a11) (b11) (a12) (b12)
(a13) (b13) (a14) (b14)
(a15) (b15) (a16) (b16)
(a17) (b17) (a18) (b18)
(a19) (b19) (a20) (b20)
(a21) (b21) (a22) (b22)
(a23) (b23) (a24) (b24)
Figure 10 Examples of images with all categories of LPs showing LPs detection
12 Mathematical Problems in Engineering
Table 2 Performance comparison of the proposed LP detectionmethodwith few of the competitive LP detectionmethods from the literature
Algorithm and references Data setNumber ofimages indata set
Types of vehiclespresent in data set LP detection rate
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
a new approach for finding and extracting the noisymissedcharacters of the LPLPs due to the presence of noise suchas dirt or screw or stamp between LP characters and LPborder The clustering techniques proposed in this paper usegeometrical properties of the components of LP characterssuch as the distance between the components the anglebetween the components and the height of the componentsto find the probable LPLPsThis is the first time that differentclustering techniques are applied on geometrical propertiesof the components of an input image for finding the probableLPLPsThe proposed geometry-based clusteringmethod forfinding the vehicle LPLPs is scale and rotational invariantand is suitable for many countries LP detection for any typeof vehicles and motorcycles having different plate variations
The performance of the proposed LP detection methodis more prominent when compared with other competitiveLP detection methods from the literature by taking intoconsideration publicly available benchmark LP databases Itis inappropriate to declare which methods are better becauseinmost of the previously publishedmethods the performanceevaluations were carried on proprietary data sets havingrestricted conditions and were not revealed to the public toassess their performance In this paper we are proposing newmethods for LP detection noisymissed character extractionand LP characters rotation correction New findings in thispaper are as follows
(i) Proposing a new method for LP detection usingdistance-based line-based and height-based cluster-ing techniques on geometrical properties of the LPcomponents
(ii) Proposing a new method to remove unwanted clus-tered components using the thinning and resizingtechnique
(iii) Proposing a new method for correcting LP rotationof the probable LP cluster components using theaverage angle amongst successive probable LP clustercomponentsrsquo left-top coordinates and 119909-axis
(iv) Proposing a new method to extract the noisymissedcharacters of the probable LP because of the presenceof noise between LP characters and LP border
The remaining sections of this paper are planned as followsSection 2 exhibits the existing similar research Section 3describes the proposed approach for multiple LPs detectionSection 4 elaborates on the proposed methodology formultiple LPs detection Section 5 describes the extraction ofnoisymissed characters due to the presence of noise betweenLP characters and LP border This section also describes amethod for probable LP characters rotation if the vehicle LPis rotated Experimental results are discussed in Section 6 andConclusion in Section 7
2 Existing Similar Research
Many LPR algorithms have been proposed in the literaturefor the past ten years and even today LP detection remainsthe challenging area due to different environmental condi-tions and plate variations In the literature there is no LP
detection method which will work for many countries forall types of vehicles and motorcycles without any constraintLP detection is challenging and crucial in LPR systemswhich influences the recognition rate Most of the existingLP detection papers from the literature are based on edgeinformation morphological operations template matchingand color information of the LP
Lee et al in paper [3] proposed a color image processing(CIP) method to extract the LP of Korean carrsquos based on LPbackground and LP characters color using the color histo-gram A Neural Network (NN) classifier is used to classifya color This paper used the aspect ratio of the LP regionto select the most probable LP and reported 9125 successrate for LP detection over 80 car images The drawback ofpaper [3] is that the LP detection will not work properlyif the vehicle color matches either background color of theLP or color of the LP characters A hybrid LP localizationscheme is presented by Bai and Liu in paper [4] based onthe edge statistics and morphology (ESM) The proposedapproach had four sections Section 1 handles the verticaledge detection Section 2 takes care of the edge statisticalanalysis Section 3 finds the hierarchical-based LP locationand Section 4 finds the morphology-based LP extractionThis paper reported 996 overall success rate for detectingthe LP out of 9825 images The drawback of paper [4] is thatit uses edge information and morphology-based approachesto detect the LP Some LPs are not so easy to detect usingedge information andmorphology-based approaches have todefine Structuring Element (SE) to perform morphologicaloperations to find the probable LPLPs from an input imageDefining a particular SE to perform morphology-basedoperations to detect the probable LPLPs is a nongenericapproach and will fail to detect the LPLPs from the inputimages under various characteristics of LPLPs in the images
Yang et al in paper [5] proposed a new method based onfixed color collocation (FCC) to locate the LP This methodused the color collocation of the platersquos background andcharacters to recognize the LPs This paper reported 95success rate for LP detection The drawback of paper [5] isalso similar to the drawback of paper [3] Anagnostopoulos etal in paper [6] proposed a new image segmentation methodcalled sliding concentric windows (SCW) for LP detectionThe SCW method works based on local irregularities in theimage The method uses statistics such as standard deviationand mean value for possible LP location SCW uses twoconcentric windows A and B with different sizes to scanthe image from left to right and top to bottom to find themean and standard deviation of the regions of the concentricwindows If the ratio of the statistical measurements exceedsa threshold value set by the user then the central pixel of thetwo concentric windows is considered to be the part of LPThis paper reported a success rate of 965 for LP detectionusing media-lab proprietary LP data set The limitation ofpaper [6] is that the statistical measurement threshold valueset by the user has to be decided according to the applicationafter a trial-and-error procedure which is not a genericsolution
Faradji et al in paper [7] proposed a real-time and robust(RTR) method to find LP location Finding LP location has
Mathematical Problems in Engineering 3
several stages with the combination of Sobelmask histogramanalysis and morphological operations The overall successrate for detecting the LP by this paper was 835The limita-tion of paper [7] is the same as that in paper [4] Huang et alin paper [8] proposed LPR strategy formotorcycles for check-ing annual inspection status The LP data set considered bythis paper contains onlymotorcycles having the LP charactersfalling in only one line The method proposed in this paperfinds the LP using search window with the help of horizontaland vertical projections (SWHVP) and reported an averageLP detection rate of 9755The drawback of paper [8] is thatit is not mentioned how to get the initial size of the searchwindow to perform horizontal and vertical projections Italso used morphology-based dilation operation which isnot a generic solution to detect the LPLPs as explained inpaper [4] drawback Wen et al in paper [9] proposed twomethods to find the LP from an input image (two pass)These two methods are based on Connected ComponentAnalysis (CCA) model Before applying these methods theinput image is binarized using improved Bernsen algorithmto remove shadows and uneven illumination Method 1 isused to find the candidate regions based on prior knowledgeof the LP The frame is detected using CCA methodologyIf the frame is broken the LP cannot be detected correctlyWhen the LP is not detected using Method 1 then Method 2is adopted Method 2 extracts the LP using large numeralextraction technique This paper reported a success rate of9716 The drawback of paper [9] is that the proposedMethod 1 fails to detect the LPLPs from the input image ifthe LP frame is brokenThe drawback of theMethod 2 is thatit will fail to detect the LPLPs from the input image if theLPLPs are not in horizontal position
Haneda and Hanaizumi in paper [10] proposed RELIPalgorithm which performs a global search for the probableLP using multiple templates 3D cross-correlation functionand Principal Component ExpansionThis paper uses cornerdetection to remove deformation of LPs RELIP reported 97LP detection success rate The drawback of paper [10] is thatit uses spatial similarity with an LPLPs template to detect theLP which is not a generic solution in the real world contextas the LPs having various deformations such as tilt rotationand pan from various viewpoints Zhou et al in paper [11]proposed a new approach for LP detection based on PrincipalVisual Word (PVW) discovering and visual word matchingIn visual word matching it will compare the extracted SIFTfeatures of the test image with all discovered PVW and locatethe LP based on matching results This method published932 success rate on the proprietary data set and 848success rate on Caltech dataset The drawback of paper [11]is that it works based on the prior knowledge of the LP
Al-Ghaili et al in paper [12] proposed a new approachin which a color image is converted to grayscale and thenthe adaptive threshold is applied on the grayscale image toconvert it into a binary image ULEA method is appliedto the grayscale image to enhance the quality by removingthe noise Next VEDA is applied to detect the LP from theinput image In order to detect the true LP some statisticaland logical operations are applied The success rate reportedby this paper was 9165 for LP detection The drawback
of paper [12] is that it extracts the LPLPs from the inputimage based on extracting vertical edges which is the sameas the drawback of paper [4] Hsu et al in paper [13] pro-posed a new approach (AOLP) for detecting LP candidatesusing Expectation-Maximization clustering method on ver-tical edges of grayscale images This paper reported 9333success rate on AOLP proprietary benchmark LP data set andreported 921 success rate onmedia-lab benchmark LP dataset for LP detection It ismentioned in the paper that the LPRsolution is designed primarily based on LPs of Taiwan and isnot optimal for other countriesThe drawback of paper [13] isthe same as that in paper [4] because its LP detection is basedon extracting vertical edges
Abo Samra and Khalefah in paper [14] proposed a newLP localization algorithm using dynamic image processingtechniques and genetic algorithms (GA) (DIP-GA) CCAtechnique is used to detect the candidate objects of the inputimage and improved the CCA technique with the help ofmodifiedGAThe system ismade adaptable to any country byintroducing a scale-invariant geometric relationship matrixto model the LP symbols The speed of the LP detection isimproved by introducing two new crossover operators Thesystem reported 9761 success rate using publicly availablemedia-lab benchmark LP database by considering only 335images out of 741 images This paper also reported 9875success rate using proprietary data set having 800 imagesamples and reported 9841 overall accuracyThe drawbackof paper [14] is that it is not able to detect multiple LPs in animage
The LP detection methods which use edge informationand morphological operations mainly focus on finding thecomponents which are rectangular in shape with specificaspect ratio Such type of LP extraction methods will failto identify the LP if the LP does not follow a rectangularshape with proper aspect ratio The problem with templatematching LP detection methods is that it will not work forall types of plate variations The color based LP extractionmethods use the background color of the LP to identify theprobable LP candidate region because some countries use aparticular background color in their LPs Such type of LPRsystems will fail to detect the LP properly if the body color ofthe vehicle matches with the LP background colorThe abovecategorized LP detection methods used the features of LP inan image to extract the LP location from an input imageThese categories of LP detection methods have limitations inextracting LPLPs from an image because of the features theyadopted to extract LPLPs
In this paper we are proposing for the first time anew LP detection method which will work for the LPdetection of many countries having any shape which usesdifferent clustering techniques on geometrical propertiesof the character components of the LPLPs in the inputimageThe advantage of using different clustering techniqueson geometrical properties of the character components ofthe LP is that they are independent of scale rotation tiltand orientation There are very few techniquespublicationsin the literature which talk about LP detection methodsunder various environmental conditions and plate variations
4 Mathematical Problems in Engineering
mentioned which will work for any type of vehicle andmotorcycle having any LP shape
3 Proposed Approach for Multiple LicensePlates Detection
This paper proposes a new method for LPLPs detectionand noisy characters extraction for any type of vehicle andmotorcycle having different plate variations under differentenvironmental and weather conditions The environmentalconditions include different illumination weather and back-ground conditions The plate variations include location ofthe plate anywhere on the vehicle many plates in singleimage different combination of vehicles with different plateorientations different sizes of plates background color ofplates plates with dirt rotated plates LPs having two linesof characters and each line of characters are of different sizeand tilted LPs The proposed method can be articulated asa generalized method for identifying the LPLPs because itis independent of plate variations under different environ-mental andweather conditions and can be applicable tomanycountries and for any vehicle having multiple lines in the LPIn the proposed approach we have not used any type of edgedetection templatematchingmorphological operations andcolor information of the LPs which are extensively used bypreviously published papers to detect the LP
The proposed method uses CCA to label the componentsand applies different clustering techniques on geometricalproperties of the labelled components such as the location ofthe components the angle between the components and theheight of the components to extract the probable LPLPs Inmost of the countries the LP characters are near to each otherare positioned along one or multiple lines and are similar inheight Based on these properties of LP characters clusteringtechniques can be applied on geometrical properties of LPcharacters to identify the LPLPs from an input image Mostof these properties are followed by many nations whiledesigning their LPs That is why the proposed approach canbe applicable to detect the LP of many nations which followthe properties of the LP characters mentioned in this paperwhile designing their LPsThe proposedmethod contains thefollowing steps
(i) Apply preprocessing steps on the input image If it isa video convert the video into different frames andthen apply preprocessing steps After completing thepreprocessing steps label the components of the inputimage to find the number of components 119873
119868and to
extract the geometrical properties of the each of 119873119868
components such as left-top coordinates width andheight
(ii) Apply newly proposed distance-based clusteringalgorithm on each of119873
119868componentsrsquo left-top coordi-
nates This algorithm divides the image componentsinto various distance-based clusters which are closeto each other Let119873
119889be the number of clusters formed
after distance-based clustering from 119873119868components
of the preprocessed input image
(iii) Recluster each 119873119889distance-based cluster into dif-
ferent line-based clusters If the components ofthe distance-based clusters subtend a similar anglebetween the lines joining left-top coordinates of thecomponents with the 119909-axis then construct line-based clusters from such components Line-basedclustering algorithm reclusters 119873
119889distance-based
cluster components into119873119897line-based clusters which
are in line and close to each other(iv) Recluster each 119873
119897line-based cluster based on the
cluster components height This is height-based clus-tering which reclusters119873
119897line-based clusters into119873
ℎ
height-based clusters(v) After the distance line and height-based clustering
techniques the resultant clustered components havethe properties such as near to each other positionedalong a line and being similar in height Theseproperties belong to the characters of LP of anyvehicle and motorcycle in the world
(vi) In the resultant 119873ℎclusters there may be a few
non-LP clusters in which all the components followthe properties of LP characters In order to removesuch non-LP clusters apply the thinning and resizingtechnique on119873
ℎheight-based clusters Let119873tr be the
number of probable LP clusters in the image afterapplying thinning and resizing technique
(vii) Refine 119873tr clusters further by finding the border ofthe cluster components If the border percentage foreach cluster is less than predefined threshold removesuch clusters from the list of probable LP cluster (119873tr)After this step let 119873 be the number of probable LPclusters
(viii) Now apply the newly proposed character extractionmethod to extract noisymissed characters of the LPon each of ldquo119873rdquo probable LPs due to the presenceof noise such as screw or dirt or stamps between LPcharacters and border of the LP
(ix) After extracting the noisymissed characters rotatethe probable LP characters using the average anglebetween the lines joining adjacent componentsrsquo left-top coordinates and the119909-axis so that all the probableLP characters will be horizontal to the 119909-axis
The above mentioned outlined procedures are explained indetail in the following sections
4 Detailed Description of the ProposedApproach for Multiple License PlatesDetection from Videos and Still Images
This section describes in detail the proposed approach to findthe probable LPLPs in an image with the help of variousclustering thinning and resizing techniques This sectionalso explains the method and the need to further refine theprobable LPLPs by finding the border of the LPLPs
Mathematical Problems in Engineering 5
Input image 1
(a)
Complemented binary image
(b)
Binarized image after removingsmall components
(c)
Figure 1 Results of preprocessing stage
41 Preprocessing Stage
Steps(1) Due to the effect of illumination in an open envi-
ronment and the presence of shadows it is verydifficult to process an input image with the help oftraditional threshold binarization methods and willnot give satisfactory results In this paper we are usingBernsen algorithm to overcome the illumination andshadowproblems in an image Let119891(119909 119910) denote grayvalue at a point (119909 119910) of an image Let (119909 119910) be thecentre of a block of size (2119908+1)times(2119908+1) in the imagewhere 119908 is the number of pixels Threshold 119879(119909 119910) atthe point (119909 119910) can be computed using
(2) Convert the input image 1 (shown in Figure 1) intograyscale If it is video convert it into frames and thenconvert each frame into grayscale Apply the Bernsenalgorithm to overcome from uneven illuminationsor shadows present in the grayscale image Comple-ment the binarized image whose output is shown inFigure 1 second column Remove the componentswhose height is less than three pixels from the com-plemented binary image because no LP character isless than three pixels in height The image in Figure 1third column shows the output after removing thecomponents that are less than three pixels in height
(3) To implement the rest of the operations on individualcomponents such distance-based clustering line-based clustering and height-based clustering extractthe geometrical properties of individual componentssuch as left-top coordinates width and height
(4) Geometrical properties of the individual componentsdescribed in Step 3 can be extracted using the follow-ing procedure
(a) To find the number of components present inan image use CCA method to apply labellingto preprocessed image Let119873
119868be the number of
components present in the input image at thisstage
(b) After labelling find left-top coordinates widthand height of119873
119868components
(c) Crop each individual component from thepreprocessed image using left-top coordinateswidth and height
(d) Save the cropped components of the inputimage as separate image components
42 Clustering Stage The purpose of the clustering stage isto prepare the probable LP clusters from 119873
119868components of
the preprocessed input image In this stage the proposedsystem performs three types of clustering techniques oneafter the other The first clustering technique is the distance-based clustering whose purpose is to divide119873
119868components
of the preprocessed input image into groups which arenear to each other The second clustering technique is theline-based clustering whose significance is to divide eachdistance-based cluster into an array of line-based clustersThecomponents of the line-based clusters are in line and closeto each other from the viewpoint of left-top coordinates ofthe componentsThe third clustering technique is the height-based clustering whose significance is to regroup the line-based cluster components which are similar in their heightAfter all these clustering techniques the resultant clustercomponents in each cluster are close to each other positionedin a line and alike in their heights which are the probableLPLPs of the input image
Context dependent variables are cluster size and maxdistance The first context dependent variable cluster sizeindicates the minimum number of characters in a row of theLP and can be tuned to satisfy country specific LP constraintsIn our experiments we have considered cluster size as 4The next context dependent variable max distance is usedduring various clustering stages which explain the maximumdistance that is allowed between the successive componentsof the LP clusters The value of the variable max distance iscomputed as one-third of the columns of the input imageTheproposed geometry-based clustering method has followingsteps
Steps
(1) After preprocessing stage cluster all the compo-nents of the image based on the distance betweenleft-top coordinates of each individual component
6 Mathematical Problems in Engineering
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 2 An example image showing the components after dis-tance-based clustering
using distance-based clustering algorithm which isexplained in detail in the following steps
(2) Distance-based clustering algorithm prepares a ma-trix (distance matrix) of size [119873
119868 119873119868] where119873
119868indi-
cates the number of components of the input imageafter the preprocessing stage In the distance matrixthe 1st row indicates the distance between the 1stlabel component and rest of the componentsrsquo left-topcoordinates 2nd row indicates the distance between2nd label component and rest of the componentsrsquo left-top coordinates and so on Distance-based clusteringalgorithm prepares maximum 119873
119868clusters one for
each component with the help of distance matrix
(3) Remove those distance-based clusters whose size isless than cluster size andwhich are the subset of otherdistance-based clusters Retain those distance-basedclusters with a large number of components in it whenperforming the subset removal operation At thisstage the 119873
119868components of the image are clustered
into groups based on the distance between the left-top coordinates of each individual component
(4) Figure 2 shows an example image with the com-ponents C1 sdot sdot sdotC13 which are clustered into twodistance-based clusters highlighted with rectangularborder (in red color) The components C1 sdot sdot sdotC9formed as first distance-based cluster and C10 sdot sdot sdotC13as second distance-based cluster
(5) Figure 3 shows the resultant image after distance-based clustering Let 119873
119889be the number of distance-
based clusters at this stage which are shown as ellipsesin Figure 3 As most of the components of the inputimage are very small in size it is not possible toobservewith the naked eye the components formed asdistance-based clusters from the input image Hencethe same is explained by taking an example as shownin Figure 2
Figure 3 Resultant image after distance-based clustering
(6) Now apply line-based clustering technique on each of119873119889distance-based clusters to find those components
which are in line with each other(7) In the line-based clustering consider individual
distance-based cluster and take each component fromthe cluster and draw a line from left-top coordinates(1199091 1199101) of one component to the left-top coordinates
(1199092 1199102) of the next component in the current cluster
Find the angle 120579 between 119909-axis and the line thatis drawn between two left-top coordinates of com-ponents as shown in (2) In the same way find theangle between rest of the components and 119909-axisas a matrix (angle matrix) of size [119873119873] where 119873indicates the number of individual components ineach distance-based cluster
120579 = (
180
120587
) lowast tanminus1 ((1199102minus 1199101)
(1199092minus 1199091)
) (2)
(8) Cluster those components as line-based clusterswhich subtend similar angle with the 119909-axis andwhich are close to each other (based on max dis-tance) for each row of the angle matrix Nowremove those line-based clusters which are less thancluster size and which are subsets to other line-basedclusters This is the line-based clustering technique
(9) Line-based clustering is used to recluster the distance-based cluster components based on the propertyof having similar angle and closeness of the clustercomponents After this stage the components ofan input image are clustered using distance-basedclustering and line-based clustering techniques
(10) Figure 4 shows an example image in which the dis-tance-based clusters (shown in the red border) aredivided into line-based clusters (shown in the greenborder) In Figure 4 the first distance-based clusterwith the components C1 sdot sdot sdotC9 is divided into twoline-based clusters with the components C1 sdot sdot sdotC5and C6 sdot sdot sdotC9 indicated with rectangular (green incolor) boxes The second distance-based cluster withthe components C10 sdot sdot sdotC13 resulted into a line-basedcluster with the components C10 C11 and C13 andthe component C12 is removed from the resultant listbecause C12 is not in line with other componentsTheresultant image after line-based clustering is shown in
Mathematical Problems in Engineering 7
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 4 An example image showing how the distance-basedclusters are divided into the line-based clusters
Figure 5 Resultant image after line-based clustering
Figure 5 in which the line-based clusters are markedin rectangular boxes
(11) There is no difference between Figures 3 and 5 basedon the number of components and positions of thecomponents are concerned The resultant distance-based clusters are shown in the ellipse shape withvarious colors in Figure 3 The resultant line-basedclusters are shown in the rectangular shape withvarious colors in Figure 5
(12) After line-based clustering apply height-based clus-tering technique to remove few unwantedjunk com-ponents which are very close to and in line with thecomponents but shows much difference in height ascompared with other components in each line-basedclusterNow remove those clusterswhich are less thancluster size and which are subsets to other clustersThis is height-based clustering
(13) Figure 6 shows an example image in which thecomponent C3 (indicated with the arrow) is showingmuch difference in height as compared to the rest ofthe components it will be removed from the line-based cluster and will result in height-based clusterwith the components C1 C2 C4 and C5 Figure 7shows the image after height-based clustering and theresultant height-based clusters are shown in ellipses
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 6 An example image showing the height-based clustering
Figure 7 Resultant image after height-based clustering
At this stage let 119873ℎbe the number of height-based
clusters(14) After height-based clustering few of the clusters
from the final cluster list may contain all unwantedcomponents which obey all the characteristics basedon distance line and height-based clustering Suchjunk component clusters can be removed by thinningand resizing technique
(15) Apply infinite thinning and resizing technique oneach 119873
ℎheight-based cluster of the image Thinning
is a morphological operation used for skeletonizationof the binary image components When we applythinning and resizing operations junk componentswill retain its shape but the LP characters will fadeaway completely Remove those clusters from thecluster list which retain its shape after thinning andresizing operations This technique removes all junkcluster components from the final cluster list Let 119873
1
be the number of clusters after removing junk clustersusing the thinning and resizing technique Figure 8shows image after thinning and resizing techniqueand the resultant clusters are shown in ellipses shape
43 Finding Border of the License Plate Contrasting toa typical LPR system the proposed system first findsthe probable LP characters and then finds the border of
8 Mathematical Problems in Engineering
Figure 8 Resultant image after thinning and resizing technique
the components The reason for finding the border of LP isthat there can be a group of non-LP characters in the imagewith similar properties of LP characters without the borderIn order to avoid such type of characters the system proposesto find the border of the LP
Predefined Border Percentage Beta Beta is a user definedvariable which decides the border percentage that an LPcan have After rigorous experimentations with the help ofmany datasets from multiple countries we have come to aconclusion to decide the value of Beta as 70
Steps
(1) Let 1198731be the number of probable LP clusters at
this stage Take each individual component fromthe cluster and traverse from left-top coordinatesof each individual component towards the upwarddirection till the traversal reaches the border pointor three times the height of each individual charactercomponent
(2) Apply the same procedure towards the downwarddirection to find the bottom border point for eachindividual component
(3) Save the top and bottom border points for all thecomponents of each cluster
(4) Find the border percentage for each cluster usingthe top and bottom border points Retain a clusteronly when the percentage is greater than or equalto the predefined border percentage ldquoBetardquo Retainedclusters indicate the LPLPs of an image and itscomponents indicate the individual characters of theLP
(5) Remove those clusters which fall below predefinedborder percentage ldquoBetardquo This is another way tofurther refine the cluster list to get the required LPregion Figure 9 shows the image after LP borderidentification
5 Noisy Characters Extraction and LicensePlate Characters Rotation
Noisymissed characters extraction stage is to extract few ofthe LP characters which may be missed during the previous
Figure 9 Resultant image after license plate border identification
stages because of the presence of noise such as screw ordirt or stamps between the LP border and LP charactersThis section proposes a new approach to extract such typeof noisymissed charactersThe proposed noisymissed char-acters extraction algorithm can be applied after Section 43
Steps
(1) The proposed algorithm uses (3) to find the noisy LPcharactersrsquo left-top coordinates (119909
2 1199102) at a distance
119889 from the leftmost cluster componentrsquos left-topcoordinates (119909
1 1199101) and the average slope119898 amongst
the cluster components
1199092= 1199091plusmn (
119889
radic (1 + 1198982)
)
1199102= 1199101plusmn (
119898 lowast 119889
radic (1 + 1198982)
)
(3)
(2) At this stage we have few probable LP clusters Takeeach probable LP cluster and find the average heightand average slope (119898) amongst the subsequent clustercomponentsrsquo left-top coordinates and the 119909-axis
(3) From each probable LP cluster take left-top coordi-nates of the first component (119909
1 1199101) and move in the
downward direction to (14)th of the average heightof probable LP cluster Now traverse the image rightside one pixel at a time using (3) mentioned aboveto find any noisymissed LP character components
(4) If any noisymissed LP character component is foundwhich is not part of the probable LP cluster com-ponent and is not a background pixel then findthe left-top coordinates (119909
2 1199102) and the width of
the noisymissed component To find the left-topcoordinates (119909
2 1199102) and thewidth of the noisymissed
component we have to perform three traversals asdescribed as follows
(i) The first traversal is towards the top side ofthe noisymissed character component to com-pensate the left-top coordinatersquos slope with theaverage slopeThe second traversal is to traversetowards the left side of themissed component inthe direction of the average slope till it reaches
Mathematical Problems in Engineering 9
leftmost pixel of the noisymissed componentwithin the average height using (3) After reach-ing the left most side of the noisymissed com-ponent fix it as left-top coordinates (119909
2 1199102) of
the noisymissed component(ii) Take the newly found left-top coordinates of
the noisymissed LP character component andmove towards the right side (third traversal)in the direction of the average slope till itreaches the right most point of the missedcomponent within the average height using (3)The difference between the 119909-coordinates of thenewly found left-top coordinate point (119909
1 1199101)
and the right most point is the width of thenoisymissed component
(iii) Crop the noisymissed component using left-top coordinates width and the average heightof the cluster components
(5) Repeat the same procedure till the traversal reachesthe last component of the probable LP cluster com-ponent Repeat the same procedure for all probableLP clusters
(6) Rotate each individual component of the probableLP cluster by using average angle 120579 which can becalculated from the average slope (119898) amongst theprobable LP cluster components using (4) Con-sider each component of the binary image as 119865 =119865(119894 119895) 119894 = 1 2 119868 119895 = 1 2 119869 and can bedefined as shown in (5) One has
120579 = (
180
120587
) lowast tanminus1 (119898) (4)
119865 (119894 119895) =
1 white pixel
0 black pixel(5)
(7) Let 119865(119909 119910) be the image component before rota-tion and let 119865(1199091015840 1199101015840) be the image component afterrotation Use average angle 120579 to rotate 119865(119909 119910) Theequation for each individual pixel of 119865(1199091015840 1199101015840) can beobtained by using
1199091015840= 119909 lowast cos (120579) + 119910 lowast sin (120579)
1199101015840= minus119909 lowast sin (120579) + 119910 lowast cos (120579)
(6)
(8) In a rotated image if the average angle of the probablecluster components is above a certain threshold thenthere is a chance that the other part of the charactercomponent will be present in the target componentIn such a case retain the bigger component fromthe target component and remove the rest of thecomponents
6 Experimental Results
The above described concepts are implemented using MAT-LAB on Intel core i3 processor machine having 4GB RAM
The performance of the proposed LP detection methodis compared with some of the competitive LP detectionmethods by taking into consideration publicly availablemedia-lab benchmark LP database Israeli LP images fromthe web and proprietary Indian LPs having different platevariations and weather conditions in an open environmentTotal images from all these data sets are 900 To furtherassess the performance of the proposed LP detectionmethodwe have considered AOLP benchmark LP database having2049 imageswith three subsets For experimentation we haveconsidered Indian Israeli and media-lab Greek benchmarkLPs as single data set having different characteristics of theLPs in images as described in Table 1
In an open environment there are many possible waysby which we can capture an image from cameras Theproposed geometry-based clustering techniques in this paperare invariant to size tilt pan and rotation That is why theproposed approach works properly with extreme observationviews There is no restriction on the size of LP characters todetect the LP which can be observed from Figures 10(a23)and 10(b23) There are very few methods in the literaturewhich talks about the LP detection of motorcycles where LPcharacters fall in two lines and each line of characters areof different size The proposed method works for any typeof vehicles motorcycles vans and trucks having multiplelines of LP characters and also each line of characters havingdifferent sizes The proposed approach will fail to detect theLPLPs from an input image if the LP characters touch theborder of the LP or there are less than cluster size charactersin the LP or there are no characters present in LP at all Table 1shows the summarization of the various characteristics ofLPs in the images described in Figure 10 For exampleFigure 10(a1) (S number 1 in Table 1) indicates an LP inan input image from Greek in an open environment In allimages we have assumed neither fixed number of charactersin the LP nor the number of lines We assumed that the sizesof the LP characters may be different due to different viewconditions
Figure 10 shows the sample results for all categories ofthe LPs which include images of different plate variationsenvironmental and weather conditions from media-labIsraeli and proprietary Indian LP databases whose summaryis given in Table 1 The odd column of Figure 10 shows theactual image and the even column shows the binarized imageRed border in the binary image of Figure 10 indicates theidentification of the LP and without a red border belongs to afailed case Most of the vehicles in many countries includingmotorcycles will have only a single line of characters in theirLPs but there may be a chance that few countries like Indiawill have LP characters that will fall into more than one lineand there may be a chance that the characters in each line ofLP may vary in size as shown in Figures 10(a8) 10(a18) and10(a19) From such type of the LPs most of the existing LPRsystems detect only the line of characters which are bigger insize Hence such type of LP detection systems will not satisfythe real-time requirementsThe proposed approachwill workin all such conditions
Figures 10(a21) 10(b21) 10(a22) and 10(b22) show imageswith failed LP detection due to overlapping of LP characters
10 Mathematical Problems in Engineering
Table 1 Summarization of the characteristics of LPs in the images described in Figure 10
S number Figure number Country Characteristics of the imageRecognized by theproposed approach
(yesno)1 Figure 10(a1) Greek Image in open environment Yes2 Figure 10(a2) Greek Image with blur Yes3 Figure 10(a3) Greek Image with shadow Yes4 Figure 10(a4) Greek Image with reflectance and illumination Yes5 Figure 10(a5) Greek Image with dirt and screw in the LP Yes6 Figure 10(a6) Greek Image with distorted LP Yes7 Figure 10(a7) Greek Image with LP at a different place Yes8 Figure 10(a8) Greek Image taken at night Yes9 Figure 10(a9) Greek Image with dirt and shadows taken on difficult tracks Yes10 Figure 10(a10) Israel Image with different tilt Yes11 Figure 10(a11) Israel Image with different tilt Yes12 Figure 10(a12) Greek Image with an extreme pan Yes13 Figure 10(a13) Indian Image with multiple LPs (two cars) Yes14 Figure 10(a14) Indian Image with multiple LPs (three motorcycles) Yes15 Figure 10(a15) Indian Image with multiple LPs and different combination of vehicles Yes
16 Figure 10(a16) Indian Image with rotation (an image containing car with leftdiagonally rotated LP) Yes
17 Figure 10(a17) Indian Image with rotation (an image containing car with rightdiagonally rotated LP) Yes
18 Figure 10(a18) Indian Image with rotation (an image containing motorcycle with leftdiagonally rotated LP) Yes
19 Figure 10(a19) Indian Image with rotation (an image containing motorcycle with rightdiagonally rotated LP) Yes
20 Figure 10(a20) Urdu lingual country Image from different country Yes21 Figure 10(a21) Greek Image with lot of dirt in the LP No22 Figure 10(a22) Greek Image with lot of dirt in the LP No23 Figure 10(a23) Israel Image with close view of LP Yes24 Figure 10(a24) Greek Image with few noisy characters in the LP Yes
with LP border because of the presence of lot of dirt betweenLP characters and LP border Figures 10(a24) and 10(b24)show images with LPs 2nd (character ldquoKrdquo) and 6th (characterldquo7rdquo) characters touching the border of the LP due to thepresence of noise such as screw or dirt or stamp betweenLP characters and border of the LP These characters areextracted successfully using the proposed noisymissed char-acters extraction algorithm from Section 5 Figures 10(a1)ndash10(a9) 10(a12) 10(a21) 10(a22) and 10(a24) show Greek LPsFigures 10(a13)ndash10(a19) show Indian LPs Figures 10(a10)10(a11) and 10(a23) show Israeli LPs and Figure 10(a20)shows an LP from Urdu lingual country
From Table 1 and Figure 10 we can conclude that theimages are considered from four different countries andwe have also considered AOLP benchmark database forperformance evaluation which contains images from Taiwancountry With these results we can claim that the proposedapproach will work successfully to identify LPLPs from animage whose individual character properties are near to eachother in line with each other and similar in height in aparticular line There is no restriction on the number of lines
present in the LP of the vehicle and the characters in each lineof the LP can be of different size
The performance comparison amongst few of the promi-nent LP detection methods and the proposed approach isshown in Table 2 The method proposed by Bai and Liuin paper [4] is superior to the proposed approach witha remarkable LP detection success rate of 996 whichsupersedes all other methods The methods proposed byHuang et al in paper [8] Wen et al in paper [9] Yang etal in paper [5] Haneda and Hanaizumi in paper [10] andAnagnostopoulos et al in paper [6] reported 9755 9716953 97 and 965 LP detection rates respectively whichare less than the proposed methodrsquos success rate of 9874Lee et al in paper [3] and Al-Ghaili et al in paper [12]reported 9125 and 9165 success rates but their data setcontains only cars Faradji et al in paper [7] reported lowersuccess rate as compared to others Zhou et al in paper [11]reported 932 success rate on the proprietary data set and848 on Caltech data set Abo Samra and Khalefah in paper[14] reported 9875 success rate which is equivalent to theproposed methods success rate of 9874 The success rates
Mathematical Problems in Engineering 11
(a1) (b1) (a2) (b2)
(a3) (b3) (a4) (b4)
(a5) (b5) (a6) (b6)
(a7) (b7) (a8) (b8)
(a9) (b9) (a10) (b10)
(a11) (b11) (a12) (b12)
(a13) (b13) (a14) (b14)
(a15) (b15) (a16) (b16)
(a17) (b17) (a18) (b18)
(a19) (b19) (a20) (b20)
(a21) (b21) (a22) (b22)
(a23) (b23) (a24) (b24)
Figure 10 Examples of images with all categories of LPs showing LPs detection
12 Mathematical Problems in Engineering
Table 2 Performance comparison of the proposed LP detectionmethodwith few of the competitive LP detectionmethods from the literature
Algorithm and references Data setNumber ofimages indata set
Types of vehiclespresent in data set LP detection rate
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
several stages with the combination of Sobelmask histogramanalysis and morphological operations The overall successrate for detecting the LP by this paper was 835The limita-tion of paper [7] is the same as that in paper [4] Huang et alin paper [8] proposed LPR strategy formotorcycles for check-ing annual inspection status The LP data set considered bythis paper contains onlymotorcycles having the LP charactersfalling in only one line The method proposed in this paperfinds the LP using search window with the help of horizontaland vertical projections (SWHVP) and reported an averageLP detection rate of 9755The drawback of paper [8] is thatit is not mentioned how to get the initial size of the searchwindow to perform horizontal and vertical projections Italso used morphology-based dilation operation which isnot a generic solution to detect the LPLPs as explained inpaper [4] drawback Wen et al in paper [9] proposed twomethods to find the LP from an input image (two pass)These two methods are based on Connected ComponentAnalysis (CCA) model Before applying these methods theinput image is binarized using improved Bernsen algorithmto remove shadows and uneven illumination Method 1 isused to find the candidate regions based on prior knowledgeof the LP The frame is detected using CCA methodologyIf the frame is broken the LP cannot be detected correctlyWhen the LP is not detected using Method 1 then Method 2is adopted Method 2 extracts the LP using large numeralextraction technique This paper reported a success rate of9716 The drawback of paper [9] is that the proposedMethod 1 fails to detect the LPLPs from the input image ifthe LP frame is brokenThe drawback of theMethod 2 is thatit will fail to detect the LPLPs from the input image if theLPLPs are not in horizontal position
Haneda and Hanaizumi in paper [10] proposed RELIPalgorithm which performs a global search for the probableLP using multiple templates 3D cross-correlation functionand Principal Component ExpansionThis paper uses cornerdetection to remove deformation of LPs RELIP reported 97LP detection success rate The drawback of paper [10] is thatit uses spatial similarity with an LPLPs template to detect theLP which is not a generic solution in the real world contextas the LPs having various deformations such as tilt rotationand pan from various viewpoints Zhou et al in paper [11]proposed a new approach for LP detection based on PrincipalVisual Word (PVW) discovering and visual word matchingIn visual word matching it will compare the extracted SIFTfeatures of the test image with all discovered PVW and locatethe LP based on matching results This method published932 success rate on the proprietary data set and 848success rate on Caltech dataset The drawback of paper [11]is that it works based on the prior knowledge of the LP
Al-Ghaili et al in paper [12] proposed a new approachin which a color image is converted to grayscale and thenthe adaptive threshold is applied on the grayscale image toconvert it into a binary image ULEA method is appliedto the grayscale image to enhance the quality by removingthe noise Next VEDA is applied to detect the LP from theinput image In order to detect the true LP some statisticaland logical operations are applied The success rate reportedby this paper was 9165 for LP detection The drawback
of paper [12] is that it extracts the LPLPs from the inputimage based on extracting vertical edges which is the sameas the drawback of paper [4] Hsu et al in paper [13] pro-posed a new approach (AOLP) for detecting LP candidatesusing Expectation-Maximization clustering method on ver-tical edges of grayscale images This paper reported 9333success rate on AOLP proprietary benchmark LP data set andreported 921 success rate onmedia-lab benchmark LP dataset for LP detection It ismentioned in the paper that the LPRsolution is designed primarily based on LPs of Taiwan and isnot optimal for other countriesThe drawback of paper [13] isthe same as that in paper [4] because its LP detection is basedon extracting vertical edges
Abo Samra and Khalefah in paper [14] proposed a newLP localization algorithm using dynamic image processingtechniques and genetic algorithms (GA) (DIP-GA) CCAtechnique is used to detect the candidate objects of the inputimage and improved the CCA technique with the help ofmodifiedGAThe system ismade adaptable to any country byintroducing a scale-invariant geometric relationship matrixto model the LP symbols The speed of the LP detection isimproved by introducing two new crossover operators Thesystem reported 9761 success rate using publicly availablemedia-lab benchmark LP database by considering only 335images out of 741 images This paper also reported 9875success rate using proprietary data set having 800 imagesamples and reported 9841 overall accuracyThe drawbackof paper [14] is that it is not able to detect multiple LPs in animage
The LP detection methods which use edge informationand morphological operations mainly focus on finding thecomponents which are rectangular in shape with specificaspect ratio Such type of LP extraction methods will failto identify the LP if the LP does not follow a rectangularshape with proper aspect ratio The problem with templatematching LP detection methods is that it will not work forall types of plate variations The color based LP extractionmethods use the background color of the LP to identify theprobable LP candidate region because some countries use aparticular background color in their LPs Such type of LPRsystems will fail to detect the LP properly if the body color ofthe vehicle matches with the LP background colorThe abovecategorized LP detection methods used the features of LP inan image to extract the LP location from an input imageThese categories of LP detection methods have limitations inextracting LPLPs from an image because of the features theyadopted to extract LPLPs
In this paper we are proposing for the first time anew LP detection method which will work for the LPdetection of many countries having any shape which usesdifferent clustering techniques on geometrical propertiesof the character components of the LPLPs in the inputimageThe advantage of using different clustering techniqueson geometrical properties of the character components ofthe LP is that they are independent of scale rotation tiltand orientation There are very few techniquespublicationsin the literature which talk about LP detection methodsunder various environmental conditions and plate variations
4 Mathematical Problems in Engineering
mentioned which will work for any type of vehicle andmotorcycle having any LP shape
3 Proposed Approach for Multiple LicensePlates Detection
This paper proposes a new method for LPLPs detectionand noisy characters extraction for any type of vehicle andmotorcycle having different plate variations under differentenvironmental and weather conditions The environmentalconditions include different illumination weather and back-ground conditions The plate variations include location ofthe plate anywhere on the vehicle many plates in singleimage different combination of vehicles with different plateorientations different sizes of plates background color ofplates plates with dirt rotated plates LPs having two linesof characters and each line of characters are of different sizeand tilted LPs The proposed method can be articulated asa generalized method for identifying the LPLPs because itis independent of plate variations under different environ-mental andweather conditions and can be applicable tomanycountries and for any vehicle having multiple lines in the LPIn the proposed approach we have not used any type of edgedetection templatematchingmorphological operations andcolor information of the LPs which are extensively used bypreviously published papers to detect the LP
The proposed method uses CCA to label the componentsand applies different clustering techniques on geometricalproperties of the labelled components such as the location ofthe components the angle between the components and theheight of the components to extract the probable LPLPs Inmost of the countries the LP characters are near to each otherare positioned along one or multiple lines and are similar inheight Based on these properties of LP characters clusteringtechniques can be applied on geometrical properties of LPcharacters to identify the LPLPs from an input image Mostof these properties are followed by many nations whiledesigning their LPs That is why the proposed approach canbe applicable to detect the LP of many nations which followthe properties of the LP characters mentioned in this paperwhile designing their LPsThe proposedmethod contains thefollowing steps
(i) Apply preprocessing steps on the input image If it isa video convert the video into different frames andthen apply preprocessing steps After completing thepreprocessing steps label the components of the inputimage to find the number of components 119873
119868and to
extract the geometrical properties of the each of 119873119868
components such as left-top coordinates width andheight
(ii) Apply newly proposed distance-based clusteringalgorithm on each of119873
119868componentsrsquo left-top coordi-
nates This algorithm divides the image componentsinto various distance-based clusters which are closeto each other Let119873
119889be the number of clusters formed
after distance-based clustering from 119873119868components
of the preprocessed input image
(iii) Recluster each 119873119889distance-based cluster into dif-
ferent line-based clusters If the components ofthe distance-based clusters subtend a similar anglebetween the lines joining left-top coordinates of thecomponents with the 119909-axis then construct line-based clusters from such components Line-basedclustering algorithm reclusters 119873
119889distance-based
cluster components into119873119897line-based clusters which
are in line and close to each other(iv) Recluster each 119873
119897line-based cluster based on the
cluster components height This is height-based clus-tering which reclusters119873
119897line-based clusters into119873
ℎ
height-based clusters(v) After the distance line and height-based clustering
techniques the resultant clustered components havethe properties such as near to each other positionedalong a line and being similar in height Theseproperties belong to the characters of LP of anyvehicle and motorcycle in the world
(vi) In the resultant 119873ℎclusters there may be a few
non-LP clusters in which all the components followthe properties of LP characters In order to removesuch non-LP clusters apply the thinning and resizingtechnique on119873
ℎheight-based clusters Let119873tr be the
number of probable LP clusters in the image afterapplying thinning and resizing technique
(vii) Refine 119873tr clusters further by finding the border ofthe cluster components If the border percentage foreach cluster is less than predefined threshold removesuch clusters from the list of probable LP cluster (119873tr)After this step let 119873 be the number of probable LPclusters
(viii) Now apply the newly proposed character extractionmethod to extract noisymissed characters of the LPon each of ldquo119873rdquo probable LPs due to the presenceof noise such as screw or dirt or stamps between LPcharacters and border of the LP
(ix) After extracting the noisymissed characters rotatethe probable LP characters using the average anglebetween the lines joining adjacent componentsrsquo left-top coordinates and the119909-axis so that all the probableLP characters will be horizontal to the 119909-axis
The above mentioned outlined procedures are explained indetail in the following sections
4 Detailed Description of the ProposedApproach for Multiple License PlatesDetection from Videos and Still Images
This section describes in detail the proposed approach to findthe probable LPLPs in an image with the help of variousclustering thinning and resizing techniques This sectionalso explains the method and the need to further refine theprobable LPLPs by finding the border of the LPLPs
Mathematical Problems in Engineering 5
Input image 1
(a)
Complemented binary image
(b)
Binarized image after removingsmall components
(c)
Figure 1 Results of preprocessing stage
41 Preprocessing Stage
Steps(1) Due to the effect of illumination in an open envi-
ronment and the presence of shadows it is verydifficult to process an input image with the help oftraditional threshold binarization methods and willnot give satisfactory results In this paper we are usingBernsen algorithm to overcome the illumination andshadowproblems in an image Let119891(119909 119910) denote grayvalue at a point (119909 119910) of an image Let (119909 119910) be thecentre of a block of size (2119908+1)times(2119908+1) in the imagewhere 119908 is the number of pixels Threshold 119879(119909 119910) atthe point (119909 119910) can be computed using
(2) Convert the input image 1 (shown in Figure 1) intograyscale If it is video convert it into frames and thenconvert each frame into grayscale Apply the Bernsenalgorithm to overcome from uneven illuminationsor shadows present in the grayscale image Comple-ment the binarized image whose output is shown inFigure 1 second column Remove the componentswhose height is less than three pixels from the com-plemented binary image because no LP character isless than three pixels in height The image in Figure 1third column shows the output after removing thecomponents that are less than three pixels in height
(3) To implement the rest of the operations on individualcomponents such distance-based clustering line-based clustering and height-based clustering extractthe geometrical properties of individual componentssuch as left-top coordinates width and height
(4) Geometrical properties of the individual componentsdescribed in Step 3 can be extracted using the follow-ing procedure
(a) To find the number of components present inan image use CCA method to apply labellingto preprocessed image Let119873
119868be the number of
components present in the input image at thisstage
(b) After labelling find left-top coordinates widthand height of119873
119868components
(c) Crop each individual component from thepreprocessed image using left-top coordinateswidth and height
(d) Save the cropped components of the inputimage as separate image components
42 Clustering Stage The purpose of the clustering stage isto prepare the probable LP clusters from 119873
119868components of
the preprocessed input image In this stage the proposedsystem performs three types of clustering techniques oneafter the other The first clustering technique is the distance-based clustering whose purpose is to divide119873
119868components
of the preprocessed input image into groups which arenear to each other The second clustering technique is theline-based clustering whose significance is to divide eachdistance-based cluster into an array of line-based clustersThecomponents of the line-based clusters are in line and closeto each other from the viewpoint of left-top coordinates ofthe componentsThe third clustering technique is the height-based clustering whose significance is to regroup the line-based cluster components which are similar in their heightAfter all these clustering techniques the resultant clustercomponents in each cluster are close to each other positionedin a line and alike in their heights which are the probableLPLPs of the input image
Context dependent variables are cluster size and maxdistance The first context dependent variable cluster sizeindicates the minimum number of characters in a row of theLP and can be tuned to satisfy country specific LP constraintsIn our experiments we have considered cluster size as 4The next context dependent variable max distance is usedduring various clustering stages which explain the maximumdistance that is allowed between the successive componentsof the LP clusters The value of the variable max distance iscomputed as one-third of the columns of the input imageTheproposed geometry-based clustering method has followingsteps
Steps
(1) After preprocessing stage cluster all the compo-nents of the image based on the distance betweenleft-top coordinates of each individual component
6 Mathematical Problems in Engineering
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 2 An example image showing the components after dis-tance-based clustering
using distance-based clustering algorithm which isexplained in detail in the following steps
(2) Distance-based clustering algorithm prepares a ma-trix (distance matrix) of size [119873
119868 119873119868] where119873
119868indi-
cates the number of components of the input imageafter the preprocessing stage In the distance matrixthe 1st row indicates the distance between the 1stlabel component and rest of the componentsrsquo left-topcoordinates 2nd row indicates the distance between2nd label component and rest of the componentsrsquo left-top coordinates and so on Distance-based clusteringalgorithm prepares maximum 119873
119868clusters one for
each component with the help of distance matrix
(3) Remove those distance-based clusters whose size isless than cluster size andwhich are the subset of otherdistance-based clusters Retain those distance-basedclusters with a large number of components in it whenperforming the subset removal operation At thisstage the 119873
119868components of the image are clustered
into groups based on the distance between the left-top coordinates of each individual component
(4) Figure 2 shows an example image with the com-ponents C1 sdot sdot sdotC13 which are clustered into twodistance-based clusters highlighted with rectangularborder (in red color) The components C1 sdot sdot sdotC9formed as first distance-based cluster and C10 sdot sdot sdotC13as second distance-based cluster
(5) Figure 3 shows the resultant image after distance-based clustering Let 119873
119889be the number of distance-
based clusters at this stage which are shown as ellipsesin Figure 3 As most of the components of the inputimage are very small in size it is not possible toobservewith the naked eye the components formed asdistance-based clusters from the input image Hencethe same is explained by taking an example as shownin Figure 2
Figure 3 Resultant image after distance-based clustering
(6) Now apply line-based clustering technique on each of119873119889distance-based clusters to find those components
which are in line with each other(7) In the line-based clustering consider individual
distance-based cluster and take each component fromthe cluster and draw a line from left-top coordinates(1199091 1199101) of one component to the left-top coordinates
(1199092 1199102) of the next component in the current cluster
Find the angle 120579 between 119909-axis and the line thatis drawn between two left-top coordinates of com-ponents as shown in (2) In the same way find theangle between rest of the components and 119909-axisas a matrix (angle matrix) of size [119873119873] where 119873indicates the number of individual components ineach distance-based cluster
120579 = (
180
120587
) lowast tanminus1 ((1199102minus 1199101)
(1199092minus 1199091)
) (2)
(8) Cluster those components as line-based clusterswhich subtend similar angle with the 119909-axis andwhich are close to each other (based on max dis-tance) for each row of the angle matrix Nowremove those line-based clusters which are less thancluster size and which are subsets to other line-basedclusters This is the line-based clustering technique
(9) Line-based clustering is used to recluster the distance-based cluster components based on the propertyof having similar angle and closeness of the clustercomponents After this stage the components ofan input image are clustered using distance-basedclustering and line-based clustering techniques
(10) Figure 4 shows an example image in which the dis-tance-based clusters (shown in the red border) aredivided into line-based clusters (shown in the greenborder) In Figure 4 the first distance-based clusterwith the components C1 sdot sdot sdotC9 is divided into twoline-based clusters with the components C1 sdot sdot sdotC5and C6 sdot sdot sdotC9 indicated with rectangular (green incolor) boxes The second distance-based cluster withthe components C10 sdot sdot sdotC13 resulted into a line-basedcluster with the components C10 C11 and C13 andthe component C12 is removed from the resultant listbecause C12 is not in line with other componentsTheresultant image after line-based clustering is shown in
Mathematical Problems in Engineering 7
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 4 An example image showing how the distance-basedclusters are divided into the line-based clusters
Figure 5 Resultant image after line-based clustering
Figure 5 in which the line-based clusters are markedin rectangular boxes
(11) There is no difference between Figures 3 and 5 basedon the number of components and positions of thecomponents are concerned The resultant distance-based clusters are shown in the ellipse shape withvarious colors in Figure 3 The resultant line-basedclusters are shown in the rectangular shape withvarious colors in Figure 5
(12) After line-based clustering apply height-based clus-tering technique to remove few unwantedjunk com-ponents which are very close to and in line with thecomponents but shows much difference in height ascompared with other components in each line-basedclusterNow remove those clusterswhich are less thancluster size and which are subsets to other clustersThis is height-based clustering
(13) Figure 6 shows an example image in which thecomponent C3 (indicated with the arrow) is showingmuch difference in height as compared to the rest ofthe components it will be removed from the line-based cluster and will result in height-based clusterwith the components C1 C2 C4 and C5 Figure 7shows the image after height-based clustering and theresultant height-based clusters are shown in ellipses
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 6 An example image showing the height-based clustering
Figure 7 Resultant image after height-based clustering
At this stage let 119873ℎbe the number of height-based
clusters(14) After height-based clustering few of the clusters
from the final cluster list may contain all unwantedcomponents which obey all the characteristics basedon distance line and height-based clustering Suchjunk component clusters can be removed by thinningand resizing technique
(15) Apply infinite thinning and resizing technique oneach 119873
ℎheight-based cluster of the image Thinning
is a morphological operation used for skeletonizationof the binary image components When we applythinning and resizing operations junk componentswill retain its shape but the LP characters will fadeaway completely Remove those clusters from thecluster list which retain its shape after thinning andresizing operations This technique removes all junkcluster components from the final cluster list Let 119873
1
be the number of clusters after removing junk clustersusing the thinning and resizing technique Figure 8shows image after thinning and resizing techniqueand the resultant clusters are shown in ellipses shape
43 Finding Border of the License Plate Contrasting toa typical LPR system the proposed system first findsthe probable LP characters and then finds the border of
8 Mathematical Problems in Engineering
Figure 8 Resultant image after thinning and resizing technique
the components The reason for finding the border of LP isthat there can be a group of non-LP characters in the imagewith similar properties of LP characters without the borderIn order to avoid such type of characters the system proposesto find the border of the LP
Predefined Border Percentage Beta Beta is a user definedvariable which decides the border percentage that an LPcan have After rigorous experimentations with the help ofmany datasets from multiple countries we have come to aconclusion to decide the value of Beta as 70
Steps
(1) Let 1198731be the number of probable LP clusters at
this stage Take each individual component fromthe cluster and traverse from left-top coordinatesof each individual component towards the upwarddirection till the traversal reaches the border pointor three times the height of each individual charactercomponent
(2) Apply the same procedure towards the downwarddirection to find the bottom border point for eachindividual component
(3) Save the top and bottom border points for all thecomponents of each cluster
(4) Find the border percentage for each cluster usingthe top and bottom border points Retain a clusteronly when the percentage is greater than or equalto the predefined border percentage ldquoBetardquo Retainedclusters indicate the LPLPs of an image and itscomponents indicate the individual characters of theLP
(5) Remove those clusters which fall below predefinedborder percentage ldquoBetardquo This is another way tofurther refine the cluster list to get the required LPregion Figure 9 shows the image after LP borderidentification
5 Noisy Characters Extraction and LicensePlate Characters Rotation
Noisymissed characters extraction stage is to extract few ofthe LP characters which may be missed during the previous
Figure 9 Resultant image after license plate border identification
stages because of the presence of noise such as screw ordirt or stamps between the LP border and LP charactersThis section proposes a new approach to extract such typeof noisymissed charactersThe proposed noisymissed char-acters extraction algorithm can be applied after Section 43
Steps
(1) The proposed algorithm uses (3) to find the noisy LPcharactersrsquo left-top coordinates (119909
2 1199102) at a distance
119889 from the leftmost cluster componentrsquos left-topcoordinates (119909
1 1199101) and the average slope119898 amongst
the cluster components
1199092= 1199091plusmn (
119889
radic (1 + 1198982)
)
1199102= 1199101plusmn (
119898 lowast 119889
radic (1 + 1198982)
)
(3)
(2) At this stage we have few probable LP clusters Takeeach probable LP cluster and find the average heightand average slope (119898) amongst the subsequent clustercomponentsrsquo left-top coordinates and the 119909-axis
(3) From each probable LP cluster take left-top coordi-nates of the first component (119909
1 1199101) and move in the
downward direction to (14)th of the average heightof probable LP cluster Now traverse the image rightside one pixel at a time using (3) mentioned aboveto find any noisymissed LP character components
(4) If any noisymissed LP character component is foundwhich is not part of the probable LP cluster com-ponent and is not a background pixel then findthe left-top coordinates (119909
2 1199102) and the width of
the noisymissed component To find the left-topcoordinates (119909
2 1199102) and thewidth of the noisymissed
component we have to perform three traversals asdescribed as follows
(i) The first traversal is towards the top side ofthe noisymissed character component to com-pensate the left-top coordinatersquos slope with theaverage slopeThe second traversal is to traversetowards the left side of themissed component inthe direction of the average slope till it reaches
Mathematical Problems in Engineering 9
leftmost pixel of the noisymissed componentwithin the average height using (3) After reach-ing the left most side of the noisymissed com-ponent fix it as left-top coordinates (119909
2 1199102) of
the noisymissed component(ii) Take the newly found left-top coordinates of
the noisymissed LP character component andmove towards the right side (third traversal)in the direction of the average slope till itreaches the right most point of the missedcomponent within the average height using (3)The difference between the 119909-coordinates of thenewly found left-top coordinate point (119909
1 1199101)
and the right most point is the width of thenoisymissed component
(iii) Crop the noisymissed component using left-top coordinates width and the average heightof the cluster components
(5) Repeat the same procedure till the traversal reachesthe last component of the probable LP cluster com-ponent Repeat the same procedure for all probableLP clusters
(6) Rotate each individual component of the probableLP cluster by using average angle 120579 which can becalculated from the average slope (119898) amongst theprobable LP cluster components using (4) Con-sider each component of the binary image as 119865 =119865(119894 119895) 119894 = 1 2 119868 119895 = 1 2 119869 and can bedefined as shown in (5) One has
120579 = (
180
120587
) lowast tanminus1 (119898) (4)
119865 (119894 119895) =
1 white pixel
0 black pixel(5)
(7) Let 119865(119909 119910) be the image component before rota-tion and let 119865(1199091015840 1199101015840) be the image component afterrotation Use average angle 120579 to rotate 119865(119909 119910) Theequation for each individual pixel of 119865(1199091015840 1199101015840) can beobtained by using
1199091015840= 119909 lowast cos (120579) + 119910 lowast sin (120579)
1199101015840= minus119909 lowast sin (120579) + 119910 lowast cos (120579)
(6)
(8) In a rotated image if the average angle of the probablecluster components is above a certain threshold thenthere is a chance that the other part of the charactercomponent will be present in the target componentIn such a case retain the bigger component fromthe target component and remove the rest of thecomponents
6 Experimental Results
The above described concepts are implemented using MAT-LAB on Intel core i3 processor machine having 4GB RAM
The performance of the proposed LP detection methodis compared with some of the competitive LP detectionmethods by taking into consideration publicly availablemedia-lab benchmark LP database Israeli LP images fromthe web and proprietary Indian LPs having different platevariations and weather conditions in an open environmentTotal images from all these data sets are 900 To furtherassess the performance of the proposed LP detectionmethodwe have considered AOLP benchmark LP database having2049 imageswith three subsets For experimentation we haveconsidered Indian Israeli and media-lab Greek benchmarkLPs as single data set having different characteristics of theLPs in images as described in Table 1
In an open environment there are many possible waysby which we can capture an image from cameras Theproposed geometry-based clustering techniques in this paperare invariant to size tilt pan and rotation That is why theproposed approach works properly with extreme observationviews There is no restriction on the size of LP characters todetect the LP which can be observed from Figures 10(a23)and 10(b23) There are very few methods in the literaturewhich talks about the LP detection of motorcycles where LPcharacters fall in two lines and each line of characters areof different size The proposed method works for any typeof vehicles motorcycles vans and trucks having multiplelines of LP characters and also each line of characters havingdifferent sizes The proposed approach will fail to detect theLPLPs from an input image if the LP characters touch theborder of the LP or there are less than cluster size charactersin the LP or there are no characters present in LP at all Table 1shows the summarization of the various characteristics ofLPs in the images described in Figure 10 For exampleFigure 10(a1) (S number 1 in Table 1) indicates an LP inan input image from Greek in an open environment In allimages we have assumed neither fixed number of charactersin the LP nor the number of lines We assumed that the sizesof the LP characters may be different due to different viewconditions
Figure 10 shows the sample results for all categories ofthe LPs which include images of different plate variationsenvironmental and weather conditions from media-labIsraeli and proprietary Indian LP databases whose summaryis given in Table 1 The odd column of Figure 10 shows theactual image and the even column shows the binarized imageRed border in the binary image of Figure 10 indicates theidentification of the LP and without a red border belongs to afailed case Most of the vehicles in many countries includingmotorcycles will have only a single line of characters in theirLPs but there may be a chance that few countries like Indiawill have LP characters that will fall into more than one lineand there may be a chance that the characters in each line ofLP may vary in size as shown in Figures 10(a8) 10(a18) and10(a19) From such type of the LPs most of the existing LPRsystems detect only the line of characters which are bigger insize Hence such type of LP detection systems will not satisfythe real-time requirementsThe proposed approachwill workin all such conditions
Figures 10(a21) 10(b21) 10(a22) and 10(b22) show imageswith failed LP detection due to overlapping of LP characters
10 Mathematical Problems in Engineering
Table 1 Summarization of the characteristics of LPs in the images described in Figure 10
S number Figure number Country Characteristics of the imageRecognized by theproposed approach
(yesno)1 Figure 10(a1) Greek Image in open environment Yes2 Figure 10(a2) Greek Image with blur Yes3 Figure 10(a3) Greek Image with shadow Yes4 Figure 10(a4) Greek Image with reflectance and illumination Yes5 Figure 10(a5) Greek Image with dirt and screw in the LP Yes6 Figure 10(a6) Greek Image with distorted LP Yes7 Figure 10(a7) Greek Image with LP at a different place Yes8 Figure 10(a8) Greek Image taken at night Yes9 Figure 10(a9) Greek Image with dirt and shadows taken on difficult tracks Yes10 Figure 10(a10) Israel Image with different tilt Yes11 Figure 10(a11) Israel Image with different tilt Yes12 Figure 10(a12) Greek Image with an extreme pan Yes13 Figure 10(a13) Indian Image with multiple LPs (two cars) Yes14 Figure 10(a14) Indian Image with multiple LPs (three motorcycles) Yes15 Figure 10(a15) Indian Image with multiple LPs and different combination of vehicles Yes
16 Figure 10(a16) Indian Image with rotation (an image containing car with leftdiagonally rotated LP) Yes
17 Figure 10(a17) Indian Image with rotation (an image containing car with rightdiagonally rotated LP) Yes
18 Figure 10(a18) Indian Image with rotation (an image containing motorcycle with leftdiagonally rotated LP) Yes
19 Figure 10(a19) Indian Image with rotation (an image containing motorcycle with rightdiagonally rotated LP) Yes
20 Figure 10(a20) Urdu lingual country Image from different country Yes21 Figure 10(a21) Greek Image with lot of dirt in the LP No22 Figure 10(a22) Greek Image with lot of dirt in the LP No23 Figure 10(a23) Israel Image with close view of LP Yes24 Figure 10(a24) Greek Image with few noisy characters in the LP Yes
with LP border because of the presence of lot of dirt betweenLP characters and LP border Figures 10(a24) and 10(b24)show images with LPs 2nd (character ldquoKrdquo) and 6th (characterldquo7rdquo) characters touching the border of the LP due to thepresence of noise such as screw or dirt or stamp betweenLP characters and border of the LP These characters areextracted successfully using the proposed noisymissed char-acters extraction algorithm from Section 5 Figures 10(a1)ndash10(a9) 10(a12) 10(a21) 10(a22) and 10(a24) show Greek LPsFigures 10(a13)ndash10(a19) show Indian LPs Figures 10(a10)10(a11) and 10(a23) show Israeli LPs and Figure 10(a20)shows an LP from Urdu lingual country
From Table 1 and Figure 10 we can conclude that theimages are considered from four different countries andwe have also considered AOLP benchmark database forperformance evaluation which contains images from Taiwancountry With these results we can claim that the proposedapproach will work successfully to identify LPLPs from animage whose individual character properties are near to eachother in line with each other and similar in height in aparticular line There is no restriction on the number of lines
present in the LP of the vehicle and the characters in each lineof the LP can be of different size
The performance comparison amongst few of the promi-nent LP detection methods and the proposed approach isshown in Table 2 The method proposed by Bai and Liuin paper [4] is superior to the proposed approach witha remarkable LP detection success rate of 996 whichsupersedes all other methods The methods proposed byHuang et al in paper [8] Wen et al in paper [9] Yang etal in paper [5] Haneda and Hanaizumi in paper [10] andAnagnostopoulos et al in paper [6] reported 9755 9716953 97 and 965 LP detection rates respectively whichare less than the proposed methodrsquos success rate of 9874Lee et al in paper [3] and Al-Ghaili et al in paper [12]reported 9125 and 9165 success rates but their data setcontains only cars Faradji et al in paper [7] reported lowersuccess rate as compared to others Zhou et al in paper [11]reported 932 success rate on the proprietary data set and848 on Caltech data set Abo Samra and Khalefah in paper[14] reported 9875 success rate which is equivalent to theproposed methods success rate of 9874 The success rates
Mathematical Problems in Engineering 11
(a1) (b1) (a2) (b2)
(a3) (b3) (a4) (b4)
(a5) (b5) (a6) (b6)
(a7) (b7) (a8) (b8)
(a9) (b9) (a10) (b10)
(a11) (b11) (a12) (b12)
(a13) (b13) (a14) (b14)
(a15) (b15) (a16) (b16)
(a17) (b17) (a18) (b18)
(a19) (b19) (a20) (b20)
(a21) (b21) (a22) (b22)
(a23) (b23) (a24) (b24)
Figure 10 Examples of images with all categories of LPs showing LPs detection
12 Mathematical Problems in Engineering
Table 2 Performance comparison of the proposed LP detectionmethodwith few of the competitive LP detectionmethods from the literature
Algorithm and references Data setNumber ofimages indata set
Types of vehiclespresent in data set LP detection rate
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
mentioned which will work for any type of vehicle andmotorcycle having any LP shape
3 Proposed Approach for Multiple LicensePlates Detection
This paper proposes a new method for LPLPs detectionand noisy characters extraction for any type of vehicle andmotorcycle having different plate variations under differentenvironmental and weather conditions The environmentalconditions include different illumination weather and back-ground conditions The plate variations include location ofthe plate anywhere on the vehicle many plates in singleimage different combination of vehicles with different plateorientations different sizes of plates background color ofplates plates with dirt rotated plates LPs having two linesof characters and each line of characters are of different sizeand tilted LPs The proposed method can be articulated asa generalized method for identifying the LPLPs because itis independent of plate variations under different environ-mental andweather conditions and can be applicable tomanycountries and for any vehicle having multiple lines in the LPIn the proposed approach we have not used any type of edgedetection templatematchingmorphological operations andcolor information of the LPs which are extensively used bypreviously published papers to detect the LP
The proposed method uses CCA to label the componentsand applies different clustering techniques on geometricalproperties of the labelled components such as the location ofthe components the angle between the components and theheight of the components to extract the probable LPLPs Inmost of the countries the LP characters are near to each otherare positioned along one or multiple lines and are similar inheight Based on these properties of LP characters clusteringtechniques can be applied on geometrical properties of LPcharacters to identify the LPLPs from an input image Mostof these properties are followed by many nations whiledesigning their LPs That is why the proposed approach canbe applicable to detect the LP of many nations which followthe properties of the LP characters mentioned in this paperwhile designing their LPsThe proposedmethod contains thefollowing steps
(i) Apply preprocessing steps on the input image If it isa video convert the video into different frames andthen apply preprocessing steps After completing thepreprocessing steps label the components of the inputimage to find the number of components 119873
119868and to
extract the geometrical properties of the each of 119873119868
components such as left-top coordinates width andheight
(ii) Apply newly proposed distance-based clusteringalgorithm on each of119873
119868componentsrsquo left-top coordi-
nates This algorithm divides the image componentsinto various distance-based clusters which are closeto each other Let119873
119889be the number of clusters formed
after distance-based clustering from 119873119868components
of the preprocessed input image
(iii) Recluster each 119873119889distance-based cluster into dif-
ferent line-based clusters If the components ofthe distance-based clusters subtend a similar anglebetween the lines joining left-top coordinates of thecomponents with the 119909-axis then construct line-based clusters from such components Line-basedclustering algorithm reclusters 119873
119889distance-based
cluster components into119873119897line-based clusters which
are in line and close to each other(iv) Recluster each 119873
119897line-based cluster based on the
cluster components height This is height-based clus-tering which reclusters119873
119897line-based clusters into119873
ℎ
height-based clusters(v) After the distance line and height-based clustering
techniques the resultant clustered components havethe properties such as near to each other positionedalong a line and being similar in height Theseproperties belong to the characters of LP of anyvehicle and motorcycle in the world
(vi) In the resultant 119873ℎclusters there may be a few
non-LP clusters in which all the components followthe properties of LP characters In order to removesuch non-LP clusters apply the thinning and resizingtechnique on119873
ℎheight-based clusters Let119873tr be the
number of probable LP clusters in the image afterapplying thinning and resizing technique
(vii) Refine 119873tr clusters further by finding the border ofthe cluster components If the border percentage foreach cluster is less than predefined threshold removesuch clusters from the list of probable LP cluster (119873tr)After this step let 119873 be the number of probable LPclusters
(viii) Now apply the newly proposed character extractionmethod to extract noisymissed characters of the LPon each of ldquo119873rdquo probable LPs due to the presenceof noise such as screw or dirt or stamps between LPcharacters and border of the LP
(ix) After extracting the noisymissed characters rotatethe probable LP characters using the average anglebetween the lines joining adjacent componentsrsquo left-top coordinates and the119909-axis so that all the probableLP characters will be horizontal to the 119909-axis
The above mentioned outlined procedures are explained indetail in the following sections
4 Detailed Description of the ProposedApproach for Multiple License PlatesDetection from Videos and Still Images
This section describes in detail the proposed approach to findthe probable LPLPs in an image with the help of variousclustering thinning and resizing techniques This sectionalso explains the method and the need to further refine theprobable LPLPs by finding the border of the LPLPs
Mathematical Problems in Engineering 5
Input image 1
(a)
Complemented binary image
(b)
Binarized image after removingsmall components
(c)
Figure 1 Results of preprocessing stage
41 Preprocessing Stage
Steps(1) Due to the effect of illumination in an open envi-
ronment and the presence of shadows it is verydifficult to process an input image with the help oftraditional threshold binarization methods and willnot give satisfactory results In this paper we are usingBernsen algorithm to overcome the illumination andshadowproblems in an image Let119891(119909 119910) denote grayvalue at a point (119909 119910) of an image Let (119909 119910) be thecentre of a block of size (2119908+1)times(2119908+1) in the imagewhere 119908 is the number of pixels Threshold 119879(119909 119910) atthe point (119909 119910) can be computed using
(2) Convert the input image 1 (shown in Figure 1) intograyscale If it is video convert it into frames and thenconvert each frame into grayscale Apply the Bernsenalgorithm to overcome from uneven illuminationsor shadows present in the grayscale image Comple-ment the binarized image whose output is shown inFigure 1 second column Remove the componentswhose height is less than three pixels from the com-plemented binary image because no LP character isless than three pixels in height The image in Figure 1third column shows the output after removing thecomponents that are less than three pixels in height
(3) To implement the rest of the operations on individualcomponents such distance-based clustering line-based clustering and height-based clustering extractthe geometrical properties of individual componentssuch as left-top coordinates width and height
(4) Geometrical properties of the individual componentsdescribed in Step 3 can be extracted using the follow-ing procedure
(a) To find the number of components present inan image use CCA method to apply labellingto preprocessed image Let119873
119868be the number of
components present in the input image at thisstage
(b) After labelling find left-top coordinates widthand height of119873
119868components
(c) Crop each individual component from thepreprocessed image using left-top coordinateswidth and height
(d) Save the cropped components of the inputimage as separate image components
42 Clustering Stage The purpose of the clustering stage isto prepare the probable LP clusters from 119873
119868components of
the preprocessed input image In this stage the proposedsystem performs three types of clustering techniques oneafter the other The first clustering technique is the distance-based clustering whose purpose is to divide119873
119868components
of the preprocessed input image into groups which arenear to each other The second clustering technique is theline-based clustering whose significance is to divide eachdistance-based cluster into an array of line-based clustersThecomponents of the line-based clusters are in line and closeto each other from the viewpoint of left-top coordinates ofthe componentsThe third clustering technique is the height-based clustering whose significance is to regroup the line-based cluster components which are similar in their heightAfter all these clustering techniques the resultant clustercomponents in each cluster are close to each other positionedin a line and alike in their heights which are the probableLPLPs of the input image
Context dependent variables are cluster size and maxdistance The first context dependent variable cluster sizeindicates the minimum number of characters in a row of theLP and can be tuned to satisfy country specific LP constraintsIn our experiments we have considered cluster size as 4The next context dependent variable max distance is usedduring various clustering stages which explain the maximumdistance that is allowed between the successive componentsof the LP clusters The value of the variable max distance iscomputed as one-third of the columns of the input imageTheproposed geometry-based clustering method has followingsteps
Steps
(1) After preprocessing stage cluster all the compo-nents of the image based on the distance betweenleft-top coordinates of each individual component
6 Mathematical Problems in Engineering
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 2 An example image showing the components after dis-tance-based clustering
using distance-based clustering algorithm which isexplained in detail in the following steps
(2) Distance-based clustering algorithm prepares a ma-trix (distance matrix) of size [119873
119868 119873119868] where119873
119868indi-
cates the number of components of the input imageafter the preprocessing stage In the distance matrixthe 1st row indicates the distance between the 1stlabel component and rest of the componentsrsquo left-topcoordinates 2nd row indicates the distance between2nd label component and rest of the componentsrsquo left-top coordinates and so on Distance-based clusteringalgorithm prepares maximum 119873
119868clusters one for
each component with the help of distance matrix
(3) Remove those distance-based clusters whose size isless than cluster size andwhich are the subset of otherdistance-based clusters Retain those distance-basedclusters with a large number of components in it whenperforming the subset removal operation At thisstage the 119873
119868components of the image are clustered
into groups based on the distance between the left-top coordinates of each individual component
(4) Figure 2 shows an example image with the com-ponents C1 sdot sdot sdotC13 which are clustered into twodistance-based clusters highlighted with rectangularborder (in red color) The components C1 sdot sdot sdotC9formed as first distance-based cluster and C10 sdot sdot sdotC13as second distance-based cluster
(5) Figure 3 shows the resultant image after distance-based clustering Let 119873
119889be the number of distance-
based clusters at this stage which are shown as ellipsesin Figure 3 As most of the components of the inputimage are very small in size it is not possible toobservewith the naked eye the components formed asdistance-based clusters from the input image Hencethe same is explained by taking an example as shownin Figure 2
Figure 3 Resultant image after distance-based clustering
(6) Now apply line-based clustering technique on each of119873119889distance-based clusters to find those components
which are in line with each other(7) In the line-based clustering consider individual
distance-based cluster and take each component fromthe cluster and draw a line from left-top coordinates(1199091 1199101) of one component to the left-top coordinates
(1199092 1199102) of the next component in the current cluster
Find the angle 120579 between 119909-axis and the line thatis drawn between two left-top coordinates of com-ponents as shown in (2) In the same way find theangle between rest of the components and 119909-axisas a matrix (angle matrix) of size [119873119873] where 119873indicates the number of individual components ineach distance-based cluster
120579 = (
180
120587
) lowast tanminus1 ((1199102minus 1199101)
(1199092minus 1199091)
) (2)
(8) Cluster those components as line-based clusterswhich subtend similar angle with the 119909-axis andwhich are close to each other (based on max dis-tance) for each row of the angle matrix Nowremove those line-based clusters which are less thancluster size and which are subsets to other line-basedclusters This is the line-based clustering technique
(9) Line-based clustering is used to recluster the distance-based cluster components based on the propertyof having similar angle and closeness of the clustercomponents After this stage the components ofan input image are clustered using distance-basedclustering and line-based clustering techniques
(10) Figure 4 shows an example image in which the dis-tance-based clusters (shown in the red border) aredivided into line-based clusters (shown in the greenborder) In Figure 4 the first distance-based clusterwith the components C1 sdot sdot sdotC9 is divided into twoline-based clusters with the components C1 sdot sdot sdotC5and C6 sdot sdot sdotC9 indicated with rectangular (green incolor) boxes The second distance-based cluster withthe components C10 sdot sdot sdotC13 resulted into a line-basedcluster with the components C10 C11 and C13 andthe component C12 is removed from the resultant listbecause C12 is not in line with other componentsTheresultant image after line-based clustering is shown in
Mathematical Problems in Engineering 7
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 4 An example image showing how the distance-basedclusters are divided into the line-based clusters
Figure 5 Resultant image after line-based clustering
Figure 5 in which the line-based clusters are markedin rectangular boxes
(11) There is no difference between Figures 3 and 5 basedon the number of components and positions of thecomponents are concerned The resultant distance-based clusters are shown in the ellipse shape withvarious colors in Figure 3 The resultant line-basedclusters are shown in the rectangular shape withvarious colors in Figure 5
(12) After line-based clustering apply height-based clus-tering technique to remove few unwantedjunk com-ponents which are very close to and in line with thecomponents but shows much difference in height ascompared with other components in each line-basedclusterNow remove those clusterswhich are less thancluster size and which are subsets to other clustersThis is height-based clustering
(13) Figure 6 shows an example image in which thecomponent C3 (indicated with the arrow) is showingmuch difference in height as compared to the rest ofthe components it will be removed from the line-based cluster and will result in height-based clusterwith the components C1 C2 C4 and C5 Figure 7shows the image after height-based clustering and theresultant height-based clusters are shown in ellipses
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 6 An example image showing the height-based clustering
Figure 7 Resultant image after height-based clustering
At this stage let 119873ℎbe the number of height-based
clusters(14) After height-based clustering few of the clusters
from the final cluster list may contain all unwantedcomponents which obey all the characteristics basedon distance line and height-based clustering Suchjunk component clusters can be removed by thinningand resizing technique
(15) Apply infinite thinning and resizing technique oneach 119873
ℎheight-based cluster of the image Thinning
is a morphological operation used for skeletonizationof the binary image components When we applythinning and resizing operations junk componentswill retain its shape but the LP characters will fadeaway completely Remove those clusters from thecluster list which retain its shape after thinning andresizing operations This technique removes all junkcluster components from the final cluster list Let 119873
1
be the number of clusters after removing junk clustersusing the thinning and resizing technique Figure 8shows image after thinning and resizing techniqueand the resultant clusters are shown in ellipses shape
43 Finding Border of the License Plate Contrasting toa typical LPR system the proposed system first findsthe probable LP characters and then finds the border of
8 Mathematical Problems in Engineering
Figure 8 Resultant image after thinning and resizing technique
the components The reason for finding the border of LP isthat there can be a group of non-LP characters in the imagewith similar properties of LP characters without the borderIn order to avoid such type of characters the system proposesto find the border of the LP
Predefined Border Percentage Beta Beta is a user definedvariable which decides the border percentage that an LPcan have After rigorous experimentations with the help ofmany datasets from multiple countries we have come to aconclusion to decide the value of Beta as 70
Steps
(1) Let 1198731be the number of probable LP clusters at
this stage Take each individual component fromthe cluster and traverse from left-top coordinatesof each individual component towards the upwarddirection till the traversal reaches the border pointor three times the height of each individual charactercomponent
(2) Apply the same procedure towards the downwarddirection to find the bottom border point for eachindividual component
(3) Save the top and bottom border points for all thecomponents of each cluster
(4) Find the border percentage for each cluster usingthe top and bottom border points Retain a clusteronly when the percentage is greater than or equalto the predefined border percentage ldquoBetardquo Retainedclusters indicate the LPLPs of an image and itscomponents indicate the individual characters of theLP
(5) Remove those clusters which fall below predefinedborder percentage ldquoBetardquo This is another way tofurther refine the cluster list to get the required LPregion Figure 9 shows the image after LP borderidentification
5 Noisy Characters Extraction and LicensePlate Characters Rotation
Noisymissed characters extraction stage is to extract few ofthe LP characters which may be missed during the previous
Figure 9 Resultant image after license plate border identification
stages because of the presence of noise such as screw ordirt or stamps between the LP border and LP charactersThis section proposes a new approach to extract such typeof noisymissed charactersThe proposed noisymissed char-acters extraction algorithm can be applied after Section 43
Steps
(1) The proposed algorithm uses (3) to find the noisy LPcharactersrsquo left-top coordinates (119909
2 1199102) at a distance
119889 from the leftmost cluster componentrsquos left-topcoordinates (119909
1 1199101) and the average slope119898 amongst
the cluster components
1199092= 1199091plusmn (
119889
radic (1 + 1198982)
)
1199102= 1199101plusmn (
119898 lowast 119889
radic (1 + 1198982)
)
(3)
(2) At this stage we have few probable LP clusters Takeeach probable LP cluster and find the average heightand average slope (119898) amongst the subsequent clustercomponentsrsquo left-top coordinates and the 119909-axis
(3) From each probable LP cluster take left-top coordi-nates of the first component (119909
1 1199101) and move in the
downward direction to (14)th of the average heightof probable LP cluster Now traverse the image rightside one pixel at a time using (3) mentioned aboveto find any noisymissed LP character components
(4) If any noisymissed LP character component is foundwhich is not part of the probable LP cluster com-ponent and is not a background pixel then findthe left-top coordinates (119909
2 1199102) and the width of
the noisymissed component To find the left-topcoordinates (119909
2 1199102) and thewidth of the noisymissed
component we have to perform three traversals asdescribed as follows
(i) The first traversal is towards the top side ofthe noisymissed character component to com-pensate the left-top coordinatersquos slope with theaverage slopeThe second traversal is to traversetowards the left side of themissed component inthe direction of the average slope till it reaches
Mathematical Problems in Engineering 9
leftmost pixel of the noisymissed componentwithin the average height using (3) After reach-ing the left most side of the noisymissed com-ponent fix it as left-top coordinates (119909
2 1199102) of
the noisymissed component(ii) Take the newly found left-top coordinates of
the noisymissed LP character component andmove towards the right side (third traversal)in the direction of the average slope till itreaches the right most point of the missedcomponent within the average height using (3)The difference between the 119909-coordinates of thenewly found left-top coordinate point (119909
1 1199101)
and the right most point is the width of thenoisymissed component
(iii) Crop the noisymissed component using left-top coordinates width and the average heightof the cluster components
(5) Repeat the same procedure till the traversal reachesthe last component of the probable LP cluster com-ponent Repeat the same procedure for all probableLP clusters
(6) Rotate each individual component of the probableLP cluster by using average angle 120579 which can becalculated from the average slope (119898) amongst theprobable LP cluster components using (4) Con-sider each component of the binary image as 119865 =119865(119894 119895) 119894 = 1 2 119868 119895 = 1 2 119869 and can bedefined as shown in (5) One has
120579 = (
180
120587
) lowast tanminus1 (119898) (4)
119865 (119894 119895) =
1 white pixel
0 black pixel(5)
(7) Let 119865(119909 119910) be the image component before rota-tion and let 119865(1199091015840 1199101015840) be the image component afterrotation Use average angle 120579 to rotate 119865(119909 119910) Theequation for each individual pixel of 119865(1199091015840 1199101015840) can beobtained by using
1199091015840= 119909 lowast cos (120579) + 119910 lowast sin (120579)
1199101015840= minus119909 lowast sin (120579) + 119910 lowast cos (120579)
(6)
(8) In a rotated image if the average angle of the probablecluster components is above a certain threshold thenthere is a chance that the other part of the charactercomponent will be present in the target componentIn such a case retain the bigger component fromthe target component and remove the rest of thecomponents
6 Experimental Results
The above described concepts are implemented using MAT-LAB on Intel core i3 processor machine having 4GB RAM
The performance of the proposed LP detection methodis compared with some of the competitive LP detectionmethods by taking into consideration publicly availablemedia-lab benchmark LP database Israeli LP images fromthe web and proprietary Indian LPs having different platevariations and weather conditions in an open environmentTotal images from all these data sets are 900 To furtherassess the performance of the proposed LP detectionmethodwe have considered AOLP benchmark LP database having2049 imageswith three subsets For experimentation we haveconsidered Indian Israeli and media-lab Greek benchmarkLPs as single data set having different characteristics of theLPs in images as described in Table 1
In an open environment there are many possible waysby which we can capture an image from cameras Theproposed geometry-based clustering techniques in this paperare invariant to size tilt pan and rotation That is why theproposed approach works properly with extreme observationviews There is no restriction on the size of LP characters todetect the LP which can be observed from Figures 10(a23)and 10(b23) There are very few methods in the literaturewhich talks about the LP detection of motorcycles where LPcharacters fall in two lines and each line of characters areof different size The proposed method works for any typeof vehicles motorcycles vans and trucks having multiplelines of LP characters and also each line of characters havingdifferent sizes The proposed approach will fail to detect theLPLPs from an input image if the LP characters touch theborder of the LP or there are less than cluster size charactersin the LP or there are no characters present in LP at all Table 1shows the summarization of the various characteristics ofLPs in the images described in Figure 10 For exampleFigure 10(a1) (S number 1 in Table 1) indicates an LP inan input image from Greek in an open environment In allimages we have assumed neither fixed number of charactersin the LP nor the number of lines We assumed that the sizesof the LP characters may be different due to different viewconditions
Figure 10 shows the sample results for all categories ofthe LPs which include images of different plate variationsenvironmental and weather conditions from media-labIsraeli and proprietary Indian LP databases whose summaryis given in Table 1 The odd column of Figure 10 shows theactual image and the even column shows the binarized imageRed border in the binary image of Figure 10 indicates theidentification of the LP and without a red border belongs to afailed case Most of the vehicles in many countries includingmotorcycles will have only a single line of characters in theirLPs but there may be a chance that few countries like Indiawill have LP characters that will fall into more than one lineand there may be a chance that the characters in each line ofLP may vary in size as shown in Figures 10(a8) 10(a18) and10(a19) From such type of the LPs most of the existing LPRsystems detect only the line of characters which are bigger insize Hence such type of LP detection systems will not satisfythe real-time requirementsThe proposed approachwill workin all such conditions
Figures 10(a21) 10(b21) 10(a22) and 10(b22) show imageswith failed LP detection due to overlapping of LP characters
10 Mathematical Problems in Engineering
Table 1 Summarization of the characteristics of LPs in the images described in Figure 10
S number Figure number Country Characteristics of the imageRecognized by theproposed approach
(yesno)1 Figure 10(a1) Greek Image in open environment Yes2 Figure 10(a2) Greek Image with blur Yes3 Figure 10(a3) Greek Image with shadow Yes4 Figure 10(a4) Greek Image with reflectance and illumination Yes5 Figure 10(a5) Greek Image with dirt and screw in the LP Yes6 Figure 10(a6) Greek Image with distorted LP Yes7 Figure 10(a7) Greek Image with LP at a different place Yes8 Figure 10(a8) Greek Image taken at night Yes9 Figure 10(a9) Greek Image with dirt and shadows taken on difficult tracks Yes10 Figure 10(a10) Israel Image with different tilt Yes11 Figure 10(a11) Israel Image with different tilt Yes12 Figure 10(a12) Greek Image with an extreme pan Yes13 Figure 10(a13) Indian Image with multiple LPs (two cars) Yes14 Figure 10(a14) Indian Image with multiple LPs (three motorcycles) Yes15 Figure 10(a15) Indian Image with multiple LPs and different combination of vehicles Yes
16 Figure 10(a16) Indian Image with rotation (an image containing car with leftdiagonally rotated LP) Yes
17 Figure 10(a17) Indian Image with rotation (an image containing car with rightdiagonally rotated LP) Yes
18 Figure 10(a18) Indian Image with rotation (an image containing motorcycle with leftdiagonally rotated LP) Yes
19 Figure 10(a19) Indian Image with rotation (an image containing motorcycle with rightdiagonally rotated LP) Yes
20 Figure 10(a20) Urdu lingual country Image from different country Yes21 Figure 10(a21) Greek Image with lot of dirt in the LP No22 Figure 10(a22) Greek Image with lot of dirt in the LP No23 Figure 10(a23) Israel Image with close view of LP Yes24 Figure 10(a24) Greek Image with few noisy characters in the LP Yes
with LP border because of the presence of lot of dirt betweenLP characters and LP border Figures 10(a24) and 10(b24)show images with LPs 2nd (character ldquoKrdquo) and 6th (characterldquo7rdquo) characters touching the border of the LP due to thepresence of noise such as screw or dirt or stamp betweenLP characters and border of the LP These characters areextracted successfully using the proposed noisymissed char-acters extraction algorithm from Section 5 Figures 10(a1)ndash10(a9) 10(a12) 10(a21) 10(a22) and 10(a24) show Greek LPsFigures 10(a13)ndash10(a19) show Indian LPs Figures 10(a10)10(a11) and 10(a23) show Israeli LPs and Figure 10(a20)shows an LP from Urdu lingual country
From Table 1 and Figure 10 we can conclude that theimages are considered from four different countries andwe have also considered AOLP benchmark database forperformance evaluation which contains images from Taiwancountry With these results we can claim that the proposedapproach will work successfully to identify LPLPs from animage whose individual character properties are near to eachother in line with each other and similar in height in aparticular line There is no restriction on the number of lines
present in the LP of the vehicle and the characters in each lineof the LP can be of different size
The performance comparison amongst few of the promi-nent LP detection methods and the proposed approach isshown in Table 2 The method proposed by Bai and Liuin paper [4] is superior to the proposed approach witha remarkable LP detection success rate of 996 whichsupersedes all other methods The methods proposed byHuang et al in paper [8] Wen et al in paper [9] Yang etal in paper [5] Haneda and Hanaizumi in paper [10] andAnagnostopoulos et al in paper [6] reported 9755 9716953 97 and 965 LP detection rates respectively whichare less than the proposed methodrsquos success rate of 9874Lee et al in paper [3] and Al-Ghaili et al in paper [12]reported 9125 and 9165 success rates but their data setcontains only cars Faradji et al in paper [7] reported lowersuccess rate as compared to others Zhou et al in paper [11]reported 932 success rate on the proprietary data set and848 on Caltech data set Abo Samra and Khalefah in paper[14] reported 9875 success rate which is equivalent to theproposed methods success rate of 9874 The success rates
Mathematical Problems in Engineering 11
(a1) (b1) (a2) (b2)
(a3) (b3) (a4) (b4)
(a5) (b5) (a6) (b6)
(a7) (b7) (a8) (b8)
(a9) (b9) (a10) (b10)
(a11) (b11) (a12) (b12)
(a13) (b13) (a14) (b14)
(a15) (b15) (a16) (b16)
(a17) (b17) (a18) (b18)
(a19) (b19) (a20) (b20)
(a21) (b21) (a22) (b22)
(a23) (b23) (a24) (b24)
Figure 10 Examples of images with all categories of LPs showing LPs detection
12 Mathematical Problems in Engineering
Table 2 Performance comparison of the proposed LP detectionmethodwith few of the competitive LP detectionmethods from the literature
Algorithm and references Data setNumber ofimages indata set
Types of vehiclespresent in data set LP detection rate
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
Steps(1) Due to the effect of illumination in an open envi-
ronment and the presence of shadows it is verydifficult to process an input image with the help oftraditional threshold binarization methods and willnot give satisfactory results In this paper we are usingBernsen algorithm to overcome the illumination andshadowproblems in an image Let119891(119909 119910) denote grayvalue at a point (119909 119910) of an image Let (119909 119910) be thecentre of a block of size (2119908+1)times(2119908+1) in the imagewhere 119908 is the number of pixels Threshold 119879(119909 119910) atthe point (119909 119910) can be computed using
(2) Convert the input image 1 (shown in Figure 1) intograyscale If it is video convert it into frames and thenconvert each frame into grayscale Apply the Bernsenalgorithm to overcome from uneven illuminationsor shadows present in the grayscale image Comple-ment the binarized image whose output is shown inFigure 1 second column Remove the componentswhose height is less than three pixels from the com-plemented binary image because no LP character isless than three pixels in height The image in Figure 1third column shows the output after removing thecomponents that are less than three pixels in height
(3) To implement the rest of the operations on individualcomponents such distance-based clustering line-based clustering and height-based clustering extractthe geometrical properties of individual componentssuch as left-top coordinates width and height
(4) Geometrical properties of the individual componentsdescribed in Step 3 can be extracted using the follow-ing procedure
(a) To find the number of components present inan image use CCA method to apply labellingto preprocessed image Let119873
119868be the number of
components present in the input image at thisstage
(b) After labelling find left-top coordinates widthand height of119873
119868components
(c) Crop each individual component from thepreprocessed image using left-top coordinateswidth and height
(d) Save the cropped components of the inputimage as separate image components
42 Clustering Stage The purpose of the clustering stage isto prepare the probable LP clusters from 119873
119868components of
the preprocessed input image In this stage the proposedsystem performs three types of clustering techniques oneafter the other The first clustering technique is the distance-based clustering whose purpose is to divide119873
119868components
of the preprocessed input image into groups which arenear to each other The second clustering technique is theline-based clustering whose significance is to divide eachdistance-based cluster into an array of line-based clustersThecomponents of the line-based clusters are in line and closeto each other from the viewpoint of left-top coordinates ofthe componentsThe third clustering technique is the height-based clustering whose significance is to regroup the line-based cluster components which are similar in their heightAfter all these clustering techniques the resultant clustercomponents in each cluster are close to each other positionedin a line and alike in their heights which are the probableLPLPs of the input image
Context dependent variables are cluster size and maxdistance The first context dependent variable cluster sizeindicates the minimum number of characters in a row of theLP and can be tuned to satisfy country specific LP constraintsIn our experiments we have considered cluster size as 4The next context dependent variable max distance is usedduring various clustering stages which explain the maximumdistance that is allowed between the successive componentsof the LP clusters The value of the variable max distance iscomputed as one-third of the columns of the input imageTheproposed geometry-based clustering method has followingsteps
Steps
(1) After preprocessing stage cluster all the compo-nents of the image based on the distance betweenleft-top coordinates of each individual component
6 Mathematical Problems in Engineering
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 2 An example image showing the components after dis-tance-based clustering
using distance-based clustering algorithm which isexplained in detail in the following steps
(2) Distance-based clustering algorithm prepares a ma-trix (distance matrix) of size [119873
119868 119873119868] where119873
119868indi-
cates the number of components of the input imageafter the preprocessing stage In the distance matrixthe 1st row indicates the distance between the 1stlabel component and rest of the componentsrsquo left-topcoordinates 2nd row indicates the distance between2nd label component and rest of the componentsrsquo left-top coordinates and so on Distance-based clusteringalgorithm prepares maximum 119873
119868clusters one for
each component with the help of distance matrix
(3) Remove those distance-based clusters whose size isless than cluster size andwhich are the subset of otherdistance-based clusters Retain those distance-basedclusters with a large number of components in it whenperforming the subset removal operation At thisstage the 119873
119868components of the image are clustered
into groups based on the distance between the left-top coordinates of each individual component
(4) Figure 2 shows an example image with the com-ponents C1 sdot sdot sdotC13 which are clustered into twodistance-based clusters highlighted with rectangularborder (in red color) The components C1 sdot sdot sdotC9formed as first distance-based cluster and C10 sdot sdot sdotC13as second distance-based cluster
(5) Figure 3 shows the resultant image after distance-based clustering Let 119873
119889be the number of distance-
based clusters at this stage which are shown as ellipsesin Figure 3 As most of the components of the inputimage are very small in size it is not possible toobservewith the naked eye the components formed asdistance-based clusters from the input image Hencethe same is explained by taking an example as shownin Figure 2
Figure 3 Resultant image after distance-based clustering
(6) Now apply line-based clustering technique on each of119873119889distance-based clusters to find those components
which are in line with each other(7) In the line-based clustering consider individual
distance-based cluster and take each component fromthe cluster and draw a line from left-top coordinates(1199091 1199101) of one component to the left-top coordinates
(1199092 1199102) of the next component in the current cluster
Find the angle 120579 between 119909-axis and the line thatis drawn between two left-top coordinates of com-ponents as shown in (2) In the same way find theangle between rest of the components and 119909-axisas a matrix (angle matrix) of size [119873119873] where 119873indicates the number of individual components ineach distance-based cluster
120579 = (
180
120587
) lowast tanminus1 ((1199102minus 1199101)
(1199092minus 1199091)
) (2)
(8) Cluster those components as line-based clusterswhich subtend similar angle with the 119909-axis andwhich are close to each other (based on max dis-tance) for each row of the angle matrix Nowremove those line-based clusters which are less thancluster size and which are subsets to other line-basedclusters This is the line-based clustering technique
(9) Line-based clustering is used to recluster the distance-based cluster components based on the propertyof having similar angle and closeness of the clustercomponents After this stage the components ofan input image are clustered using distance-basedclustering and line-based clustering techniques
(10) Figure 4 shows an example image in which the dis-tance-based clusters (shown in the red border) aredivided into line-based clusters (shown in the greenborder) In Figure 4 the first distance-based clusterwith the components C1 sdot sdot sdotC9 is divided into twoline-based clusters with the components C1 sdot sdot sdotC5and C6 sdot sdot sdotC9 indicated with rectangular (green incolor) boxes The second distance-based cluster withthe components C10 sdot sdot sdotC13 resulted into a line-basedcluster with the components C10 C11 and C13 andthe component C12 is removed from the resultant listbecause C12 is not in line with other componentsTheresultant image after line-based clustering is shown in
Mathematical Problems in Engineering 7
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 4 An example image showing how the distance-basedclusters are divided into the line-based clusters
Figure 5 Resultant image after line-based clustering
Figure 5 in which the line-based clusters are markedin rectangular boxes
(11) There is no difference between Figures 3 and 5 basedon the number of components and positions of thecomponents are concerned The resultant distance-based clusters are shown in the ellipse shape withvarious colors in Figure 3 The resultant line-basedclusters are shown in the rectangular shape withvarious colors in Figure 5
(12) After line-based clustering apply height-based clus-tering technique to remove few unwantedjunk com-ponents which are very close to and in line with thecomponents but shows much difference in height ascompared with other components in each line-basedclusterNow remove those clusterswhich are less thancluster size and which are subsets to other clustersThis is height-based clustering
(13) Figure 6 shows an example image in which thecomponent C3 (indicated with the arrow) is showingmuch difference in height as compared to the rest ofthe components it will be removed from the line-based cluster and will result in height-based clusterwith the components C1 C2 C4 and C5 Figure 7shows the image after height-based clustering and theresultant height-based clusters are shown in ellipses
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 6 An example image showing the height-based clustering
Figure 7 Resultant image after height-based clustering
At this stage let 119873ℎbe the number of height-based
clusters(14) After height-based clustering few of the clusters
from the final cluster list may contain all unwantedcomponents which obey all the characteristics basedon distance line and height-based clustering Suchjunk component clusters can be removed by thinningand resizing technique
(15) Apply infinite thinning and resizing technique oneach 119873
ℎheight-based cluster of the image Thinning
is a morphological operation used for skeletonizationof the binary image components When we applythinning and resizing operations junk componentswill retain its shape but the LP characters will fadeaway completely Remove those clusters from thecluster list which retain its shape after thinning andresizing operations This technique removes all junkcluster components from the final cluster list Let 119873
1
be the number of clusters after removing junk clustersusing the thinning and resizing technique Figure 8shows image after thinning and resizing techniqueand the resultant clusters are shown in ellipses shape
43 Finding Border of the License Plate Contrasting toa typical LPR system the proposed system first findsthe probable LP characters and then finds the border of
8 Mathematical Problems in Engineering
Figure 8 Resultant image after thinning and resizing technique
the components The reason for finding the border of LP isthat there can be a group of non-LP characters in the imagewith similar properties of LP characters without the borderIn order to avoid such type of characters the system proposesto find the border of the LP
Predefined Border Percentage Beta Beta is a user definedvariable which decides the border percentage that an LPcan have After rigorous experimentations with the help ofmany datasets from multiple countries we have come to aconclusion to decide the value of Beta as 70
Steps
(1) Let 1198731be the number of probable LP clusters at
this stage Take each individual component fromthe cluster and traverse from left-top coordinatesof each individual component towards the upwarddirection till the traversal reaches the border pointor three times the height of each individual charactercomponent
(2) Apply the same procedure towards the downwarddirection to find the bottom border point for eachindividual component
(3) Save the top and bottom border points for all thecomponents of each cluster
(4) Find the border percentage for each cluster usingthe top and bottom border points Retain a clusteronly when the percentage is greater than or equalto the predefined border percentage ldquoBetardquo Retainedclusters indicate the LPLPs of an image and itscomponents indicate the individual characters of theLP
(5) Remove those clusters which fall below predefinedborder percentage ldquoBetardquo This is another way tofurther refine the cluster list to get the required LPregion Figure 9 shows the image after LP borderidentification
5 Noisy Characters Extraction and LicensePlate Characters Rotation
Noisymissed characters extraction stage is to extract few ofthe LP characters which may be missed during the previous
Figure 9 Resultant image after license plate border identification
stages because of the presence of noise such as screw ordirt or stamps between the LP border and LP charactersThis section proposes a new approach to extract such typeof noisymissed charactersThe proposed noisymissed char-acters extraction algorithm can be applied after Section 43
Steps
(1) The proposed algorithm uses (3) to find the noisy LPcharactersrsquo left-top coordinates (119909
2 1199102) at a distance
119889 from the leftmost cluster componentrsquos left-topcoordinates (119909
1 1199101) and the average slope119898 amongst
the cluster components
1199092= 1199091plusmn (
119889
radic (1 + 1198982)
)
1199102= 1199101plusmn (
119898 lowast 119889
radic (1 + 1198982)
)
(3)
(2) At this stage we have few probable LP clusters Takeeach probable LP cluster and find the average heightand average slope (119898) amongst the subsequent clustercomponentsrsquo left-top coordinates and the 119909-axis
(3) From each probable LP cluster take left-top coordi-nates of the first component (119909
1 1199101) and move in the
downward direction to (14)th of the average heightof probable LP cluster Now traverse the image rightside one pixel at a time using (3) mentioned aboveto find any noisymissed LP character components
(4) If any noisymissed LP character component is foundwhich is not part of the probable LP cluster com-ponent and is not a background pixel then findthe left-top coordinates (119909
2 1199102) and the width of
the noisymissed component To find the left-topcoordinates (119909
2 1199102) and thewidth of the noisymissed
component we have to perform three traversals asdescribed as follows
(i) The first traversal is towards the top side ofthe noisymissed character component to com-pensate the left-top coordinatersquos slope with theaverage slopeThe second traversal is to traversetowards the left side of themissed component inthe direction of the average slope till it reaches
Mathematical Problems in Engineering 9
leftmost pixel of the noisymissed componentwithin the average height using (3) After reach-ing the left most side of the noisymissed com-ponent fix it as left-top coordinates (119909
2 1199102) of
the noisymissed component(ii) Take the newly found left-top coordinates of
the noisymissed LP character component andmove towards the right side (third traversal)in the direction of the average slope till itreaches the right most point of the missedcomponent within the average height using (3)The difference between the 119909-coordinates of thenewly found left-top coordinate point (119909
1 1199101)
and the right most point is the width of thenoisymissed component
(iii) Crop the noisymissed component using left-top coordinates width and the average heightof the cluster components
(5) Repeat the same procedure till the traversal reachesthe last component of the probable LP cluster com-ponent Repeat the same procedure for all probableLP clusters
(6) Rotate each individual component of the probableLP cluster by using average angle 120579 which can becalculated from the average slope (119898) amongst theprobable LP cluster components using (4) Con-sider each component of the binary image as 119865 =119865(119894 119895) 119894 = 1 2 119868 119895 = 1 2 119869 and can bedefined as shown in (5) One has
120579 = (
180
120587
) lowast tanminus1 (119898) (4)
119865 (119894 119895) =
1 white pixel
0 black pixel(5)
(7) Let 119865(119909 119910) be the image component before rota-tion and let 119865(1199091015840 1199101015840) be the image component afterrotation Use average angle 120579 to rotate 119865(119909 119910) Theequation for each individual pixel of 119865(1199091015840 1199101015840) can beobtained by using
1199091015840= 119909 lowast cos (120579) + 119910 lowast sin (120579)
1199101015840= minus119909 lowast sin (120579) + 119910 lowast cos (120579)
(6)
(8) In a rotated image if the average angle of the probablecluster components is above a certain threshold thenthere is a chance that the other part of the charactercomponent will be present in the target componentIn such a case retain the bigger component fromthe target component and remove the rest of thecomponents
6 Experimental Results
The above described concepts are implemented using MAT-LAB on Intel core i3 processor machine having 4GB RAM
The performance of the proposed LP detection methodis compared with some of the competitive LP detectionmethods by taking into consideration publicly availablemedia-lab benchmark LP database Israeli LP images fromthe web and proprietary Indian LPs having different platevariations and weather conditions in an open environmentTotal images from all these data sets are 900 To furtherassess the performance of the proposed LP detectionmethodwe have considered AOLP benchmark LP database having2049 imageswith three subsets For experimentation we haveconsidered Indian Israeli and media-lab Greek benchmarkLPs as single data set having different characteristics of theLPs in images as described in Table 1
In an open environment there are many possible waysby which we can capture an image from cameras Theproposed geometry-based clustering techniques in this paperare invariant to size tilt pan and rotation That is why theproposed approach works properly with extreme observationviews There is no restriction on the size of LP characters todetect the LP which can be observed from Figures 10(a23)and 10(b23) There are very few methods in the literaturewhich talks about the LP detection of motorcycles where LPcharacters fall in two lines and each line of characters areof different size The proposed method works for any typeof vehicles motorcycles vans and trucks having multiplelines of LP characters and also each line of characters havingdifferent sizes The proposed approach will fail to detect theLPLPs from an input image if the LP characters touch theborder of the LP or there are less than cluster size charactersin the LP or there are no characters present in LP at all Table 1shows the summarization of the various characteristics ofLPs in the images described in Figure 10 For exampleFigure 10(a1) (S number 1 in Table 1) indicates an LP inan input image from Greek in an open environment In allimages we have assumed neither fixed number of charactersin the LP nor the number of lines We assumed that the sizesof the LP characters may be different due to different viewconditions
Figure 10 shows the sample results for all categories ofthe LPs which include images of different plate variationsenvironmental and weather conditions from media-labIsraeli and proprietary Indian LP databases whose summaryis given in Table 1 The odd column of Figure 10 shows theactual image and the even column shows the binarized imageRed border in the binary image of Figure 10 indicates theidentification of the LP and without a red border belongs to afailed case Most of the vehicles in many countries includingmotorcycles will have only a single line of characters in theirLPs but there may be a chance that few countries like Indiawill have LP characters that will fall into more than one lineand there may be a chance that the characters in each line ofLP may vary in size as shown in Figures 10(a8) 10(a18) and10(a19) From such type of the LPs most of the existing LPRsystems detect only the line of characters which are bigger insize Hence such type of LP detection systems will not satisfythe real-time requirementsThe proposed approachwill workin all such conditions
Figures 10(a21) 10(b21) 10(a22) and 10(b22) show imageswith failed LP detection due to overlapping of LP characters
10 Mathematical Problems in Engineering
Table 1 Summarization of the characteristics of LPs in the images described in Figure 10
S number Figure number Country Characteristics of the imageRecognized by theproposed approach
(yesno)1 Figure 10(a1) Greek Image in open environment Yes2 Figure 10(a2) Greek Image with blur Yes3 Figure 10(a3) Greek Image with shadow Yes4 Figure 10(a4) Greek Image with reflectance and illumination Yes5 Figure 10(a5) Greek Image with dirt and screw in the LP Yes6 Figure 10(a6) Greek Image with distorted LP Yes7 Figure 10(a7) Greek Image with LP at a different place Yes8 Figure 10(a8) Greek Image taken at night Yes9 Figure 10(a9) Greek Image with dirt and shadows taken on difficult tracks Yes10 Figure 10(a10) Israel Image with different tilt Yes11 Figure 10(a11) Israel Image with different tilt Yes12 Figure 10(a12) Greek Image with an extreme pan Yes13 Figure 10(a13) Indian Image with multiple LPs (two cars) Yes14 Figure 10(a14) Indian Image with multiple LPs (three motorcycles) Yes15 Figure 10(a15) Indian Image with multiple LPs and different combination of vehicles Yes
16 Figure 10(a16) Indian Image with rotation (an image containing car with leftdiagonally rotated LP) Yes
17 Figure 10(a17) Indian Image with rotation (an image containing car with rightdiagonally rotated LP) Yes
18 Figure 10(a18) Indian Image with rotation (an image containing motorcycle with leftdiagonally rotated LP) Yes
19 Figure 10(a19) Indian Image with rotation (an image containing motorcycle with rightdiagonally rotated LP) Yes
20 Figure 10(a20) Urdu lingual country Image from different country Yes21 Figure 10(a21) Greek Image with lot of dirt in the LP No22 Figure 10(a22) Greek Image with lot of dirt in the LP No23 Figure 10(a23) Israel Image with close view of LP Yes24 Figure 10(a24) Greek Image with few noisy characters in the LP Yes
with LP border because of the presence of lot of dirt betweenLP characters and LP border Figures 10(a24) and 10(b24)show images with LPs 2nd (character ldquoKrdquo) and 6th (characterldquo7rdquo) characters touching the border of the LP due to thepresence of noise such as screw or dirt or stamp betweenLP characters and border of the LP These characters areextracted successfully using the proposed noisymissed char-acters extraction algorithm from Section 5 Figures 10(a1)ndash10(a9) 10(a12) 10(a21) 10(a22) and 10(a24) show Greek LPsFigures 10(a13)ndash10(a19) show Indian LPs Figures 10(a10)10(a11) and 10(a23) show Israeli LPs and Figure 10(a20)shows an LP from Urdu lingual country
From Table 1 and Figure 10 we can conclude that theimages are considered from four different countries andwe have also considered AOLP benchmark database forperformance evaluation which contains images from Taiwancountry With these results we can claim that the proposedapproach will work successfully to identify LPLPs from animage whose individual character properties are near to eachother in line with each other and similar in height in aparticular line There is no restriction on the number of lines
present in the LP of the vehicle and the characters in each lineof the LP can be of different size
The performance comparison amongst few of the promi-nent LP detection methods and the proposed approach isshown in Table 2 The method proposed by Bai and Liuin paper [4] is superior to the proposed approach witha remarkable LP detection success rate of 996 whichsupersedes all other methods The methods proposed byHuang et al in paper [8] Wen et al in paper [9] Yang etal in paper [5] Haneda and Hanaizumi in paper [10] andAnagnostopoulos et al in paper [6] reported 9755 9716953 97 and 965 LP detection rates respectively whichare less than the proposed methodrsquos success rate of 9874Lee et al in paper [3] and Al-Ghaili et al in paper [12]reported 9125 and 9165 success rates but their data setcontains only cars Faradji et al in paper [7] reported lowersuccess rate as compared to others Zhou et al in paper [11]reported 932 success rate on the proprietary data set and848 on Caltech data set Abo Samra and Khalefah in paper[14] reported 9875 success rate which is equivalent to theproposed methods success rate of 9874 The success rates
Mathematical Problems in Engineering 11
(a1) (b1) (a2) (b2)
(a3) (b3) (a4) (b4)
(a5) (b5) (a6) (b6)
(a7) (b7) (a8) (b8)
(a9) (b9) (a10) (b10)
(a11) (b11) (a12) (b12)
(a13) (b13) (a14) (b14)
(a15) (b15) (a16) (b16)
(a17) (b17) (a18) (b18)
(a19) (b19) (a20) (b20)
(a21) (b21) (a22) (b22)
(a23) (b23) (a24) (b24)
Figure 10 Examples of images with all categories of LPs showing LPs detection
12 Mathematical Problems in Engineering
Table 2 Performance comparison of the proposed LP detectionmethodwith few of the competitive LP detectionmethods from the literature
Algorithm and references Data setNumber ofimages indata set
Types of vehiclespresent in data set LP detection rate
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
Figure 2 An example image showing the components after dis-tance-based clustering
using distance-based clustering algorithm which isexplained in detail in the following steps
(2) Distance-based clustering algorithm prepares a ma-trix (distance matrix) of size [119873
119868 119873119868] where119873
119868indi-
cates the number of components of the input imageafter the preprocessing stage In the distance matrixthe 1st row indicates the distance between the 1stlabel component and rest of the componentsrsquo left-topcoordinates 2nd row indicates the distance between2nd label component and rest of the componentsrsquo left-top coordinates and so on Distance-based clusteringalgorithm prepares maximum 119873
119868clusters one for
each component with the help of distance matrix
(3) Remove those distance-based clusters whose size isless than cluster size andwhich are the subset of otherdistance-based clusters Retain those distance-basedclusters with a large number of components in it whenperforming the subset removal operation At thisstage the 119873
119868components of the image are clustered
into groups based on the distance between the left-top coordinates of each individual component
(4) Figure 2 shows an example image with the com-ponents C1 sdot sdot sdotC13 which are clustered into twodistance-based clusters highlighted with rectangularborder (in red color) The components C1 sdot sdot sdotC9formed as first distance-based cluster and C10 sdot sdot sdotC13as second distance-based cluster
(5) Figure 3 shows the resultant image after distance-based clustering Let 119873
119889be the number of distance-
based clusters at this stage which are shown as ellipsesin Figure 3 As most of the components of the inputimage are very small in size it is not possible toobservewith the naked eye the components formed asdistance-based clusters from the input image Hencethe same is explained by taking an example as shownin Figure 2
Figure 3 Resultant image after distance-based clustering
(6) Now apply line-based clustering technique on each of119873119889distance-based clusters to find those components
which are in line with each other(7) In the line-based clustering consider individual
distance-based cluster and take each component fromthe cluster and draw a line from left-top coordinates(1199091 1199101) of one component to the left-top coordinates
(1199092 1199102) of the next component in the current cluster
Find the angle 120579 between 119909-axis and the line thatis drawn between two left-top coordinates of com-ponents as shown in (2) In the same way find theangle between rest of the components and 119909-axisas a matrix (angle matrix) of size [119873119873] where 119873indicates the number of individual components ineach distance-based cluster
120579 = (
180
120587
) lowast tanminus1 ((1199102minus 1199101)
(1199092minus 1199091)
) (2)
(8) Cluster those components as line-based clusterswhich subtend similar angle with the 119909-axis andwhich are close to each other (based on max dis-tance) for each row of the angle matrix Nowremove those line-based clusters which are less thancluster size and which are subsets to other line-basedclusters This is the line-based clustering technique
(9) Line-based clustering is used to recluster the distance-based cluster components based on the propertyof having similar angle and closeness of the clustercomponents After this stage the components ofan input image are clustered using distance-basedclustering and line-based clustering techniques
(10) Figure 4 shows an example image in which the dis-tance-based clusters (shown in the red border) aredivided into line-based clusters (shown in the greenborder) In Figure 4 the first distance-based clusterwith the components C1 sdot sdot sdotC9 is divided into twoline-based clusters with the components C1 sdot sdot sdotC5and C6 sdot sdot sdotC9 indicated with rectangular (green incolor) boxes The second distance-based cluster withthe components C10 sdot sdot sdotC13 resulted into a line-basedcluster with the components C10 C11 and C13 andthe component C12 is removed from the resultant listbecause C12 is not in line with other componentsTheresultant image after line-based clustering is shown in
Mathematical Problems in Engineering 7
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 4 An example image showing how the distance-basedclusters are divided into the line-based clusters
Figure 5 Resultant image after line-based clustering
Figure 5 in which the line-based clusters are markedin rectangular boxes
(11) There is no difference between Figures 3 and 5 basedon the number of components and positions of thecomponents are concerned The resultant distance-based clusters are shown in the ellipse shape withvarious colors in Figure 3 The resultant line-basedclusters are shown in the rectangular shape withvarious colors in Figure 5
(12) After line-based clustering apply height-based clus-tering technique to remove few unwantedjunk com-ponents which are very close to and in line with thecomponents but shows much difference in height ascompared with other components in each line-basedclusterNow remove those clusterswhich are less thancluster size and which are subsets to other clustersThis is height-based clustering
(13) Figure 6 shows an example image in which thecomponent C3 (indicated with the arrow) is showingmuch difference in height as compared to the rest ofthe components it will be removed from the line-based cluster and will result in height-based clusterwith the components C1 C2 C4 and C5 Figure 7shows the image after height-based clustering and theresultant height-based clusters are shown in ellipses
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 6 An example image showing the height-based clustering
Figure 7 Resultant image after height-based clustering
At this stage let 119873ℎbe the number of height-based
clusters(14) After height-based clustering few of the clusters
from the final cluster list may contain all unwantedcomponents which obey all the characteristics basedon distance line and height-based clustering Suchjunk component clusters can be removed by thinningand resizing technique
(15) Apply infinite thinning and resizing technique oneach 119873
ℎheight-based cluster of the image Thinning
is a morphological operation used for skeletonizationof the binary image components When we applythinning and resizing operations junk componentswill retain its shape but the LP characters will fadeaway completely Remove those clusters from thecluster list which retain its shape after thinning andresizing operations This technique removes all junkcluster components from the final cluster list Let 119873
1
be the number of clusters after removing junk clustersusing the thinning and resizing technique Figure 8shows image after thinning and resizing techniqueand the resultant clusters are shown in ellipses shape
43 Finding Border of the License Plate Contrasting toa typical LPR system the proposed system first findsthe probable LP characters and then finds the border of
8 Mathematical Problems in Engineering
Figure 8 Resultant image after thinning and resizing technique
the components The reason for finding the border of LP isthat there can be a group of non-LP characters in the imagewith similar properties of LP characters without the borderIn order to avoid such type of characters the system proposesto find the border of the LP
Predefined Border Percentage Beta Beta is a user definedvariable which decides the border percentage that an LPcan have After rigorous experimentations with the help ofmany datasets from multiple countries we have come to aconclusion to decide the value of Beta as 70
Steps
(1) Let 1198731be the number of probable LP clusters at
this stage Take each individual component fromthe cluster and traverse from left-top coordinatesof each individual component towards the upwarddirection till the traversal reaches the border pointor three times the height of each individual charactercomponent
(2) Apply the same procedure towards the downwarddirection to find the bottom border point for eachindividual component
(3) Save the top and bottom border points for all thecomponents of each cluster
(4) Find the border percentage for each cluster usingthe top and bottom border points Retain a clusteronly when the percentage is greater than or equalto the predefined border percentage ldquoBetardquo Retainedclusters indicate the LPLPs of an image and itscomponents indicate the individual characters of theLP
(5) Remove those clusters which fall below predefinedborder percentage ldquoBetardquo This is another way tofurther refine the cluster list to get the required LPregion Figure 9 shows the image after LP borderidentification
5 Noisy Characters Extraction and LicensePlate Characters Rotation
Noisymissed characters extraction stage is to extract few ofthe LP characters which may be missed during the previous
Figure 9 Resultant image after license plate border identification
stages because of the presence of noise such as screw ordirt or stamps between the LP border and LP charactersThis section proposes a new approach to extract such typeof noisymissed charactersThe proposed noisymissed char-acters extraction algorithm can be applied after Section 43
Steps
(1) The proposed algorithm uses (3) to find the noisy LPcharactersrsquo left-top coordinates (119909
2 1199102) at a distance
119889 from the leftmost cluster componentrsquos left-topcoordinates (119909
1 1199101) and the average slope119898 amongst
the cluster components
1199092= 1199091plusmn (
119889
radic (1 + 1198982)
)
1199102= 1199101plusmn (
119898 lowast 119889
radic (1 + 1198982)
)
(3)
(2) At this stage we have few probable LP clusters Takeeach probable LP cluster and find the average heightand average slope (119898) amongst the subsequent clustercomponentsrsquo left-top coordinates and the 119909-axis
(3) From each probable LP cluster take left-top coordi-nates of the first component (119909
1 1199101) and move in the
downward direction to (14)th of the average heightof probable LP cluster Now traverse the image rightside one pixel at a time using (3) mentioned aboveto find any noisymissed LP character components
(4) If any noisymissed LP character component is foundwhich is not part of the probable LP cluster com-ponent and is not a background pixel then findthe left-top coordinates (119909
2 1199102) and the width of
the noisymissed component To find the left-topcoordinates (119909
2 1199102) and thewidth of the noisymissed
component we have to perform three traversals asdescribed as follows
(i) The first traversal is towards the top side ofthe noisymissed character component to com-pensate the left-top coordinatersquos slope with theaverage slopeThe second traversal is to traversetowards the left side of themissed component inthe direction of the average slope till it reaches
Mathematical Problems in Engineering 9
leftmost pixel of the noisymissed componentwithin the average height using (3) After reach-ing the left most side of the noisymissed com-ponent fix it as left-top coordinates (119909
2 1199102) of
the noisymissed component(ii) Take the newly found left-top coordinates of
the noisymissed LP character component andmove towards the right side (third traversal)in the direction of the average slope till itreaches the right most point of the missedcomponent within the average height using (3)The difference between the 119909-coordinates of thenewly found left-top coordinate point (119909
1 1199101)
and the right most point is the width of thenoisymissed component
(iii) Crop the noisymissed component using left-top coordinates width and the average heightof the cluster components
(5) Repeat the same procedure till the traversal reachesthe last component of the probable LP cluster com-ponent Repeat the same procedure for all probableLP clusters
(6) Rotate each individual component of the probableLP cluster by using average angle 120579 which can becalculated from the average slope (119898) amongst theprobable LP cluster components using (4) Con-sider each component of the binary image as 119865 =119865(119894 119895) 119894 = 1 2 119868 119895 = 1 2 119869 and can bedefined as shown in (5) One has
120579 = (
180
120587
) lowast tanminus1 (119898) (4)
119865 (119894 119895) =
1 white pixel
0 black pixel(5)
(7) Let 119865(119909 119910) be the image component before rota-tion and let 119865(1199091015840 1199101015840) be the image component afterrotation Use average angle 120579 to rotate 119865(119909 119910) Theequation for each individual pixel of 119865(1199091015840 1199101015840) can beobtained by using
1199091015840= 119909 lowast cos (120579) + 119910 lowast sin (120579)
1199101015840= minus119909 lowast sin (120579) + 119910 lowast cos (120579)
(6)
(8) In a rotated image if the average angle of the probablecluster components is above a certain threshold thenthere is a chance that the other part of the charactercomponent will be present in the target componentIn such a case retain the bigger component fromthe target component and remove the rest of thecomponents
6 Experimental Results
The above described concepts are implemented using MAT-LAB on Intel core i3 processor machine having 4GB RAM
The performance of the proposed LP detection methodis compared with some of the competitive LP detectionmethods by taking into consideration publicly availablemedia-lab benchmark LP database Israeli LP images fromthe web and proprietary Indian LPs having different platevariations and weather conditions in an open environmentTotal images from all these data sets are 900 To furtherassess the performance of the proposed LP detectionmethodwe have considered AOLP benchmark LP database having2049 imageswith three subsets For experimentation we haveconsidered Indian Israeli and media-lab Greek benchmarkLPs as single data set having different characteristics of theLPs in images as described in Table 1
In an open environment there are many possible waysby which we can capture an image from cameras Theproposed geometry-based clustering techniques in this paperare invariant to size tilt pan and rotation That is why theproposed approach works properly with extreme observationviews There is no restriction on the size of LP characters todetect the LP which can be observed from Figures 10(a23)and 10(b23) There are very few methods in the literaturewhich talks about the LP detection of motorcycles where LPcharacters fall in two lines and each line of characters areof different size The proposed method works for any typeof vehicles motorcycles vans and trucks having multiplelines of LP characters and also each line of characters havingdifferent sizes The proposed approach will fail to detect theLPLPs from an input image if the LP characters touch theborder of the LP or there are less than cluster size charactersin the LP or there are no characters present in LP at all Table 1shows the summarization of the various characteristics ofLPs in the images described in Figure 10 For exampleFigure 10(a1) (S number 1 in Table 1) indicates an LP inan input image from Greek in an open environment In allimages we have assumed neither fixed number of charactersin the LP nor the number of lines We assumed that the sizesof the LP characters may be different due to different viewconditions
Figure 10 shows the sample results for all categories ofthe LPs which include images of different plate variationsenvironmental and weather conditions from media-labIsraeli and proprietary Indian LP databases whose summaryis given in Table 1 The odd column of Figure 10 shows theactual image and the even column shows the binarized imageRed border in the binary image of Figure 10 indicates theidentification of the LP and without a red border belongs to afailed case Most of the vehicles in many countries includingmotorcycles will have only a single line of characters in theirLPs but there may be a chance that few countries like Indiawill have LP characters that will fall into more than one lineand there may be a chance that the characters in each line ofLP may vary in size as shown in Figures 10(a8) 10(a18) and10(a19) From such type of the LPs most of the existing LPRsystems detect only the line of characters which are bigger insize Hence such type of LP detection systems will not satisfythe real-time requirementsThe proposed approachwill workin all such conditions
Figures 10(a21) 10(b21) 10(a22) and 10(b22) show imageswith failed LP detection due to overlapping of LP characters
10 Mathematical Problems in Engineering
Table 1 Summarization of the characteristics of LPs in the images described in Figure 10
S number Figure number Country Characteristics of the imageRecognized by theproposed approach
(yesno)1 Figure 10(a1) Greek Image in open environment Yes2 Figure 10(a2) Greek Image with blur Yes3 Figure 10(a3) Greek Image with shadow Yes4 Figure 10(a4) Greek Image with reflectance and illumination Yes5 Figure 10(a5) Greek Image with dirt and screw in the LP Yes6 Figure 10(a6) Greek Image with distorted LP Yes7 Figure 10(a7) Greek Image with LP at a different place Yes8 Figure 10(a8) Greek Image taken at night Yes9 Figure 10(a9) Greek Image with dirt and shadows taken on difficult tracks Yes10 Figure 10(a10) Israel Image with different tilt Yes11 Figure 10(a11) Israel Image with different tilt Yes12 Figure 10(a12) Greek Image with an extreme pan Yes13 Figure 10(a13) Indian Image with multiple LPs (two cars) Yes14 Figure 10(a14) Indian Image with multiple LPs (three motorcycles) Yes15 Figure 10(a15) Indian Image with multiple LPs and different combination of vehicles Yes
16 Figure 10(a16) Indian Image with rotation (an image containing car with leftdiagonally rotated LP) Yes
17 Figure 10(a17) Indian Image with rotation (an image containing car with rightdiagonally rotated LP) Yes
18 Figure 10(a18) Indian Image with rotation (an image containing motorcycle with leftdiagonally rotated LP) Yes
19 Figure 10(a19) Indian Image with rotation (an image containing motorcycle with rightdiagonally rotated LP) Yes
20 Figure 10(a20) Urdu lingual country Image from different country Yes21 Figure 10(a21) Greek Image with lot of dirt in the LP No22 Figure 10(a22) Greek Image with lot of dirt in the LP No23 Figure 10(a23) Israel Image with close view of LP Yes24 Figure 10(a24) Greek Image with few noisy characters in the LP Yes
with LP border because of the presence of lot of dirt betweenLP characters and LP border Figures 10(a24) and 10(b24)show images with LPs 2nd (character ldquoKrdquo) and 6th (characterldquo7rdquo) characters touching the border of the LP due to thepresence of noise such as screw or dirt or stamp betweenLP characters and border of the LP These characters areextracted successfully using the proposed noisymissed char-acters extraction algorithm from Section 5 Figures 10(a1)ndash10(a9) 10(a12) 10(a21) 10(a22) and 10(a24) show Greek LPsFigures 10(a13)ndash10(a19) show Indian LPs Figures 10(a10)10(a11) and 10(a23) show Israeli LPs and Figure 10(a20)shows an LP from Urdu lingual country
From Table 1 and Figure 10 we can conclude that theimages are considered from four different countries andwe have also considered AOLP benchmark database forperformance evaluation which contains images from Taiwancountry With these results we can claim that the proposedapproach will work successfully to identify LPLPs from animage whose individual character properties are near to eachother in line with each other and similar in height in aparticular line There is no restriction on the number of lines
present in the LP of the vehicle and the characters in each lineof the LP can be of different size
The performance comparison amongst few of the promi-nent LP detection methods and the proposed approach isshown in Table 2 The method proposed by Bai and Liuin paper [4] is superior to the proposed approach witha remarkable LP detection success rate of 996 whichsupersedes all other methods The methods proposed byHuang et al in paper [8] Wen et al in paper [9] Yang etal in paper [5] Haneda and Hanaizumi in paper [10] andAnagnostopoulos et al in paper [6] reported 9755 9716953 97 and 965 LP detection rates respectively whichare less than the proposed methodrsquos success rate of 9874Lee et al in paper [3] and Al-Ghaili et al in paper [12]reported 9125 and 9165 success rates but their data setcontains only cars Faradji et al in paper [7] reported lowersuccess rate as compared to others Zhou et al in paper [11]reported 932 success rate on the proprietary data set and848 on Caltech data set Abo Samra and Khalefah in paper[14] reported 9875 success rate which is equivalent to theproposed methods success rate of 9874 The success rates
Mathematical Problems in Engineering 11
(a1) (b1) (a2) (b2)
(a3) (b3) (a4) (b4)
(a5) (b5) (a6) (b6)
(a7) (b7) (a8) (b8)
(a9) (b9) (a10) (b10)
(a11) (b11) (a12) (b12)
(a13) (b13) (a14) (b14)
(a15) (b15) (a16) (b16)
(a17) (b17) (a18) (b18)
(a19) (b19) (a20) (b20)
(a21) (b21) (a22) (b22)
(a23) (b23) (a24) (b24)
Figure 10 Examples of images with all categories of LPs showing LPs detection
12 Mathematical Problems in Engineering
Table 2 Performance comparison of the proposed LP detectionmethodwith few of the competitive LP detectionmethods from the literature
Algorithm and references Data setNumber ofimages indata set
Types of vehiclespresent in data set LP detection rate
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
Figure 4 An example image showing how the distance-basedclusters are divided into the line-based clusters
Figure 5 Resultant image after line-based clustering
Figure 5 in which the line-based clusters are markedin rectangular boxes
(11) There is no difference between Figures 3 and 5 basedon the number of components and positions of thecomponents are concerned The resultant distance-based clusters are shown in the ellipse shape withvarious colors in Figure 3 The resultant line-basedclusters are shown in the rectangular shape withvarious colors in Figure 5
(12) After line-based clustering apply height-based clus-tering technique to remove few unwantedjunk com-ponents which are very close to and in line with thecomponents but shows much difference in height ascompared with other components in each line-basedclusterNow remove those clusterswhich are less thancluster size and which are subsets to other clustersThis is height-based clustering
(13) Figure 6 shows an example image in which thecomponent C3 (indicated with the arrow) is showingmuch difference in height as compared to the rest ofthe components it will be removed from the line-based cluster and will result in height-based clusterwith the components C1 C2 C4 and C5 Figure 7shows the image after height-based clustering and theresultant height-based clusters are shown in ellipses
C1 C2 C3 C4 C5
C6C7
C8C9
C10 C11C12
C13
Figure 6 An example image showing the height-based clustering
Figure 7 Resultant image after height-based clustering
At this stage let 119873ℎbe the number of height-based
clusters(14) After height-based clustering few of the clusters
from the final cluster list may contain all unwantedcomponents which obey all the characteristics basedon distance line and height-based clustering Suchjunk component clusters can be removed by thinningand resizing technique
(15) Apply infinite thinning and resizing technique oneach 119873
ℎheight-based cluster of the image Thinning
is a morphological operation used for skeletonizationof the binary image components When we applythinning and resizing operations junk componentswill retain its shape but the LP characters will fadeaway completely Remove those clusters from thecluster list which retain its shape after thinning andresizing operations This technique removes all junkcluster components from the final cluster list Let 119873
1
be the number of clusters after removing junk clustersusing the thinning and resizing technique Figure 8shows image after thinning and resizing techniqueand the resultant clusters are shown in ellipses shape
43 Finding Border of the License Plate Contrasting toa typical LPR system the proposed system first findsthe probable LP characters and then finds the border of
8 Mathematical Problems in Engineering
Figure 8 Resultant image after thinning and resizing technique
the components The reason for finding the border of LP isthat there can be a group of non-LP characters in the imagewith similar properties of LP characters without the borderIn order to avoid such type of characters the system proposesto find the border of the LP
Predefined Border Percentage Beta Beta is a user definedvariable which decides the border percentage that an LPcan have After rigorous experimentations with the help ofmany datasets from multiple countries we have come to aconclusion to decide the value of Beta as 70
Steps
(1) Let 1198731be the number of probable LP clusters at
this stage Take each individual component fromthe cluster and traverse from left-top coordinatesof each individual component towards the upwarddirection till the traversal reaches the border pointor three times the height of each individual charactercomponent
(2) Apply the same procedure towards the downwarddirection to find the bottom border point for eachindividual component
(3) Save the top and bottom border points for all thecomponents of each cluster
(4) Find the border percentage for each cluster usingthe top and bottom border points Retain a clusteronly when the percentage is greater than or equalto the predefined border percentage ldquoBetardquo Retainedclusters indicate the LPLPs of an image and itscomponents indicate the individual characters of theLP
(5) Remove those clusters which fall below predefinedborder percentage ldquoBetardquo This is another way tofurther refine the cluster list to get the required LPregion Figure 9 shows the image after LP borderidentification
5 Noisy Characters Extraction and LicensePlate Characters Rotation
Noisymissed characters extraction stage is to extract few ofthe LP characters which may be missed during the previous
Figure 9 Resultant image after license plate border identification
stages because of the presence of noise such as screw ordirt or stamps between the LP border and LP charactersThis section proposes a new approach to extract such typeof noisymissed charactersThe proposed noisymissed char-acters extraction algorithm can be applied after Section 43
Steps
(1) The proposed algorithm uses (3) to find the noisy LPcharactersrsquo left-top coordinates (119909
2 1199102) at a distance
119889 from the leftmost cluster componentrsquos left-topcoordinates (119909
1 1199101) and the average slope119898 amongst
the cluster components
1199092= 1199091plusmn (
119889
radic (1 + 1198982)
)
1199102= 1199101plusmn (
119898 lowast 119889
radic (1 + 1198982)
)
(3)
(2) At this stage we have few probable LP clusters Takeeach probable LP cluster and find the average heightand average slope (119898) amongst the subsequent clustercomponentsrsquo left-top coordinates and the 119909-axis
(3) From each probable LP cluster take left-top coordi-nates of the first component (119909
1 1199101) and move in the
downward direction to (14)th of the average heightof probable LP cluster Now traverse the image rightside one pixel at a time using (3) mentioned aboveto find any noisymissed LP character components
(4) If any noisymissed LP character component is foundwhich is not part of the probable LP cluster com-ponent and is not a background pixel then findthe left-top coordinates (119909
2 1199102) and the width of
the noisymissed component To find the left-topcoordinates (119909
2 1199102) and thewidth of the noisymissed
component we have to perform three traversals asdescribed as follows
(i) The first traversal is towards the top side ofthe noisymissed character component to com-pensate the left-top coordinatersquos slope with theaverage slopeThe second traversal is to traversetowards the left side of themissed component inthe direction of the average slope till it reaches
Mathematical Problems in Engineering 9
leftmost pixel of the noisymissed componentwithin the average height using (3) After reach-ing the left most side of the noisymissed com-ponent fix it as left-top coordinates (119909
2 1199102) of
the noisymissed component(ii) Take the newly found left-top coordinates of
the noisymissed LP character component andmove towards the right side (third traversal)in the direction of the average slope till itreaches the right most point of the missedcomponent within the average height using (3)The difference between the 119909-coordinates of thenewly found left-top coordinate point (119909
1 1199101)
and the right most point is the width of thenoisymissed component
(iii) Crop the noisymissed component using left-top coordinates width and the average heightof the cluster components
(5) Repeat the same procedure till the traversal reachesthe last component of the probable LP cluster com-ponent Repeat the same procedure for all probableLP clusters
(6) Rotate each individual component of the probableLP cluster by using average angle 120579 which can becalculated from the average slope (119898) amongst theprobable LP cluster components using (4) Con-sider each component of the binary image as 119865 =119865(119894 119895) 119894 = 1 2 119868 119895 = 1 2 119869 and can bedefined as shown in (5) One has
120579 = (
180
120587
) lowast tanminus1 (119898) (4)
119865 (119894 119895) =
1 white pixel
0 black pixel(5)
(7) Let 119865(119909 119910) be the image component before rota-tion and let 119865(1199091015840 1199101015840) be the image component afterrotation Use average angle 120579 to rotate 119865(119909 119910) Theequation for each individual pixel of 119865(1199091015840 1199101015840) can beobtained by using
1199091015840= 119909 lowast cos (120579) + 119910 lowast sin (120579)
1199101015840= minus119909 lowast sin (120579) + 119910 lowast cos (120579)
(6)
(8) In a rotated image if the average angle of the probablecluster components is above a certain threshold thenthere is a chance that the other part of the charactercomponent will be present in the target componentIn such a case retain the bigger component fromthe target component and remove the rest of thecomponents
6 Experimental Results
The above described concepts are implemented using MAT-LAB on Intel core i3 processor machine having 4GB RAM
The performance of the proposed LP detection methodis compared with some of the competitive LP detectionmethods by taking into consideration publicly availablemedia-lab benchmark LP database Israeli LP images fromthe web and proprietary Indian LPs having different platevariations and weather conditions in an open environmentTotal images from all these data sets are 900 To furtherassess the performance of the proposed LP detectionmethodwe have considered AOLP benchmark LP database having2049 imageswith three subsets For experimentation we haveconsidered Indian Israeli and media-lab Greek benchmarkLPs as single data set having different characteristics of theLPs in images as described in Table 1
In an open environment there are many possible waysby which we can capture an image from cameras Theproposed geometry-based clustering techniques in this paperare invariant to size tilt pan and rotation That is why theproposed approach works properly with extreme observationviews There is no restriction on the size of LP characters todetect the LP which can be observed from Figures 10(a23)and 10(b23) There are very few methods in the literaturewhich talks about the LP detection of motorcycles where LPcharacters fall in two lines and each line of characters areof different size The proposed method works for any typeof vehicles motorcycles vans and trucks having multiplelines of LP characters and also each line of characters havingdifferent sizes The proposed approach will fail to detect theLPLPs from an input image if the LP characters touch theborder of the LP or there are less than cluster size charactersin the LP or there are no characters present in LP at all Table 1shows the summarization of the various characteristics ofLPs in the images described in Figure 10 For exampleFigure 10(a1) (S number 1 in Table 1) indicates an LP inan input image from Greek in an open environment In allimages we have assumed neither fixed number of charactersin the LP nor the number of lines We assumed that the sizesof the LP characters may be different due to different viewconditions
Figure 10 shows the sample results for all categories ofthe LPs which include images of different plate variationsenvironmental and weather conditions from media-labIsraeli and proprietary Indian LP databases whose summaryis given in Table 1 The odd column of Figure 10 shows theactual image and the even column shows the binarized imageRed border in the binary image of Figure 10 indicates theidentification of the LP and without a red border belongs to afailed case Most of the vehicles in many countries includingmotorcycles will have only a single line of characters in theirLPs but there may be a chance that few countries like Indiawill have LP characters that will fall into more than one lineand there may be a chance that the characters in each line ofLP may vary in size as shown in Figures 10(a8) 10(a18) and10(a19) From such type of the LPs most of the existing LPRsystems detect only the line of characters which are bigger insize Hence such type of LP detection systems will not satisfythe real-time requirementsThe proposed approachwill workin all such conditions
Figures 10(a21) 10(b21) 10(a22) and 10(b22) show imageswith failed LP detection due to overlapping of LP characters
10 Mathematical Problems in Engineering
Table 1 Summarization of the characteristics of LPs in the images described in Figure 10
S number Figure number Country Characteristics of the imageRecognized by theproposed approach
(yesno)1 Figure 10(a1) Greek Image in open environment Yes2 Figure 10(a2) Greek Image with blur Yes3 Figure 10(a3) Greek Image with shadow Yes4 Figure 10(a4) Greek Image with reflectance and illumination Yes5 Figure 10(a5) Greek Image with dirt and screw in the LP Yes6 Figure 10(a6) Greek Image with distorted LP Yes7 Figure 10(a7) Greek Image with LP at a different place Yes8 Figure 10(a8) Greek Image taken at night Yes9 Figure 10(a9) Greek Image with dirt and shadows taken on difficult tracks Yes10 Figure 10(a10) Israel Image with different tilt Yes11 Figure 10(a11) Israel Image with different tilt Yes12 Figure 10(a12) Greek Image with an extreme pan Yes13 Figure 10(a13) Indian Image with multiple LPs (two cars) Yes14 Figure 10(a14) Indian Image with multiple LPs (three motorcycles) Yes15 Figure 10(a15) Indian Image with multiple LPs and different combination of vehicles Yes
16 Figure 10(a16) Indian Image with rotation (an image containing car with leftdiagonally rotated LP) Yes
17 Figure 10(a17) Indian Image with rotation (an image containing car with rightdiagonally rotated LP) Yes
18 Figure 10(a18) Indian Image with rotation (an image containing motorcycle with leftdiagonally rotated LP) Yes
19 Figure 10(a19) Indian Image with rotation (an image containing motorcycle with rightdiagonally rotated LP) Yes
20 Figure 10(a20) Urdu lingual country Image from different country Yes21 Figure 10(a21) Greek Image with lot of dirt in the LP No22 Figure 10(a22) Greek Image with lot of dirt in the LP No23 Figure 10(a23) Israel Image with close view of LP Yes24 Figure 10(a24) Greek Image with few noisy characters in the LP Yes
with LP border because of the presence of lot of dirt betweenLP characters and LP border Figures 10(a24) and 10(b24)show images with LPs 2nd (character ldquoKrdquo) and 6th (characterldquo7rdquo) characters touching the border of the LP due to thepresence of noise such as screw or dirt or stamp betweenLP characters and border of the LP These characters areextracted successfully using the proposed noisymissed char-acters extraction algorithm from Section 5 Figures 10(a1)ndash10(a9) 10(a12) 10(a21) 10(a22) and 10(a24) show Greek LPsFigures 10(a13)ndash10(a19) show Indian LPs Figures 10(a10)10(a11) and 10(a23) show Israeli LPs and Figure 10(a20)shows an LP from Urdu lingual country
From Table 1 and Figure 10 we can conclude that theimages are considered from four different countries andwe have also considered AOLP benchmark database forperformance evaluation which contains images from Taiwancountry With these results we can claim that the proposedapproach will work successfully to identify LPLPs from animage whose individual character properties are near to eachother in line with each other and similar in height in aparticular line There is no restriction on the number of lines
present in the LP of the vehicle and the characters in each lineof the LP can be of different size
The performance comparison amongst few of the promi-nent LP detection methods and the proposed approach isshown in Table 2 The method proposed by Bai and Liuin paper [4] is superior to the proposed approach witha remarkable LP detection success rate of 996 whichsupersedes all other methods The methods proposed byHuang et al in paper [8] Wen et al in paper [9] Yang etal in paper [5] Haneda and Hanaizumi in paper [10] andAnagnostopoulos et al in paper [6] reported 9755 9716953 97 and 965 LP detection rates respectively whichare less than the proposed methodrsquos success rate of 9874Lee et al in paper [3] and Al-Ghaili et al in paper [12]reported 9125 and 9165 success rates but their data setcontains only cars Faradji et al in paper [7] reported lowersuccess rate as compared to others Zhou et al in paper [11]reported 932 success rate on the proprietary data set and848 on Caltech data set Abo Samra and Khalefah in paper[14] reported 9875 success rate which is equivalent to theproposed methods success rate of 9874 The success rates
Mathematical Problems in Engineering 11
(a1) (b1) (a2) (b2)
(a3) (b3) (a4) (b4)
(a5) (b5) (a6) (b6)
(a7) (b7) (a8) (b8)
(a9) (b9) (a10) (b10)
(a11) (b11) (a12) (b12)
(a13) (b13) (a14) (b14)
(a15) (b15) (a16) (b16)
(a17) (b17) (a18) (b18)
(a19) (b19) (a20) (b20)
(a21) (b21) (a22) (b22)
(a23) (b23) (a24) (b24)
Figure 10 Examples of images with all categories of LPs showing LPs detection
12 Mathematical Problems in Engineering
Table 2 Performance comparison of the proposed LP detectionmethodwith few of the competitive LP detectionmethods from the literature
Algorithm and references Data setNumber ofimages indata set
Types of vehiclespresent in data set LP detection rate
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
Figure 8 Resultant image after thinning and resizing technique
the components The reason for finding the border of LP isthat there can be a group of non-LP characters in the imagewith similar properties of LP characters without the borderIn order to avoid such type of characters the system proposesto find the border of the LP
Predefined Border Percentage Beta Beta is a user definedvariable which decides the border percentage that an LPcan have After rigorous experimentations with the help ofmany datasets from multiple countries we have come to aconclusion to decide the value of Beta as 70
Steps
(1) Let 1198731be the number of probable LP clusters at
this stage Take each individual component fromthe cluster and traverse from left-top coordinatesof each individual component towards the upwarddirection till the traversal reaches the border pointor three times the height of each individual charactercomponent
(2) Apply the same procedure towards the downwarddirection to find the bottom border point for eachindividual component
(3) Save the top and bottom border points for all thecomponents of each cluster
(4) Find the border percentage for each cluster usingthe top and bottom border points Retain a clusteronly when the percentage is greater than or equalto the predefined border percentage ldquoBetardquo Retainedclusters indicate the LPLPs of an image and itscomponents indicate the individual characters of theLP
(5) Remove those clusters which fall below predefinedborder percentage ldquoBetardquo This is another way tofurther refine the cluster list to get the required LPregion Figure 9 shows the image after LP borderidentification
5 Noisy Characters Extraction and LicensePlate Characters Rotation
Noisymissed characters extraction stage is to extract few ofthe LP characters which may be missed during the previous
Figure 9 Resultant image after license plate border identification
stages because of the presence of noise such as screw ordirt or stamps between the LP border and LP charactersThis section proposes a new approach to extract such typeof noisymissed charactersThe proposed noisymissed char-acters extraction algorithm can be applied after Section 43
Steps
(1) The proposed algorithm uses (3) to find the noisy LPcharactersrsquo left-top coordinates (119909
2 1199102) at a distance
119889 from the leftmost cluster componentrsquos left-topcoordinates (119909
1 1199101) and the average slope119898 amongst
the cluster components
1199092= 1199091plusmn (
119889
radic (1 + 1198982)
)
1199102= 1199101plusmn (
119898 lowast 119889
radic (1 + 1198982)
)
(3)
(2) At this stage we have few probable LP clusters Takeeach probable LP cluster and find the average heightand average slope (119898) amongst the subsequent clustercomponentsrsquo left-top coordinates and the 119909-axis
(3) From each probable LP cluster take left-top coordi-nates of the first component (119909
1 1199101) and move in the
downward direction to (14)th of the average heightof probable LP cluster Now traverse the image rightside one pixel at a time using (3) mentioned aboveto find any noisymissed LP character components
(4) If any noisymissed LP character component is foundwhich is not part of the probable LP cluster com-ponent and is not a background pixel then findthe left-top coordinates (119909
2 1199102) and the width of
the noisymissed component To find the left-topcoordinates (119909
2 1199102) and thewidth of the noisymissed
component we have to perform three traversals asdescribed as follows
(i) The first traversal is towards the top side ofthe noisymissed character component to com-pensate the left-top coordinatersquos slope with theaverage slopeThe second traversal is to traversetowards the left side of themissed component inthe direction of the average slope till it reaches
Mathematical Problems in Engineering 9
leftmost pixel of the noisymissed componentwithin the average height using (3) After reach-ing the left most side of the noisymissed com-ponent fix it as left-top coordinates (119909
2 1199102) of
the noisymissed component(ii) Take the newly found left-top coordinates of
the noisymissed LP character component andmove towards the right side (third traversal)in the direction of the average slope till itreaches the right most point of the missedcomponent within the average height using (3)The difference between the 119909-coordinates of thenewly found left-top coordinate point (119909
1 1199101)
and the right most point is the width of thenoisymissed component
(iii) Crop the noisymissed component using left-top coordinates width and the average heightof the cluster components
(5) Repeat the same procedure till the traversal reachesthe last component of the probable LP cluster com-ponent Repeat the same procedure for all probableLP clusters
(6) Rotate each individual component of the probableLP cluster by using average angle 120579 which can becalculated from the average slope (119898) amongst theprobable LP cluster components using (4) Con-sider each component of the binary image as 119865 =119865(119894 119895) 119894 = 1 2 119868 119895 = 1 2 119869 and can bedefined as shown in (5) One has
120579 = (
180
120587
) lowast tanminus1 (119898) (4)
119865 (119894 119895) =
1 white pixel
0 black pixel(5)
(7) Let 119865(119909 119910) be the image component before rota-tion and let 119865(1199091015840 1199101015840) be the image component afterrotation Use average angle 120579 to rotate 119865(119909 119910) Theequation for each individual pixel of 119865(1199091015840 1199101015840) can beobtained by using
1199091015840= 119909 lowast cos (120579) + 119910 lowast sin (120579)
1199101015840= minus119909 lowast sin (120579) + 119910 lowast cos (120579)
(6)
(8) In a rotated image if the average angle of the probablecluster components is above a certain threshold thenthere is a chance that the other part of the charactercomponent will be present in the target componentIn such a case retain the bigger component fromthe target component and remove the rest of thecomponents
6 Experimental Results
The above described concepts are implemented using MAT-LAB on Intel core i3 processor machine having 4GB RAM
The performance of the proposed LP detection methodis compared with some of the competitive LP detectionmethods by taking into consideration publicly availablemedia-lab benchmark LP database Israeli LP images fromthe web and proprietary Indian LPs having different platevariations and weather conditions in an open environmentTotal images from all these data sets are 900 To furtherassess the performance of the proposed LP detectionmethodwe have considered AOLP benchmark LP database having2049 imageswith three subsets For experimentation we haveconsidered Indian Israeli and media-lab Greek benchmarkLPs as single data set having different characteristics of theLPs in images as described in Table 1
In an open environment there are many possible waysby which we can capture an image from cameras Theproposed geometry-based clustering techniques in this paperare invariant to size tilt pan and rotation That is why theproposed approach works properly with extreme observationviews There is no restriction on the size of LP characters todetect the LP which can be observed from Figures 10(a23)and 10(b23) There are very few methods in the literaturewhich talks about the LP detection of motorcycles where LPcharacters fall in two lines and each line of characters areof different size The proposed method works for any typeof vehicles motorcycles vans and trucks having multiplelines of LP characters and also each line of characters havingdifferent sizes The proposed approach will fail to detect theLPLPs from an input image if the LP characters touch theborder of the LP or there are less than cluster size charactersin the LP or there are no characters present in LP at all Table 1shows the summarization of the various characteristics ofLPs in the images described in Figure 10 For exampleFigure 10(a1) (S number 1 in Table 1) indicates an LP inan input image from Greek in an open environment In allimages we have assumed neither fixed number of charactersin the LP nor the number of lines We assumed that the sizesof the LP characters may be different due to different viewconditions
Figure 10 shows the sample results for all categories ofthe LPs which include images of different plate variationsenvironmental and weather conditions from media-labIsraeli and proprietary Indian LP databases whose summaryis given in Table 1 The odd column of Figure 10 shows theactual image and the even column shows the binarized imageRed border in the binary image of Figure 10 indicates theidentification of the LP and without a red border belongs to afailed case Most of the vehicles in many countries includingmotorcycles will have only a single line of characters in theirLPs but there may be a chance that few countries like Indiawill have LP characters that will fall into more than one lineand there may be a chance that the characters in each line ofLP may vary in size as shown in Figures 10(a8) 10(a18) and10(a19) From such type of the LPs most of the existing LPRsystems detect only the line of characters which are bigger insize Hence such type of LP detection systems will not satisfythe real-time requirementsThe proposed approachwill workin all such conditions
Figures 10(a21) 10(b21) 10(a22) and 10(b22) show imageswith failed LP detection due to overlapping of LP characters
10 Mathematical Problems in Engineering
Table 1 Summarization of the characteristics of LPs in the images described in Figure 10
S number Figure number Country Characteristics of the imageRecognized by theproposed approach
(yesno)1 Figure 10(a1) Greek Image in open environment Yes2 Figure 10(a2) Greek Image with blur Yes3 Figure 10(a3) Greek Image with shadow Yes4 Figure 10(a4) Greek Image with reflectance and illumination Yes5 Figure 10(a5) Greek Image with dirt and screw in the LP Yes6 Figure 10(a6) Greek Image with distorted LP Yes7 Figure 10(a7) Greek Image with LP at a different place Yes8 Figure 10(a8) Greek Image taken at night Yes9 Figure 10(a9) Greek Image with dirt and shadows taken on difficult tracks Yes10 Figure 10(a10) Israel Image with different tilt Yes11 Figure 10(a11) Israel Image with different tilt Yes12 Figure 10(a12) Greek Image with an extreme pan Yes13 Figure 10(a13) Indian Image with multiple LPs (two cars) Yes14 Figure 10(a14) Indian Image with multiple LPs (three motorcycles) Yes15 Figure 10(a15) Indian Image with multiple LPs and different combination of vehicles Yes
16 Figure 10(a16) Indian Image with rotation (an image containing car with leftdiagonally rotated LP) Yes
17 Figure 10(a17) Indian Image with rotation (an image containing car with rightdiagonally rotated LP) Yes
18 Figure 10(a18) Indian Image with rotation (an image containing motorcycle with leftdiagonally rotated LP) Yes
19 Figure 10(a19) Indian Image with rotation (an image containing motorcycle with rightdiagonally rotated LP) Yes
20 Figure 10(a20) Urdu lingual country Image from different country Yes21 Figure 10(a21) Greek Image with lot of dirt in the LP No22 Figure 10(a22) Greek Image with lot of dirt in the LP No23 Figure 10(a23) Israel Image with close view of LP Yes24 Figure 10(a24) Greek Image with few noisy characters in the LP Yes
with LP border because of the presence of lot of dirt betweenLP characters and LP border Figures 10(a24) and 10(b24)show images with LPs 2nd (character ldquoKrdquo) and 6th (characterldquo7rdquo) characters touching the border of the LP due to thepresence of noise such as screw or dirt or stamp betweenLP characters and border of the LP These characters areextracted successfully using the proposed noisymissed char-acters extraction algorithm from Section 5 Figures 10(a1)ndash10(a9) 10(a12) 10(a21) 10(a22) and 10(a24) show Greek LPsFigures 10(a13)ndash10(a19) show Indian LPs Figures 10(a10)10(a11) and 10(a23) show Israeli LPs and Figure 10(a20)shows an LP from Urdu lingual country
From Table 1 and Figure 10 we can conclude that theimages are considered from four different countries andwe have also considered AOLP benchmark database forperformance evaluation which contains images from Taiwancountry With these results we can claim that the proposedapproach will work successfully to identify LPLPs from animage whose individual character properties are near to eachother in line with each other and similar in height in aparticular line There is no restriction on the number of lines
present in the LP of the vehicle and the characters in each lineof the LP can be of different size
The performance comparison amongst few of the promi-nent LP detection methods and the proposed approach isshown in Table 2 The method proposed by Bai and Liuin paper [4] is superior to the proposed approach witha remarkable LP detection success rate of 996 whichsupersedes all other methods The methods proposed byHuang et al in paper [8] Wen et al in paper [9] Yang etal in paper [5] Haneda and Hanaizumi in paper [10] andAnagnostopoulos et al in paper [6] reported 9755 9716953 97 and 965 LP detection rates respectively whichare less than the proposed methodrsquos success rate of 9874Lee et al in paper [3] and Al-Ghaili et al in paper [12]reported 9125 and 9165 success rates but their data setcontains only cars Faradji et al in paper [7] reported lowersuccess rate as compared to others Zhou et al in paper [11]reported 932 success rate on the proprietary data set and848 on Caltech data set Abo Samra and Khalefah in paper[14] reported 9875 success rate which is equivalent to theproposed methods success rate of 9874 The success rates
Mathematical Problems in Engineering 11
(a1) (b1) (a2) (b2)
(a3) (b3) (a4) (b4)
(a5) (b5) (a6) (b6)
(a7) (b7) (a8) (b8)
(a9) (b9) (a10) (b10)
(a11) (b11) (a12) (b12)
(a13) (b13) (a14) (b14)
(a15) (b15) (a16) (b16)
(a17) (b17) (a18) (b18)
(a19) (b19) (a20) (b20)
(a21) (b21) (a22) (b22)
(a23) (b23) (a24) (b24)
Figure 10 Examples of images with all categories of LPs showing LPs detection
12 Mathematical Problems in Engineering
Table 2 Performance comparison of the proposed LP detectionmethodwith few of the competitive LP detectionmethods from the literature
Algorithm and references Data setNumber ofimages indata set
Types of vehiclespresent in data set LP detection rate
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
leftmost pixel of the noisymissed componentwithin the average height using (3) After reach-ing the left most side of the noisymissed com-ponent fix it as left-top coordinates (119909
2 1199102) of
the noisymissed component(ii) Take the newly found left-top coordinates of
the noisymissed LP character component andmove towards the right side (third traversal)in the direction of the average slope till itreaches the right most point of the missedcomponent within the average height using (3)The difference between the 119909-coordinates of thenewly found left-top coordinate point (119909
1 1199101)
and the right most point is the width of thenoisymissed component
(iii) Crop the noisymissed component using left-top coordinates width and the average heightof the cluster components
(5) Repeat the same procedure till the traversal reachesthe last component of the probable LP cluster com-ponent Repeat the same procedure for all probableLP clusters
(6) Rotate each individual component of the probableLP cluster by using average angle 120579 which can becalculated from the average slope (119898) amongst theprobable LP cluster components using (4) Con-sider each component of the binary image as 119865 =119865(119894 119895) 119894 = 1 2 119868 119895 = 1 2 119869 and can bedefined as shown in (5) One has
120579 = (
180
120587
) lowast tanminus1 (119898) (4)
119865 (119894 119895) =
1 white pixel
0 black pixel(5)
(7) Let 119865(119909 119910) be the image component before rota-tion and let 119865(1199091015840 1199101015840) be the image component afterrotation Use average angle 120579 to rotate 119865(119909 119910) Theequation for each individual pixel of 119865(1199091015840 1199101015840) can beobtained by using
1199091015840= 119909 lowast cos (120579) + 119910 lowast sin (120579)
1199101015840= minus119909 lowast sin (120579) + 119910 lowast cos (120579)
(6)
(8) In a rotated image if the average angle of the probablecluster components is above a certain threshold thenthere is a chance that the other part of the charactercomponent will be present in the target componentIn such a case retain the bigger component fromthe target component and remove the rest of thecomponents
6 Experimental Results
The above described concepts are implemented using MAT-LAB on Intel core i3 processor machine having 4GB RAM
The performance of the proposed LP detection methodis compared with some of the competitive LP detectionmethods by taking into consideration publicly availablemedia-lab benchmark LP database Israeli LP images fromthe web and proprietary Indian LPs having different platevariations and weather conditions in an open environmentTotal images from all these data sets are 900 To furtherassess the performance of the proposed LP detectionmethodwe have considered AOLP benchmark LP database having2049 imageswith three subsets For experimentation we haveconsidered Indian Israeli and media-lab Greek benchmarkLPs as single data set having different characteristics of theLPs in images as described in Table 1
In an open environment there are many possible waysby which we can capture an image from cameras Theproposed geometry-based clustering techniques in this paperare invariant to size tilt pan and rotation That is why theproposed approach works properly with extreme observationviews There is no restriction on the size of LP characters todetect the LP which can be observed from Figures 10(a23)and 10(b23) There are very few methods in the literaturewhich talks about the LP detection of motorcycles where LPcharacters fall in two lines and each line of characters areof different size The proposed method works for any typeof vehicles motorcycles vans and trucks having multiplelines of LP characters and also each line of characters havingdifferent sizes The proposed approach will fail to detect theLPLPs from an input image if the LP characters touch theborder of the LP or there are less than cluster size charactersin the LP or there are no characters present in LP at all Table 1shows the summarization of the various characteristics ofLPs in the images described in Figure 10 For exampleFigure 10(a1) (S number 1 in Table 1) indicates an LP inan input image from Greek in an open environment In allimages we have assumed neither fixed number of charactersin the LP nor the number of lines We assumed that the sizesof the LP characters may be different due to different viewconditions
Figure 10 shows the sample results for all categories ofthe LPs which include images of different plate variationsenvironmental and weather conditions from media-labIsraeli and proprietary Indian LP databases whose summaryis given in Table 1 The odd column of Figure 10 shows theactual image and the even column shows the binarized imageRed border in the binary image of Figure 10 indicates theidentification of the LP and without a red border belongs to afailed case Most of the vehicles in many countries includingmotorcycles will have only a single line of characters in theirLPs but there may be a chance that few countries like Indiawill have LP characters that will fall into more than one lineand there may be a chance that the characters in each line ofLP may vary in size as shown in Figures 10(a8) 10(a18) and10(a19) From such type of the LPs most of the existing LPRsystems detect only the line of characters which are bigger insize Hence such type of LP detection systems will not satisfythe real-time requirementsThe proposed approachwill workin all such conditions
Figures 10(a21) 10(b21) 10(a22) and 10(b22) show imageswith failed LP detection due to overlapping of LP characters
10 Mathematical Problems in Engineering
Table 1 Summarization of the characteristics of LPs in the images described in Figure 10
S number Figure number Country Characteristics of the imageRecognized by theproposed approach
(yesno)1 Figure 10(a1) Greek Image in open environment Yes2 Figure 10(a2) Greek Image with blur Yes3 Figure 10(a3) Greek Image with shadow Yes4 Figure 10(a4) Greek Image with reflectance and illumination Yes5 Figure 10(a5) Greek Image with dirt and screw in the LP Yes6 Figure 10(a6) Greek Image with distorted LP Yes7 Figure 10(a7) Greek Image with LP at a different place Yes8 Figure 10(a8) Greek Image taken at night Yes9 Figure 10(a9) Greek Image with dirt and shadows taken on difficult tracks Yes10 Figure 10(a10) Israel Image with different tilt Yes11 Figure 10(a11) Israel Image with different tilt Yes12 Figure 10(a12) Greek Image with an extreme pan Yes13 Figure 10(a13) Indian Image with multiple LPs (two cars) Yes14 Figure 10(a14) Indian Image with multiple LPs (three motorcycles) Yes15 Figure 10(a15) Indian Image with multiple LPs and different combination of vehicles Yes
16 Figure 10(a16) Indian Image with rotation (an image containing car with leftdiagonally rotated LP) Yes
17 Figure 10(a17) Indian Image with rotation (an image containing car with rightdiagonally rotated LP) Yes
18 Figure 10(a18) Indian Image with rotation (an image containing motorcycle with leftdiagonally rotated LP) Yes
19 Figure 10(a19) Indian Image with rotation (an image containing motorcycle with rightdiagonally rotated LP) Yes
20 Figure 10(a20) Urdu lingual country Image from different country Yes21 Figure 10(a21) Greek Image with lot of dirt in the LP No22 Figure 10(a22) Greek Image with lot of dirt in the LP No23 Figure 10(a23) Israel Image with close view of LP Yes24 Figure 10(a24) Greek Image with few noisy characters in the LP Yes
with LP border because of the presence of lot of dirt betweenLP characters and LP border Figures 10(a24) and 10(b24)show images with LPs 2nd (character ldquoKrdquo) and 6th (characterldquo7rdquo) characters touching the border of the LP due to thepresence of noise such as screw or dirt or stamp betweenLP characters and border of the LP These characters areextracted successfully using the proposed noisymissed char-acters extraction algorithm from Section 5 Figures 10(a1)ndash10(a9) 10(a12) 10(a21) 10(a22) and 10(a24) show Greek LPsFigures 10(a13)ndash10(a19) show Indian LPs Figures 10(a10)10(a11) and 10(a23) show Israeli LPs and Figure 10(a20)shows an LP from Urdu lingual country
From Table 1 and Figure 10 we can conclude that theimages are considered from four different countries andwe have also considered AOLP benchmark database forperformance evaluation which contains images from Taiwancountry With these results we can claim that the proposedapproach will work successfully to identify LPLPs from animage whose individual character properties are near to eachother in line with each other and similar in height in aparticular line There is no restriction on the number of lines
present in the LP of the vehicle and the characters in each lineof the LP can be of different size
The performance comparison amongst few of the promi-nent LP detection methods and the proposed approach isshown in Table 2 The method proposed by Bai and Liuin paper [4] is superior to the proposed approach witha remarkable LP detection success rate of 996 whichsupersedes all other methods The methods proposed byHuang et al in paper [8] Wen et al in paper [9] Yang etal in paper [5] Haneda and Hanaizumi in paper [10] andAnagnostopoulos et al in paper [6] reported 9755 9716953 97 and 965 LP detection rates respectively whichare less than the proposed methodrsquos success rate of 9874Lee et al in paper [3] and Al-Ghaili et al in paper [12]reported 9125 and 9165 success rates but their data setcontains only cars Faradji et al in paper [7] reported lowersuccess rate as compared to others Zhou et al in paper [11]reported 932 success rate on the proprietary data set and848 on Caltech data set Abo Samra and Khalefah in paper[14] reported 9875 success rate which is equivalent to theproposed methods success rate of 9874 The success rates
Mathematical Problems in Engineering 11
(a1) (b1) (a2) (b2)
(a3) (b3) (a4) (b4)
(a5) (b5) (a6) (b6)
(a7) (b7) (a8) (b8)
(a9) (b9) (a10) (b10)
(a11) (b11) (a12) (b12)
(a13) (b13) (a14) (b14)
(a15) (b15) (a16) (b16)
(a17) (b17) (a18) (b18)
(a19) (b19) (a20) (b20)
(a21) (b21) (a22) (b22)
(a23) (b23) (a24) (b24)
Figure 10 Examples of images with all categories of LPs showing LPs detection
12 Mathematical Problems in Engineering
Table 2 Performance comparison of the proposed LP detectionmethodwith few of the competitive LP detectionmethods from the literature
Algorithm and references Data setNumber ofimages indata set
Types of vehiclespresent in data set LP detection rate
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
Table 1 Summarization of the characteristics of LPs in the images described in Figure 10
S number Figure number Country Characteristics of the imageRecognized by theproposed approach
(yesno)1 Figure 10(a1) Greek Image in open environment Yes2 Figure 10(a2) Greek Image with blur Yes3 Figure 10(a3) Greek Image with shadow Yes4 Figure 10(a4) Greek Image with reflectance and illumination Yes5 Figure 10(a5) Greek Image with dirt and screw in the LP Yes6 Figure 10(a6) Greek Image with distorted LP Yes7 Figure 10(a7) Greek Image with LP at a different place Yes8 Figure 10(a8) Greek Image taken at night Yes9 Figure 10(a9) Greek Image with dirt and shadows taken on difficult tracks Yes10 Figure 10(a10) Israel Image with different tilt Yes11 Figure 10(a11) Israel Image with different tilt Yes12 Figure 10(a12) Greek Image with an extreme pan Yes13 Figure 10(a13) Indian Image with multiple LPs (two cars) Yes14 Figure 10(a14) Indian Image with multiple LPs (three motorcycles) Yes15 Figure 10(a15) Indian Image with multiple LPs and different combination of vehicles Yes
16 Figure 10(a16) Indian Image with rotation (an image containing car with leftdiagonally rotated LP) Yes
17 Figure 10(a17) Indian Image with rotation (an image containing car with rightdiagonally rotated LP) Yes
18 Figure 10(a18) Indian Image with rotation (an image containing motorcycle with leftdiagonally rotated LP) Yes
19 Figure 10(a19) Indian Image with rotation (an image containing motorcycle with rightdiagonally rotated LP) Yes
20 Figure 10(a20) Urdu lingual country Image from different country Yes21 Figure 10(a21) Greek Image with lot of dirt in the LP No22 Figure 10(a22) Greek Image with lot of dirt in the LP No23 Figure 10(a23) Israel Image with close view of LP Yes24 Figure 10(a24) Greek Image with few noisy characters in the LP Yes
with LP border because of the presence of lot of dirt betweenLP characters and LP border Figures 10(a24) and 10(b24)show images with LPs 2nd (character ldquoKrdquo) and 6th (characterldquo7rdquo) characters touching the border of the LP due to thepresence of noise such as screw or dirt or stamp betweenLP characters and border of the LP These characters areextracted successfully using the proposed noisymissed char-acters extraction algorithm from Section 5 Figures 10(a1)ndash10(a9) 10(a12) 10(a21) 10(a22) and 10(a24) show Greek LPsFigures 10(a13)ndash10(a19) show Indian LPs Figures 10(a10)10(a11) and 10(a23) show Israeli LPs and Figure 10(a20)shows an LP from Urdu lingual country
From Table 1 and Figure 10 we can conclude that theimages are considered from four different countries andwe have also considered AOLP benchmark database forperformance evaluation which contains images from Taiwancountry With these results we can claim that the proposedapproach will work successfully to identify LPLPs from animage whose individual character properties are near to eachother in line with each other and similar in height in aparticular line There is no restriction on the number of lines
present in the LP of the vehicle and the characters in each lineof the LP can be of different size
The performance comparison amongst few of the promi-nent LP detection methods and the proposed approach isshown in Table 2 The method proposed by Bai and Liuin paper [4] is superior to the proposed approach witha remarkable LP detection success rate of 996 whichsupersedes all other methods The methods proposed byHuang et al in paper [8] Wen et al in paper [9] Yang etal in paper [5] Haneda and Hanaizumi in paper [10] andAnagnostopoulos et al in paper [6] reported 9755 9716953 97 and 965 LP detection rates respectively whichare less than the proposed methodrsquos success rate of 9874Lee et al in paper [3] and Al-Ghaili et al in paper [12]reported 9125 and 9165 success rates but their data setcontains only cars Faradji et al in paper [7] reported lowersuccess rate as compared to others Zhou et al in paper [11]reported 932 success rate on the proprietary data set and848 on Caltech data set Abo Samra and Khalefah in paper[14] reported 9875 success rate which is equivalent to theproposed methods success rate of 9874 The success rates
Mathematical Problems in Engineering 11
(a1) (b1) (a2) (b2)
(a3) (b3) (a4) (b4)
(a5) (b5) (a6) (b6)
(a7) (b7) (a8) (b8)
(a9) (b9) (a10) (b10)
(a11) (b11) (a12) (b12)
(a13) (b13) (a14) (b14)
(a15) (b15) (a16) (b16)
(a17) (b17) (a18) (b18)
(a19) (b19) (a20) (b20)
(a21) (b21) (a22) (b22)
(a23) (b23) (a24) (b24)
Figure 10 Examples of images with all categories of LPs showing LPs detection
12 Mathematical Problems in Engineering
Table 2 Performance comparison of the proposed LP detectionmethodwith few of the competitive LP detectionmethods from the literature
Algorithm and references Data setNumber ofimages indata set
Types of vehiclespresent in data set LP detection rate
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
Open environmentdifferent plate variations 965 921 973
AOLP benchmark LP database 2049 images havingvans trucks and cars
Open environmentdifferent plate variations 8167 9333 937
Average of success rates 8909 9272 9466
of the above mentioned LP detection systems are based onproprietary data sets
It is impractical to compare the performances of differentLP detection systems which evaluated their performancesusing proprietary LP data sets There should be a commontrue benchmark LP database openly available to assess theperformance of the proposed LP detection systems A com-mon publicly available media-lab benchmark LP databasefor the research community is initiated by Anagnostopouloset al in paper [1] which contains Greek vehicle LP imagesAs the media-lab benchmark LP database is not satisfyingall plate variations mentioned in this paper we coupled theimages of Israeli and Indian LPs having cars vans trucksand motorcycles with media-lab benchmark LP databaseto attain all plate variations mentioned Table 3 shows theperformance comparison between SCW method [6] AOLP
method [13] and the proposed approach usingmedia-lab andAOLP benchmark LP databases Usingmedia-lab benchmarkLP database the proposed methodrsquos success rate of 973is better when compared to SCW methodrsquos success rate of965 (number of images taken by SCW method is 1334)and is more than AOLP methodrsquos success rate of 921Using AOLP benchmark LP database the success rate of theproposed approach is 937 which is close to the successrate of 9333 of the AOLP approach and better than 8167which is the success rate of SCWmethodThe average successrate of the proposed approach which is based on both thebenchmark LP databases is 9466 and is a bit more thanAOLPrsquos average success rate of 9272 and is better thanSCWrsquos average success rate of 8909 as shown in Table 3Abo Samra and Khalefah in paper [14] also tested theirproposed methods performance using media-lab benchmark
Mathematical Problems in Engineering 13
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
LP database and reported a success rate of 9761 using 335images only (instead of 741 images) whereas the proposedmethod used 741 images frommedia-lab benchmark LP dataset and reported 973 success rate which is almost equivalentto the success rate reported in paper [14]
The success rate of the proposed approach using media-lab benchmark LP data set and the proprietary LP data setis 9756 which is a bit more as compared to the successrate of 965 of the SCW approach The success rate of SCWmethod is reported using 1334 images whereas they madeavailable only 741 images online as media-lab benchmarkLP database We do not have clarification on rest of (1334 minus741) 593 images During our experimentation we observedaround 2 of the media-lab LP benchmark database havingnoisymissed characters and we have achieved 100 noisycharacters extraction from the input images using the pro-posed noisymissed characters extraction method
We observed that most of the LP detection papers fromthe literature vastly used edge information template infor-mation morphological operations and color information ofthe LPsThese types of methodologies have restrictions whendetecting the LPLPs from the input images as explained inSection 2 In order to overcome the shortcomings of the LPdetection methods from the literature which are enlightenedin this paper we have proposed geometry-based clusteringtechniques which are invariant to color scale rotation andscale variances of the LPs and also proved from Figure 10that the proposedmultiple LPs detectionmethod successfullydetects the LPs from the input images taken from the openenvironment all weather conditions and all plate variationsmentioned in this paper Hence the proposed method hasthe ability to detect multiple LPs from an input imagewhich follow the properties of the proposed geometry-basedclustering techniques
7 Conclusion
In this paper we have proposed a new method for LPLPsdetection and noisymissed characters extraction due to thepresence of noise between LP characters and LP border Theproposed methodrsquos performance is evaluated on media-laband AOLP benchmark LP data sets and reported successrates of 973 and 937 respectively which are shown inperformance comparison Table 2 The average success rateof the proposed approach (9466) is more as compared toSCW (8909) and AOLP (9272) approaches using bothbenchmark LP data sets which are shown in Table 3 Theproposed approach can detect multiple LPs in an image andis not specific to any country there is no restriction on thenumber of characters present in the LPs the number of linespresent in the LPs and the size of the characters in eachline of the LP As demonstrated in the results the proposedapproach is less restrictive as compared with most of thepreviously published work and it works for many countrieshaving different plate variations under different environmen-tal and weather conditions The proposed approach fails toidentify the LPLPs if the LP characters are missed due to
the presence of noise such as extremely dirty or blur orcharacters of LP touches the LP border
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
References
[1] C-N E Anagnostopoulos I E Anagnostopoulos I DPsoroulas V Loumos and E Kayafas ldquoLicense plate recog-nition from still images and video sequences a surveyrdquo IEEETransactions on Intelligent Transportation Systems vol 9 no 3pp 377ndash391 2008
[2] I E Anagnostopoulos I D Psoroulas V Loumos E Kayafasand C-N E Anagnostopoulos ldquoMedialab LPRDatabaserdquoMul-timedia Technology Laboratory National Technical Universityof Athens httpwwwmedialabntuagrresearchLPRdatabasehtml
[3] E R Lee P K Kim and H J Kim ldquoAutomatic recognition of acar license plate using color image processingrdquo in Proceedings ofthe 1st IEEE International Conference on Image Processing (ICIPrsquo94) vol 2 pp 301ndash305 Austin Tex USA 1994
[4] H Bai and C Liu ldquoA hybrid license plate extraction methodbased on edge statistics and morphologyrdquo in Proceedings of the17th International Conference on Pattern Recognition (ICPR rsquo04)vol 2 pp 831ndash834 Cambridge UK August 2004
[5] Y-Q Yang J Bai R-L Tian and N A Liu ldquoA vehicle licenseplate recognition system based on fixed color collocationrdquo inProceedings of the International Conference onMachine Learningand Cybernetics (ICMLC rsquo05) pp 5394ndash5397 August 2005
[6] C N E Anagnostopoulos I E Anagnostopoulos V Loumosand E Kayafas ldquoA license plate-recognition algorithm for intel-ligent transportation system applicationsrdquo IEEE Transactionson Intelligent Transportation Systems vol 7 no 3 pp 377ndash3912006
[7] F Faradji A H Rezaie and M Ziaratban ldquoA morphological-based license plate locationrdquo in Proceedings of the IEEE Inter-national Conference on Image Processing pp I57ndashI60 IEEE SanAntonio Tex USA September 2007
[8] Y-P Huang C-H Chen Y-T Chang and F E Sandnes ldquoAnintelligent strategy for checking the annual inspection status ofmotorcycles based on license plate recognitionrdquo Expert Systemswith Applications vol 36 no 5 pp 9260ndash9267 2009
[9] Y Wen Y Lu J Yan Z Zhou K M von Deneen and P ShildquoAn algorithm for license plate recognition applied to intelligenttransportation systemrdquo IEEE Transactions on Intelligent Trans-portation Systems vol 12 no 3 pp 830ndash845 2011
[10] K Haneda and H Hanaizumi ldquoA flexible method for recogniz-ing four-digit numbers on a license-plate in a video scenerdquo inProceedings of the IEEE International Conference on IndustrialTechnology (ICIT rsquo12) pp 112ndash116 IEEE Athens Ga USAMarch 2012
[11] W Zhou H Li Y Lu and Q Tian ldquoPrincipal visual word dis-covery for automatic license plate detectionrdquo IEEE Transactionson Image Processing vol 21 no 9 pp 4269ndash4279 2012
[12] A M Al-Ghaili S Mashohor A R Ramli and A IsmailldquoVertical-edge-based car-license-plate detectionmethodrdquo IEEETransactions on Vehicular Technology vol 62 no 1 pp 26ndash382013
14 Mathematical Problems in Engineering
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
[13] G-S Hsu J-C Chen and Y-Z Chung ldquoApplication-orientedlicense plate recognitionrdquo IEEE Transactions on Vehicular Tech-nology vol 62 no 2 pp 552ndash561 2013
[14] G Abo Samra and F Khalefah ldquoLocalization of license platenumber using dynamic image processing techniques andgenetic algorithmsrdquo IEEE Transactions on Evolutionary Compu-tation vol 18 no 2 pp 244ndash257 2014
![Page 1: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/1.jpg)
![Page 2: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/2.jpg)
![Page 3: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/3.jpg)
![Page 4: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/4.jpg)
![Page 5: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/5.jpg)
![Page 6: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/6.jpg)
![Page 7: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/7.jpg)
![Page 8: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/8.jpg)
![Page 9: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/9.jpg)
![Page 10: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/10.jpg)
![Page 11: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/11.jpg)
![Page 12: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/12.jpg)
![Page 13: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/13.jpg)
![Page 14: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/14.jpg)
![Page 15: Research Article Color, Scale, and Rotation Independent ...downloads.hindawi.com/journals/mpe/2016/9306282.pdfAbo Samra and Khalefah in paper [] proposed a new LP localization algorithm](https://reader034.documents.pub/reader034/viewer/2022052012/60294f69176dd906162926b9/html5/thumbnails/15.jpg)
