Zhongyan Liang, Sanyuan ZhangUnder review for Journal of Zhejiang University Science C (Computers & Electronics)

Publisher: Springer

A Credible Tilt License Plate Correction Method Based on Pairwise Parallel Lines

Andy {andrey.korea@gmail.com}

2Intelligent Systems

Lab.

Problem setting

Goal: License Plate(LP) tilt correction algorithm robust under various angles.

LP localization problem is considered to be solved

3Intelligent Systems

Lab.

Main algorithm scheme

4Intelligent Systems

Lab.

PreprocessingLow-pass Wiener filter

222 /,, yxIyxW

- local mean 22

- variance

- average of all estimated variances for each pixel in the neighborhood

yxI , - pixel intensity

Applied filter size: 3x3

5Intelligent Systems

Lab.

BinarizationSauvola’s threshold

11,,R

skyxyxT

- local mean

s - standard deviation

R = 128 (for grayscale image)

k – takes values from [0.2,0.5]

6Intelligent Systems

Lab.

Find fitting points1. Find bounding boxes of connected pixels

2. Count bounding boxes heights distribution for the following intervals:[0.05H, 0.2H]; [0.15H, 0.3H]; [0.25H, 0.4H]; [0.35H, 0.5H]; [0.45H, 0.6H];[0.55H, 0.7H]; [0.65H, 0.8H]; [0.75H, H]Where H is the height of LP

3. Select the interval with the maximum value as the candidate interval

4. Use bounding boxes of selected interval to draw upper and lower lines by selecting highest and lowest points of bounding box.

5. Find fitting lines using Least Squares method

7Intelligent Systems

Lab.

Fitted lines verification

dr

dr

B

T

BT

BT

0

T - angle of the top fitted line

B - angle of the bottom fitted line

Lines are considered to be parallel if

Tn

iTiT r

Tnr

1

1

Bn

iBiB r

Bnr

1

1

- average distance from bounding boxes to the top line

- average distance from bounding boxes to the bottom line

Bir - distance from i th bounding boxes to the bottom line

Tir - distance from i th bounding boxes to the top line

BnTn, - number of fitted points in top and bottom lines

Rotation angle of LP:

2/BT

8Intelligent Systems

Lab.

Feature extractionUsed if the previous algorithm failed to estimate angle1. Find vertical edges using Sobel edge detector.

2. Use Otsu method for binarization.

3. Remove objects with height of bounding box less than 8 pixels.

4. Use foreground points of binarized Sobel image as feature points.

5. Use Principle Component Analysis (PCA) to find best fitting line.

9Intelligent Systems

Lab.

Experimental results

(a) Original image

(b) The lines fitted and failure by using the method based on parallel lines

(c) The vertical edges

(d) Correction results

10Intelligent Systems

Lab.

Comparison with other methods

(a) Original Image (b) Method by using Harris Feature and PCA (c) Method by using One Fitted Straight Line (d) Method by using Vertical Edges and PCA (Stage II only) (e) Method by using

Two Fitted Straight Lines (Stage I only) (f) The proposed method

11Intelligent Systems

Lab.

Experimental resultsData set 1: tilt license plates (k >= 0.03) in a variety of environments.Data set 2: non-tilt license plates (k < 0.03) in the case of sufficient sunshine.

Harris & PCA Fitting One Line Stage I only Stage II only Proposed

Set 1 36.99% 85.62% 88.36% 60.27% 92.47%

Set 2 40.00% 98.18% 98.18% 65.45% 98.18%

03.0tan kAccuracy

kstdDev tan

Robustness

Harris & PCA Fitting One Line Stage I only Stage II only Proposed

Set 1 0.07597 0.02781 0.02833 0.02892 0.02441

Set 2 0.13714 0.00675 0.0071 0.02531 0.0071

12Intelligent Systems

Lab.

Experimental resultsData set 1: tilt license plates (k >= 0.03) in a variety of environments.Data set 2: non-tilt license plates (k < 0.03) in the case of sufficient sunshine.

Total Real correct Reported correct Confidence of credibility

Set 1 146 134 129 92.27%

Set 2 56 55 55 100%

ResultsCorrectionCorrectReported

ResultsCorrectionCorrectRealycredibilitofConfidence

Credibility

13Intelligent Systems

Lab.

Conclusions

Advantages :

• Combined method for LP tilt correction proposed

• Result verification allows to use additional correction algorithm if needed

• Experiments shown promising results

Disadvantages: • Goal setting is uncertain. Input images and initial conditions not described.

• What error level is acceptable for further recognition?

• Number of experiments is insufficient to prove the effectiveness of algorithm.

• Not enough analysis and discussions.

14Intelligent Systems

Lab.

Wiener filter example

Original image Filtered image

Difference Difference with enhanced contrast

Magnified parts

Original Filtered

15Intelligent Systems

Lab.

Binarization

Radius 5, k=0.2 Radius 5, k=0.5

16Intelligent Systems

Lab.

Otsu methodIn Otsu's method we exhaustively search for the threshold that minimizes the intra-class variance, defined as a weighted sum of variances of the two classes:

ttwttwtw222

211

2

1

01

t

i

iptw

Where

Algorithm

N.Otsu, “A Threshold Selection Method from Gray-Level Histograms”, IEEE Transactions on Systems, Man and Cybernetics, vol.9, issue 1, pp.62 – 66, Jan. 1979

Compute histogram and probabilities of each intensity level1. Set up initial and 2. Step through all possible thresholds t=0…maximum intensity

- Update and - Compute

3. Desired threshold corresponds to the maximum

iw i

iw i tw2

tw2

255

2t

iptw

17Intelligent Systems

Lab.

OLS vs PCA