The Viola/Jones Face Detector

Prepared with figures taken from“Robust real-time object detection”

CRL 2001/01, February 2001

Three Measures Toward Speeded Up Detection

• Integral image: a fast way to compute simple “features”

• In Adaboost the weak learner is nothing but a feature selector. The advantage is that if there are N weak learners there are merely N features to compute.

• Cascaded combination of classifiers. Most of true negatives are rejected very fast at the at the first few stages. Can keep high detection rate and low false positive rate.

Image Features

Rectangle filters

Similar to Haar wavelets

Base resolution is 24-by-24

11 scales, scaling factor of 1.25

45396 features

Rectangular Features for Face Detection

Forehead, eye features can be captured

Fast Feature Computation: Integral Image

• Integral image value at a pixel (x, y) is the sum of the pixel values of the original image above and to the left of (x, y), inclusive.

• Integral image can be computed by one pass through the image

Computing Sum within a Rectangle by Integral Image

• The sum of the pixels within rectangle D can be computed with four array references.

• The value of the integral image at location 1 is the sum of the pixels in rectangle A.

• The value at location 2 is A + B, at location 3 is A + C, and at location 4 is A + B + C + D.

• The sum within D can be computed as 4 + 1 - (2 + 3).

Constrained Classifier: Feature Selection

• Restrict the weak learner to a single feature

• A weak classifier hj(x) consists of a feature fj, a threshold j, and a parity pj indicating the direction of inequality sign:

• x is a 24-by-24 sub-window of an image

.otherwise,0

,)(if1)( jjjj

j

θpxfpxh

Boosting Algorithm

Learning Result

Must do better

Cascading Classifiers

The initial classifier eliminates a large number of negative examples with very little processing.

Subsequent layers eliminate additional negatives but require additional computation.

After several stages of processing the number of sub-windows have been reduced radically.

How Cascading Can Meet Performance?

K

iifF

1

For K stages of cascading with each stage having fi as the false positive rate, the overall false positive rate for the cascade is

Similarly, the overall detection rate is

K

iidD

1

To keep F very low and D very high, for each stage the goal is to have very high detection rate (close to 100%),

but moderate false positive rate (say, 30%)

Cascaded Classifier

1 Feature 5 Features

F

50%20 Features

20% 2%FACE

NON-FACE

F

NON-FACE

F

NON-FACE

IMAGESUB-WINDOW

• A 1 feature classifier achieves 100% detection rate and about 50% false positive rate.

• A 5 feature classifier achieves 100% detection rate and 40% false positive rate (20% cumulative)– using data from previous stage.

• A 20 feature classifier achieve 100% detection rate with 10% false positive rate (2% cumulative)

Building A Cascaded Detector

Classifier is Learned from Labeled Data

• Training Data– 4916 hand labeled faces

• All frontal

– 10000 non faces– Faces are normalized

• Scale, translation

• Many variations– Across individuals– Illumination– Pose (rotation both in plane and out)

Boosted Face Detection

• For each round of boosting:– Evaluate each rectangle filter on each example– Sort examples by filter values– Select best threshold for each filter (min Z)– Select best filter/threshold (= Feature) – Reweight examples

• Weeks to learn train• 15 frames per second to detect faces from unknown

images.

Performance

Output of Face Detector on Test Images