Download - Face Recognition CPSC 4600/5600 @ UTC/CSE. 2 Face Recognition Introduction Introduction Face recognition algorithms Face recognition algorithms Comparison.

Face RecognitionFace Recognition

CPSC 4600/5600 @ UTC/CSECPSC 4600/5600 @ UTC/CSE

22

Face RecognitionFace Recognition

IntroductionIntroductionFace recognition algorithmsFace recognition algorithmsComparisonComparisonShort summaryShort summary

33

Face Recognition AlgorithmsFace Recognition Algorithms

We will introduceWe will introduceEigenfacesEigenfacesFisherfacesFisherfacesElastic Bunch-Graph Matching Elastic Bunch-Graph Matching

44

EigenfacesEigenfaces

Developed in 1991 by M.TurkDeveloped in 1991 by M.TurkBased on Principal Component Analysis Based on Principal Component Analysis

(PCA)(PCA)Relatively simpleRelatively simpleFastFastRobustRobust

55


PCA seeks directions that are efficient for PCA seeks directions that are efficient for representing the datarepresenting the data

efficientnot efficient

Class A

Class B

Class A

Class B

66


PCA maximizes the total scatter

scatter

Class A

Class B

77


PCA reduces the dimension of the dataPCA reduces the dimension of the dataSpeeds up the computational timeSpeeds up the computational time

88

99

Eigenfaces, the algorithmEigenfaces, the algorithm

AssumptionsAssumptionsSquare images with Width = Height = N Square images with Width = Height = N M is the number of images in the databaseM is the number of images in the databaseP is the number of persons in the databaseP is the number of persons in the database

1010


The databaseThe database

2

1

2

N

b

b

b

2

1

2

N

c

c

c

2

1

2

N

d

d

d

2

1

2

N

e

e

e

2

1

2

N

a

a

a

2

1

2

N

f

f

f

2

1

2

N

g

g

g

2

1

2

N

h

h

h

1111


We compute the average faceWe compute the average face

2 2 2

1 1 1

2 2 21, 8

N N N

a b h

a b hm where M

M

a b h

1212


Then subtract it from the training facesThen subtract it from the training faces

2 2 2 2 2 2 2 2

2 2

1 1 1 1 1 1 1 1

2 2 2 2 2 2 2 2

1 1 1 1

2 2

, , , ,

,

m m m m

N N N N N N N N

m m

N N

a m b m c m d m

a m b m c m d ma b c d

a m b m c m d m

e m f m

e m fe f

e m

2 2 2 2 2 2

1 1 1 1

2 2 2 2 2 2, ,m m

N N N N N N

g m h m

m g m h mg h

f m g m h m

1313


Now we build the matrix which is Now we build the matrix which is NN22 by by MM

The covariance matrix which is The covariance matrix which is NN22 by by NN22

m m m m m m m mA a b c d e f g h

Cov AA

1414


Find eigenvalues of the covariance matrixFind eigenvalues of the covariance matrixThe matrix is very largeThe matrix is very largeThe computational effort is very bigThe computational effort is very big

We are interested in at most We are interested in at most MM eigenvalues eigenvaluesWe can reduce the dimension of the matrixWe can reduce the dimension of the matrix

1515


Compute another matrix which is Compute another matrix which is MM by by MM

Find the Find the MM eigenvalues and eigenvectors eigenvalues and eigenvectorsEigenvectors of Eigenvectors of Cov Cov andand L L are are equivalentequivalent

Build matrix Build matrix V V from the eigenvectors offrom the eigenvectors of L L

L A A

1616


Eigenvectors of Eigenvectors of CovCov are linear combination are linear combination of image space with the eigenvectors of of image space with the eigenvectors of LL

Eigenvectors represent the variation in the Eigenvectors represent the variation in the facesfaces

U AV V is Matrix of eigenvectors

m m m m m m m mA a b c d e f g h

1717


A: collection of the training faces

U: Face Space / Eigen Space

1818


Eigenface of original facesEigenface of original faces

1919


Compute for each face its Compute for each face its projection onto projection onto the face spacethe face space

Compute the Compute the thresholdthreshold

1 2 3 4

5 6 7 8

, , , ,

, , ,

m m m m

m m m m

U a U b U c U d

U e U f U g U h

1max , 1..

2 i j for i j M

2020

Eigenfaces: Eigenfaces: Recognition ProcedureRecognition Procedure

To recognize a faceTo recognize a face

Subtract the average face from itSubtract the average face from it

2

1

2

N

r

r

r

2 2

1 1

2 2

m

N N

r m

r mr

r m

2121


Compute its Compute its projectionprojection onto the onto the face space Uface space U

mU r

22 1..i i for i M

Compute the distance in the face space Compute the distance in the face space between between the facethe face and and all known facesall known faces

2222


Reconstruct the face from eigenfacesReconstruct the face from eigenfaces

Compute the distance between Compute the distance between the facethe face and and its reconstructionits reconstruction

s U

22mr s

2323


Distinguish betweenDistinguish between If then it’s not a face; the distance If then it’s not a face; the distance

between between the facethe face and and its reconstructionits reconstruction is is larger than thresholdlarger than threshold

If then it’s a new faceIf then it’s a new face If then it’s a known If then it’s a known

face because the distance in the face space face because the distance in the face space between between the facethe face and and all known facesall known faces is larger is larger than thresholdthan threshold

, ( 1.. )iand i M min iand

2424


Problems with eigenfacesProblems with eigenfacesDifferent illuminationDifferent illumination

2525


Problems with eigenfacesProblems with eigenfacesDifferent head poseDifferent head poseDifferent alignmentDifferent alignmentDifferent facial expressionDifferent facial expression

2626

FisherfacesFisherfaces

Developed in 1997 by P.Belhumeur et al.Developed in 1997 by P.Belhumeur et al.Based on Fisher’s Linear Discriminant Based on Fisher’s Linear Discriminant

Analysis (LDA)Analysis (LDA)Faster than eigenfaces, in some casesFaster than eigenfaces, in some casesHas lower error ratesHas lower error ratesWorks well even if different illuminationWorks well even if different illuminationWorks well even if different facial express.Works well even if different facial express.

2727


LDA seeks directions that are efficient for LDA seeks directions that are efficient for discrimination between the datadiscrimination between the data

Class A

Class B

2828


LDA maximizes the between-class scatterLDA maximizes the between-class scatterLDA minimizes the within-class scatterLDA minimizes the within-class scatter

Class A

Class B

2929

Fisherfaces, the algorithmFisherfaces, the algorithm

AssumptionsAssumptionsSquare images with Width=Height=N Square images with Width=Height=N M is the number of images in the databaseM is the number of images in the databaseP is the number of persons in the databaseP is the number of persons in the database

3030


The databaseThe database

2

1

2

N

b

b

b

2

1

2

N

c

c

c

2

1

2

N

d

d

d

2

1

2

N

e

e

e

2

1

2

N

a

a

a

2

1

2

N

f

f

f

2

1

2

N

g

g

g

2

1

2

N

h

h

h

3131

Fisherfaces, the algorithm Fisherfaces, the algorithm

We compute the average We compute the average of all facesof all faces

2 2 2

1 1 1

2 2 21, 8

N N N

a b h

a b hm where M

M

a b h

3232


Compute the average face Compute the average face of each personof each person

2 2 2 2

2 2 2 2

1 1 1 1

2 2 2 2

1 1 1 1

2 2 2 2

1 1, ,

2 2

1 1,

2 2

N N N N

N N N N

a b c d

a b c dx y

a b c d

e f g h

e f g hz w

e f g h

3333


And subtract them from the training facesAnd subtract them from the training faces

2 2 2 2 2 2 2 2

2 2

1 1 1 1 1 1 1 1

2 2 2 2 2 2 2 2

1 1 1 1

2 2

, , , ,

,

m m m m

N N N N N N N N

m m

N N

a x b x c y d y

a x b x c y d ya b c d

a x b x c y d y

e z f z

e z fe f

e z

2 2 2 2 2 2

1 1 1 1

2 2 2 2 2 2, ,m m

N N N N N N

g w h w

z g w h wg h

f z g w h w

3434


We build scatter matrices We build scatter matrices SS11, , SS22, , SS33, , SS44

And the And the within-class within-class scatter matrix scatter matrix SSWW

1 2

3 4

, ,

,

m m m m m m m m

m m m m m m m m

S a a b b S c c d d

S e e f f S g g h h

1 2 3 4WS S S S S

3535


The between-class scatter matrixThe between-class scatter matrix

We are seeking the matrix We are seeking the matrix WW maximizing maximizing

2 2 2 2BS x m x m y m y m z m z m w m w m

B

W

W S WJ W

W S W

3636


If If SSW W is nonsingular ( ):is nonsingular ( ):Columns of Columns of W W are eigenvectors ofare eigenvectors of

We have to compute the inverse of We have to compute the inverse of SSWW

We have to multiply the matricesWe have to multiply the matricesWe have to compute the eigenvectorsWe have to compute the eigenvectors

1W BS S

2M N

3737


If If SSW W is nonsingular ( ):is nonsingular ( ):Simpler:Simpler:

Columns of Columns of W W are eigenvectors satisfyingare eigenvectors satisfying

The eigenvalues are roots ofThe eigenvalues are roots of

Get eigenvectors by solvingGet eigenvectors by solving

B i i W iS w S w

0B i WS S

0B i W iS S w

2M N

3838


If If SSW W is singular ( ):is singular ( ):Apply PCA firstApply PCA first

Will reduce the dimension of faces from Will reduce the dimension of faces from NN22 to to MMThere are There are MM MM-dimensional vectors-dimensional vectors

Apply LDA as describedApply LDA as described

2M N

3939


Project faces onto the LDA-spaceProject faces onto the LDA-space

To classify the faceTo classify the faceProject it onto the LDA-spaceProject it onto the LDA-spaceRun a nearest-neighbor classifierRun a nearest-neighbor classifier

, ,

,

LDA LDA

LDA LDA

x W x y W y

z W z w W w

4040


ProblemsProblemsSmall databases Small databases The face to classify must be in the DBThe face to classify must be in the DB

4141

PCA & Fisher’s Linear Discriminant

4242

PCA & Fisher’s Linear Discriminant

4343

ComparisonComparison

FERET databaseFERET database

best ID rate: eigenfaces 80.0%, fisherfaces 93.2%best ID rate: eigenfaces 80.0%, fisherfaces 93.2%

4444


EigenfacesEigenfacesproject faces onto a lower dimensional sub-project faces onto a lower dimensional sub-

spacespaceno distinction between inter- and intra-class no distinction between inter- and intra-class

variabilitiesvariabilities optimal for representation but not for optimal for representation but not for

discriminationdiscrimination

4545


FisherfacesFisherfaces find a sub-space which maximizes the ratio of find a sub-space which maximizes the ratio of

inter-class and intra-class variabilityinter-class and intra-class variabilitysame intra-class variability for all classessame intra-class variability for all classes

4646

Local Feature AnalysisLocal Feature Analysis-- Elastic Bunch-Graph Matching-- Elastic Bunch-Graph Matching

4747

Face FeaturesFace Features

Facial recognition utilizes distinctive features of Facial recognition utilizes distinctive features of the face – including: distinct the face – including: distinct micro elementsmicro elements like: like: Mouth, Nose, Eye, Cheekbones, Chin, Lips, Forehead, Mouth, Nose, Eye, Cheekbones, Chin, Lips, Forehead,

EarsEars

Upper outlines of the eye sockets, the areas Upper outlines of the eye sockets, the areas surrounding the cheekbones, the sides of the surrounding the cheekbones, the sides of the mouth, and the location of the nose and eyes. mouth, and the location of the nose and eyes.

The distance between the eyes, the length of the The distance between the eyes, the length of the nose, and the angle of the jaw.nose, and the angle of the jaw.

4848

Face FeaturesFace Features Some technologies do Some technologies do notnot utilize utilize areas of the face areas of the face

located near the hairlinelocated near the hairline, so they are somewhat resistant , so they are somewhat resistant to moderate changes in hairstyle. to moderate changes in hairstyle.

When used in identification mode, facial recognition When used in identification mode, facial recognition technology generally returns technology generally returns candidate lists of close candidate lists of close matchesmatches as opposed to returning a single definitive as opposed to returning a single definitive match as does fingerprint and iris-scan. match as does fingerprint and iris-scan.

The file containing The file containing facialfacial micro featuresmicro features is called a is called a "template." "template."

Using templates, the software then compares that image Using templates, the software then compares that image with another image and produces with another image and produces a scorea score that measures that measures how similar the images are to each other. how similar the images are to each other.

4949

Typical sources of images for use in facial Typical sources of images for use in facial recognition include video camera signals recognition include video camera signals and pre-existing photos such as those in and pre-existing photos such as those in driver's license databases. driver's license databases. including:including:Distance between the micro elementsDistance between the micro elementsA reference featureA reference featureSize of the micro elementSize of the micro elementAmount of head radiated from the face Amount of head radiated from the face

(unseen by human eye). Heat can be (unseen by human eye). Heat can be measured using an infrared camera. measured using an infrared camera.

Face FeaturesFace Features

5050

A face recognition based on local feature analysis

A face is represented as a graph, whose nodes, positioned in correspondence to the facial fiducial points. A fiducial point is a point or line on a scale used for

reference or comparison purposes.

A face recognition system uses an automatic approach to localize the facial fiducial points.

It then determines the head pose and compares the face with the gallery images.

This approach is invariant to rotation, light and scale.

5151A template for the 34 fiducial points on a face image:

5252

EBGM

Elastic Bunch-Graph Matching (EBGM) algorithm locates landmarks on an image, such as the eyes, nose, and mouth.

Gabor jets are extracted from each landmark and are used to form a face graph for each image. A face graph serves the same function as the projected vectors in the PCA or LDA algorithm; they represent the image in a low dimensional space.

After a face graph has been created for each test image, the algorithm measures the similarity of the face graphs.

Paper:Paper:http://www.snl.salk.edu/~fellous/posters/Bu97poster/BUPoster.pdf

5353

SummarySummary

Three algorithms have been introducedThree algorithms have been introducedEigenfacesEigenfaces

Reduce the dimension of the data from Reduce the dimension of the data from NN22 to to MMVerify if the image is a face at allVerify if the image is a face at allAllow online trainingAllow online trainingFast recognition of facesFast recognition of facesProblems with illumination, head pose etc Problems with illumination, head pose etc

5454

SummarySummaryFisherfaces

Reduce dimension of the data from Reduce dimension of the data from NN22 toto P-1 P-1Can outperform eigenfaces on a representative Can outperform eigenfaces on a representative

DBDBWorks also with various illuminationsWorks also with various illuminationsCan only classify a face which is “known” to DBCan only classify a face which is “known” to DB

Elastic Bunch-Graph MatchingReduce the dimension of the data from Reduce the dimension of the data from NN22 to to MMRecognize face with different posesRecognize face with different posesRecognize face with different expressionsRecognize face with different expressions

5555

ReferencesReferences

[1] M. Turk, A. Pentland, “Face Recognition Using Eigenfaces”[1] M. Turk, A. Pentland, “Face Recognition Using Eigenfaces”

[2] J. Ashbourn, Avanti, V. Bruce, A. Young, ”Face Recognition Based [2] J. Ashbourn, Avanti, V. Bruce, A. Young, ”Face Recognition Based on Symmetrization and Eigenfaces”on Symmetrization and Eigenfaces”

[3] http://www.markus-hofmann.de/eigen.html [3] http://www.markus-hofmann.de/eigen.html

[4] [4] P. Belhumeur, J. Hespanha, D. Kriegman, “Eigenfaces vs Eigenfaces vs Fisherfaces: Recognition using Class Specific Linear Projection”Fisherfaces: Recognition using Class Specific Linear Projection”

[5] R. Duda, P. Hart, D. Stork, “Pattern Classification”, ISBN 0-471-[5] R. Duda, P. Hart, D. Stork, “Pattern Classification”, ISBN 0-471-05669-3, pp. 121-12405669-3, pp. 121-124

[6] F.[6] F. Perronin, J.-L. Dugelay, “Deformable Face Mapping For Person Deformable Face Mapping For Person Identification”, ICIP 2003, BarcelonaIdentification”, ICIP 2003, Barcelona

[7] B. Moghaddam, C. Nastar, and A. Pentland. A bayesian similarity [7] B. Moghaddam, C. Nastar, and A. Pentland. A bayesian similarity measure for direct image matching. ICPR, B:350–358, 1996.measure for direct image matching. ICPR, B:350–358, 1996.

http://www.face-rec.org/interesting-papers/

5656

Hands-on Lab of Face BiometricsHands-on Lab of Face Biometrics

Present one of the following algorithmsElastic Bunch-Graph Matching (EBGM)

algorithmBayesian Intrapersonal/Extrapersonal

Classifier, orOne from

http://www.face-rec.org/interesting-papers/ Hands-on Lab of Face BiometricsHands-on Lab of Face Biometrics

http://www.cs.colostate.edu/evalfacerec/User Guide User Guide