Face Recognition:An Introduction

Face

Face RecognitionFace is the most common biometric used by humansApplications range from static, mug-shot verification to a dynamic, uncontrolled face identification in a cluttered backgroundChallenges: automatically locate the face recognize the face from a general view point under different illumination conditions, facial expressions, and aging effects

Authentication vs IdentificationFace Authentication/Verification (1:1 matching)

Face Identification/Recognition (1:N matching)

Applicationswww.viisage.com Access Controlwww.visionics.com

Applications Video Surveillance (On-line or off-line)Face Scan at Airportswww.facesnap.de

Why is Face Recognition Hard? Many faces of Madonna

Face Recognition DifficultiesIdentify similar faces (inter-class similarity)Accommodate intra-class variability due to:head poseillumination conditionsexpressionsfacial accessoriesaging effectsCartoon faces

Inter-class SimilarityDifferent persons may have very similar appearanceTwins Father and son www.marykateandashley.comnews.bbc.co.uk/hi/english/in_depth/americas/2000/us_elections

Intra-class VariabilityFaces with intra-subject variations in pose, illumination, expression, accessories, color, occlusions, and brightness

Sketch of a Pattern Recognition ArchitectureFeatureExtractionClassificationImage(window)ObjectIdentityFeature Vector

Example: Face DetectionScan window over image

Classify window as either:FaceNon-face

Detection Test Sets

Profile viewsSchneidermans Test set

Face Detection: Experimental Results Test sets: two CMU benchmark data setsTest set 1: 125 images with 483 facesTest set 2: 20 images with 136 faces[See also work by Viola & Jones, Rehg, more recentby Schneiderman]

Example: Finding skinNon-parametric Representation of CCDSkin has a very small range of (intensity independent) colors, and little textureCompute an intensity-independent color measure, check if color is in this range, check if there is little texture (median filter)See this as a classifier - we can set up the tests by hand, or learn them.get class conditional densities (histograms), priors from data (counting)

Classifier is

Figure from Statistical color models with application to skin detection, M.J. Jones and J. Rehg, Proc. Computer Vision and Pattern Recognition, 1999 copyright 1999, IEEE

Face Detection

Face Detection AlgorithmFace LocalizationLighting CompensationSkin Color DetectionColor Space TransformationVariance-based Segmentation Connected Component &GroupingFace Boundary DetectionVerifying/ WeightingEyes-Mouth TrianglesEye/ Mouth DetectionFacial Feature DetectionInput ImageOutput Image

Canon Powershot

Face Recognition: 2-D and 3-D

2-D

Face Database

2-D

RecognitionDataRecognitionComparisonPrior knowledgeof face class

Pose-dependentAlgorithmsPose-invariantPose-dependencyMatching featuresAppearance-based (Holistic)-- Gordon et al., 1995Feature-based (Analytic)HybridViewer-centered Images-- Lengagne et al., 1996-- Atick et al., 1996Object-centered Models-- Yan et al., 1996-- Zhao et al., 2000Face representation-- Zhang et al., 2000PCA, LDALFAEGBMTaxonomy of Face Recognition

Image as a Feature VectorConsider an n-pixel image to be a point in an n-dimensional space, x Rn.Each pixel value is a coordinate of x.

Nearest Neighbor Classifier

{ Rj } are set of training images.

CommentsSometimes called Template MatchingVariations on distance function (e.g. L1, robust distances)Multiple templates per class- perhaps many training images per class.Expensive to compute k distances, especially when each image is big (N dimensional).May not generalize well to unseen examples of class.Some solutions:Bayesian classificationDimensionality reduction

Eigenfaces (Turk, Pentland, 91) -1Use Principle Component Analysis (PCA) to reduce the dimsionality

How do you construct Eigenspace?[ ] [ ][ x1 x2 x3 x4 x5 ]WConstruct data matrix by stacking vectorized images and then apply Singular Value Decomposition (SVD)

EigenfacesModelingGiven a collection of n labeled training images,Compute mean image and covariance matrix.Compute k Eigenvectors (note that these are images) of covariance matrix corresponding to k largest Eigenvalues.Project the training images to the k-dimensional Eigenspace.RecognitionGiven a test image, project to Eigenspace.Perform classification to the projected training images.

Eigenfaces: Training Images[ Turk, Pentland 01

EigenfacesMean ImageBasis Images

Difficulties with PCAProjection may suppress important detailsmallest variance directions may not be unimportantMethod does not take discriminative task into accounttypically, we wish to compute features that allow good discriminationnot the same as largest variance

Fisherfaces: Class specific linear projection An n-pixel image xRn can be projected to a low-dimensional feature space yRm by

y = Wx

where W is an n by m matrix.

Recognition is performed using nearest neighbor in Rm.

How do we choose a good W?

PCA & Fishers Linear Discriminant Between-class scatter

Within-class scatter

Total scatter

Wherec is the number of classesi is the mean of class i| i | is number of samples of i..

PCA & Fishers Linear Discriminant PCA (Eigenfaces)

Maximizes projected total scatter

Fishers Linear Discriminant

Maximizes ratio of projected between-class to projected within-class scatter12

Four Fisherfaces From ORL Database

Eigenfaces and Fisherfaces

OO Face recognition can be categorized into appearance-based, geometry-based, and hybrid approaches.

OO Face recognition can be categorized into appearance-based, geometry-based, and hybrid approaches.

OO Face recognition can be categorized into appearance-based, geometry-based, and hybrid approaches.

OO Face recognition can be categorized into appearance-based, geometry-based, and hybrid approaches.

OO Face recognition can be categorized into appearance-based, geometry-based, and hybrid approaches.

Play this, its animated Green denotes topics covered subsequent slides22.10 - Two classes indicated by * and o; the first principal component captures all the variance,but completely destroys any ability to discriminate. The second is close to whats required.Use this slide to explain the three types of scatterPoint to emphasize is thatPCA maximizes the projected total scatter I.e., it presserves the information in the training set, and is optimal in a least squares sense for reconstruction. But in dropping dimensions, it may smear classes together.FLD trades off two desirable effects for recognition. A. The within class scatter over all classes is minimized this makes classification using nearest neighbor effective. B. The between class scatter is maximized this causes classes to be far apart in the feature space.