Face RecognitionFace Recognition

IntroductionIntroduction

Why we are interested in face Why we are interested in face recognition?recognition?Passport control at terminals in Passport control at terminals in

airportsairportsParticipant identification in Participant identification in

meetingsmeetingsSystem access controlSystem access controlScanning for criminal personsScanning for criminal persons

Face Recognition Face is the most common biometric used by

humans Applications range from static, mug-shot

verification to a dynamic, uncontrolled face identification in a cluttered background

Challenges: automatically locate the face recognize the face from a general view point

under different illumination conditions, facial expressions, and aging effects

Authentication vs Identification

Face Authentication/Verification (1:1 matching)

Face Identification/recognition(1:n matching)

Applications

It is not fool proof – many have been It is not fool proof – many have been fooled by identical twinsfooled by identical twins

Because of these, use of facial biometrics Because of these, use of facial biometrics for identification is often questioned.for identification is often questioned.

ApplicationApplication Video Surveillance (On-line or off-line) http://www.crossmatch.com/facesnap-fotoshot.phphttp://www.crossmatch.com/facesnap-fotoshot.php

locates and extracts images from video footage for identification and verification

Why is Face Recognition Hard?

Many faces of Madonna

Why is Face Recognition Hard?

Face Recognition Difficulties

Identify similar faces (inter-class similarity)

Accommodate intra-class variability due to: head pose illumination conditions expressions facial accessories aging effects

Cartoon faces

Inter-class Similarity Different persons may have very similar appearance

Twins Father and son

Intra-class Variability Faces with intra-subject variations in pose,

illumination, expression, accessories, color, occlusions, and brightness

Sketch of a Pattern RecognitionArchitecture

Example: Face Detection Scan window over image. Classify window as either:

Face Non-face

Profile views

Schneiderman’s Test set as an example

Example: Finding skinNon-parametric

Representation of CCD Skin has a very small range of (intensity

independent) colors, and little texture Compute an intensity-independent color

measure, check if color is in this range, check if there is little texture (median filter)

See this as a classifier - we can set up the tests by hand, or learn them.

get class conditional densities (histograms), priors from data (counting)

Classifier is

Face Detection Algorithm

Face RecognitionFace Recognition

Face Recognition: 2-D and 3-D

Consider an n-pixel image to be a point in an n-dimensional space,

Each pixel value is a coordinate of x.

Image as a Feature Vector

nRx

Nearest Neighbor Classifier

Rj is the training datasetRj is the training dataset The match for I is R1, who is closer than R2The match for I is R1, who is closer than R2

Comments

Sometimes called “Template Matching” Variations on distance function

Multiple templates per class- perhaps many training images per class.

Expensive to compute k distances, especially when each image is big (N dimensional).

May not generalize well to unseen examples of class.

Some solutions: Bayesian classification Dimensionality reduction

Holistic or Appearance-based Face Holistic or Appearance-based Face recognitionrecognition EigenFaceEigenFace LDALDA

Feature-basedFeature-based

Face Recognition Face Recognition SolutionsSolutions

EigenFaceEigenFace

EigenFaceEigenFace Use Principle Component Analysis

(PCA) to determine the most discriminating features between images of faces.

The principal component analysis or The principal component analysis or Karhunen-Loeve transformKarhunen-Loeve transform is a is a mathematical way of determining that mathematical way of determining that linear transformation of a sample of linear transformation of a sample of points in points in LL-dimensional space which -dimensional space which exhibits the properties of the sample exhibits the properties of the sample most clearly along the coordinate axes.most clearly along the coordinate axes.

PCAPCA

http://www.cs.otago.ac.nz/cosc453/http://www.cs.otago.ac.nz/cosc453/student_tutorials/student_tutorials/principal_components.pdf principal_components.pdf

More New Techniques in More New Techniques in Face BiometricsFace Biometrics

Facial geometry, 3D face recognitionFacial geometry, 3D face recognition

http://www-users.cs.york.ac.uk/~nep/research/3Dface/tomh/3DFaceDatabase.html

3D reconstruction

Skin pattern recognitionSkin pattern recognition

using the details of the skin for using the details of the skin for authentication authentication

http://pagesperso-orange.fr/http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htmfingerchip/biometrics/types/face.htm

Facial thermogramFacial thermogram Facial thermogram requires an (expensive) Facial thermogram requires an (expensive)

infrared camera to detect the facial heat infrared camera to detect the facial heat patterns that are unique to every human patterns that are unique to every human being. Technology Recognition Systems being. Technology Recognition Systems worked on that subject in 1996-1999. Now worked on that subject in 1996-1999. Now disappeared.disappeared.

http://pagesperso-orange.fr/http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htmfingerchip/biometrics/types/face.htm

Side effect of Facial Side effect of Facial thermogramthermogram

can detect liescan detect lies The image on the left shows his normal The image on the left shows his normal

facial thermogram, and the image on the facial thermogram, and the image on the right shows the temperature changes right shows the temperature changes when he lied. when he lied.

http://pagesperso-orange.fr/http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htmfingerchip/biometrics/types/face.htm

Smile recognitionSmile recognition Probing the characteristic pattern of muscles Probing the characteristic pattern of muscles

beneath the skin of the face.beneath the skin of the face. Analyzing how the skin around the subject's Analyzing how the skin around the subject's

mouth moves between the two smiles. mouth moves between the two smiles. Tracking changes in the position of tiny wrinkles Tracking changes in the position of tiny wrinkles

in the skin, each just a fraction of a millimetre in the skin, each just a fraction of a millimetre wide. wide.

The data is used to produce an image of the face The data is used to produce an image of the face overlaid with tiny arrows that indicate how overlaid with tiny arrows that indicate how different areas of skin move during a smile. different areas of skin move during a smile.

http://pagesperso-orange.fr/http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htmfingerchip/biometrics/types/face.htm

Dynamic facial featuresDynamic facial features

They track the motion of certain features They track the motion of certain features on the face during a facial expression on the face during a facial expression (e.g., smile) and obtain a vector field that (e.g., smile) and obtain a vector field that characterizes the deformation of the face. characterizes the deformation of the face.

http://pagesperso-orange.fr/http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htmfingerchip/biometrics/types/face.htm