Person-Specific Domain Adaptation with Applications to Heterogeneous Face Recognition (HFR)

Person-Specific Domain Adaptation with Applications to Heterogeneous Face Recognition (HFR)

Presenter: Yao-Hung Tsai

Dept. of Electrical Engineering, NTU

Oral Presentation:

2014.05.02

Dept. of Electrical Engineering 2Yao-Hung Tsai (蔡曜宏 )

Outline

• Face Recognition• Conventional Approach• Heterogeneous Face Recognition• Domain Adaptation Approach• Proposed Approach

– Domain-independent Component Analysis– Person-specific Classifier– Combinational Framework

• Experiments


Outline



• Experiments


Heterogeneous Face Recognition

• Face Recognition – Face Identification– Identify the subject from the captured images



• Face Recognition – Face Verification– Verify a specific subject with respect to the captured image



• Face Recognition Application

Access Control System

Photo auto-tagging

Crime investigation

……


Outline



• Experiments


Conventional Approach• Direct method

– Direct compare two images based on their pixel values

v.s.

– Advantages :• Naïve, simple to implement

– Disadvantages• Require lots of computation effort


Conventional Approach• A common method : Eigenface method

– Representation: pixel intensity

– Collecting several images as the training set:

– Then we apply PCA to this set.

= …

n

d


Conventional Approach• PCA

– PCA projects columns of X from high-dimension ( ) to low dimension ( ).

– PCA make projection variance maximized by optimize:

– After solving the optimization we will get a set of basis vectors (faces):

– We can reconstruct the images by:

Ex: 2 dim to 1 dim

Note: v1 will capture most data variance


Conventional Approach– The combinational coefficients will be the new feature of face:

– For recognition, we simply project all images into this k-dimensional space and apply classifiers.

Note: Same class cluster together.


Conventional Approach• However, there exist several problems

– Traditional pattern recognition problems typically deal with• Training and test data collected from the same feature space

– In real word applications, these data are • Collected from different feature domains• Exhibiting distinct feature distributions

• We call this cross-domain recognition problems– Also called Heterogeneous Face Recognition (HFR)


Outline



• Experiments



• HFR is an emerging task in biometrics

Sketches in Criminal Cases Night Vision Camera



• Face recognition conduct on different domains



• When conventional FR meets HFR …– If directly apply PCA on images cross domains (e.x. infra-red v.s.

visible spectrum)– We visualize the data distribution of first 3 dimensions :

VISNIR

VIS Domain

NIR Domain


Heterogeneous Face Recognition– Observing the difference between domains :

• Instances with same class are far from each others.• Data from same domain close to each others.• That is,

• Domain difference dominates the data variance.– So, we need to conduct domain adaptation approach

• For comparing images from source and target domain

and


Outline



• Experiments


Domain Adaptation Approach• There are numerous approaches of domain adaptation

– Observing domain invariant features• Local Binary Patterns (LBP) – PAMI 2006

– Projecting images on common feature space• Canonical Correlation Analysis (CCA)• Partial Least Squares (PLS) – CVPR 2011• Semi-coupled Dictionary Learning (SCDL) – CVPR 2012• Coupled Dictionary Learning (CDL) – ICCV 2013

– Match distributions between cross domain images• Match marginal distributions (TCA) – TNN 2011• Match also joint distributions (JDA) – ICCV 2014


Domain Adaptation Approach• Illustrate the notation of external data

– Take access control system (ACS) as an example– At first, we usually cannot get the user’s images in advance– Thus, we need to use images from other subjects collected in

advance to model the system– Let us call it external data

Note: Images from both domains need to be collected.

External Data… …


Domain Adaptation Approach• Most of the approaches require a large number of paired

external data– However, it is very difficult to collect them !– Collecting external data with no labeled information is much

easier

• Moreover, direct use of external data might be non-preferable– There’s no guarantee of the same feature distribution among

external data and test data– For example, the common feature space observed from the face

images of females will not generalize well to those of males.


Domain Adaptation Approach• So, I proposed an approach with the following properties

– Require no labeled information in external data– Advocate the learning of person-specific domain adaptation

model for HFR– DiCA ( Domain-independent Components Analysis) is proposed

to build a common feature space


Outline



• Experiments


Domain-independent Component Analysis

• Review the observations in HFR problems–

– Domain difference dominates the data variance.• We check differences of projected means (MMD) for

every dimensions of PCA space:

and

…

: mean of NIR

: mean of VIS

MMD:

-



• Then we can discard the components with high MMD value.

• We get the final domain-independent projection matrix:

External Data… …

Domain-independent Components Analysis: DiCA



• So far, we can directly project user’s images to DiCA space and match test images.

• However, to address the issue that subjects from external data are different from users and to improve the performance.

• I further proposed– Person-specific Classifier (PC)


Outline



• Experiments


Person-specific Classifier• Forming a specific classifier for the input test image, for

this specific classifier outperforms than the general one• SVM (support vector machine) classifier is chose to be

this person-specific classifier– Choose test data as positive instance.– User defined negative instances could be chosen for different

usage.

Positive Negative

Person-specific classifier


Person-specific Classifier• Support Vector Machines (SVM)

– Classifier to discriminate two categories data– Training dataset xi A∈ + ⇔ yi = 1 & xi A∈ - ⇔ yi = -1


Person-specific Classifier• Goal : Predict the unseen class label for new data

– Find a function f : Rn → R by learning from dataf(x) ≥ 0 x A⇒ ∈ + and f(x) < 0 x A⇒ ∈ -

– Simplest function is linear : f (x) = w⊤x + b


Outline



• Experiments


Combinational Framework

External Data

…

…

NIR DataVIS DataUser’s images

…

Test

positive negative

Sim

ilari

ty S

core

User’s images

Form DiCA

Subspace

NIR

VIS


Outline



• Experiments


Experiments• Two HFR scenario:

– Photo – sketch (CUHK database)– VIS – NIR (CASIA 2.0 database)

• Identification Task– For photo-sketch, there are 100 gallery images and 100 test

images.– For VIS-NIR, there are 359 gallery images and 6200 test images

(with different occlusions)


Experiments• Two HFR scenario:

– Photo – sketch (CUHK database)– VIS – NIR (CASIA 2.0 database)

• Identification Task– For photo-sketch, there are 100 gallery images and 100 test

images.– For VIS-NIR, there are 359 gallery images and 6200 test images

(with different occlusions)


Experiments• Sketch-to-photo Dataset

• NIR-to-VIS Dataset


The End

Thank You!

Date post:	23-Mar-2016
Category:	Documents
Upload:	valin
View:	113 times
Download:	0 times

Person-Specific Domain Adaptation with Applications to Heterogeneous Face Recognition (HFR)

Documents