The Impact of Image Quality on the Performance of Face ... · The Impact of Image Quality on the...

The Impact of Image Quality on the Performanceof Face Recognition

Abhishek Dutta1, Raymond Veldhuis, Luuk Spreeuwers

University of Twente, Netherlands.

May 25, 2012

33rd WIC Symposium on Information Theory in the Benelux, May 24-25, 2012

1http://abhishekdutta.org

1 / 28

Introduction

What is the impact of image quality on the performance of acommercial face recognition system2?

Motion Blur (angle = 0)

length = 03 length = 17

Gaussian Noise (mean= 0)

var. = 0.007 var. = 0.3

Pose

Illumination

60 x 45 120 x 90

Resolution

2Cognitec FaceVACS SDK 8.4.0

2 / 28

Introduction

What is the impact of image quality on the performance of acommercial face recognition system2?




var. = 0.007 var. = 0.3

Pose

Illumination

60 x 45 120 x 90

Resolution

2Cognitec FaceVACS SDK 8.4.0

3 / 28

A Quick Question: Fill in the blankFace recognition is

I difficult

I very difficult

[MultiPIE]

face recognition performance depends on image quality of the pairof images participating in the comparison process.

4 / 28


I difficult

I very difficult

[MultiPIE]


5 / 28


I difficult

I very difficult

[MultiPIE]


6 / 28

Measure of Face Recognition Performance

7 / 28


8 / 28


9 / 28


10 / 28

Pose and Illumination

* : Illumination (flash) placed just above the respective camera

-

6

Test Illumination*

Ref

.Il

lum

ina

tio

n*

11 / 28


gradual performance degradation as ref. pose moves away from test pose12 / 28


gradual performance degradation as ref. pose moves away from test pose13 / 28

Pose and Illumination ...

I gradual reduction in recognition performanceas the reference set pose moves away fromthe pose in the test set

I role of illumination insignificant for same pose(and camera3) in the test and reference set

Test and reference images having similar pose and captured bysame camera can greatly improve recognition performance.

3in the MultiPIE data set, if we exactly match the pose, we are alsomatching all the imaging characteristics

14 / 28

Pose and Illumination ...

I gradual reduction in recognition performanceas the reference set pose moves away fromthe pose in the test set

I role of illumination insignificant for same pose(and camera3) in the test and reference set

Test and reference images having similar pose and captured bysame camera can greatly improve recognition performance.

3in the MultiPIE data set, if we exactly match the pose, we are alsomatching all the imaging characteristics

15 / 28

Resolution

19_0 05_1

0.5

0.6

0.7

0.840x30

60x45

80x60

100x75

120x90

160x120

200x150

640x480

40x30

60x45

80x60

100x75

120x90

160x120

200x150

640x480

Reference set resolution

Are

a U

nder

RO

C (

AU

C)

Test set res.

(dist. between

eyes in pixels)

40x30 (13)

60x45 (16)

80x60 (18)

100x75 (20)

120x90 (22)

160x120 (25)

200x150 (28)

640x480 (50)




var. = 0.007 var. = 0.3

Pose

Illumination

60 x 45 120 x 90

Resolution

Tes

t&

ref.

po

se

16 / 28

Resolution ...

19_0 05_1

0.5

0.6

0.7

0.840x30

60x45

80x60

100x75

120x90

160x120

200x150

640x480

40x30

60x45

80x60

100x75

120x90

160x120

200x150

640x480

Reference set resolution

Are

a U

nder

RO

C (

AU

C)

Test set res.

(dist. between

eyes in pixels)

40x30 (13)

60x45 (16)

80x60 (18)

100x75 (20)

120x90 (22)

160x120 (25)

200x150 (28)

640x480 (50)

Tes

t&

ref.

po

se

Analysis:I recognition performance improves with the resolution of the

test and reference set (as expected).I relative difference in pose between the test and reference

image determines the extent of influence of resolution.17 / 28

Noise

19_0 05_1

0.5

0.6

0.7

0.8

0 0.007 0.03 0.07 0.1 0.3 0 0.007 0.03 0.07 0.1 0.3

Ref. set Gaussian noise variance (mean = 0)

Are

a U

nder

RO

C (

AU

C) Test set Gaussian

noise variance

(mean = 0)

0

0.007

0.03

0.07

0.1

0.3




var. = 0.007 var. = 0.3

Pose

Illumination

60 x 45 120 x 90

Resolution

Tes

t&

ref.

po

se

18 / 28

Noise ...

19_0 05_1

0.5

0.6

0.7

0.8

0 0.007 0.03 0.07 0.1 0.3 0 0.007 0.03 0.07 0.1 0.3

Ref. set Gaussian noise variance (mean = 0)

Are

a U

nder

RO

C (

AU

C) Test set Gaussian

noise variance

(mean = 0)

0

0.007

0.03

0.07

0.1

0.3

Tes

t&

ref.

po

se

Analysis:

I recognition performance degrades with the noise in the testand reference set (as expected).

I relative difference in pose between the test and referenceimage determines the extent of influence of zero meanGaussian noise.

19 / 28

Blur

19_0 05_1

0.60

0.65

0.70

0.75

0.80

0.85

0 3 5 7 13 17 0 3 5 7 13 17

Ref. set motion blur length (angle = 0)

Are

a U

nder

RO

C (

AU

C) Test set motion

blur length

(angle = 0)

0

3

5

7

13

17




var. = 0.007 var. = 0.3

Pose

Illumination

60 x 45 120 x 90

Resolution

Tes

t&

ref.

po

se

20 / 28

Blur ...

19_0 05_1

0.60

0.65

0.70

0.75

0.80

0.85

0 3 5 7 13 17 0 3 5 7 13 17

Ref. set motion blur length (angle = 0)

Are

a U

nder

RO

C (

AU

C) Test set motion

blur length

(angle = 0)

0

3

5

7

13

17

Tes

t&

ref.

po

se

Analysis:

I recognition performance degrades with the blur in the testand reference set (as expected).

I relative difference in pose between the test and referenceimage determines the extent of influence of image blur.

21 / 28

Summary

Quality Difference in Area Under ROC

Pose ∼ 50%Resolution ∼ 35%Noise (Gaussian) ∼ 35%Blur (Motion) ∼ 20%Illumination ∼ 20%

I approximately matching non-frontal pose between test andreference images (and using same camera) can greatlyimprove recognition performance.

I it is the relative difference in pose between the test andreference image that determines the extent of influence thatother quality parameters like illumination, noise, motion blur,and resolution have on the face recognition performance

I face recognition performance depends on quality of image pair

22 / 28

Summary






23 / 28

Summary






24 / 28

Limitations

I We have assumed that the image quality parameters areindependent. In reality, all the quality parameters co-exist andpresence or absence of one quality parameter (like pose, blur,etc) might affect the behavior of other quality parameters(like resolution, noise, etc)

I All the images used in this study were taken from a singleimage data set. Although test and reference images differedby session, ideally both test and reference images should havebeen taken from the different data set in order to simulate theconditions present in a real forensic case.

I these findings are limited by the inclusion of a specificcommercial face recognition system in this study.

25 / 28

Limitations




26 / 28

Limitations




27 / 28

Thank You

For more details, refer to:

The Impact of Image Quality on the Performance of Face Recognition, A. Dutta, R. Veldhuis, L. Spreeuwers, 33rd

WIC Symposium on Information Theory in the Benelux, May 2012.

28 / 28

Notes for Q/A Session

Table: Image quality variations included in this study.

QualityCamera Flash Resolution Motion Blur Gaus. Noise

Testi , Refj Testi , Refj Testi , Refj Testi , Refj Testi , RefjPose and Illumina-tion

ci , cj ∈ C fi , fj ∈ F ri , rj = D0 0, 0 0, 0

Resolution 19 1, {∗} 18, {∗∗} ri , rj ∈ R 0, 0 0, 0

Gaussian Noise 19 1, {∗} 18, {∗∗} ri , rj = D0 0, 0 σ̄i , σ̄j ∈ Nσ̄

Motion Blur 19 1, {∗} 18, {∗∗} ri , rj = D0 li , lj ∈ Bl 0, 0

C = [19 1, 19 0, 04 1, 05 0, 05 1, 14 0, 13 0, 08 0, 08 1],F = [02, 04, 14, 05, 15, 06, 07, 16, 08, 09, 17, 10, 18, 12], R = [640× 480, · · · , 60× 45],Bl (length in pixels) = [1, 3, 5, 7, 13, 17, 21], Note: angle= 0Nσ̄ (variance) = [0.001, 0.007, 0.03, 0.07, 0.1, 0.2]. Note: mean = 0{∗} = {19 1, 05 1}, {∗∗} = {10, 07}, D0 = 640× 480

Table: Test and reference set specifications (source: MultiPIE)

Test Set (Probe) Reference Set (Gallery)

size (image count) 479 442person count 319 268session 01,03 02,04

expression neutral neutraleye annotation manual manual

29 / 28

Notes for Q/A Session ...

05_0 04_1 19_0

19_108_1

08_0 13_0 14_0 05_1

chair withhead rest

04 05 0607

08 09 10 1202

14 15 16 1718

camera

flash

Figure: Camera and flash location for all the images used in thisexperiment (source : MultiPIE data set)

30 / 28

Date post:	21-Jun-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

The Impact of Image Quality on the Performance of Face ... · The Impact of Image Quality on the...

Documents