+ All Categories
Home > Documents > [Lecture Notes in Computer Science] Adaptive Multimedia Retrieval: User, Context, and Feedback...

[Lecture Notes in Computer Science] Adaptive Multimedia Retrieval: User, Context, and Feedback...

Date post: 08-Dec-2016
Category:
Upload: marcin
View: 212 times
Download: 0 times
Share this document with a friend
11
3D Face Recognition by Modeling the Arrangement of Concave and Convex Regions Stefano Berretti, Alberto Del Bimbo, and Pietro Pala Dipartimento di Sistemi e Informatica University of Firenze Firenze, Italy Abstract. In this paper, we propose an original framework for three dimensional face representation and similarity matching. Basic traits of a face are encoded by extracting convex and concave regions from the surface of a face model. A compact graph representation is then con- structed from these regions through an original modeling technique ca- pable to quantitatively measure spatial relationships between regions in a three dimensional space and to encode this information in an attributed relational graph. In this way, the structural similarity between two face models is evaluated by matching their corresponding graphs. Experimen- tal results on a 3D face database show that the proposed solution attains high retrieval accuracy and is reasonably robust to facial expression and pose changes. 1 Introduction Representation and matching of face models has been an active research area in the last years, with a major emphasis targeting detection and recognition of faces in still images and videos (see [1] for an updated survey). More recently, the increasing availability of three-dimensional (3D) data, has paved the way to the use of 3D face models to improve the effectiveness of face recognition systems (see [2] for a recent survey). In fact, solutions based on 3D face models, feature less sensitivity—if not invariance—to lighting conditions and pose. This is particularly relevant in real contexts of use, where face images are usually captured in non-controlled environments, without any particular cooperation by human subjects. Generally, three main classes of approaches can be identified to distinguish the way in which 3D face models can improve face identification with respect to traditional solutions. A first class of approaches relies on a generic 3D face model to match two 2D face images. For example, in [3] a method is proposed for face recognition across variations in pose, ranging from frontal to profile views, and across a wide range of illuminations, including cast shadows and specular This work is partially supported by the Information Society Technologies (IST) Program of the European Commission as part of the DELOS Network of Excellence on Digital Libraries (Contract G038-507618). S. Marchand-Maillet et al. (Eds.): AMR 2006, LNCS 4398, pp. 108–118, 2007. c Springer-Verlag Berlin Heidelberg 2007
Transcript

3D Face Recognition by Modeling theArrangement of Concave and Convex Regions

Stefano Berretti, Alberto Del Bimbo, and Pietro Pala�

Dipartimento di Sistemi e InformaticaUniversity of Firenze

Firenze, Italy

Abstract. In this paper, we propose an original framework for threedimensional face representation and similarity matching. Basic traits ofa face are encoded by extracting convex and concave regions from thesurface of a face model. A compact graph representation is then con-structed from these regions through an original modeling technique ca-pable to quantitatively measure spatial relationships between regions in athree dimensional space and to encode this information in an attributedrelational graph. In this way, the structural similarity between two facemodels is evaluated by matching their corresponding graphs. Experimen-tal results on a 3D face database show that the proposed solution attainshigh retrieval accuracy and is reasonably robust to facial expression andpose changes.

1 Introduction

Representation and matching of face models has been an active research areain the last years, with a major emphasis targeting detection and recognition offaces in still images and videos (see [1] for an updated survey). More recently,the increasing availability of three-dimensional (3D) data, has paved the wayto the use of 3D face models to improve the effectiveness of face recognitionsystems (see [2] for a recent survey). In fact, solutions based on 3D face models,feature less sensitivity—if not invariance—to lighting conditions and pose. Thisis particularly relevant in real contexts of use, where face images are usuallycaptured in non-controlled environments, without any particular cooperation byhuman subjects.

Generally, three main classes of approaches can be identified to distinguishthe way in which 3D face models can improve face identification with respectto traditional solutions. A first class of approaches relies on a generic 3D facemodel to match two 2D face images. For example, in [3] a method is proposed forface recognition across variations in pose, ranging from frontal to profile views,and across a wide range of illuminations, including cast shadows and specular

� This work is partially supported by the Information Society Technologies (IST)Program of the European Commission as part of the DELOS Network of Excellenceon Digital Libraries (Contract G038-507618).

S. Marchand-Maillet et al. (Eds.): AMR 2006, LNCS 4398, pp. 108–118, 2007.c© Springer-Verlag Berlin Heidelberg 2007

3D Face Recognition by Modeling the Arrangement 109

reflections. To account for these variations, the algorithm simulates the processof image formation in 3D space, using computer graphics, and it estimates 3Dshape and texture of faces from single images.

A different class of approaches relies on using multiple imaging modalities inwhich information extracted from 3D shapes and 2D images of the face are com-bined together to attain better recognition results. In [4], face recognition in videosis obtained under variations in pose and lighting by using 3D face models. In thisapproach, 3D database models are used to capture a set of projection images takenfrom different point of views. Similarity between a target image and 3D models iscomputed by matching the query with the projection images of the models. In [5],Gabor filter responses in the 2D domain, and “point signature” in the 3D are usedto perform face recognition. Extracted 2D and 3D features are then combined to-gether to form an augmented vector which is used to represent each facial image.PCA-based recognition experiments, performed using 3D and 2D images are re-ported in [6]. The multi-modal result was obtained using a weighted sum of thedistances from the individual 3D and 2D face spaces. A large experimentation interms of number of subjects, gallery and probe images, and the time lapse betweengallery and probe image acquisition, is also presented in this work.

Finally, another class of methods relies on using only 3D shapes for the pur-pose of face recognition. Early works focused on the use of surface curvatureinformation and the Extended Gaussian Image, which provide one-to-one map-ping between curvature normals of the surface and the unit sphere. Followinga similar solution, 3D face recognition is approached in [7], by first segmentingthe shape based on Gaussian curvature, and then creating a feature vector fromthe segmented regions. This set of features is then used to represent faces inrecognition experiments. However, a key limitation of such approaches is that toenable reliable extraction of curvature data, accurate 3D acquisition is required.Other solutions have used registration techniques to align 3D models or clouds ofpoints. In [8], face recognition is performed using Iterative Closest Point (ICP)matching of face surfaces with resolution levels typical of the irregular pointcloud representations provided by structured light scanning.

In this paper, we propose an original solution to retrieval by similarity of3D faces based on description and matching of the relative position of salientanatomical facial structures. In the proposed model, these structures relate toconvex and concave regions that are identified on a 3D face by means of surfacecurvature analysis. Facial information captured by these regions is then repre-sented in a compact form evaluating spatial relationships between every pair ofregions. To this end, we propose an original modeling technique capable to quan-titatively measure the spatial relationships between three dimensional entities.The model develops on the theory of weighted walkthroughs (WWs), originallyproposed to represent spatial relationships between two-dimensional extendedentities [9]. Then, we show how to extend the model so as to capture relation-ships between 2D surface regions in a 3D space. Finally, mapping regions andtheir relationships to a graph model and defining a distance measure between3DWWs allows for the effective comparison of face models.

110 S. Berretti, A. Del Bimbo, and P. Pala

The paper is organized in four Sections and a Conclusion. In Sect.2, a methodis presented for extracting salient convex and concave regions from a dense trian-gular mesh. The theory of 3DWWs is then developed and proposed to representspatial relationships between surface regions in a 3D space. This enables theeffective representation of a face model through an attributed relational graphaccounting for face regions and their relationships. Based on this model, a simi-larity measure between 3DWWs is defined in Sect.3, and a method for the effi-cient comparison of graph representations of facial models is discussed in Sect.4.Face recognition results obtained on a 3D face database are reported in Sect.5.Finally, conclusions are outlined in Sect.6.

2 Extraction and Description of Convex and ConcaveFace Regions

The relative position and shape of convex and concave regions of a face, capturegeometric elements of a face that can be used to enable face identification. Lociof convex and concave surface have been intensively studied in connection withresearches on surface mathematics, [10,11], human perception of shapes, [12],quality-control of free-form surfaces, [13], image and data analysis [14], facerecognition [15] and many other applications.

In the proposed solution, identification of convex and concave surface regionsis accomplished by means of curvature based segmentation of model surface.For this purpose the mean shift segmentation procedure [16], [17] is adopted,so as to avoid use of a predefined—although parameterized—model to fit thedistribution of curvature values. Specifically, the mean shift procedure relies onestimation of the local density gradient. Gradient estimation is used within aniterative procedure to find local peaks of the density. All points that convergeto the same peak are then considered to be members of the same segment.

Use of the mean shift procedure to segment a 3D surface requires the definitionof a radially symmetric kernel to measure the distance—both spatially and inthe curvature space—between mesh vertices. This kernel is used to associatewith every mesh vertex vi a mean shift vector. During the iterative stage of themean shift procedure, the mean shift vector associated with each vertex climbsto the hilltops of the density function. At each iteration, each mean shift vectoris attracted by the sample point kernels centered at nearby vertices.

For 3D surface segmentation, the feature space is composed of two indepen-dent domains: the spatial/lattice domain and the range/curvature domain. Thus,every mesh vertex is mapped into a multi-dimensional feature point character-ized by the 3-dimensional spatial lattice and 1-dimensional curvature space. Dueto the different nature of the two domains, the kernel is usually broken intothe product of two different radially symmetric kernels (ks(.) and kr(.) are theprofiles of the kernel):

Khs hr(vi) =c

(hs)3(hr)ks

(∥∥∥∥xs

hs

∥∥∥∥2)

kr

(∥∥∥∥xr

hr

∥∥∥∥2)

3D Face Recognition by Modeling the Arrangement 111

where superscript ‘s’ refers to the spatial domain, and ‘r’ to the curvaturerange, xs and xr are the spatial and range parts of a feature vector, hs and hr

are the bandwidths in the two domains and c is a normalization coefficient.As an example, Fig.1 shows results of the detection of convex (a) and concave

(b) regions on a face model.Regions extracted from a 3D face are 2D surface portions in a 3D reference

space. Information captured by these regions is represented by modeling regionsand their mutual spatial relationships. To this end, we propose a theory of 3Dspatial relationships between surface entities, which develops on the model ofweighted walkthroughs (WWs) originally defined for two-dimensional extendedentities [9]. Description of spatial relationships through 3DWWs is invariant totranslation and scale but not to rotation. Therefore, in order to enable invarianceof face matching with respect to translation, scale and rotation, face models arefirst normalized: models are scaled and rotated so as to fit within a sphere ofunit radius centered at the nose tip and aligning the nose ridge along the Z axis.

(a) (b)

Fig. 1. Salient curvature extrema detected on a face model: triangles corresponding toconvex (a) and concave (b) regions

2.1 3D Weighted Walkthroughs

In a three dimensional Cartesian reference system, with coordinate axes X, Y, Z,projections of two points, a = 〈xa, ya, za〉 and b = 〈xb, yb, zb〉 on each axis,can take three different orders: before, coincident, or after. The combinationof the three projections results in 27 different three-dimensional displacements(primitive directions), which can be encoded by a triple of indexes 〈i, j, k〉:

i =

⎧⎨⎩

−1 xb < xa

0 xb = xa

+1 xb > xa

j =

⎧⎨⎩

−1 yb < ya

0 yb = ya

+1 yb > ya

k =

⎧⎨⎩

−1 zb < za

0 zb = za

+1 zb > za

In general, pairs of points in two sets A and B, can be connected throughmultiple different primitive directions. According to this, the triple 〈i, j, k〉, isa walkthrough from A to B if it encodes the displacement between at leastone pair of points belonging to A and B, respectively. In order to account forits perceptual relevance, each walkthrough 〈i, j, k〉 is associated with a weightwi,j,k(A, B) measuring the number of pairs of points belonging to A and B,whose displacement is captured by the direction 〈i, j, k〉.

112 S. Berretti, A. Del Bimbo, and P. Pala

The weight is evaluated as an integral measure over the six-dimensional setof point pairs in A and B (see Fig.2(a)):

wijk(A,B)=1

Kijk(A, B)

∫A

∫B

Ci(xb − xa)Cj(yb − ya)Ck(zb − za)dxadxbdyadybdzadzb

(1)

where Kijk(A, B) acts as dimensional normalization factor, and C±1(.) are thecharacteristic functions of the positive and negative real semi-axis (0, +∞) and(−∞, 0), respectively. In particular, C0(·) = δ(·) denotes the Dirac’s function,and acts as a characteristic function of the singleton set {0}. Weights between Aand B are organized in a 3×3×3 matrix (w(A, B)), of indexes i, j, k (see Fig.2).As a particular case, Eq.(1) also holds if A and B are coincident (i.e., A ≡ B).

In Eq.(1), the weights with one, two or three null indexes (i.e., wi,0,0, wi,j,0,wi,0,k, w0,j,0, w0,j,k, w0,0,k and w0,0,0) are computed by integrating a quasi-everywhere-null function (the set of point pairs that are aligned or coincidenthas a null measure in the six-dimensional space of Eq.(1)). The Dirac functionappearing in the expression of C0(·) reduces the dimensionality of the integra-tion domain to enable a finite non-null measure. To compensate this reduction,normalization factors Ki,j,k(A, B) (Ki,j,k in the following) have different dimen-sionality whether indexes i, j and k are equal to zero or take non-null values:

K±1,±1,0 = LALBHAHBDAB K±1,0,0 = LALBHABDAB K±1,±1,±1 = |A||B|K±1,0,±1 = LALBHABDADB K0,±1,0 = LABHAHBDAB K0,0,0 = (|A||B|) 1

2

K0,±1,±1 = LABHAHBDADB K0,0,±1 = LABHABDADB

(2)

where (see Fig.2(b)): |A| and |B| are the volumes of A, and B; LA, HA, DA, LB,HB and DB are the width, height and depth of the 3D minimum embeddingrectangles of A and B, respectively; LAB, HAB and DAB are the width, heightand depth of the 3D minimum embedding rectangles of the union of A and B,respectively.

Developing on the properties of integrals, it can be easily proven that thetwenty-seven weights of 3DWWs are reflexive (i.e., wi,j,k(A, B) = w−i,−j,−k(B, A)),and invariant with respect to shifting and scaling.

2.2 WWs Between 3D Surfaces

In that Eq.(1) accounts for the contribution of individual pairs of 3D points,computation of spatial relationships between surface entities in 3D directly de-scends from the general case. For 3D surfaces, Eq.(1) can be written by replacingvolumetric integrals with surface integrals extended to the area of two surfaces.

In practice, the complexity in computing Eq.(1) is managed by reducing theintegral to a double summation over a discrete domain obtained by uniformlypartitioning the 3D space. In this way, volumetric-pixels vxyz (voxels) of uniformsize are used to approximate entities (i.e., A =

⋃n An, where An are voxels with

a non-null intersection with the entity: vxyz ∈ {An} iff vxyz∩A �= ∅). According

3D Face Recognition by Modeling the Arrangement 113

(a) 0

01/4 1/2 1/4

0

<1, −1, 1>

B

Z

Y

A

X

w (A,B) =

w (A,B) =

w (A,B) =

k=−1

0 0 0000

1/4 1/2 1/4

k=0

0 0 000

1/2 1 1/2

k=1

0 0 00

.

.(b)

D

HA

ABD

ABL

HB

DB

BL

HAB

A

Y

X

Z

BAL

A

Fig. 2. (a) Walkthrough connecting a points in A with a point in B. The relationshipmatrix between A and B is expressed by three matrixes for k = 1, 0, −1, respectively.(b) Measures on A and B appearing in the normalization factors Kijk of Eq.(2).

to this, 3DWWs between A =⋃

n An, and B =⋃

m Bm can be derived aslinear combination of the 3DWWs between individual voxel pairs 〈An, Bm〉:

wijk(⋃n

An,⋃m

Bm) =1

Kijk(A, B)

∑n

∑m

Kijk(An, Bm) · w(An, Bm) (3)

as can be easily proven by the properties of integrals. Terms w(An, Bm), indi-cating 3DWWs between individual voxel pairs, are computed in closed form inthat they represent the relationships occurring among elementary cubes (voxels)and only twenty-seven basic mutual-positions are possible between voxels in 3D.

3 Similarity Measure for 3DWWs

Three directional weights, taking values within 0 and 1, can be computed onthe eight corner weights of the 3DWWs matrix (all terms are intended to becomputed between two surface regions A and B, i.e., wi,j,k = wi,j,k(A, B)):

wH = w1,1,1 + w1,−1,1 + w1,1,−1 + w1,−1,−1

wV = w−1,1,1 + w1,1,1 + w−1,1,−1 + w1,1,−1 (4)wD = w1,1,1 + w1,−1,1 + w−1,1,1 + w−1,−1,1

which account for the degree by which B is on the right, up and in front of A,respectively. Similarly, seven weights account for the alignment along the threereference directions of the space:

wH0 = w0,1,1 + w0,−1,1 + w0,1,−1 + w0,−1,−1 wHV0 = w0,0,1 + w0,0,−1wV0 = w1,0,1 + w−1,0,1 + w−1,0,−1 + w1,0,−1 wHD0 = w0,1,0 + w0,−1,0wD0 = w1,1,0 + w1,−1,0 + w−1,1,0 + w−1,−1,0 wV D0 = w1,0,0 + w−1,0,0

wHV D0 = w0,0,0

(5)

where wH0 , wV0 , wD0 measure alignments in which the coordinates X , Y andZ do not change, respectively; wHV0 , wHD0 , wV D0 , measure alignments where

114 S. Berretti, A. Del Bimbo, and P. Pala

coordinates XY , XZ and Y Z do not change, respectively; and wHV D0 accountsfor overlap between points of A and B.

Based on the previous weights, similarity in the arrangement of pairs of sur-faces (A, B) and (A′, B′) is evaluated by a distance D(w, w′) which combines thedifferences between homologous weights in the 3DWWs w(A, B) and w(A′, B′).In terms of the weights of Eqs.(4)-(5), this is expressed as:

D(w, w′) = λH |wH − w′H | + λV |wV − w′

V | + λD|wD − w′D|

+ λH0 |wH0 − w′H0 | + λV0 |wV0 − w′

V0 | + λD0 |wD0 − w′D0 |

+ λHV0 |wHV0 − w′HV0 | + λHD0 |wHD0 − w′

HD0 | + λV D0 |wV D0 − w′V D0 |

+ λHV D0 |wHV D0 − w′HV D0 |

where λH , λV , λD, λH0 , λV0 , λD0 , λHV0 , λHD0 , λV D0 and λHV D0 , are non-negative numbers with sum equal to 1.

Distance D can be proven to exhibit the five properties that are commonlyassumed as axiomatic basis of metric distances, i.e., positivity (D(w, w′) ≥ 0),normality (∀w, w′, D(w, w′) ≤ 1), auto-similarity (D(w, w′) = 0 iff w = w′),symmetry (D(w, w′) = D(w′, w)), and triangularity (D(w, w′) + D(w′, w) ≥D(w, w)). Each property is proven to separately hold for each of the distancecomponents, and it is then extended to the sum D.

In addition, due to the integral nature of weights wijk, D satisfies a propertyof continuity which ensures that slight changes in the mutual positioning or inthe distribution of points in two sets A and B result in slight changes in their3DWWs. If the set B is modified by the addition of Bε, the relationship withrespect to a set A changes up to a distance which is limited by a bound tendingto zero when Bε becomes small with respect to B. This has a main relevance inensuring robustness of comparison.

4 Matching Face Representations

According to the modeling technique of Sect.2, a generic face model F , is de-scribed by a set of NF regions. In that WWs are computed for every pairs ofregions (including the pair composed by a region and itself), a face is representedby a set of NF · (NF + 1)/2 relationship matrixes. This model is cast to a graphrepresentation by regarding face regions as graph nodes and their mutual spatialrelationships as graph edges:

Gdef= < N, E, α, β >, N = set of nodes

E ⊆ N × N = set of edgesγ : N → LN , nodes labeling functionδ : E → LE , edge labeling function

where LN and LE are the sets of nodes and edge labels, respectively. In ourframework, γ is the function that assigns to a node nk the self-relationshipmatrix w(nk, nk) computed between the region associated to the node and itself.In addition, γ associates the node with the area of the region and a type whichdistinguishes between nodes corresponding to concave and convex regions. The

3D Face Recognition by Modeling the Arrangement 115

edge labeling function δ assigns to an edge [nj , nk], connecting nodes nj and nk,the relationship matrix w(nj , nk) occurring between the regions associated tothe two nodes.

In order to compare graph representations, distance measures for node labelsand for edge labels have been defined. Both of them, rely on the distance measureD between 3DWWs defined in Sect.3.

Matching a template face graph T , and a gallery reference face graph R, in-volves the association of the nodes in the template with a subset of the nodes inthe reference. Using an additive composition, and indicating with Γ an injectivefunction which associates nodes tk in the template graph with a subset of thenodes Γ (tk) in the reference graph, this is expressed as follows:

μΓ (T, R)def=

λ

NT·

NT∑k=1

D(w(tk , tk), w(Γ (tk), Γ (tk))) + (6)

+2(1 − λ)

NT (NT − 1)·

NT∑k=1

k−1∑h=1

D(w(tk , th), w(Γ (tk), Γ (th)))

where the first summation accounts for the average distance scored by matchingnodes of the two graphs, and the second double summation evaluates the meandistance in the arrangements of pairs of nodes in the two graphs. In this equation,NT is the number of nodes in the template graph T , and λ ∈ [0, 1] balances themutual relevance of edge and node distance.

In general, given two graphs T and R, a combinatorial number of different in-terpretations Γ are possible, each scoring a different value of distance. Accordingto this, the distance μ between T and R is defined as the minimum under anypossible interpretation Γ : μ(T, R) = minΓ μΓ (T, R). In so doing, computation ofthe distance becomes an optimal error-correcting (sub)graph isomorphism prob-lem, which is a NP-complete problem with exponential time solution algorithms.Since the proposed modeling technique results into complete graphs with a rel-atively large number of nodes (i.e., typical models have more than 20 regions,almost equally divided between concave and convex regions), to improve thecomputational efficiency, we relaxed the requirement of optimality by acceptingsub-optimal matches. This is obtained by imposing that cross-matches betweennodes of different type is not allowed, and renouncing to include in the distanceminimization the relationships between nodes of different type. According to this,the distance μ(T, R) is computed as the sum of three separated components:

μ(T, R) = minΓa

[μΓa(Ta, Ra)] + minΓb

[μΓb(Tb, Rb)] + (7)

+ (1 − λ) · μs(w(Ta, Tb), w(Γa(Ta), Γb(Tb)))

where Ta, Ra and Tb, Rb are the sub-graphs composed by nodes of concaveand convex regions in the template and reference models, respectively (i.e., T =Ta ∪ Tb, R = Ra ∪ Rb). Optimal solutions minΓa and minΓb

in matching sub-graphs are computed by using the algorithm in [18]. Finally, the third term of

116 S. Berretti, A. Del Bimbo, and P. Pala

Eq.(7), accounts for the relationship distance occurring between concave nodesand convex nodes in the matched sub-graphs:

μs(w(Ta, Tb), w(Γa(Ta), Γb(Tb))) =1

NTa · NTb

· (8)

·∑

tk∈Ta

∑th∈Tb

D(w(tk , th), w(Γa(tk), Γb(th)))

Without loss of generality, Eqs.(6)-(8) assume that the number of nodes in thetemplate graph (NTa , NTb

), are not greater than the number of nodes in thereference graph (NRa , NRb

). In fact, if NTa > NRa or NTb> NRb

, graphs canbe exchanged due to the reflexivity of 3DWWs, and the normality in the sum oftheir eight corner weights.

5 Experimental Results

The proposed approach for description and matching of faces has been exper-imented using models from the GavabDB database [19]. This includes three-dimensional facial surface of 61 people (45 male and 16 female). The whole setof people are Caucasian and most of them are aged between 18 and 40. For eachperson, 7 different models are taken—differing in terms of viewpoint or facialexpression—resulting in 427 facial models. In particular, there are 2 frontal and2 rotated models with neutral facial expression, and 3 frontal models in whichthe person laughs, smiles or exhibits a random gesture. All models are auto-matically processed, as described in the previous sections, so as to extract agraph based description of their content, encoding prominent characteristics ofindividual convex and concave regions as well as their relative arrangement.

In order to assess the effectiveness of the proposed solution for face identi-fication, we performed a set of recognition experiments. In these experiments,one of the two frontal models with neutral expression provided for each personis assumed as reference (gallery) model for the identification. Results are givenin Tab.1 as matching accuracy for different categories of test models.

It can be noted that the proposed approach provides a quite high recognitionaccuracy also for variations in face expression.

Table 1. Matching accuracy for different categories

Test category Matching Accuracyfrontal - neutral gesture 94%frontal - smile gesture 85%frontal - laugh gesture 81%frontal - random gesture 77%rotated looking down - neutral gesture 80%rotated looking up - neutral gesture 79%

3D Face Recognition by Modeling the Arrangement 117

In Fig.3, recognition examples are reported for four test faces of differentsubjects. For each case, on the left the probe face is shown, while on the rightthe correctly identified reference face is reported. These models also provideexamples of the variability in terms of facial expression of face models includedin the gallery.

Fig. 3. Four recognition examples. For each pair, the probe (on the left) and the cor-rectly identified model (on the right) are reported

6 Conclusions

In this paper, we have proposed an original solution to the problem of 3D facerecognition. The basic idea is to compare 3D face models by using the infor-mation provided by their salient convex and concave regions. To this end, anoriginal framework has been developed which provides two main contributions.First, 3D face models are described by regions which are extracted as zones ofconvex and concave curvature of 3D dense meshes through a 3D mean-shift likeprocedure. Then, a theory for modeling spatial relationships between surfaces in3D has been developed, in the form of 3DWWs. Finally, we proposed a graphmatching solution for the comparison between 3DWWs computed on regionsextracted from a template model and those of reference models. The viability ofthe approach has been validated in a set of recognition experiments.

Future work will address an extended experimental validation in order tocompare the proposed approach with respect to existing solutions. How issues ofillumination and pose variations affect the performance of the proposed solutionwill be also considered.

References

1. Zhao, W., Chellappa, R., Phillips, P., Rosenfeld, A.: Face recognition: A literaturesurvey. ACM Computing Survey 35(4) (2003) 399–458

2. Bowyer, K., Chang, K., Flynn, P.: A survey of approaches to three dimensionalface recognition. In: Proc. International Conference on Pattern Recognition, Cam-bridge, UK (2004) 358–361

3. Blanz, V., Vetter, T.: Face recognition based on fitting a 3d morphable model.IEEE Transactions on Pattern Analysis and Machine Intelligence 25(9) (2003)1063–1074

4. Park, U., Chen, H., Jain, A.: 3d model-assisted face recognition in video. In: Proc.Canadian Conference on Computer and Robot Vision. (2005) 322–329

118 S. Berretti, A. Del Bimbo, and P. Pala

5. Wang, Y., Chua, C., Ho, Y.: Facial feature detection and face recognition from 2dand 3d images. Pattern Recognition Letters 23(10) (2002) 1191–1202

6. Chang, K., Bowyer, K., Flynn, P.: Face recognition using 2d and 3d facial data.In: Proc. Multimodal User Authentication Workshop. (2003) 25–32

7. Moreno, A., Sanchez, A., Velez, J., Dıaz, F.: Face recognition using 3d surface-extracted descriptor. In: Proc. Irish Machine Vision and Image Processing. (2003)

8. Cook, J., Chandran, V., Sridharan, S., Fookes, C.: Face recognition from 3d datausing iterative closest point algorithm and gaussian mixture models. In: Proc.Symp. on 3D Data Processing, Visualization and Transmission. (2004) 502–509

9. Berretti, S., Del Bimbo, A., Vicario, E.: Weighted walkthroughs between extendedentities for retrieval by spatial arrangement. IEEE Transactions on Multimedia5(1) (2003) 52–70

10. Koenderink, J.: Solid Shapes. MIT press, Boston, USA (1990)11. Porteous, I.: Differentiation for the Intelligence of Curves and Surfaces. Cambridge

University Press, Cambridge, UK (1994)12. Hoffman, D., Richards, W.: Parts of recognition. Cognition 18 (1985) 65–9613. Hosaka, M.: Modeling of Curves and Surfaces in CAD/CAM. Springer, Berlin

(1992)14. Eberly, D.: Ridges in Image and Data Analysis. Kluwer (1996)15. Halliman, P., Gordon, G., Yuille, A., Giblin, P., Mumford, D.: Two- and Three-

Dimensional Patterns of the Face. A.K. Peters, Natick, MA, USA (1999)16. Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space

analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(5)(2002) 603–619

17. Yamauchi, H., Lee, S., Lee, Y., Ohtake, Y., Belyaev, A., Seidel, H.P.: Feature sen-sitive mesh segmentation with mean shift. In: Proc. Shape Modeling International.(2005) 236–243

18. Berretti, S., Del Bimbo, A., Vicario, E.: Efficient matching and indexing of graphmodels in content based retrieval. IEEE Transactions on Pattern Analysis andMachine Intelligence 23(10) (2001) 1089–1105

19. Moreno, A., Sanchez, A.: Gavabdb: A 3d face database. In: Proc. 2nd COST275Workshop on Biometrics on the Internet, Vigo (Spain) (2004) 75–80


Recommended