[IEEE 2013 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR) - Tianjin,...

Proceedings of the 2013 International Conference on Wavelet Analysis and Pattern Recognition, Tianjin, 14-17 July, 2013

JOINT IRIS AND FACIAL RECOGNITION BASED ON FEATURE FUSION

AND BIOMIMETIC PATTERN RECOGNITION

YING XU1,2, FEI LU01, YI-KUI ZHAI2 and JUN-YING GAN2

1 School of Automation, South China University of Technology, Guangzhou 510000, Guangdong, China 2 School of Information and Engineering, Wuyi University, Jiangmen 529020, Guangdong, China

E-MAIL: [email protected]@[email protected]@163.com

Abstract:

Fusion biometric recognition modal contributes in two as

pects. It can not only improve the biometric recognition accu

racy, but also gives a comparatively safe strategy, since it is dif

ficult for intruders to achieve multi-biometric information simul

taneously, especially the iris information. In this paper, a nov

el biometric fusion recognition modal with iris and facial images

based on biomimetic patteru recognition is proposed. The Con

tourlet transform (CT) and two directional two dimensional prin

cipal component analysis (2D)2PCA are used here to extract the

iris feature and the facial feature respectively, and a new fusion

feature vector was formed on the combination of the previous iris

and facial features. Lastly, the fusion feature vector is used to

construct the covering of high dimensional space using biomimet

ic patteru recognition method, in which the hyper-sausage neu

ron is adopted. Furthermore, a fixed random matrix is used here

to reduce the computational complexity and improve the recogni

tion efficiency. Experiments on the public union database show

that the proposed modal can achieve the state-of-the-art recog

nition accuracy while keeping the enrollment process safe.

Keywords:

Multi-modal biometric; Contourlet transform; (2D)2PCA;

Feature fusion

978-1-4799-0417-4/13/$31.00 ©2013 IEEE

1. Introduction

With the rapid development of the biometric recognition

technology, more and more multi-modal biometric recognition

systems [1] have been proposed for the following triple reason

s. Firstly, multi-modal biometric recognition techniques use

multi-source features, such as iris, face, finger print, palm, and

voice etc together in order to obtain integrated information of

the same object more essentially; secondly, it can improve the

recognition accuracy since it can fully utilize and fusing the

correlation information from different biometric images; Third

ly, it also gives a relatively safe strategy compare with the sin

gle biometric recognition, for single biometric information can

easily acquired by the intruders.

In the literature, there are several papers that address

multi-modal ear and face recognition. Superresolution based

multi-modal recognition has been proposed for face and iris

image fusion, and is shown to further improve recognition per

formance by capitalising on direct super-resolving the features

which are used for recognition [2] . Wang et. al [3] adopt an effi

cient feature-level fusion scheme for iris and face in series, and

normalizes the original features of iris and face using z-score

model to eliminate the unbalance in the order of magnitude and

the distribution between two different kinds of feature vectors,

and then connect the normalized feature vectors in serial rule,

202


which has proved to be effective. Lian et. al [4] proposes face

iris multimodal biometric system based on fusion at matching

score level using support vector machine (SVM). The perfor

mances of face and iris recognition can be enhanced using a

proposed feature selection method to select an optimal subset

of features. The results show that the proposed feature selection

method is able improve the classification accuracy in terms of

total error rate. The design of a multi-biometric system is de

pendent on the application. They differ from one another in

terms of their architecture, the number and the choice of the

biometric modalities and the methods used for the integration

tion, Hough transform and normalize the iris image by

the Daugman's Rubber Sheet Model, and extract the

iris feature with the Contourlet Transform.

Step 3 Apply the illumination normalization Walsh Hadamard

transform as the facial preprocessing, and then extrac

t (2D)2PCA vector of the training image as the facial

feature.

Step 4 Form a new feature vector based on the iris and facial

feature extracted above by the fusion strategy.

Step 5 Construct the High Dimensional Covering Space by

the new feature vectors trained by biomimetic pattern

or fusion of information. recognition algorithm.

In this paper, a multi-modal face and ear biometric system Step 6 Achieved the final recognition result by the judgment

is presented, the proposed biometric technique tends to make

full use of the Contourlet Transform [5] and (2D)2PCA [6-7]

to extract the features of iris and face images, and then a new

feature vector was form by the simply weighted concatenated

way. The novel fusion feature is sent to the high dimension

al covering space which is trained by the biomimetic pattern

recognition algorithm, to decide which class this sample be

longs with the rejected mode, rather than the traditional clas

sifier (such as K-nearest neighbor, SVM, etc. with only the

classification mode.

2. The Proposed Multimodal Biometric Recognition

Algorithm

of whether the probe image is in or out of the High

Dimensional Covering Space.

The Complex Geometry Covering in High

Dimensional Space

Figure 1. Framework of multi modal biometric

recognition process with iris and facial images

The multimodal biometric recognition framework presented in 2.1. Preprocess and Feature Extraction of Iris Images this paper is illustrated in Fig. 1, in which the Canny Edge De-

tection, Hough Transform and the Daugman's Rubber Sheet Iris recognition has been considered as an effective way for us

Model is utilized in the iris preprocess, while in the illumination er authentication. One important characteristic of the iris is

normalization Walsh-Hadamard transform is adopted in the fa- that, it is so unique that no two irises are alike, even among i

cial preprocess. Detail steps of multimodal biometric recogni- dentical twins, in the entire human popUlation [8] . The visible

tion process is as follows: characteristics of human iris, such as freckles, coronas, stripes

Step 1 Acquire an iris image and a facial image with cameras and et.al, which are generally called the texture of the iris, are

respectively. unique to each subject [9-10] . In this paper, the Canny Edge

Step 2 Preprocess the iris image with the canny edge detec- Detection and Hough Transform is adopted to estimate the iris

203


(g)

Figure 2. Preprocessing of the iris image, (a) In

put iris image; (b) Edge detected image; (c) lo

cate pupil and iris boundary; (d) Detected top

and bottom eyelid region; (e) Segmented iris im

age; (f) Normalized iris image

boundary, then the Daugman's Rubber Sheet Model is used to

normalize the iris image, and finally the Contourlet Transform

is employed to extract the iris feature. Details involve in prepro-

ber Sheet Model to achieve this goal.

In feature extraction process of the iris image, here we use the

Contourlet Transform, developed by Do and Vetterli [5] . It is a

new two-dimensional extension of the wavelet transform using

multi-scale and directional filter banks that can deal effective

ly with images having smooth contours. The Contourlet expan

sion is composed of basis images oriented at various directions

in multiple scales, with flexible aspect ratios. Given this rich set

of basis images, the contourlet transform effectively captures s

mooth contours that are the dominant feature in natural images.

The main difference between contourlets and other multi-scale

directional systems is that the contourlet transform allows for

different and flexible number of directions at each scale, while

achieving nearly critical sampling. Fig. 3 shows a multiscale

and directional decomposition using a combination of a Lapla

cian pyramid (LP) and a directional filter bank (DFB) . Band

pass images from the LP are fed into a DFB so that directional

cess and feature extraction of the iris image are given as below.

Preprocess of the iris image, as shown in Fig. 2, including the

following steps: information can be captured. The scheme can be iterated on Step 1 Edge Map Generation: due to the significant feature the coarse image. Specifically, let Specifically, let ao[n] be the

of handling spurious noisy images of canny operator, input image. The output after the LP stage is J bandpass images canny edge detection algorithm is used to generate the bj [n] ; j = 1,2, . . . ,J (in the fine-to-coarse order) and a lowpass edge map of the iris image here. image aj [n] . That means, the j-th level of the LP decomposes

Step 2 Circular Boundaries and Parameters Estimation: the the image aj-l [n] into a coarser image aj [n] and a detail image edge map above is used to locate the exact boundary bj[n] . Each bandpass image bj[n] is further decomposed by an of pupil and iris by Hough transform for its ability of ij-Ievel DFB into 21jbandpass directional images c1j j,k [n] ; k =

identifying positions of circular shapes, the parameters 0,1, . . . ,21j -1. of the radius and centre coordinates of the circle best

defined by the edge points can be achieved by the a 2.2. Preprocess and Feature Extraction of Facial Images maximum point in the Hough space.

Step 3 Eyelids and Eyelashes Isolation: the eyelids and eye- The illumination normalization of the facial image is crucial

lashes may corrupt the iris pattern, removal of such important in practical face recognition system, because it is

noises is essential for reliable iris information. Here, one of the key factors influences the recognition performance.

eyelids are isolated by fitting a line to the upper and Here, we adopted the block-wise two-dimensional Walsh

lower eyelid using the linear Hough transform, and a Hadamard transform (2D-WHT) to handle the illumination

threshold is used to isolate eyelashes. variation problem. Then a two-directional two-dimensional

Step 4 Iris image Normalization: normalization of the iris im- PCA, also named (2D)2PCA, proposed in [6] by Zhang and

age involves unwrapping the iris and converting it into Zhou, is used to extract the facial feature after the illumina

its polar equivalent. Here we use the Daugman's Rub- tion normalization process. (2D)2PCA, which considered si-

204


image

bandpass directional subbands

_,-----·IJ bandpass directional subbands

2.3. Fusion feature vector construction via combination

of the iris and facial vector

After extracting the iris and face feature via 2DPCA and Con

tourlet transform respectively, we can get a d dimensional fea

ture F face and x dimensional feature Firisrespectively, where

Fface={Fpl, Fp2, ... , Fpd}, Firis={FLl' FL2, ... , FLx}, then

the max-min regularization principle is adopted to normalize "' the feature vector, thus the following equation can be acquired

Figure 3. The contourlet filter bank: first, a mul

tiscale decomposition by the Laplacian pyramid

is computed, and then a directional filter bank is

applied to each bandpass channel.

multaneously the row and column directions was a generaliza

tion two-dimensional principal component analysis (2DPCA)

method [7], and reported to be efficient while the number of

coefficients is smaller than the previous method. The whole

process of illumination normalization and feature extraction

of facial images on combination of block-wise 2D-WHT and

(2D)2PCA, including the following steps:

Step 1 For a given facial image A', transform it into the loga-

I Fface - min(Fface) Fface = (F ) . (F ) max face - mln face

I Firis - min( Firis) Firis = (F ) . (F ) max iris - mln iris

(1)

(2)

Finally, the new fusion vector can be acquired via the weighted

concatenated way by the following equation

(3)

where Wl and W2 is the weight value of face and iris feature

while maintaining Wl +W2=1. The fusion feature F in equation

(3) will be the final multi-modal feature adopted in the follow

ing recognition process. For the high dimensionality brought

by feature fusion, we just simply use principal component anal

ysis (PCA) method to extract the statistically indenpendent fea

tures here.

rithm domain and divide it into 8x 8 blocks, apply the 2.4. Multi-modal Biometric Recognition via Biomimet-

WHT to each block. Zero k dimension low-frequency

WHT coefficients as the zigzag mode. ic Pattern Recognition

Step 2 Apply the inverse WHT to each block, the illumination Biomimetic Pattern Recognition (BPR) [11] was first proposed image A' can be achieved.

Step 3 Apply 2DPCA and Alternative 2DPCA to facial image,

we obtain projection matrices X and Z.

Step 4 Project the mby nimage A' onto X and Z simulta

neously, yielding a q by d matrix X and C where

C = ZT A' X, it is called the coefficient matrix in im

age representation and the feature matrix in face recog

nition. According to above steps, we can obtain the

facial feature vector by column remap the matrix C.

as a new pattern recognition model by academician Wang

Shoujue in 2002. Different from the "division" concept of tra

ditional pattern recognition, BPR emphasizes the view point of

the function and mathematical model of pattern recognition on

the concept of "cognition", which is much closer to the function

of human being.

In this paper, an n-dimensional hyper-sausages neuron is

applied in the implementation of BPR. According to the theo

ry of high dimensional hyper-surfaces, a neuron can construct

205


various types of complex closed hyper-surface. For easier im- respectively in the following experiments to evaluate the per

plementing the BPR, an n-dimensional hyper-sausages neuron formance of the proposed modal and algorithm. CASIA iris

is introduced here. So the union of hyper-sausages, the topo-

logical product of line segments and hyper-surface, can be a

suitable basic shape (sets Pi) to cover the region of samples

of the same class in the feature space approximately (set P' a ) .

Specifically, consider the original samples set Y, let Y' be a sub-

set of Y with} elements, as follows

B = {xl x = ¥i' (i = 0, 1,2, ... , n),

database collected by the Institute of Automation, Chinese A

cademy of Sciences [12] contains 756 gray-scale eye images

( 320x 280 pixels) with 108 unique eyes and 7 different im

ages of each eye, these images are mainly from Asians differ

from ages, and taken twice separately in two months, half in a

month. Here 700 images (100 unique eyes, each eye has 7 im

ages) were selected randomly to perform our experiments. For

the face images, we used a subset of AR face database here,

(4) since it is substantially more challenging. AR database [13]

where y' c Y, d is selected constant. Let} neurons cover P a

approximately, and then the covering of i-th neuron Pj is

consists of over 4,000 frontal images for 126 individuals. For

each individual, 26 pictures were taken in two separate session

s. These images include more facial variations, including illu

Pi = {xl = p(x,y) ::; k,y E Bi,x ERn}

Bi = { xl x = o:Y' + (1 - o:)¥i�l' 0: E (0,1) } The covering of all} neurons is: P� = ui�� Pi. The function of Hyper-Sausages Neuron (HSN) is

Where

d2(x,�) fHSN(X) = sgn(2 r6 - 0.5)

Ilx - xlI12,q(x, Xl, X2) < 0

Ilx - x2112,q(x, Xl, X2) > Ilx - x211 Ilx - xll12 - q2(X, Xl, x2),otherwise

q(X, Xl, X2) = \ X - Xl, II�: = ��II )

(5) mination change, expressions, and facial disguises comparing

to other existing databases. A subset of the data set consist(6)

ing of 50 male subjects and 50 female subjects (each person

has 7 images without occlusion) are chosen for the experiment.

Fig. 4 shows some examples of face and iris images in the ex

periments. All experiments in the iris feature extraction process

(7) using the contourlet transform, in the LP stage we use the "9-7"

filters. We choose "9-7" biorthogonal filters because they have

been shown to provide the best results for images, partly be-

(8)

cause they are linear phase and are close to being orthogonal.

In the DFB stage we use the "pkva" filters. We then choose

the Low frequency coefficients after 3 level Laplacian pyramid

(LP) decomposition as features of the iris image, and remap it

(9) into a vector. For the face feature extraction process, 90% of

Where x is feature vector of a testing sample. While Xl and X2 are feature vectors of two training samples which determine a

line segment. ( - ) in equation (9) represents the inner product

operation. II . II denotes 12 norm.

3. Experimental results

In this paper, we used the face and iris images from t

wo public standard AR face database and CASIA iris database

206

Figure 4. Some examples of face and iris images

from AR face database and CASIA iris database.


the energy of (2D)2PCA is adopted. All the experiments are all varying conditions of TTR*. By fusing iris and face biomet

implemented for 100 times, the recognition rate here takes the ric, the performance of the system can increase to 100% when

average results. the TTR * equals to 512 or 6/1. Though the single iris recogni-

Table 1. Results of average true recognition rate

(%) by the proposed single and multi-modal

based biometric recognition algorithm under d

ifferent training and testing sample ratio (TTR*)

without dimension reduction via 100 times inde

pendent experiments.

Recognition Modal TIR * 611 5/2 4/3 3/4 215 116

Face modal 98.69% 98.12% 96.75% 95.49% 93.28% 89.62%

Iris modal

Joint face and iris modal

100%

100%

100% 98.89% 97.63% 95.32% 93.25%

100% 99.54% 98.67% 98.73% 97.57%

Table 2. True recognition rate (%) comparison of

proposed algorithm and other algorithms under

different feature dimension via peA dimension

reduction with a training and testing sample ra

tio equal to TTR*=4/3.

Algorithm Dimension 64 128 256 512

peA plus BP - iris only 90.51% 93.37% 94.26% 94.88%

peA plus REF - iris only 90.69% 94.91% 95.01% 95.77%

PCA plus BP - face only 86.98% 90.71% 90.24% 92.66%

peA plus REF - face only 89.56% 92.45% 92.89% 93.09%

peA plus BP -Joint face and iris 94.94% 95.09% 96.58% 97.64%

peA plus REF -Joint face and iris 96.17% 97.73% 98.26% 98.85%

Proposed -Joint face and iris 98.81% 98.68% 99.15% 99.32%

We tested the performance of our approach for both sin-

tion modal can achieve 100% true recognition rate in the same

condition, the proposed multi-modal recognition modal outper

forms the iris one in a low TTR *. Thus it can overcome the con

dition of deficiency of training samples and maintain a robust

result in all conditions.

To further justify the performance of the multi-model bio

metric recognition system in this work, we compare its perfor

mance to the well known Back propagation (BP) and radius

basis function (RBF) neural network method. The results are

presented in Table 2. It indicated that the multimodal feature

fusion will benefit the recognition result. The proposed multi

model biometric recognition method outperforms than the sin

gle face or iris recognition model based biometric recognition

method, which yielded a true recognition of above 99% when

the dimension higher than 256 dimension. For the lower di

mensional feature, the joint face and iris modal also performs

better than the single one, though the recognition rate decrease

a little bit.

4. Conclusions

In this paper, a novel joint face and iris based biometric

recognition approach based on the Contourlet transform and

biomimetic pattern recognition was proposed. The iris and

face feature are extracted based on the Contourlet transform

and (2D)2PCA respectively, and then fused in a weighted con

catenated way to achieve robust performance to single biomet

ric variations. Finally, the high dimensional space covering

gle and multi-modal biometric recognition separately and then method based on biomimetic pattern recognition was adopted

compared their performances in terms of varying training and here to finish the recognition task. Experiments results show

testing sample ratio (TTR*). The results of our experiments that, the proposed multi-modal biometric recognition method

are reported in Table 1. It indicated that the joint iris and face can achieve relatively high performance, thus it could be wide

modal outperforms the single face modal and iris modal under ly used in personal authentication applications in the future.

207


Acknowledgement

This work is supported by National Natural Sci

ence Foundation Under Grand No.61072127; and Natural

Science Foundation of Guangdong Province, P.R.e. Un

der Grand NO.10152902001000 002, No.S2011040004211,

NO.S2011010001085 and No. 10151064101000000; and Foun

dation for Distinguished Young Talents in Higher Education

of Guangdong, China Under Grand NO.2012LYM _0127; and

Special fund of Zhujiang science and technology new star of

Guangzhou city NO.2011122 00084; and Science and technol

ogy plan project of Guangdong, No. 201IB080701045.

References

[1] A. K. Jain, A. Ross, Multibiometric systems, Communi

cations of the ACM, Special Issue on Multimodal Inter

faces, 2004, 47:34-40

[2] K. Nguyen, S. Sridharan, S. Denman, and e. Fookes,

"Feature-domain super-resolution framework for Gabor

based face and iris recognition," in Proc. of CVPR, 2012,

pp. 2642-2649

[3] Z. F. Wang, E. F. Wang, S. S. Wang, and Q. Ding, "Multi

modal Biometric System Using Face-Iris Fusion Feature,"

Journal of Computers, 2011, 6(5):931-938

[4] H. F. Lian, D. Isa, "Feature Selection for Support Vector

Machine-based Face-iris Multimodal Biometrci system,"

Expert Systems with Applications, 2011, 38: 11105-

11111

[5] M. N. Do and M. Vetterli, "The Contourlet transform:

an efficient directional multi-resolution image represen

tation", IEEE Transactions Image on Processing, 2005,

14(12):2091-2106

[6] D. Zhang and Z. H. Zhou, "(2D)2PCA: Two-directional

two-dimensional PCA for efficient face representation

and recognition," Neurocomputing, 2005, 69: 224-231

[7] J. Yang, D. Zhang, A. F. Frangi, and 1. Yang, "Two

dimensional PCA: A new approach to appearance-based

face representation and recognition," IEEE Trans. Pattern

Anal. Mach. Intell., 2004, 26(1):131-137

[8] J. Daugman, "High confidence visual recognition by test

of statistical independence", IEEE Transactions on Pat

tern Analysis and Machine Intelligence, 1993, 15:1148-

1161

[9] S. Lim, K. Lee and etc. , "Efficient iris recognition through

improvement of feature vector and classifier", ETRI Jour

nal, 2001, 23(2):123-134

[10] L. Ma, T. Tan, Y. Wang, and etc. , "Efficient Iris Recogni

tion by Characterizing Key Local Variations", IEEE Tran

s. Image Processing, 2004, 13(6):739-750

[11] Shoujue Wang, Jian Xu, et aI. , "Biomimeitic Pattern

Recognition-A New Model of Pattern Recognition The

ory and Its Applications", Acta Electronca Sinica, 2002,

30:2258-2262

[12] Institute of Automation, Chinese Academy of Sci-

ences, "CASIA iris database", http://nlprweb. ia

. ac.cnienglishiirds/irisdatabase.htm

[13] A. Martinez and R. Benavente, "The AR Face Database",

VC Technical Report 24, 1998

208

Date post:	07-Jan-2017
Category:	Documents
Upload:	doandiep
View:	214 times
Download:	1 times

[IEEE 2013 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR) - Tianjin,...

Documents