2010 5th International Symposium on Telecommunications (IST'2010)

A Low Complexity and Efficient Face Recognition Approach in JPEG Compressed

Domain Using Quantized Coefficients

Alireza Sepasmoghaddam

Islamic Azad University Qazvin Branch

Qazvin, Iran sepasmoghaddam@gmail.com

M. Shahram Moin Multimedia Systems Research

Group, IT Faculty

Hamidreza Rashidy Kanan

Electrical Engineering Department, Bu-Ali Sina University

Hamedan, Iran h.rashidykanan@basu.ac.ir

Iran Telecom Research Center Tehran, Iran

moin@itrc.ac.ir

Abstract- Computational and space complexities and storage space are amongst the most important issues in designing face recognition systems. A common method for storing images in face recognition systems is compressing images using JPEG standard. Usually, the compressed images are fully

decompressed for recognition, so that the recognition process is done in decompressed domain. This procedure causes a high computational overhead. In this paper, we have studied the face recognition procedure using quantized coefficients in

JPEG compressed domain for reducing the computational overhead caused by decompression process. In addition, in order to reduce the matching stage computational and space complexities, variance analysis and principle components

analysis methods on quantized coefficients have been applied to reduce the dimension of images subspace. The experiments in this research have been done on four datasets of FERET database. Experimental results show that the proposed method

outperforms existing methods in recognition rates, storage space, computational and space complexity aspects.

Keywords-Face recognitio; JPEG Compressed Domain;

Principle Components Analysis; Variance Analysis; FERET database (key words)

I. INTRODUCTION

Computational complexity, space complexity and storage space are amongst the most important challenges in software systems design. The direct relation between computational and space complexity and the real runtime of programs and also the relation between storage space and the necessary space for storing programs and data are to be considered in this challenge. Thus, the system with high space complexity, computational complexity and storage space will face many problems during implementation, particularly in on-line applications. This issue is quite tangible in biometric systems, since these systems should have suitable response times for input queries. Also, they need a considerable space for database information.

Simultaneously, with biometric systems improvements, there were different compressing methods and standards for various data including video, audio, images and ASCII codes. These compressing standards can be useful in decreasing the biometric systems storage space. One of the common compressing standards for still images is JPEG [1]. This standard is based on Discrete Cosine Transform (OCT) and encodes the images with considerable compression ratio,

978-1-4244-8185-9/101$26.00 ©2010 IEEE 781

without noticeable degradation in their quality. Nowadays, JPEG is considered as a common method for still images' compression in Internet, and most of the operating systems are able to decoding images encoded using this standard. Also, most of the modem cameras store directly the output images based on this method by embedded software or hardware.

Face recognition systems are a type of biometric systems that are widely used in legal and commercial applications with a very high acceptability [2]. Using the JPEG compressing standard, one can considerably solve the storing space problem in face recognition systems. However, since JPEG is a lossy compression method, a portion of information in the original image can be eliminated during encoding procedure. Thus, this omitted information is expected to affect the recognition results. However, some experiments have been done in pixel domains, showing surprisingly that not only using these compressing methods in suitable compressing rates does not reduce the recognition rates, but also improve it in some cases ([3],[5]and [11]). Despite this advantage, compression and decompression cause some problems. Block diagrams of coding and decoding in JPEG compression method are shown in Fig. 1. Traditionally, in face recognition systems, the compressed images must be fully decompressed, which means a considerable computational overhead in recognition process. To reduce the impact of this overhead, researchers have proposed to perform the face recognition in compressed domain.

In this paper, a new method for face recognition in JPEG compressed domain using quantized coefficients has been proposed for reducing the computational complexity, space complexity and storage space along with improving the recognition results. In addition, a comparison between different distance metrics and various coefficients' preselection in face recognition system performance has been done.

Block diagram of the proposed face recognition system is shown in Fig. 2. First, the uncompressed images are normalized and then, are JPEG compressed. At next step, the normalized and compressed images will be decompressed before inverse quantization stage. Then all quantized coefficients, or preselected coefficients using the variance analysis and the Principle Components Analysis methods

will be used to calculate the distance between the probe image and reference images, using different distance metrics.

The rest of this paper is organized as follows: Section II is dedicated to a review of related works. Section III contains a description of principles and methods used in this paper. Section IV presents simulations, results, comparisons and their analysis and Section V concludes the paper.

Header Tables

Data

Header Tables

Data

D

Fq(u. v)

D

(B)

Figure l. Block diagram of JPEG (A) coding (B) decoding procedures.

Calculating the Distance Between Probe hnages

and Refe."ellCe Images Space

Ranking and Evaluating

Figure 2. Block diagram of proposed face recognition system

782

II. RELATED WORKS

In this section, we present a review of researches done on face recognition in JPEG compressed domain. In [6], the DCT coefficients are used as input for HMM recognition algorithm [7]. The results show that using these coefficients up to 6% improvement in recognition rates can be obtained, compared to uncompressed domain recognition. In another research [8], there is a study based on DCT recognition coefficients with LDA [9] and PCA [10] feature extraction methods and the results show approximately 8% of improvement in frrst rank rate. However, in the most important study in this field [11], a comprehensive experiment in face recognition domain has been done before inverse DCT stage. In this study, researchers have used the FERET database [12] and PCA features with Cityblock distance metric and ICA [13] with Cosine distance metric. The results show that recognition using DCT coefficients in compressed domain cause improvements in recognition rates. According to JPEG compressing block diagram in Fig. 1, in previous works, face recognition has been done using DCT coefficients. In our presented method, to reduce more the computational overhead, face recognition is done using quantized coefficients, i.e. before inverse quantization stage. Consequently, in JPEG decoding, the inverse quantization will be avoided besides the inverse DCT.

III. DESCRIPTION OF METHODS

In this section, we will introduce the proposed methods and components used in our simulations. The block diagram of the proposed face recognition system is shown in Fig. 2. First, the images (8bit/pixel) in database will be normalized and registered (based on methods mentioned in section III­B), and then compressed with the compression rate of 8, which means 1 bit/pixel, by JPEG compression standard. In next step, decoding will be done before inverse quantization step. After fmding quantization coefficients, three different experiments will be simulated. First, all quantized coefficients are selected. In second method, the Principle Components Analysis method (section III-C), will be applied on coefficients. In third method, the analysis of variance method (section III-D) will be applied on quantized coefficients for preselecting the coefficients. Then different distance metrics (section III-E) are applied on subspaces obtained in three methods to perform the face recognition.

A. Face image database

FERET is the database of the face images used in this research. We used grayscale images of this database with size of 384x256 pixels and 256 gray level/pixel, i.e. 8bit/pixel. The experiments have been done on four datasets in a standard test space, including images of subjects taken under various conditions and times [4]. These are Fb (different expression test), Fc (different illumination), Dupl (images taken anywhere between one minute and 1,031 days after the gallery image) and Dup2 (images taken at least 18 months after the gallery image was taken). These datasets contain respectively 1195, 194, 722 and 234 images that are compared with 1196 reference images in Fa dataset.

B. Image registration and normalization

Before recognition process, face images should be normalized, so that the undesirable impact of superfluous elements can be eliminated. In this research, we have used the approaches presented in [14] for registering and normalizing the images. For this purpose, after placing eyes in one line in images, the distance between eyes is converted to 70 pixels, and then the picture size is modified to 152 x 128 pixels after moving eyes to fixed coordinates. Finally, the histogram equalization is used for images contrast adjustment.

C. Principle Component Analysis (PCA)

Principle component analysis (PCA), is a subspace projection technique that is widely used in face recognition. Given an s-dimensional vector representation of each face in a training set of images, PCA tends to fmd a t-dimensional subspace whose basis vectors correspond to the maximum variance direction in the original image space. This new subspace has smaller dimensions compared to primary space, due to omitting the principle components with smaller eigen values. The vectors obtained in this subspace should have the most information from the primary image. So the energy of image remains almost unchanged and the subspace of the images can be compared effectively in matching stage.

D. Variance Analysis

One of the most important methods for coefficients preselection in face recognition systems is using variance analysis in the pixels levels in images space. This idea is because that those parts of the face image with higher variance include more information. In this paper, 1196 reference images have been used for computing the variance of all coordinates of the image and only coefficients with highest variances have been selected for recognition. Different numbers of coefficients with highest variances are tested in experiments.

E. Distance metrics

One of the most important issues in face recognition systems is the selection of distance metric used in matching phase, for comparing a probe image with available images in reference images space. As mentioned before, the comparison between these metrics for selecting the most suitable metric(s) in different conditions is one of the goals of this research. For this purpose, the Cityblock (Eq.(1)), Euclidean (Eq.(2)), Cosine (Eq.(3)) and standard correlation in (Eq.(4)) have been used. In these metrics, the distance between two vectors, X and Y with n available elements, is calculated. Another metric is Spearman correlation [15]. For measuring distance with this metric, first the elements of X and Y vectors are sorted in ascending or descending order and correspondent components in primary vectors are replaced by their corresponding indexes in sorted vectors. Then, the distance between new vectors will be calculated by standard correlation using Eq.(4).

783

n dCityblock(X,Y) = �]x; - y;1

;=1

n dEuc!;dean (X, Y) = �)X; - y; f

;=1

n

LX;y;

dCosine (X, Y) = 1- r=::::::::;==I===

dCo"elat;on(X,Y) =

F. Ranking and performace evaluation

(1)

(2)

(3)

(4)

After calculating the distance between input face image (probe image) and reference images in compressed domain, a sorted list L = {Lh L2, ... , L,,} is obtained, where L1 is an image in the reference images set with maximum similarity with probe image and Ln is the image with minimum similarity. If the identity of L1 is equal to that of probe image, the algorithm has done the recognition in first rank correctly. In a similar manner, if the identity of Ln is equal to that of probe image, the algorithm has done the recognition in rank n correctly. In general, rank n of recognition is calculated per input images according to (5):

(5)

where Rn is the number of correct recognition in rank n on images probe collection and IFI is the total number of probe images.

Another evaluation metric is cumulative recognition rank n (Eq. 6), which has a suitable potential for presenting the recognition histogram and curves.

n CRRn= L RR;

;=1

(6)

IV. EXPRIMENTAL RESULTS AND ANALYSIS

A. Results of proposed method

The results of using all quantized coefficients with different distance metrics tested on four datasets Fb, Fe, Dupl and Dup2, are shown in Table I, and the results of applying PCA on quantized coefficients are shown in Table 2. Also, the experiments results using different numbers of preselected coefficients by variance analysis and Cityblock

and Spearman correlation distance metrics are shown in Tables 3 and 4, respectively. Selecting these two distance metrics is because of their better recognition rates compared to other distance metrics in last two experiments. In Tables 1-4, RRJ and CRR]o show the first rank recognition (Eq. 5) and the cumulative rank recognition to 20tli rank (Eq. 6), respectively. The best results in these tables are shown using bold and italic font, for each dataset.

TABLE I. RECOGNITION RATES WITH ALL QUANTIZED COEFFICIENTS, USING DIFFERENT DICTANCE METRICS, APPLIED ON 4 DATASETS OFFERET

Distance metric

Dataset Cityblock Euclidean Cosine Correlation Spearman

RRI CRRlo RRI CRR20 RRI CRRlo RRI CRRlo RRI CRRlo

Fb 83.2 95.4 75.3 92.2 75.4 92.2 75.4 92.4 83.5 94.8

Fe 72.2 91.2 68.1 90.2 68.1 90.2 69.2 91.2 63.5 89.2

Dupl 33.6 53.4 29.5 47.6 29.6 48.2 30.2 53.4 46.1 66.5

Dup2 19.7 38.1 17.6 30.8 17.1 32.5 17.6 38.1 29.1 49.6

TABLE II. RECOGNITION RATES WITH APPLYING PCA ON QUANTIZED COEFFICIENTS, USING DIFFERENT DICTANCE METRICS, APPLIED ON 4 DATASETS OF FERET

Distance metric

Dataset Cityblock Euclidean Cosine Correlation Spearman

RRI CRRlo RRI CRR20 RRI CRRlo RRI CRRlo RRI CRRlo

Fb 75.3 91.7 74.5 92.3 74.2 91.9 74.1 91.8 78 93.6

Fe 70.2 91.8 68.1 89.7 58.3 85.6 58.3 85.1 71.7 89.7

Dupl 30.2 50.2 30 47.6 29.3 46.4 29.6 46.3 38.9 55.9

Dup2 15.4 33.8 12.9 30.4 10.7 24 10.7 23.6 20.8 44.5

TABLE III. RECOGNITION RATES AFTER APPLYING VARIANCE ANALYSIS METHOD ON QUANTIZED COEFFICIENTS, USING CITYBLOCK DICTANCE METRIC, APPLIED ON 4 DATASETS OF FERET

The number of preselected coefficients

Dataset 512 1024 2048 4096 8192

RRI CRRlo RRI CRR20 RRI CRRlo RRI CRRlo RRI CRRlo

Fb 83.2 95.2 84 95.3 85.1 95.5 84.1 94.8 83.6 94.4

Fe 69.6 91.8 73.8 92.4 76.9 92.9 74.8 91.8 72.2 91.8

Dupl 31.5 49.1 33 51.6 33.4 53 33.3 52.4 33.2 52.2

Dup2 10.9 26.9 11.3 29.4 13.9 30.6 11.1 28.3 11.1 28.3

784

TABLE IV. RECOGNITION RATES AFTER APPLYING VARIANCE ANALYSIS METHOD ON QUANTIZED COEFFICIENTS, USING SPEARMAN CORRELATION DICT ANCE METRIC, APPLIED ON 4 OAT ASETS OF FERET

The number of preselected coefficients

512 1024 2048 4096 8192 Dataset

RRl CRRzo RRl CRRzo RRl CRRzo RRl CRRzo RRl CRRzo

Fb 76 92.2 79.7 93.3 84.8 94.8 85 94.8 84.3 94.4

Fc 61.4 82.5 63.5 84.6 67.6 88.7 67.6 88.7 61.1 88.2

Dupl 30.9 48.5 33.6 53.5 43.7 64.6 46.2 66.5 46.1 66.5

Dup2 25.7 48.6 26.5 53.1 27.3 52.9 28.3 53 27.8 50.9

TABLE V. FIRST RANK RECOGNITION RATES OF 3 DIFFERENT VARIANTS OF THE PROPOSED METHOD AND FIVE OTHER METHODS

Dataset PCA(3) LDA(3) ICA (3)

Fb 79.4 75.4 83

Fc 47.9 11.3 68.3

Dupl 38.5 35.6 44.3

Dup2 19.7 12.8 30.8

To analyze the results of using all quantized coefficients for recognition, the fIrst recognition ranks with different distance metrics are shown in Fig. 3. We can see that the Spearman correlation metric in 3 data sets Fb, Dup 1 and Dup2 has the highest fIrst rank recognition rates. In Fc data set, the Cityblock distance metric gives the best results. Results obtained by applying PCA method on quantized coefficients (Table 2) show that using PCA decrease the recognition rates. Contrary to PCA method, using coefficient preselecting method by variance analysis improves the recognition results along with an improvement in computational complexity.

-+- Cityblock -+- Euclidean --+- Cosine ___ Correlation - Spearm

�.----------------------------------------, I

80

70

60

50

40

30

20

10

O+---------�--------�--------_r--------� I Fb Fc Dupl Dup2

Figure 3. First rank recognition rates of different distance metrics on four datasets Fb, Fc, Dupl and Dup2

Method

Proposed Proposed Proposed OCT_PCA method method method

DCT_PCA With (Using all (Applying (Applying (11)

80.3

62.9

40.3

21.4

785

preselecting quantized PCA on variance (11) coer.) quantized analysis on

coer.) quantized coer.)

82.5 83.5 78 85.1

77.3 72.2 71.7 76.9

39.1 46.1 38.9 46.2

22.2 29.1 20.8 28.3

B. A comparison between proposed method and other related works

The results of the proposed method in face recognition are compared with those of related works, in Table 5 and Fig. 4. It can be seen that the proposed method outperforms three linear subspace methods, i.e. PCA, LDA and ICA, reported in [3]. Also, it is slightly better than [ 11], whilst it has a lower computational cost, thanks to avoiding inverse quantization.

__ PCA[3] - ICA[3] __ OCT _PCA]reselect[ I J] -t- Quantize_Preselect [proposed method]

__ LOA[3] __ OCT]CA[II] -.- All Quantize [proposed method] -Quantize_ PCA [proposed method]

90 .---�----------------------------------�

80

70

60

50

40

30

20

10

O+---------�--------�--------,_--------� Fb Fc Dup1 Dup2

Figure 4. First rank recognitiom rates of 3 different variants of the proposed method and five other methods

C. Computational and space complexity of proposed method

If we assume that the number of images' pixels is N, the number of reference images is G, and all the pixels of probe images are compared with their corresponding pixel in reference images, the time complexity order of this method is equal to O(NxG) per image in matching stage. Now, if we use the coefficients preselecting methods with P coefficient (P < N), only P coefficients will be compared instead of N coefficients. Therefore, the computational and space complexity order will be O(pxG), that cause aN IP speedup ratio in program runtime and a ratio of PIN reduction in required memory for running recognition algorithms.

Numerically, using 2048 to 4096 coefficients instead of 19456 coefficients, proposed method will have a speedup rate of approximately 5 to 10 times, compared to the standard method in [11].

Another improvement is related to decompression computational complexity. The complexities of the major operations in JPEG decompression are as follows: entropy decoding O(N log N), inverse quantization O(N) and inverse OCT O(N2), in standard implementation [11]. Obviously, by eliminating the inverse OCT and inverse quantization steps, the proposed method's complexity is O(N2 + N) lower than that of the recognition in pixel domain and O(N) lower than that of the other compressed domain recognition method, reported in [11].

V. CONCLUSIONS

In this paper, a new approach for face recognition in JPEG compressed images domain has been proposed. With simulating different variant of proposed method, the results were presented using different distance metrics on four datasets of FERET database and fmally these results were compared with those of similar related works.

We observed that results of face recognition using quantized coefficients in JPEG decompressed images are satisfactory, and avoiding inverse quantization and inverse OCT causes a computational complexity reduction up to, O(N2 + N) without any notable degradation in recognition results. Therefore, regarding number of preselecting coefficients by variance analysis method, it was found that using 2048 to 4098 coefficients in all the experiments, a reduction of 5 to 10 times in the computational cost is reachable. Spearman correlation and Cityblock metrics gave the best results on datasets of FE RET.

As future works, to reduce more the computational complexity, recognition in compressed domains on AC and OC coefficients will be studied. Also we will investigate the effects of illumination, pose angle, and occlusion in the new method's performance.

786

REFERENCES

[I] G.K. Wallace, "The JPEG still picture compression standard," Communications of the ACM, Vol. 34, No. 4, pp. 30-44,1991.

[2] A K. Jain, A Ross and S. Prabhakar, "An introduction to biometric recognition," IEEE Transaction on Circuits and Systems for Video Technology, Vol. 14, No. I, pp. 4-19, 2004.

[3] K.Delac and M. Grgic, "Image compression effects in face recognition systems," in: Face Recognition, Vienna: I-Tech Education and Publishing, 2007, pp. 75-92.

[4] PJ Phillips, H. Moon, S.A Rizvi and PJ Rauss, "The FERET evaluation methodology for face recognition algorithms," IEEE Transactions on Pattern Recognition and Machine Intelligence, Vol. 22, No. 10, pp. 1090-1104,2000.

[5] D. McGarry, C. Arndt, S. McCabe and D. D'Amato, "Effects of compression and individual variability on face recognition performance," SPIE Defense & Security Symposium, pp. 362-372, 2004.

[6] S. Eickeler, S. Muller and G. Rigoll, "High quality face recognition in JPEG compressed images," International Conference on Image Processing (ICIP'99), pp. 672-676, 1999.

[7] B. Moghaddam and Gregory Shakhnarovich, "Face recognition in subspaces," IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, pp. 696-710,2004.

[8] J Choi, YS Chung, KH Kim and JH Yoo , "Face recognition using energy probability in OCT domain," IEEE International Conference on Multimedia, pp. 249-256, 2006.

[9] P. N. Belhumeur, J. P. Hespanha and D. J. Kriegman, "Eigenfaces vs. fisherfaces: Recognition using class specific linear projection," IEEE Trans on Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, pp. 711-720, 1997.

[10] M. Turk and A Pentland, "Eigenfaces for recognition," Journal of Cognitive Neuroscience, Vol. 3, No. I, pp. 71-86, 1991.

[II] K Delac, M Grgic and S Grgic, "Face recognition in JPEG and JPEG2000 compressed domain," Image and Vision Computing, Vol.27, no. 8, pp. 1108-1120,2009.

[12] P. J. Phillips and P. Rauss. "The face recognition technology (FERET) program," CTAC International Technoogy Symposium, pp. 8-20, August 1997.

[13] M. Bartlett , J.R. Movellan and T. Sejnowski, "Face recognition by Independent component analysis," IEEE Transactions on Neural Network, Vol. 13, pp. 1450-1464,2002.

[14] PJ Phillips, H. Wechsler, J. Huang, and P. Rauss, "The FERET database and evaluation procedure for face recognition algorithms," Image and Vision Computing, Vol. 16, No. 5, pp 295-306,1998.

[15] J.1. Myers, AD. Well, Research Design and Statistical Analysis, 2nd edition, Lawrence Erlbaum, 2003, pp.508.