Post on 28-Oct-2014
transcript
A Project Report
On
College Attendance System based on Facial Recognition
Submitted to
Amity University, Uttar Pradesh
In partial fulfilment of the requirements for the award of the degree of
Bachelor of Technology
In
Computer Science & Engineering
By
Rohan Khurana
Under the guidance of
Ms. Divya Sharma
Department of Computer Science & Engineering
Amity School of Engineering & Technology
Amity University, Noida
Uttar Pradesh
April, 2012
ii
DECLARATION
I, Rohan Khurana, student of Bachelor of Technology (Computer Science &
Engineering) hereby declare that the project titled “College Attendance System based
on Facial Recognition” which is submitted by me to Department of Computer Science
& Engineering, Amity School of Engineering and Technology, Amity University
Uttar Pradesh, Noida, in partial fulfilment of the requirement for the award of the
degree of Bachelor of Technology in Computer Science & Engineering, has not been
previously formed the basis for the award of any degree, diploma or other similar title
or recognition.
Place: Noida
Date: 30th April 2012 Rohan Khurana
iii
CERTIFICATE
On the basis of declaration submitted by Rohan Khurana, student of B. Tech
(Computer Science & Engineering), I hereby certify that the project titled “College
Attendance System based on Facial Recognition” which is submitted to Department
of Computer Science & Engineering, Amity School of Engineering and Technology,
Amity University Uttar Pradesh, Noida, in partial fulfilment of the requirement for the
award of the degree of Bachelor of Technology in Computer Science & Engineering,
is an original contribution with existing knowledge and faithful record of work carried
out by him/them under my guidance and supervision.
To the best of my knowledge this work has not been submitted in part or full for any
Degree or Diploma to this University or elsewhere.
Noida (Ms. Divya Sharma)
Date Department of Computer Science and Engineering
Amity School of Engineering and Technology
Amity University Uttar Pradesh, Noida
iv
ACKNOWLEDGEMENT
“College Attendance System based on Facial Recognition” is a biometrics oriented
project aimed at presenting some concepts from various computer science disciplines
with a programming approach. An effort has been made to present this material in a
very lucid manner followed by scrutinized observation to every step of the source
code so implemented.
I take this opportunity to express my deep regards to respected Ms. Divya Sharma,
Dept. of Computer Science & Engineering, A.S.E.T, AUUP. I am indebted to her for
providing me the necessary counselling and support. I am deeply indebted to our
mentor Miss. Divya Sharma for encroachment upon her time. I would like to express
a heartfelt gratitude for her insight, expertise and energy contributed greatly to this
project. In addition to this I acknowledge my debt of gratitude to the numerous
authors whose great masterly work I have consulted during the preparation of this
project.
Last but not the least; I would like to thank my parents for their sound counselling and
cheerful support.
I hope that this project on “College Attendance System based on Facial Recognition”
in the present form caters to the needs of all concerned.
Rohan Khurana
v
ABSTRACT
This research aims at providing a system to automatically record the students’
attendance during lecture hours in a hall or room using facial recognition technology
instead of the traditional manual methods.
The objective behind this research is to thoroughly study the field if pattern
recognition (facial recognition) which is very important and is used in various
applications like identification and detection.
vi
CONTENTS
Declaration iii
Certificate iv
Acknowledgements v
Abstract vi
Contents vii
CHAPTER 1 Introduction 1
1.1 Background of the Study 1
1.2 Problem Definition 1
1.3 Proposed Scope and Enhancement 2
CHAPTER 2 Literature Review and Prior Research 3
2.1 Biometric Recognition 3
2.1.1 Emergence 3
2.1.2 Advantages 4
2.1.3 Types 5
2.1.4 Modules of a Biometric System 8
2.2 Facial Recognition 10
2.2.1 Automatic Facial Recognition Process Flow 10
2.2.2 Facial Recognition Techniques 12
2.2.3 Limitations of Face Recognition Technologies 14
CHAPTER 3 Materials and Methodology 18
3.1 System Development Requirements 18
3.1.1 Hardware 18
3.1.2 Software 18
3.1.3 Internet Connection 18
3.2 System Analysis and Design 19
3.2.1 Logical Data Flow (DFD) 19
3.2.2 Data Structures 23
3.2.3 Database Design(ERD) 24
vii
CHAPTER 4 Implementation 25
4.1 Introduction 25
4.2 Colour Based Techniques 25
4.2.1 Description 25
4.2.2 Problems Faced 26
4.2.3 Limitations 26
4.2.4 Recommendations 26
4.2.5 Flow Chart 27
4.3 Principal Component Analysis 28
4.3.1 Description 28
4.3.2 PCA based Face Recognition Algorithm 29
4.3.3 Calculation of Eigenfaces with PCA 35
4.3.4 Classifying Images 39
4.3.5 Input and Output 40
4.3.6 MATLAB Implementation 41
4.3.7 Database Implementation 41
CHAPTER 5 Discussion of Result 42
CHAPTER 6 Presentation of Result 43
6.1 The Basic User Interface 43
6.2 Average Face Output 44
6.3 Recognition Output 45
CHAPTER 7 Conclusions 46
CHAPTER 8 Future Prospects 47
CHAPTER 9 References 48
1
1. Introduction
1.1 Background of the study
The rapid development in the field of pattern recognition and its uses in different
areas e.g. (signature recognition, facial recognition), arises the importance of the
utilization of this technology in different areas in large organizations. This is mainly
because these applications help the top-management take decisions that improve the
performance and effectiveness of the organization. On the other hand, for an
organization to be effective, it needs accurate and fast means of recording the
performance of the people inside this organization. Biometric recognition has the
potential to become an irreplaceable part of many identification systems used for
evaluating the performance of those people working within the organization.
Although biometric technologies are being applied in many fields it has not yet
delivered its promise of guaranteeing automatic human recognition. This research is
the first of its kind to attempt to provide an automated attendance system that
recognizes students using face recognition technology through an image/video stream
to record their attendance in lectures or sections and evaluating their performance
accordingly.
1.2 Problem definition
Every time a lecture or laboratory starts, the lecturer delays the lecture to record
students’ attendance. This is a lengthy process and takes a lot of time and effort,
especially if it is a lecture with a huge number of students. It also causes a lot of
disturbance and interruption when an exam is held. Moreover the attendance sheet is
subjected to damage and loss while being passed on between different students or
teaching staff. And when the number of students enrolled in a certain course is huge,
the lecturers tend to call a couple of student names at random which is not a fair
student evaluation process either. Finally, these attendance records are used by the
staff to monitor the students’ attendance rates. This process could be easy and
effective with a small number of students but on the other hand, dealing with the
records of a large number of students often leads to human errors.
2
1.3 Proposed scope and enhancement
This project proposes solutions to all the above mentioned problems by providing an
automated attendance system for all the students that attend a certain lecture,
laboratory or exam at its specific time, thus saving time, effort and reducing
distractions and disturbance. Another advantage concerning exams, is when the
invigilator or the advisor accidentally loses an exam paper or the student lies about
attending the exam, there will be a record of the students’ attendance for the exam at
that time, thus protecting both lecturer’s and student’s rights. In addition, an
automated performance evaluation would provide more accurate and reliable results
avoiding human error.
3
2. Literature review and prior research
I have studied the literature available in the fields of pattern recognition and biometric
systems with a focus on facial recognition. In addition, a study of previous attendance
systems and its atomization attempts was conducted to examine the similar previous
systems. The literature is divided into two main parts; the first part examines the
different biometric characteristics and systems. In the second part, an in-depth study
on facial recognition is conducted for it is the technology to be used by our proposed
project.
2.1 Biometric recognition:
There are three different types of authentication: something you know, such as
passwords, something you have, such as badges or cards, and finally something you
are, which mainly depends on biometrics and physical traits. Each of these three
authentication methods has its advantages and disadvantages, and each is considered
appropriate for certain types of application.
Among there three types, scientists and researchers consider biometric recognition
systems as high-level security systems. They define the biometric term as the
“science, involving the statistical analysis of biological characteristics”. Biometric
recognition generally matches a live digital image of a portion from a certain physical
body part with a previously recorded image of that same portion; whether it is in
identification mode where one-to-many comparisons take place, or verification
(authentication) mode where one-to-one comparisons occur.
2.1.1 Emergence
For thousands of years, humans have used biometrics and physical characteristics
such as face and voice to identify other human beings for a wide variety of purposes,
ranging from simple business interactions to criminal investigations. The study of
fingerprint dates back to ancient China. While the ancient Jordanian have commonly
used physical traits such as height, scars, eye and hair colour to identify individuals
for business transactions. Generally speaking, the human mind automatically
recognizes different people depending on their external physical and behavioural
features.
4
2.1.2 Advantages
The main advantage of biometric systems over normal automated systems is that they
really do what they are supposed to do, which is authenticating the user, in a way
imitating the human capabilities; and using real human physical characteristics, which
are almost impossible to change. In addition, some researchers proposed that
biometrics are not subjected to theft, loss or passing to anyone else like what is done
with cards or passwords. While some others object and point out that they are not a
secret and could be in many cases falsified or stolen from computer systems.
Another advantage of using biometrics would be speed in doing work. For example,
authentication of a regular user with iris recognition would take two to three seconds,
while using a key to open the door would need five to ten seconds. Moreover the
normal human perception system has an impressive capability of recognizing and
distinguishing different faces, but this capability is limited by the number and type of
faces that can be easily processed. This limitation is overcome by the computerized
facial recognition systems that can mimic the human mind capability in addition to
storing and processing as much people as necessary. And finally, it is important to
point out that those technologies that are able to support biometric based applications
are becoming more and more available, which make biometric based systems more
available to users.
5
2.1.3 Types
Any human body characteristic that is considered universal, distinctive, almost
permanent, collectable and clear enough not to fool the capturing system, can be used
as a biometric feature if it is accepted to be used by humans. Besides that, biometric
systems now come in all different shapes and sizes. It ranges from hardware,
software, OEMs, certain modules or complete solutions. In this section of our
literature review, we will quickly go through the different biometric types used for
person detection.
2.1.3.1 The Eye
Analysing the eye is generally thought to present the highest levels of accuracy and
uniqueness. They can be divided into two different technologies: iris biometrics and
retina biometrics.
Iris: It is the coloured tissue representing the ring surrounding the eye pupil.
Each person’s iris has a unique structure and a complex pattern. In addition, it
is believed that artificially duplicating an iris is virtually impossible. It is also
known that the iris is from the first body parts decaying after death, therefore
it is unlikely that a dead iris could be used to by-pass a biometric system.
Retina: It is the layer of blood vessels situated at the back of the eye. Just as
the iris, the retina forms a unique pattern and decays quickly after death.
Retina recognition systems are complex but at the same time regarded as the
most secure biometric system.
2.1.3.2 Face
For a computerized system to mimic the human ability in recognizing faces,
sophisticated and complex artificial intelligence and machine learning is needed to be
able to compare and match human faces with different poses, facial hair, glasses, etc.
That is why these systems depend greatly on the extraction and comparison engines.
Different tools maybe used in these systems such as standard video, or still imaging or
thermal imaging.
6
2.1.3.3 Finger
In this type of study, the finger tips are analysed for its unique pattern and print
produced by the finger minutiae, in addition to finger geometry which concentrates on
the shape of the finger rather than the print itself.
2.1.3.4 Hand
There are mainly three biometric systems using the characteristics of the human hand.
These are: Hand geometry, Vein pattern analysis, and Palm identification.
Hand geometry: Just as with the finger geometry, the hand geometry
analysing system captures a three-dimensional image of the hand and
measures the shape and length of fingers and knuckles. Hand geometry was
from those first biometric systems used for access control applications.
Although it is not very accurate, it is thought to be very convenient and large
amounts can be processed quickly.
Vein pattern analysis: The past number of years has witnessed much
development in the area of analysing the patterns of the veins in the back of
the human hand. This technique is called the “vein check” which examines
the unique pattern of the blood vessels or what is called the “vein tree” which
can be captured by the infra-red light.
Palm identification: Palm biometrics is very similar to the study of finger
print; where the print and the patterns of ridges, valleys and minutiae found
on the palm are analysed and studied.
2.1.3.5 Voice
Voice biometrics focus on the sound of the voice, not on what is being said. That is
why it is important to distinguish between this technology and those technologies
recognizing words and commands. The sound of the human voice is caused by
vibration in the vocal chords. The shape and size of the vocal tract, in addition, the
shape of the mouth and the nasal cavities all contribute in the way a voice sounds.
Voice recognition techniques may either use text-dependent or text-independent
methods. In other words, voice may be captured by speaking out any phrase, word or
number (text-independent) or by specifically saying a certain password combining
phrases, words or numbers (text-dependent). However this biometric system may be
7
challenged by the background noise that reduces the quality of the data and the
system’s performance.
8
2.1.4 Modules of a Biometric System
Biometric systems are designed using the following four modules as explained in
figure 1:
Figure 1: Typical biometric recognition system
2.1.4.1 Sensor or Capturing Module
This module captures the biometric data. Different devices are used for different
biometric system according to the feature that is being captured; for example, video
camera, thermal camera, voice recorder, infrared… etc.
2.1.4.2 Feature Extraction Module
This module processes the captured biometric data to extract a set of discriminatory
features forming a template. For example: the position and orientation of minutiae
points in a fingerprint image are extracted in the feature extraction module of a
fingerprint-based biometric system.
2.1.4.3 Matcher Module
In this module, the features extracted from the previous module are compared to those
stored and decisions regarding matching score are made. For example: in a finger-
print based biometric system, the number of matching minutiae points determine the
matching score and consequently the match or not results.
9
2.1.4.4 System Database Module
It is used by the biometric system to store the templates of the enrolled users. The
enrolment module is responsible for enrolling individuals into the biometric system
database, where the biometric characteristic of an individual is first scanned by the
sensor to produce a digital representation (initial stored template) of the characteristic.
10
2.2 Facial Recognition
Face recognition is considered to be one of the most successful applications of image
analysis and processing. That is the main reason behind the great attention it has been
given in the past several years.
2.2.1 Automatic Facial Recognition Process Flow
Generally any biometric system goes through the same processes of the four modules
explained earlier, biometric capture, feature extraction, and comparison with
templates available in the database as explained in figure 1. The facial recognition
process is similar to the general biometric recognition process. In the face-base
biometric systems detection, alignment, feature extraction, and matching take place.
Figure 2.5.2 Face recognition processing flow
The facial recognition process can be divided into two main stages: processing before
detection where face detection and alignment take place (localization and
normalization), and afterwards recognition occur through feature extraction and
matching steps.
Face Detection: This process separates the facial area from the rest of the
background image. In the case of video streams, faces can be tracked using a
face tracking component.
Face Alignment: This process focus on finding the best localization and
normalization of the face; where the detection step roughly estimates the
position of the face, this step outlines the facial components, such as face
outline, eyes, nose, ears and mouth. Afterwards normalization with respect to
geometrical transforms such as size and pose, in addition to photometrical
properties such as illumination and grey scale take place.
11
Feature Extraction: After the previous two steps, feature extraction is
performed resulting in effective information that is useful for distinguishing
between faces of different persons and stable with respect to the geometrical
and photometrical variations.
Face Matching: The extracted features are compared to those stored in the
database, and decisions are made according to the sufficient confidence in the
match score.
12
2.2.2. Facial Recognition Techniques
This section gives an overview on the major human face recognition techniques that
apply mostly to frontal faces.
2.2.2.1 Eigen faces
Eigen face is one of the most thoroughly investigated approaches to face recognition.
This method would be less sensitive to appearance changes than the standard Eigen
face method.
2.2.2.2 Neural Networks
The attractiveness of using neural networks could be due to its non-linearity in the
network. In general, neural network approaches encounter problems when the number
of classes (i.e., individuals) increases.
2.2.2.3 Graph Matching
Graph matching is another approach to face recognition presented a dynamic link
structure. In general, dynamic link architecture is superior to other face recognition
techniques in terms of rotation invariance; however, the matching process is
computationally expensive.
2.2.2.4 Hidden Markov Models (HMMs)
Stochastic modelling of non-stationary vector time series based on (HMM) has been
very successful for speech applications applied this method to human face
recognition. Faces were intuitively divided into regions such as the eyes, nose,
mouth, etc., which can be associated with the states of a hidden Markov model.
2.2.2.5 Geometrical Feature Matching
Geometrical feature matching techniques are based on the computation of a set of
geometrical features from the picture of a face. Current automated face feature
location algorithms do not provide a high degree of accuracy and require considerable
computational time.
13
2.2.2.6 Template Matching
A simple version of template matching is that a test image represented as a two-
dimensional array of intensity values is compared using a suitable metric, such as the
Euclidean distance, with a single template representing the whole face. In general,
template-based approaches compared to feature matching are a more logical
approach.
2.2.2.7 3D Morphable Model
The morphable face model is based on a vector space representation of faces that is
constructed such that any convex combination of shape and texture vectors of a set of
examples describes a realistic human face.
14
2.2.3 Limitations and Challenges of Face Recognition Technologies
As mentioned earlier, face recognition technology, just as any other biometric
technology, has not yet delivered it promise. In spite of all its potentials, it is still
quite limited in its applied scope. Many researchers have identified different problems
for the biometric system; they can be categorized in four main challenges:
2.2.3.1 Accuracy
Two biometric samples collected from the same person are not exactly the same due
to the imperfect imaging conditions. In addition, the face recognition technology is
not robust enough to handle uncontrolled and unconstrained environments. In
consequence, the results accuracy is not acceptable.
As explained in figure 3, inaccuracy can occur in two different forms, either False
Non-Match (false reject / Type 1 error) in which the system falsely declares the
failure of match between the instance and the correct stored template; or False Match
(false accept / Type 2 error) in which the system incorrectly declares a successful
match between the instance and one of the templates in the database.
Figure 3: Biometrics System Errors
15
These errors are mainly caused by the complexity and difficulties of the recognition
process because of the uncontrollable variables such as lighting, pose, expression,
aging, weight gain or loss, hairstyle and accessories; figures 4, 5 and 6 present
examples for some types of variations. This challenge is reduced as more training data
is available to enhance the performance of the biometric system.
Figure 4: Position, lighting, expression
Figure 5: Occlusion/blockage/hiding of some features
Figure 6: Ageing
16
2.2.3.2. Scale
The number of entities enrolled in the database of the biometric system greatly affects
the speed, accuracy and performance in case of identification systems; where each
new instance is compared to all those in the database to find a match. Unlike the
verification where one to one comparison takes place. As mentioned in this article,
according to some studies conducted, a facial authentication takes about 90 msec,
while a large scale ID throughput needs about 0.66 min. That is why, in biometric
identification systems, it is absolutely necessary to find more efficient ways for
biometric comparison. This can be reached by performing course pattern
classification. For example: the captured face is firstly classified into male or female,
then, an age range can be determined… etc. In this way, the search in the system’s
database can be narrowed down in an efficient way. The large intra-class variation
still represents a great challenge in the field of pattern recognition, where many
studies are being conducted to find the best way for indexing patterns similar to the
ways used in conventional databases.
2.2.3.3. Security
Facial recognition and other biometric systems are used for many security
applications, claiming that biometrics is a secure way of authenticating access. But in
fact, security of biometrics (especially face), is very questionable. This is caused by
two main reasons:
Biometrics is not a secret: This means that anyone including the attacker
knows exactly the biometric features of the targeted user.
Biometrics is not recoverable: This means that one cannot change his face
in case it became compromised.
2.2.3.4 Privacy
The issue of using recognition based systems has raised many concerns of possible
privacy violations; which is a major concern in many locations, such as the American
Civil Liberties Union (ACLU) which opposes the use of face recognition software at
airports due to privacy concern. The database of a biometric system hold irrefutable
proof of one’s identity; and there are no regulations or guarantees on how these
17
information might be used or what it could be used for. These privacy issues mostly
result in the reluctance of users to use these biometric systems.
On the other hand some people propose that biometrics is a privacy protection tool
rather than intrusion to civil rights. This would be achieved through managing data
protection and encryption along the biometric system.
18
3 Materials and Methodology
This chapter enlists the system requirements for the development of this project as
well as the system designs implemented for the project.
3.1 System Development Requirements
3.1.1 Hardware
1) A camera
2) A desktop or laptop computer with the following specifications:
Intel core 2 duo processor with 2.4 MHz or higher clock rate and 2 MB or
higher cache
3GB RAM or higher
250GB storage space
3.1.2 Software
1) Operating system (Windows 7)
2) MATLAB
3) Database (MySQL)
4) Video for Windows (VFW)
5) Camera Driver
3.1.3 Internet Connection
1) Any way of connecting to the internet (For confirmation emails)
19
3.2 System Analysis and Design
3.2.1 Logical Data Flow (DFD)
The data flow diagram is a diagram that shows how the data manipulated by a system
flows through the various processes. It provides no information about the timing or
ordering of processes, or about whether processes will operate in sequence or in
parallel. It is therefore quite different from a flowchart, which shows the flow of
control through an algorithm, allowing a reader to determine what operations will be
performed, in what order, and under what circumstances, but not what kinds of data
will be input to and output from the system, nor where the data will come from and go
to, nor where the data will be stored (all of which are shown on a DFD). DFD consists
of four main symbols which are:-
External Entities
Data stores
Processes
Dataflow lines
The system will be illustrated on three different levels of DFD which are Context
diagram (level 0)( Figure 7) which consists of all the external entities, one main
process only and the data flow between them. It has no further details. Then is drawn
the next level of DFD which is High level diagram (level 1) ( Figure 8) in which the
data stores are shown and the main single process is broken down into the major high-
level, processes that are done by the system and the dataflow between them. The third
level of DFD, Low diagram (level 2) ( Figures 9 and 10) has a purpose of breaking
down the small, main processes into even smaller processes (children processes) to
simplify their work to become clearer to the reader. Together they wholly and
completely describe the parent process, and combined must perform its full capacity.
This decomposition of the parent process is called explosion of the process.
System with proposed automated attendance module: In the figures below is the
proposed system data flow that will be operating on automated facial recognition
attendance recording methods.
20
Context Diagram (Level 0):
Figure 7: (Context Diagram- Level 0)
High Diagram (Level 1):
Figure 8: (High Diagram- level 1)
21
Low level Diagram (Level 2): This level is the level where we explode or break
down the process into smaller ones. We will only examine the segmentation fig and
the face detection fig 3.4 processes because these are the processes where the
automation and most of the project work will be take place.
Figure 9: (Low Level Diagram- Level 2 (Process 2))
22
Figure 10: (Low Level Diagram- Level 2 (Process 3))
23
3.2.2 Data Structure
Course DS = Course ID + Course Name
Student DS = Student ID + First Name + Last Name + Email
Attendance = Attendance ID + Attendance Date
Attendance Date = Day + Month + Year
24
3.2.3 Database Design (ERD)
A diagram that pictorially represents entities, the relationships between them and the
attributes used to describe them in a database. The figure below illustrates the
attributes of each entity and the relationships between them. This figure shows the
types of the entities and their lengths, data types and relations between the tables.
Figure 11: (ERD)
25
4 Implementation
4.1 Introduction
In this project different techniques have being studied like colour based detection and
Principle Component Analysis (PCA) for face detection and for feature extraction,
PCA and Linear Discriminate Analysis (LDA). For detection, Colour based technique
was implemented, which depends on the detection of the human skin colour with all
its different variations in the image. The skin area of the image is then segmented and
passed to the recognition process. For recognition, PCA technique has been
implemented which a statistical approach that deals with pure mathematical matrixes
not image processing like the colour based technique used for detection. PCA can also
be used for detection.
4.2 Colour based technique
4.2.1 Description
In this technique the image is converted into a new colour space, HCbCr where H is
the Hue component of the HIS colour space and the Cb and Cr components are
calculated by the formulas stated in Equation 4.1.
Cb= (0.148 * R) - (0.291*G) + (0.439 * B) + 128
Cr= (0.439 * R) - (0.368 * G) - (0.071 * B) + 128
Equation 4.1
The new image is scanned pixel by pixel and if each pixel satisfies the condition
stated in Equation 4.2, which is the range of the human skin, its value is set to white
else it is set to black.
26
If (H(x,y)>=.01) && (H(x,y)<=0.1 ) &&
(Cr(x,y)>=140) && (Cr(x,y)<=165) &&
(Cb(x,y)>=140) && (Cb(x,y)<=195)
Set pixel value to white
Else
Set pixel value to black
Equation 4.2
The resultant image is converted into black white colour space in which pixels have
values of 0 and 1 only (black and white). The black white image is sent to the
segmenting function which gets the x axis and y axis coordinates of the face area to
crop it out of the original image to remove the rest of the data in the image that is not
important like the background.
4.2.2 Problems faced
After applying this technique there was a problem that there was noise that
tremendously affected the segmentation process so a noise removal filter was
implemented to successfully eliminate this problem. But it reduces the face area a
little, which is acceptable.
4.2.3 Limitations
1. This algorithm should be applied on one face at a time. It doesn’t support the
detection of multiple faces.
2. It is tremulously affected by the environment’s lighting conditions.
4.2.4 Recommendations
1. To use this technique the lighting conditions should be well controlled.
2. A better technique to solve these problems is to use PCA in the detection of the
face, which also supports multiple face detection.
27
4.2.5 Flow chart
Figure 12: Colour based technique
28
4.3 Principal Component Analysis
4.3.1 Description
Based on an information theory approach, PCA decomposes face images into a small
set of characteristic features images called “Eigenfaces” that can be described as the
principal components of the initial training set of images. In the language of
information theory, the extractions of the most applicable information in a face image
is desired, and then encoded as effectively as possible. Then it compares one face
encoding with a database of models encoded similarly. In PCA, one can transform
each original image of the training set into a corresponding eigenface. Therefore, one
important feature of PCA is that the reconstruction of any original image from the
training set by combining the eigenfaces is possible. Eigenfaces are the characteristic
features of the faces. Therefore one could say that the original face image can be
reconstructed form eigenfaces if one adds up all the eigenfaces (features) in the right
proportion. Each eigenface represents only certain features if the face, which may or
may not be present in the original image.
If the feature is present in the original image to a higher degree, the share of the
corresponding eigenface in the sum of the eigenfaces should be greater. On the other
hand if the particular feature is not present in the original image, the corresponding
eigenface should contribute a smaller part to the sum of eigenfaces. Indeed, in order to
reconstruct the original image form the eigenfaces, one has to build a kind of
weighted sum of all the eigenfaces. That is, the reconstructed original image is equal
to a sum of all eigenfaces, with each eigenface having a certain weight. This weight
specifies, to what degree the specific features (eigenface) is present in the original
image. If all the eigenfaces extracted from the original images are used, one can
reconstruct the original images from the eigenfaces exactly. But using only a part of
the eigenfaces is applicable. Hence, the reconstructed image is an approximation of
the original image. However, losses due to omitting some of the eigenfaces can be
minimized, which is achieved by selecting only the most important features
(eigenfaces).
29
Moreover, it is possible not only to extract the face from eigenfaces given a set of
weights, but to the extract the weights from eigenfaces and the face to be recognized.
These weights act as the amount by which the face differs from the “typical” face
represented by the eigenfaces. Therefore, using these weights one can determine two
important things:
1. Check whether the image is a face. In the case the weights of the image differ too
much from the weights of face images, the image probably not a face.
2. Similar faces (images) possess similar features (eigenfaces) to similar degrees
(weights). If weights from all images available are extracted, the images could be
grouped to clusters. Thus, all images having similar weights are likely to be a similar
face.
4.3.2 PCA-based face recognition algorithm
The approach to PCA-based face recognition involved the following initialization
operations:
4.3.2.1 Initialization
Acquire an initial set of face images (the training set). Calculate the eigenfaces from
the training set, keeping only the M images that correspond to the highest
eigenvalues. These M images define the face space. As new faces are experienced, the
eigenvalues can be updated or recalculated. Calculate the corresponding distribution
in M-dimensional weight space for each known individual, by projecting their face
images onto the face space. Having initialized the system, the following steps are used
to recognize new faces: Calculate a set of weights based on the input image and the M
eigenfaces by projecting the input image onto each of the eigenfaces. Determine if the
image is a face at all (whether known or unknown) by checking to see if the image is
sufficiently close to the face space. If it is a face, classify the weight pattern as either a
known person or as unknown. The coming sections will elaborate the steps needed to
perform the PCA using eigenfaces on a set of images in detail.
30
4.3.2.2 Main Algorithm phases
The previous initialization processes can be summed up into three main phases. Three
main functional units are involved in these phases. The characteristics of these phases
are described below:
4.3.2.2.1 Face database formation phase
During this phase, the gathering and the pre-processing of the face images that are
going to be added to the face database are performed. Face images are stored in a face
library (file system) in the system. Every action such as training set or eigenface
formation is performed on this face library. In order to start the face recognition
process, the face library has to be filled with face images. Weight vectors of the face
library members are empty until a training set is chosen and eigenvectors are formed.
4.3.2.2.2 Training phase
Images that are going to be in the training set are chosen from the entire face library.
After choosing the training set, eigenfaces are formed and stored for later calculations.
Eigenfaces are calculated from the training set, keeping only the M images that
correspond to the highest eigenvalues. These M eigenfaces define the M-dimensional
face space. When the new faces are acknowledged, the eigenfaces can be updated or
recalculated. The corresponding weight vector of each face library member has now
been updated.
4.3.2.2.3 Recognition and learning phase
After choosing a training set and constructing the weight vectors of face library
members, now the system is ready to perform the recognition process. The
recognition process in initialized by choosing the input image (the image, one seeks to
recognize). The weight vector is constructed with the aid of the eigenfaces that were
already stored during the training phase. After obtaining the weight vector, it is
compared with the weight vector of every face library member with a user defined
“threshold”. If there exists at least one face library member that is similar to the
acquired image within that threshold then, the face image is classified as known”.
Otherwise, a miss has occurred and the face image with its corresponding weight
vector for later use. This process is called learning to recognize.
31
4.3.2.3 The Use of Eigenfaces for Recognition
4.3.2.3.1. Overview of the algorithm using eigenfaces
The algorithm for the facial recognition system using eigenfaces is basically described
as follows:
First, the original images of the training set are transformed into a set of eigenfaces E.
Afterwards, the weights are calculated for each image of the training set and stored in
the set W.
When observing an unknown image X, the weights are calculated for that particular
image and stored in the vector Wx.
Afterwards, Wx is compared with the weights of images, of which one knows for
certain that they are faces (the weights of the training set W).One way to do it would
be to regard each weight vector as a point in space and calculate an average distance
D between the weight vectors from Wx and the weight vector of the unknown image
Wx (the Euclidean distance described in the appendix B would be a measure for that).
If this average distance exceeds some threshold value, then the weight vector of the
unknown image Wx lays too far apart form the weights of the faces. In this case, the
unknown X is considered not a face. Otherwise (if X is actually a face), its weight
vector Wx is stored for later classification.
4.3.2.3.2 Eigenvectors and eigenvalues definitions
An eigenvector of a matrix is a vector such that, if multiplied with the matrix, the
result is always an integer multiple of that vector. Its direction is not changed by that
transformation. This integer value is the corresponding eigenvalue of the eigenvector.
The corresponding eigenvalue is the proportion by which an eigenvector’s magnitude
is changed. This relationship can be described by the equation M * u =v * u, where u
is an eigenvector of the matrix M and v is the corresponding eigenvalues. This means,
an eigenvalue of 2 states that the length of the eigenvector has been doubled. An
eigenvalue of 1 means that the length of the eigenvector stays the same.
32
Eigenvectors possess following properties:
They can be determined only for square matrices.
There are n eigenvectors (and corresponding eigenvalues) in an n * n matrix.
All the eigenvectors are perpendicular, i.e. at right angle with each other.
4.3.2.3.3 Problems faced
Calculating the average and weights of the faces takes so much processing and time
when run for the first time to run the program.
4.3.2.3.4 Limitations
Non-uniform backgrounds and lighting conditions affect the recognition process.
4.3.2.3.5 Recommendation
Using black or uniform background used in the image.
Use sufficient light to illuminate the scene
33
4.3.2.3.6 Flow chart
Figure 13: PCA Training Phase
34
Figure 14: PCA Recognition Phase
35
4.3.3 Calculation of Eigenfaces with PCA
4.3.3.1 Basic Definitions
Let a face image I(x,y) be a two-dimensional N * N array of 8-bit intensity values. An
image can also be considered as a vector of dimension N2, so that a typical image of
256 * 256 becomes a vector if dimension 65,536 or equivalently a point in 65,536
dimensional space. The basic idea of the principal component analysis is to find the
vectors that best account for the distribution of face images within the entire image
space. These vectors define the subspace of face images, which is known as “face
space”. Each vector is of length N2, that describes an N * N image, and is a linear
combination of the original face images. As a matter of fact, these vectors are the
eigenvectors of the covariance matrix corresponding to the original face images, and
because they are face-like in appearance, we refer to them as “eigenfaces”. Some
examples of eigenfaces are shown in Figure 15.
Figure 15: sample eigenfaces form the sample training set
36
4.3.3.2 Computation steps
For being acquainted with the terms and notations that are going to be used during the
eigenfaces algorithm evaluation, refer to Appendices A and B.
Step 1: Gather and prepare the data.
Let the training set of face images be T1, T2… TM
Step 2: Subtract the mean.
The average matrix � has to be calculated, then subtracted from the original faces
(Ti) and the result is stored in the variable � i.
The average of the set is defined by
Equation 4.3
Each face differs from the average by the vector
Equation 4.4
37
Step 3: Calculate the eigenvectors and eigenvalues of the covariance matrix.
This set of very large vectors is then subject to principal component analysis, which
seeks a set of M orthonormal vectors, Um which best describes the distribution of the
data. The Kth vector, Uk, is chosen such that
Equation 4.5
is a maximum, subject to
Equation 4.6
The vectors uk and λk are the eigenvectors and eigenvalues.
Step 4: Calculate the covariance matrix.
The covariance matrix C is calculated according to
Equation 4.7
Where the matrix A= [� 1,� 2,� 3…� M]
38
Following the matrix analysis, the M * M matrix L= A TA is constructed, where
Lmn=� Tm� n , and the M eigenvectors, V1 of L is computed. These vectors
determine linear combinations of the M training set face images to form the
eigenfaces U1.
Equation 4.8
The success of this algorithm is based on the evaluation of the eigenvalues and the
eigenvectors of the real symmetric matrix L that is composed from the training set of
images. After this step, the “training” phase of the algorithm is accomplished.
39
4.3.4 Classifying Images
Classifying an image with eigenfaces
A new face image (Γ) is transformed into its eigenface e components (projected onto
“face space”)
Equation 4.9
The weights form a feature vector,
Equation 4.10
describes the share of each eigenface in representing the input face image, treating the
eigenfaces as a foundation set for image images. The face classes Wi can be
calculated by taking the average of the results of the eigenface representation over a
small number of face images (as few as one) of each individual. Classification is
performed by comparing the feature vectors of the face library members with the
feature vector of the input face image.
This comparison is based on the Euclidean distance between the two members to be
smaller than a user defined threshold �k. If the comparison falls within the user
defined threshold, then face image is classified as “known”, otherwise it is classified
as “unknown” and can be added to face library with its feature vector for later use,
thus making the system learning to recognize new face images.
40
4.3.5 Input and Output
The inputs of our system consist of bit streams representing the image to be analysed,
an average face, an Eigenvector matrix, and a set of projections. The image to be
analysed, as well as the average face, will consist of 122 x 92 =10304 pixels, each
being 8-bit grayscale (28=256 levels ranging from 0 to 255). These figures were
chosen because they provide a good balance between size and accuracy. Additionally,
these values were used successfully by many research groups. In a database with M
faces, the Eigenvector matrix will be of size 10304 x M. Finally, the set of projections
will consist of M vectors, each having M values.
The output of our system will consist of the face ID with the closest match, as well as
the image from the database with which the closest match is found. All this
information will be displayed on the GUI of the PC. It will always call a stored
procedure in the database with image ID as its parameter to record the student’s
attendance in the database.
Furthermore, an email sent to the student’s registered email address confirming his
attendance for the lecture.
41
4.3.6 Matlab Implementation
The PCA algorithm was implemented on MATLAB and tested using AT&T ORL
face database. This database is composed of 400 images of 112 x 92 pixels each,
where these images represent 40 individuals with 10 different images for each
individual, representing various poses and facial expressions. A set of 4 images out of
each individual’s 10 images was taken as the training set, which was chosen as an
intermediate number to properly evaluate the performance of the algorithm. The
remaining 6 images of the individual were used for the testing of the recognition
phase.
4.3.7 Database Implementation
A dummy database was implemented on MySQL workbench to support recording the
attendance of the students. The actual database will be the university’s database which
contains real data of the students, courses, timetables and attendances records.
42
5 Discussion of Result
The initial phase of the project was faced by many challenges concerning the
materials and the methodology to be used. Building the project step-by-step or unit-
by-unit created concerns regarding the division of whole project into smaller modules
and the way of integrating them into a one complete working project. Separating the
design from its implementation was yet another challenge.
Division of the entire project into various sub-modules was done by recognising
various tasks needed to be performed. Each task was then handled with a different
script.
Integration of these sub modules was done basically by providing calls to and from
various method. The final GUI was so constructed that it handled the final integration
of both of the most important tasks, namely, Average Face Calculation and
Recognition. The constructed GUI also included the ways to access the information
stored in the database at real time along with calling functions for updating the
database.
It was noted that when a legal image was introduced to the system, 9.5 out of 10 times
it responded with the right match.
It was noted that if a match was found, the student’s email address was retrieved from
the database and an email was sent to his address confirming his attendance in the
particular lecture.
At present, the project cannot implement its tasks on an image with multiple faces.
43
6 Presentation of Results
6.1 The basic user interface
Figure 16: Basic GUI
This is the basic Graphical User Interface of the project for the user. Here the blank
area is to present the result upon clicking of any of the 2 push buttons provided.
44
6.2 Average face output
Figure 17: Average face output
The use of push button with text “Press to find the average face” gives the above
output. This is the calculation of average face from all the images present in the test
data.
45
6.3 Recognition output
Figure 18: Recognition output
Use of “Press for Recognition” push button prompts the user to enter the image.
When the user enters an image, its difference with the average calculated in the
previous step is calculated and checked for an image with the minimum variation of
this difference from the test data.
46
7 Conclusions
The working model of this project can perform all the intended tasks on the standard
face image database provided by the AT&T ORL.
The project is able to perform all the tasks described in the problem description with
sustained accuracy.
Though it is not yet ready for the real time, online processing of class attendances, it
is one of the optimal basic systems to provide a path for such a project.
47
8 Future Prospects
Some of the future prospects for this system are:
Processing real time, online data with capturing live images.
Integrating time table processing with this system to provide a single system
for a college for its 2 major functions i.e. scheduling of classes and marking of
attendance.
Integrating it with faculty attendance system by providing 2 different
databases.
48
9 References
[1] Alterman, A (2003). “A piece of yourself”: Ethical issues in biometric
identification. In: Ethics and Information Technology, Vol 5, pp: 139–150.
[2] Anhonen, T, Hadid, A, Pietikainen, M (2006). Face Description with Local Binary
Patterns: Application to Face Recognition. IEEE members
[3] Bennett, M.J (1996). Time and Attedance System and Method therefor. US Patent.
[4] Chalker, Jr, Spooner R.J(1982). Automated time and attendance systems. US
Patent.
[5] Cherry, K (2003). Biometrics: An In Depth Examination. In: InfoSec Reading
Room, SANS Institure. Version 1.4b, Option 1 pp. 3-23
[6] Graham, D (2003). It’s All About Authentication. In: InfoSec Reading Room,
SANS Institure. Version 1.4b, Option 1 pp. 2-12
[7]Jain, A.K., Pankanti, S, Prabhakar, S, Hong, L, Ross, A, Wayman J.L. (2004).
Biometrics: A Grand Challenge. In: International Conference on Pattern Recognition,
Cambridge, UK, pp 1-10.
[8] Jain, K, Ross, A, Prabhakar, S (2004). An Introduction to Biometric Recognition.
In: IEEE Transactions on Circuits and Systems for Video Technology, Special Issue
on Image- and Video-Based Biometrics, Vol. 14 No. 1 pp 1-29
[9] Li, S.Z., Jain, A.K. (2004). Handbook of face recognition. Springer Science and
Business Media, Inc. New York, USA.
[11] Maytas V, Riha, Z. Biometric Authentication- Security and Usability. Faculty of
Informatics, Masaryk University Brno, Czech Republic.
[12] Roethenbaugh, G (2005). Biometrics Explained. In: InterNational Committee for
Information Technology Standards.
[13] Tsalakanidou, F, Malassiotis , S, Strintzis, M.G. Face Recognition - Face
Detection, Global Approaches for, Feature Based Techniques, Problems and
Considerations, Conclusions and Future Developments.
http://encyclopedia.jrank.org/articles/pages/6741/Face-Recognition.html
[14] Zhao, W, Chellappa, R, Phillips, P.J., Rosenfeld, A (2003). Face Recognition: A
Literature Survey. In: ACM Computing Surveys.