* Hema.B **Gopi - IJMRijmr.net.in/download.php?filename=IQonm0SrUNry3or... · 2.1 Eye-blink sensor...

IJITE Vol.01 Issue-01, (May, 2013) ISSN: 2321–1776

International Journal in IT and Engineering

http://www.ijmr.net 13

A NOVEL AWARENESS AND ALERTNESS IMPLEMENTATION ON BIOMETRIC AUTHENTICATION IN MOVING VEHICLE

* Hema.B

**Gopi.V

ABSTRACT

Driver drowsiness is among the leading causal factors in traffic accidents occurring worldwide. In

this project an advance robust wireless Bluetooth communication system is used to control the

vehicle and prevent the accident. There are two distinct methods which are eye movement

monitoring and bio-signal processing are used to monitor the driver safety through analyzing the

information related to fatigue. An infrared sensor and respiration, heart rate sensor are connected

with controller, which is continuously reading the bio signal of the driver. An interface Bluetooth

module continuously transmits the bio signal with the help of microcontroller. In the receiver side an

android based Dynamic Bayesian Network is used to monitor the information of the driver status. An

alertness alarm is initiated if the driver fatigue is believed to reach a defined threshold. If the driver is

not in a position to control the vehicle, the alert will be given to nearby following vehicles by proper

indication and the vehicle shall be stopped gradually.

General Terms - Android based smart phone, Dynamic Bayesian network, and fatigue.

* PG Scholar, PSN College of Engineering and Technology, Tirunelveli. ** Professor, PSN College of Engineering and Technology, Tirunelveli.




1. INTRODUCTION

Sensor and network-based information technology growth has widened the reach of wireless sensor

networks into countless areas such as healthcare monitoring, remote control monitoring, wildlife

monitoring, detection of military explosion, intelligent home monitoring devices, and environment

observation and forecasting system [1]–[2]. In 2012, National Highway Traffic Safety

Administration (NHTSA) records say that there were 33,808 vehicle causalities. Those accidents are

due to driver fatigue which was reported by over 56,000 people. This results in 1800 deaths, 73,000

injuries and 14.5 billion dollars public property loss. It is difficult to estimate that how many

accidents were really caused by driver fatigue. According to police, a fatigued driver and drunk

driver had a same behavior like reacting slowly; go off from lanes, and carelessly slowing or

speeding up the vehicle. Driver Fatigue is caused by four main reasons. The reasons are sleep, work,

physical condition, and time of day. Normally people work more in day time and taking a rest in

night time. If rest times not enough to a person, the fatigue will cause.

A fuzzy-control massage seat was developed to keep drowsy drivers awake by Lai et al. [3]. (Luis et

al. [4]) developed a nonintrusive prototype computer vision system for monitoring driver’s

attentiveness in real-time. A system with visual, cognitive, and decision making functions for elderly

drivers to recognize various objects encountered during driving was proposed by Kasukabe et al. [5].

A traffic-simulation model was designed in a vehicle which is equipped with an adaptive cruise-

control (ACC) and lane departure warning (LDW) system for monitor a driver behavior in a real

traffic environment by Pauwelussen et al. [6]. A system with two fixed cameras to capture images

of the driver and the road respectively, and then the images were mapped to global coordinates to

monitor the driver sight line is proposed by Lee et al. [7]. These authors found four distinctive

driving patterns through analysis by a hidden Markov model (HMM). The reliability of steering

behavior to detect driver fatigue by multi wavelet packet energy spectrum using a support vector

machine (SVM) was designed by Zhao et al. [8]. A video sensor based eye- tracking and blink-

detection system with Haar-like features and template matching for an automated drowsiness

warning system was developed by Lee et al. [9]. Drowsiness has a greater effect on rule-based




driving tasks than on skill-based tasks using a Bayesian network (BN) paradigm through simulator-

based human-in-the-loop experiments was proposed by Yang et al. [10].

A latent variable to represent the attributes of individual drivers for recognizing the emotional state

of drivers using four sensors, specifically for respiration, skin conductance, temperature, and blood

pressure was developed by Wang et al. [11]. The design of an electrocardiograph (ECG) and

photoplethysmograph (PPG) sensor to measure the driver’s metabolic condition was developed by

Shin et al. [12]. An overall design of classification based on multiple features such as

electroencephalography (EEG) signals, steering wheel correction movements, lateral position,

average velocity change trends and weaving, position within the traffic lane and analysis results on

recorded videos was presented by Bouchner et al. [13]. The drowsiness-related information extracted

from electrooculogram (EOG), EEG and ECG signals to classify driver attentiveness was maximized

by Khushaba et al. [14]. A brain-computer interface (BCI) system that can analyze EEG signals in

real time to monitor a driver’s physiological and cognitive states was proposed by Lin et al. [15].

Bundele et al. [16] proposed a neural network approach to classify mental fatigue and drowsiness in

driver, where they focused on skin conductance and pulse oximetry. A first-order HMM to compute

the dynamics of BN for compiling information about multiple physiological characteristics such as

ECG and EEG to infer the level of driver fatigue was designed by Yang et al. [17]. Meanwhile, a

system to analyze a driver’s eye-lid movement, jaw movement, and variation in pulse was developed

by Deshmukh et al. [18]. An intelligent system that compiled physiological data acquired from a

sensor on the steering wheel, as well as mechanical data from a simulation platform to evaluate a

driver’s level of attentiveness was developed by Giusti et al. [19]. Additionally, a method for

detecting a driver’s distraction and drowsiness levels by analyzing several parameters using an

artificial neural network (ANN) was proposed by Eskandarian et al. [20]. Liang et al. [21] proposed

similar fusion approaches can be applied using SVM, which is a data mining method for detecting

cognitive distraction using driver eye movement. Driver stress in term of physical appearance using a

visual sensor, physiological conditions collected from emotional mouse, and behavioral data from

user interaction activities by using DBN was developed by Zhang et al. [22].




There is more number of methods for detection of driver fatigue in real time. The first method is

monitoring the driver’s physiological signals such as brain waves, heart rate, respiration rate, as well

a variety of physiological signals. The second factor is monitoring the physical changes of driver

such as mouth for yawning, head position, sagging posture, eye open or close status, and a variety of

other factors. The third method is sensing the driver operation and vehicle behavior such as steering

wheel movement and driver condition. The fourth method is monitoring the response of the driver. In

these paper only two methods is taken for driver fatigue detection. These two method is more

reliable, robust and non intrusive. This makes the driver to feel most comfortable.

2. SYSTEM ARCHITECTURE

There are four modules consists of hardware and software for driver monitoring complete system

design. The modules are bio sensor module, microcontroller module, and android application

development and information fusion by DBN.

Fig 1: Block diagram overview of system architecture

The DBN in android based smart phone receives the bio signal value from Bluetooth transmitter.

DBN fusions the parameter and calculate the probabilistic value. The DBN gives the final output as

the status of driver alertness. The warning alarm is initiated to alert the driver if the output reached

particular threshold.




2.1 Eye-blink sensor

Eye-blink sensor is used to monitor the alertness of driving person through eye lid movement status.

When the person feels drowsy or when he is unconscious the eye lid will be in closed position. An IR

sensor consists of IR transmitter and receiver. IR receiver is used as the sensing unit. The IR

transmitter continuously transmits the infrared rays towards the eye ball. When the eye lid is closed

due to drowsiness the signal is reflected by the eye lid to the receiver. The sensor is connected with

the microcontroller. Every 3 seconds the eye blink is monitored. When the sensing occurs, according

to DBN result the alarm will be sounded for alertness.

2.2 Pulse rate sensor

Pulse Rate Sensor monitors the flow of blood through a part of the body. It continuously monitors

the driver’s heart rate. It is also connected to microcontroller. The heart rate is the number of heart

beats per minute. Normal heart rate of the person is 65 to 75 beats per minute. Every 5 seconds the

heart rate is monitored. When the heart rate is goes down from normal, according to DBN result

alarm will be sounded for alertness.

2.3 Respiration sensor

A respiration sensor also connected with controller which is continuously reading the respiration of

driver. Driver’s respiration is continuously monitored for every 30 seconds. When respiration goes

down from normal, according to DBN result alarm will be sounded to alert the driver.

2.4 Signal conditioning unit

The signal conditioning unit accepts input signals from the analog sensors and gives a conditioned

output of 0-5V DC corresponding to the entire range of each parameter. It also accepts the digital

sensor inputs and gives outputs in 10 bit binary with a positive logic level of +5V. The calibration

voltages* (0, 2.5 and 5V) and the health bits are also generated in this unit.

Microcontrollers are widely used for control in electronics system. It provides real time control by

processing analog signals obtained from the system. In between a control circuit and hardware unit




there is a suitable isolation is needed to be designed. A signal conditioning unit acts as an interface

circuit in between hardware and control unit.

2.5 SMCL-LCD

AT89C51 is the 40 pins, 8 bit Microcontroller which is manufactured by Atmel group. It has the

flash type reprogrammable memory. By using this we can erase the program within few minutes. It

has 128 bytes internal Random Access Memory and 4kb on chip Read Only Memory. The 32 I/O pin

as arranged as port 0 to port 3 each has 8 bit bin. Port 0 contain 8 data line (D0-D7) as well as low

order address line (AO-A7). Port 2 contain higher order address line (A8-A15). Port 3 contains

special purpose register such as serial data transmitter/receiver register SBUF, two external and three

internal interrupt sources, and two 16 bit timers (T0, T1), control registers. It has clock and oscillator

circuit also.

The heart of the micro controller is the circuitries which generate the clock pulse. The micro

controller has the two pins of XTAL1 and XTAL2 which are connected to crystal oscillator. The

clock frequency of the microcontroller is the crystal frequency.

Here we interface LCD display to microcontroller through port0 and port2. LCD control lines are

connected in port2 and Data lines are connected in port0. Liquid Crystal Display has 16 pins in

which first three and 15th pins are used for power supply. 4th pin is RS (Register Selection). 5th pin is

Read/Write. This pin has value of 1 means read operation is done. If it is low means it performs write

operation. 6th pin is act as enable pin. Remaining pins are data lines. In microcontroller Port (1.0 and

1.1) pin is connected to eye-blink sensor and respiration sensor output value and Port (3.0 and 3.1)

pin is connected to Bluetooth transmitter hardware module. Microcontroller is worked as a

coordinator of all function.

2.6 Bluetooth transmitter

Bluetooth transmitter module in hardware gets the bio parameters value from microcontroller.

Bluetooth is an open wireless technology standard for exchanging data over short distances (using

short wavelength radio transmissions) from fixed and mobile devices, creating personal area




networks (PANs) with high levels of security. Here it acts as a wireless network instead to RS-232

data cables. It don’t have synchronization problem. Frequency-hopping spread spectrum is one of the

radio technologies which are used by Bluetooth. The data transmission rate is up to 79 bands (1 MHz

each) in the range 2402-2480 MHz it uses 2.4 GHz short-range radio frequency band.

Generally Bluetooth is a packet-based protocol. It has a master-slave structure. One master may

communicate at a time 7 slaves in a piconet. In piconet all devices share the master's clock. Packet

exchange is based on the master defined basic clock which notes at 312.5 µs intervals. The two slot

pair has a 1250 µs. In Bluetooth the master transmits packets in even slots and receiver transmits in

odd slots; the slave receives packets in even slots and transmits packets in odd slots. In all cases the

master begins to transmit even slots and the slave transmits the odd slots.

2.7 Android based smart phone

The smart phone has the facilities like 3G/4G connectivity, Wi-Fi connectivity, Bluetooth

connectivity, accelerometer w/compass, ambient light sensor, proximity sensor, GPS, Gyroscope,

and GSM.

2.8 Android architecture

Fig 2: Android architecture




In smart phone particularly Android OS is selected for this work. Because Android is an open source

operating system which is created by Google specifically for use on mobile devices (cell phones and

tablets). It is Linux based (2.6 kernels). It can be programmed in C/C++ but most application

development is done in Java. It has open source libraries like SQLite, Web Kit and OpenGL.

The benefits of Android OS over other mobile OS are it has familiar programming environment. The

Android development tools are open source. It is free even for commercial use. Android has a simple

and powerful SDK (Software Development Kit). It has no licensing, distribution, or development

fees. Android development over many platforms Linux, Mac OS, windows is possible.

2.9 Android application development

There are three tools is used for Android application development. The first tool is Eclipse Platform.

It is the platform upon which the plug-in runs. The second tool is Android Emulator it is used to

implement the Android virtual machine and used to test and debug android applications. The third

tool is Android SDK. Here the Android Developer Tools (the Eclipse plug-in) is installed by the

Android SDK.

In Android, applications are packaged in .apk format and it is downloaded to mobile and installed.

The .apks contains .dex files (byte codes), manifest and other files. Manifest contains security, link,

hardware access and minimum OS related information etc.

Fig 3: Android application development

Eclipse IDE

Android emulator

Android SDK

Android mobile device




2.10 Dynamic Bayesian network

There are various parameters can be used to detect the driver alertness level. Here those parameters

are divided into two groups: eye blink parameters and bio signal. In our work, a dynamic Bayesian

network paradigm is programmed in smart phone is used for driver fatigue analysis. DBN paradigm

is a probabilistic graphical model. It uses different mathematical techniques to integrate a given input

data. The main reason for adapting DBN is that it has an ability to integrate distinct categories of

parameters. The final output of the system defining the driver status at a specific time is estimated

with a dynamic Bayesian network paradigm.

2.11 DBN algorithm & implementation

The parameters for fatigue level analysis is blink frequency(BF), blink rate(BR), percentage of eye

closure(PC), average eye closure speed(AC), heart rate variability(HV), root mean square(RM), first-

order-derivation(FD), power spectrum density(PD), Respiration (RESP), Temperature (TEMP).

DBN calculates its probability based on density of joint probability function which is the product of

the individual density function and parent variables conditional function. The joint probability

density function can be written as, P (Y1 = y1….. YN = yN) = ∏ 푃(Yu = yu|Yν = yν) Here each Yν

is a parent of Yu. In this case, Yν is the parent node. In this the input parameters are declared. Yu is

the child node. In this status of the driver is declared. The correlations for parameters are calculated

for to detect the dependencies relationships among them.

The Pearson’s correlation among the parameters are defined as, ρX, Y = E [(XμX) (Y − μY)]/ (σXσY)

where μX and μY is the mean of X and Y parameters. Meanwhile, σX and σY stand for the standard

derivation for X and Y parameters. The correlation value is zero if the parameters are totally

independent. The negative sign expresses the inverse relationship between the parameters.




Fig 4: Structure of DBN with five parameters as input of parent nodes and the final output of

child nodes

BF and BR shows the closest linear relationship while calculating the correlation value. The high

correlation parameters should be removed to reduce the repeated calculation. Finally, the selected

highly independent parameters are PC, AC, PD, HV and RESP as inputs to the DBN paradigm. DBN

calculation is written using java eclipse language in android platform and it is implemented in

android based smart phone. The driver bio parameters are given to smart phone by using Bluetooth

transmitter. The DBN in smart phone combined the parameters and calculates the result of single

probabilistic value.

In order for the DBN to perform analysis, a conditional probability table (CPT) is required for each

and every parent node. Conditional probability is defined as it is the probability of an event occurring

given that another event has already occurred. Experiment’s data is filtering for any incomplete or

missing parts is necessary before a CPT can be constructed. Some of the eye blinks captured are not

able to process due to the sudden huge movement of driver or affected by the changes of light in the

surrounding environment. Moreover, parts of the heart & respiration rate signals are missing

sometimes. When constructing the CPT, only meaningful data is extracted and labeled.

2.12 Vehicle stopping

A warning alarm is initiated to alert the driver whenever he feels drowsiness. If the abnormal bio

value continues for a particular time period, the vehicle stopping is more important to prevent the

accident. So after giving proper indication to nearby vehicles, the vehicle will be stopped gradually.

PC AC

RESP

HV

PD

FL

PC AC

HV

RESP

PD FL

Time t-1 Time t




3. EXPERIMENTS AND EVALUATION DETAILS

The eye blink and respiration sensors are placed in a helmet which was wore by the driver and the

pulse rate sensor is connected to helmet via driver’s finger. During the experiment the helmet is wore

by the driver. The experimental evaluation details are given below: Figure 5 shows the overall

hardware kit architecture for driver fatigue level monitoring.

Fig 5: Hardware kit for driver fatigue level monitoring

Fig 6: Android based smart phone monitoring system for driver alertness




Fig 7: Screenshot for the android programming using eclipse for driver alert monitoring

Case 1: Both the eye blink and bio signal value is normal.

The DBN in android mobile calculates the combined probability value of received parameters. In

normal case the result of DBN is below the threshold value. So the status ‘Driver Safe’ will be

displayed in the mobile screen.

Case 2: When eyes are closed due to drowsiness.

In this case the final result of DBN is above the threshold level. But the alarm waits 3 seconds for

eye opening and afterwards the alarm will be sounded to alert the driver and others. Same time

driver’s ‘Eyes closed’ status is displayed in the mobile screen. The corresponding LED will be set

ON.




Fig 8: The snapshot of driver’s eyes are closed due to drowsiness

Case 3: When respiration or heart pulse go down.

When heart rate or respiration level goes down below the normal level, the DBN produces the

probability figure which was above the threshold level. In this case the alarm waits 5 seconds for

normal beats of heart and respiration. If the abnormal condition continues, alarm will be on. In the

same time the driver’s ‘Heart rate is Abnormal’ or ‘Respiration is Abnormal’ will be displayed in the

mobile screen. The corresponding LED will be set ON.

Fig 9: The snapshot of driver’s respiration in abnormal status.




4. RESULT

The system is effectively designed. It takes a quick action to avoid an accident. Dynamic Bayesian

Network programmed in smart phone performs the statistical analysis according to the extracted

information. This approach avoids the false detection rate that annoys the driver. For a true detection

only it initiates the alarm. The driver status is continuously displayed in the mobile.

5. CONCLUSION

A fatigue monitoring system was designed and implemented in Android-based smart phone. The

final output of the system produces the driver status in specific time by using dynamic Bayesian

network paradigm. It is more reliable and can be easily implemented in all automobiles. In future

work we have planned to implement a system which informs the condition of the driver to nearby

emergency ambulance, base station and the rescue guards.

REFERENCES

[1] N. CSU, “A survey of sensor network applications,” IEEE Communication. August 2002.

[2] M. Ying, “Application of sensor networks,” Department of Computer Science, Friedrich-

Alexander-Univ. Erlangen Nuremberg, Nuremberg, Germany, May 2005.

[3] R. L. Lai and C. L. Liu, “A fuzzy control massage seat for awaking drowsy drivers”, in 2006.

[4] L. M. Bergasa, J. Nuevo, M. A. Sotelo, R. Barea, and M. E. Lopez, “Real-time system for

monitoring driver vigilance,” March 2006.

[5] T. Kasukabe, M. Hiraoka, O. Yamamoto, M. Yamada, and T. Nakano, “Development of system

for comprehensively measuring driving ability for elderly safe driving” in May 2009.

[6] J. Pauwelussen and P. J. Feenstra, “Driver behavior analysis during ACC activation and

deactivation in a real traffic environment, June 2010.

[7] J. D. Lee, J. D. Li, L. C. Liu, and C. M. Chen, “A novel driving pattern recognition and status

monitoring system”, December 2006.

[8] S. F. Zhao, G. H. Xu, and T. F. Tao, “Detecting driver’s drowsiness using multi wavelet packet

energy spectrum,” in October 2009




[9] Y. S. Lee and W. Y. Chung, “Video sensor based eye tracking and blink detection to automated

drowsy driving warning system using image processing,” July 2010.

[10] J. H. Yang, Z. H. Mao, L. Tijerina, T. Pilutti, J. F. Coughlin, and E. Feron, “Detection of driver

fatigue caused by sleep deprivation,” July 2009.

[11] J. Wang and Y. Gong, “Recognition of multiple drivers’ emotional state,” in December 2008.

[12] H. S. Shin, S. J. Jung, J. J. Kim, and W. Y. Chung, “Real time car driver’s condition monitoring

system,” November 2010.

[13] P. Bouchner, R. Pieknik, S. Novontny, J. Pekny, M. Hajny, and C. Borzová, “Fatigue of car

drivers - detection and classification based on experiments on car simulators,” in September 2006.

[14] R. N. Khushaba, S. Kodagoda, S. Lal, and G. Dissanayake, “Driver drowsiness classification

using fuzzy wavelet-packet-based featureextraction algorithm,” January 2011.

[15] C. T. Lin, Y. C. Chen, T. Y. Huang, T. T. Chiu, L. W. Ko, and S. F. Liang, “Development of

wireless brain computer interface with embedded multitask scheduling and its application on real-

time driver’s drowsiness detection and warning,” May 2008.

[16] M. M. Bundele and R. Banerjee, “Detection of fatigue vehicular driver using skin conductance

and oximetry pulse: A neural network approach,” 2009.

[17] G. S. Yang, Y. Z. Lin, and P. Bhattacharya, “A driver fatigue recognition model based on

information fusion and dynamic Bayesian network”, May 2010.

[18] S. V. Deshmukh, D. P. Radake, and K. N. Hande, “Driver fatigue detection using sensor

network,” February 2011.

[19] A. Giusti, C. Zocchi, and A. Rovetta, “A noninvasive system for evaluating driver vigilance

level examining both physiological and mechanical data,” March 2009.

[20] A. Eskandarian and R. A. Sayed, “Analysis of driver impairment, fatigue, and drowsiness and a

unobstrusive vehicle-based detection scheme December 2005.

[21] Y. L. Liang, M. L. Reyes, and J. D. Lee, “Real-time detection of driver cognitive distraction

using support vector machines,” June 2007.

[22] W. H. Liao, W. H. Zhang, Z. W. Zhu, and Q. Ji, “A real-time human stress monitoring system

using dynamic Bayesian network,” June 2005.

[23] W. Y. Chung, Y. D. Lee, and S. J. Jung, “A wireless sensor network compatible wearable u-

healthcare monitoring system using integrated ECG, accelerometer and SpO2,”, August 2008.




[24] A. Shabtai, Y. Fledel, U. Kanonov, Y. Elovici, and S. Dolev, “Google android: A state-of-the-

art review of security mechanisms,” 2009.

[25] Z. L. Zheng and F. Yang, “Enhanced active shape model for facial feature localization,” July

2008.

[26] C. Zocchi, A. Rosetta, and F. Fanfold, “Physiological parameters variation during driving

simulations,” September 2007.

[27] H. Park, S. Oh, and M. Hahn, “Drowsy driving detection based on human pulse wave by

photoplethysmography” 2009.

[28] Q. Wu, B. X. Sun, B. Xie, and J. J. Zhao, “A perclos-based driver fatigue recognition

application for smart vehicle space,” October 2010.

Date post:	14-Jul-2020
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

* Hema.B **Gopi - IJMRijmr.net.in/download.php?filename=IQonm0SrUNry3or... · 2.1 Eye-blink sensor...

Documents