Recognition of Human Gait From Video

Marina GavrilovaComputer Science

University of Calgary

OutlineMotivationDistinguishing featuresRecognition process

Silhouette extractionHuman model initializationExtracting joint angles over image sequencesRecognition

Preliminary Results

Motivation

The goal is to detect and identify humans by the way they walk.The walking pattern (gait) is unique enough to identify a person.Such capabilities will enhance:

Human identification.Abnormal behavior detection.

Gait Cycle

Distinguishing features

Features that seem unique to each person:

Joint angle between the upper and lower legsRelationship between the knee joints and the feet over time

Elevation of knee joint over the ankle (i.e., vertical distance between knee and ankle) shows a distinctive temporal pattern

Elevation over ankle is distinctive

Transition from swingleg to stance leg is noticeably

different across different peopleover time

Gait Recognition Procedure

Image sequences

Background image

Silhouette images

Joint angles

Human model Recognition

Silhouette Extraction

Image Background

After background subtraction Final result

Human ModelHuman is modeled by five connected trapezoids. Each trapezoid (body part) is represented by

li

r1i

r2i

},,,{ 21 iiiii lrrbp θ=

iθ

Human ModelEach configuration of human body is represented by

where , and c as the center of the body.

},,,,{ 521 bpbpbpcH L=

},,,{ 21 iiiii lrrbp θ=

Human Model Initialization

2D Model-based Human tracking

Methods Cardboard person model Scaled Prismatic Model Twist and exponential maps Condensation-BasedTracking via Gibbs sampling (probabilistic)

Sample Tracking Results

Sample Tracking Results

Recognition

Collect feature vector with:Elevation over ankleJoint angles between upper and lower leg

Use left-right hidden Markov models for recognitionOne HMM per person, trained on a minimum of 4-5 full step cycles from that person

Recognition (continued)

Use algorithm similar to isolated speech recognition to identify people:

Collect a step cycle from test subjectFor each HMM in the database, compute likelihood that it matches signal of this step cycleSelect HMM with maximum likelihoodPerson corresponding to that HMM is identified subject

End of Gait Recognition

Infrared Recognition

Marina GavrilovaComputer Science

University of Calgary

#19

Primary Applications

Biometric IdentificationPasswords/PINs.Tokens (like ID cards).You can be your own password.

SurveillanceOff-the-shelf facial recognition system that identifies humans as they pass through a camera’s field of view.

#20Novel ApplicationsWearable Recognition Systems

Adapt to a specific user and be more intimately and actively involved in the user's activities. Face recognition software can help you remember the name of the person you are looking at.

Useful for Alzheimer's patients.

• Smart Systems– Key goal is to give machines perceptual abilities that allow them to

function naturally with people. – Critical for a variety of human-machine interfaces.

Why Infrared?

• Visible light has no effect on images taken in the thermal infrared spectrum.

• Even images taken in total darkness are clear in the thermal infrared.

Why Infrared? (Contd..)

Illumination InvarianceMajor problem in visible domain.

Uniqueness and RepeatabilitySense thermal patterns of blood vessels under the skin, which transport warm blood throughout the body. Remain relatively unaffected by aging. Even identical twins have different thermograms.

Immune from ForgeryDisguises can be easily detected.

Related WorkLot of research was done in the visible band but little attention was given in the infrared spectrum.Recent reduction in the cost of infrared cameras and availability of large data sets encouraged active research in infrared face recognition. Low-Level Models

Directly analyze the image pixels and impose probabilities on the features.Examples are PCA, ICA, and FDA.Not good in challenging conditions.

High-Level ModelsSynthesize images from 3D templates of known objects and imposeprobabilities on transformations.Template matching approaches.Computationally expensive.

IR Face Recognition – Training Phase

IR Face Recognition – Test Phase

Segmentation

Noise in the background may effect the performance of a face recognition system.

Remove the background.

Use thermal information on face to compute the features.

• Adaptive Fuzzy Segmentation (kakadiaris02)– Fuzzy affinity is assigned to spels w.r.t. target object spel.– Affinity is computed as weighted sum of the temperature and the

temperature gradient in the neighborhood of the target spel.– Minimal user interaction because of dynamically assigned weights.

Problem with Single Seed

Temperatures on face are different at different regions.

• If a single seed is chosen in a particular region, then the connectivity stretches only along this region and the segmentation goes wrong.

Multiple SeedsSolution to this problem is to choose multiple seeds in different regions on face and merge the resulting segmented parts .

• Choose a seed pixel on face wherever there is sharp change in gradient.

• Works well even when the subject is wearing glasses.

• Robust to variation of poses.

Choosing Multiple Seeds

Assumptions

Merge all resultant segmented regions to form final image.

ASSUMPTIONS

• The center of the image contains the pixel from facial region.• The temperatures at all pixels are mapped between 0 and 255.

– If this mapped temperature at a pixel is between 175 - 200, it is classified to be in blue region.

– If this mapped temperature at a pixel is between 200 - 225, it is classified to be in pink region.

– If this mapped temperature at a pixel is between 225 - 255, it is classified to be in cyan region.

Spectral Components

Bessel Parameters

The filtered images are modeled using Bessel parameters:

SK – Sample Kurtosis

SV – Sample Variance

• Each segmented image in training set is convolved with the filters in Gabor filter bank to obtain Gabor filtered images.

Bessel Model

Using the bessel parameters p and c, the filtered image I(j)(x,y) is modeled as:

Γ(p) is gamma function Iv(z) is modified bessel function of first kind given by:

Bessel Model

Comparing IR Images

Images modeled into Bessel parameters can be compared by:

• L2-metric between two Bessel forms f(x;p1,c1) and f(x;p2,c2) in D:

Hypothesis Pruning

Applying a high-level classifier on entire database is computationally very expensive.

Pruning of hypotheses can be achieved by using Bessel parameters (anuj01).

Helps in short listing best matches.

Bessel parameters for images in database can be computed offline which helps in saving a lot of computation time.

Hypothesis Pruning (Contd..) Shortlist the subjects of A with P1(α/I) greater than a specific threshold:

Sample Experimentswww.equinoxsensors.com

Image frame sequences were acquired at 10 frames/sec while the subject was reciting the vowels ‘a’,’e’,’i’,’o’,’u’.

End of Infrared Recognition

DNA Recognition

Marina GavrilovaComputer Science

University of Calgary

Identifiable biometric characteristics

Biological Background of DNA

Human Genome

Chromosome and DNA

DNA – The Double Helix

DNA

DNA Components

DNA Replication

Genetic Information

Characteristics of DNA

DNA in the Cell

DNA for Identification

What type of Genetic Information

Short Tandem Repeats (STR)

Short Tandem Repeats (STR)

STR Database

Comparison

Example – Two Suspects

Thank You