My Project Title

-Sridhar Godavarthy

Contents• A Little Background: Blink• A Lot More Background: Strain as a Soft

Forensic Evidence– Facial Recognition– Culprits– Human anatomy as a feature– Strain Measurement

• Micro expression Detection using Strain Patterns– Challenges– Sample Strain patterns

• References

Contents• A Little Background: Blink• A Lot More Background: Strain as a Soft

Forensic Evidence– Facial Recognition– Culprits– Human anatomy as a feature– Strain Measurement

• Micro expression Detection using Strain Patterns– Challenges– Sample Strain patterns

• References

BLINK!!!

A Little Background

Introduction: Blink

• Why are some people brilliant decision makers?• How do some people act upon instincts?• Why are we unable to explain some decisions?

Blink Contd…

• Great decision makers are not ones that process the most information– Malcolm Gladwell’s ‘The statue that didn’t look

right’

• They are those who have perfected the art of “Thin Slicing”– Filtering out the very few factors that matter.

Navarasas – the Nine Emotions

Contents• A Little Background: Blink• A Lot More Background: Strain as a Soft

Forensic Evidence– Facial Recognition– Culprits– Human anatomy as a feature– Strain Measurement

• Micro expression Detection using Strain Patterns– Challenges– Sample Strain patterns

• References

Facial Strain Pattern as a Soft Forensic Evidence

V.Manohar, D.B.Goldgof, S.Sarkar,Y.Zhang

Some slides have been adapted from the Authors’ presentation

A Lot More Background

Facial Recognition• Face recognition has made huge advances– Picasa’s Web Albums– Sony’s “say cheese”( or is it CHEERS) detection

• “Almost” perfect– Picasa still confuses between closely related faces– Canon almost always never detects my face• Some say - might be because of my hair ;-)

• Has anyone used the Lenovo Face ID?• Because they use static images• Could be supplemented for better

performance.

Culprits (ICHE)• Illumination• Camouflage(Makeup/glasses)• Facial Hair• Expressions

• The Solution: Use methods based on Human Anatomy

Methods based on Human Anatomy

• Iris scan• Retina scan• Skull X-ray

• Disadvantage– Require Specialized equipment– Intrusive

• Proposed Alternative– Skin and tissues of the face

Elasticity

• Different materials have different elasticity• Elasticity can be modeled

strain

stressElasticity Known

Calculate

Authentic Author Slide

Facial Strain

• What is Facial Strain?– Strain on soft tissue when expressions are made.– Anatomical method– Uses a pair of frames to measure deformation

Facial Strain

• Why Facial Strain?– As it is a difference, it is independent of all the

earlier mentioned culprits(ICHE)

Facial Strain– ‘Visual Pattern’ is unique to

every face.• Easily quantifiable by

‘elasticity’• Hard to measure – non-

linear, inverse equations• Can be represented by strain

pattern under specific boundary conditions• Is unique to a person.

Measurement of Facial Strain

• Contact strain measurement equipment is already available.– Cannot be used if we are looking to identify

people at a Casino/Airport– Did I mention the actual applications of this paper• Soft forensics based on surveillance videos

Measurement of Facial Strain Contd…

• Two major steps1. Obtain motion field between two frames2. Compute strain image from above Motion field.

First Step – Obtaining Motion Field• Feature Based– Need to identify features – Difficult!– Features may be ill defined( when camouflaged)– Usually requires manual intervention– Produces a sparse motion field– Produce Good correspondence in large motion

• Optical Flow based– Fully automated– Dense Motion field.– Requires constant illumination

First Step – Optical Flow

• Observed motion over sequential image frames

Adapted Author Slide

Second Step – Strain Computation Type

• 3D Strain– Ideal– No high speed equipment available to capture

range images

• 2D Strain– Well – not much of a choice– Authors could use existing data.

Second Step – Optical Strain

• Variation of displacement values obtained from optical flow– Calculated by taking the derivative of each pixel

Sobel operator (central difference)

Authentic Author Slide

Strain Computation - methods• Finite Element Method– Forward modeling when Dirichlet condition is satisfied– Good at handling irregular shapes– Computationally expensive– This method is an approximation to the solution

• Finite Difference Method– Strain, a tensor, can be expressed derivatives of the

displacement vector– This can be approximated by a Finite Difference Method.– Very efficient when carried out on a regular grid.– This method is an approximation to the differential

equation

Finite Difference Method

• Finite Strain tensor

• Cauchy tensor

Integrating Strain Patterns

• Motion is mostly vertical– Strain pattern is dominated by its normal

components

• The strain magnitudes are scaled to gray levels– White = highest strain– Black = lowest strain

• It is now a pattern matching problem.

Review of Choices

• Motion field : Based on Optical flow• Strain Type: 2-D• Computation: Finite Difference Method

Examples

Identification and matching

• Strain Magnitude is now 1-D• Use PCA to perform matching

Experiments

• Experiments performed on– Normal light– Low light– Shadow light– Regular face– Camouflaged face– Frontal view– Profile view– Neutral expression– Open mouth

Experiments Contd…

• Subject may not perform the expression to the same extent every time– Experiments repeated on shorter, subsampled

videos

Results

• Strain measurement seems to be logically correct

• We do not discuss the PCA and hence the recognition results as they are outside the scope of this discussion.( But they were good)

• Acts as a supplement to existing recognition methods.

Contents• A Little Background: Blink• A Lot More Background: Strain as a Soft

Forensic Evidence– Facial Recognition– Culprits– Human anatomy as a feature– Strain Measurement

• Micro expression Detection using Strain Patterns– Challenges– Sample Strain patterns

• References

Micro expression Detection using Strain Patterns

Macro Vs Micro expressions

• Macro Expressions:– Large movement• Smile• Talking• Shaking head

• Micro expressions– Raising eyebrow– Fast blinking

Can you classify?

Where will it be used?

• Supplement lie detection– Very little noise

• As part of a general discussion• Bond might not have lost even the first time!

Ideal Frame Sequence

1 2 3 4 5 n6

a. 1-2 b. 3-4

c. 1-4

d. 1-3e. 4-6

f. 1-6

a 1-2 100

b 3-4 200

c 1-4 300

d 1-3 200

e 4-6 200

f 1-6 400

Strain Measurement for a Practical Frame Sequence

Macro Expression

Micro Expression

Noise

Challenges

• Small movements are inevitable• Macro expressions also possible• Eyes always blink. Need to detect changes in

speed of blinking• Need to identify the frames to be used

Solution: Normalize

References• V.manohar, D.B. Goldgof, S.Sarkar, Y. Zhang, "Facial Strain

Pattern as a Soft Forensic Evidence", IEEE Workshop on Applications of Computer Vision (WACV'07),pp 42-42

• Vasant Manohar, Matthew Shreve, Dmitry Goldgof and Sudeep Sarkar, "Finite Element Modeling of Facial Deformation in Videos for Computing Strain Pattern", International Conference on Pattern Recognition, Dec. 2008

• Matthew A. Shreve, Shaun J. Canavan, Yong Zhang, John R. Sullins, and Rupali Patil, "Imaging And Characterization Of Facial Strain In Long Video Sequences",xxxx

• Malcolm Gladwell,” Blink: The Power of Thinking Without Thinking”, Back Bay Books (April 3, 2007)

Thank You!

Sridhar GodavarthyDept. Of Computer Science and Engineering

University of South Floridasgodavar@cse.usf.edu