-Sridhar Godavarthy

My Project Title

A Little Background: BlinkA Lot More Background: Strain as a Soft

Forensic EvidenceFacial RecognitionCulpritsHuman anatomy as a featureStrain Measurement

Micro expression Detection using Strain PatternsChallengesSample Strain patterns

References

Contents

A Little Background: BlinkA Lot More Background: Strain as a Soft

Forensic EvidenceFacial RecognitionCulpritsHuman anatomy as a featureStrain Measurement

Micro expression Detection using Strain PatternsChallengesSample Strain patterns

References

Contents

BLINK!!!

A Little Background

Why are some people brilliant decision makers?How do some people act upon instincts?Why are we unable to explain some decisions?

Introduction: Blink

Great decision makers are not ones that process the most informationMalcolm Gladwell’s ‘The statue that didn’t look

right’

They are those who have perfected the art of “Thin Slicing”Filtering out the very few factors that matter.

Blink Contd…

Navarasas – the Nine Emotions

A Little Background: BlinkA Lot More Background: Strain as a Soft

Forensic EvidenceFacial RecognitionCulpritsHuman anatomy as a featureStrain Measurement

Micro expression Detection using Strain PatternsChallengesSample Strain patterns

References

Contents

V.Manohar, D.B.Goldgof, S.Sarkar,Y.Zhang

Facial Strain Pattern as a Soft Forensic Evidence

Some slides have been adapted from the Authors’ presentation

A Lot More Background

Face recognition has made huge advancesPicasa’s Web AlbumsSony’s “say cheese”( or is it CHEERS)

detection“Almost” perfect

Picasa still confuses between closely related faces

Canon almost always never detects my faceSome say - might be because of my hair ;-)

Has anyone used the Lenovo Face ID?Because they use static imagesCould be supplemented for better

performance.

Facial Recognition

IlluminationCamouflage(Makeup/glasses)Facial HairExpressions

The Solution: Use methods based on Human Anatomy

Culprits (ICHE)

Iris scanRetina scanSkull X-ray

DisadvantageRequire Specialized equipmentIntrusive

Proposed AlternativeSkin and tissues of the face

Methods based on Human Anatomy

ElasticityDifferent materials have different elasticityElasticity can be modeled

strain

stressElasticity Known

Calculate

Authentic Author Slide

What is Facial Strain?Strain on soft tissue when expressions are

made.Anatomical methodUses a pair of frames to measure deformation

Facial Strain

Why Facial Strain?As it is a difference, it is independent of all the

earlier mentioned culprits(ICHE)

Facial Strain

‘Visual Pattern’ is unique to every face.Easily quantifiable by

‘elasticity’Hard to measure – non-

linear, inverse equationsCan be represented by

strain pattern under specific boundary conditions

Is unique to a person.

Facial Strain

Contact strain measurement equipment is already available.Cannot be used if we are looking to identify

people at a Casino/AirportDid I mention the actual applications of this

paperSoft forensics based on surveillance videos

Measurement of Facial Strain

Two major steps1. Obtain motion field between two frames2. Compute strain image from above Motion

field.

Measurement of Facial Strain Contd…

Feature BasedNeed to identify features – Difficult!Features may be ill defined( when

camouflaged)Usually requires manual interventionProduces a sparse motion fieldProduce Good correspondence in large motion

Optical Flow basedFully automatedDense Motion field.Requires constant illumination

First Step – Obtaining Motion Field

Observed motion over sequential image frames

First Step – Optical Flow Adapted

Author Slide

3D StrainIdealNo high speed equipment available to capture

range images2D Strain

Well – not much of a choiceAuthors could use existing data.

Second Step – Strain Computation Type

Variation of displacement values obtained from optical flow Calculated by taking the derivative of each pixel

Second Step – Optical Strain

Sobel operator (central difference)

Authentic Author Slide

Finite Element MethodForward modeling when Dirichlet condition is satisfiedGood at handling irregular shapesComputationally expensiveThis method is an approximation to the solution

Finite Difference MethodStrain, a tensor, can be expressed derivatives of the

displacement vectorThis can be approximated by a Finite Difference

Method.Very efficient when carried out on a regular grid.This method is an approximation to the differential

equation

Strain Computation - methods

Finite Strain tensor

Cauchy tensor

Finite Difference Method

Motion is mostly verticalStrain pattern is dominated by its normal

componentsThe strain magnitudes are scaled to gray

levelsWhite = highest strainBlack = lowest strain

It is now a pattern matching problem.

Integrating Strain Patterns

Motion field : Based on Optical flowStrain Type: 2-DComputation: Finite Difference Method

Review of Choices

Examples

Strain Magnitude is now 1-DUse PCA to perform matching

Identification and matching

Experiments performed onNormal lightLow lightShadow lightRegular faceCamouflaged faceFrontal viewProfile viewNeutral expressionOpen mouth

Experiments

Subject may not perform the expression to the same extent every timeExperiments repeated on shorter, subsampled

videos

Experiments Contd…

Strain measurement seems to be logically correct

We do not discuss the PCA and hence the recognition results as they are outside the scope of this discussion.( But they were good)

Acts as a supplement to existing recognition methods.

Results

A Little Background: BlinkA Lot More Background: Strain as a Soft

Forensic EvidenceFacial RecognitionCulpritsHuman anatomy as a featureStrain Measurement

Micro expression Detection using Strain PatternsChallengesSample Strain patterns

References

Contents

Micro expression Detection using Strain

Patterns

Macro Expressions:Large movement

SmileTalkingShaking head

Micro expressionsRaising eyebrowFast blinking

Macro Vs Micro expressions

Can you classify?

Supplement lie detectionVery little noise

As part of a general discussionBond might not have lost even the first time!

Where will it be used?

Ideal Frame Sequence

1 2 3 4 5 n6

a. 1-2

b. 3-4

c. 1-4

d. 1-3 e. 4-6

f. 1-6

Frame

Strain

a 1-2 100

b 3-4 200

c 1-4 300

d 1-3 200

e 4-6 200

f 1-6 400

Strain Measurement for a Practical Frame Sequence

Macro Expression

Micro Expression

Noise

Small movements are inevitableMacro expressions also possibleEyes always blink. Need to detect changes in

speed of blinkingNeed to identify the frames to be used

Challenges

Solution: Normalize

V.manohar, D.B. Goldgof, S.Sarkar, Y. Zhang, "Facial Strain Pattern as a Soft Forensic Evidence", IEEE Workshop on Applications of Computer Vision (WACV'07),pp 42-42

Vasant Manohar, Matthew Shreve, Dmitry Goldgof and Sudeep Sarkar, "Finite Element Modeling of Facial Deformation in Videos for Computing Strain Pattern", International Conference on Pattern Recognition, Dec. 2008

Matthew A. Shreve, Shaun J. Canavan, Yong Zhang, John R. Sullins, and Rupali Patil, "Imaging And Characterization Of Facial Strain In Long Video Sequences",xxxx

Malcolm Gladwell,” Blink: The Power of Thinking Without Thinking”, Back Bay Books (April 3, 2007)

References

Sridhar Godavarthy

Dept. Of Computer Science and Engineering

University of South Florida

sgodavar@cse.usf.edu

Thank You!