Ansari Automated Pain Detection FIT2011!12!20

8/3/2019 Ansari Automated Pain Detection FIT2011!12!20

http://slidepdf.com/reader/full/ansari-automated-pain-detection-fit20111220 1/41

Multi-disciplinary work on Video Analysis

Use of Information Technology for

Monitoring Pain in Patient Care

Rashid Ansari

Dept. of Electrical & Computer EngineeringUniversity of Illinois at Chicago

Work done with

Diana J. Wilkie, Chaired Professor in Bio-behavioral Health Sci.

and

Zhanli Chen, ECE PhD student



Outline

Larger Context

Motivation for video analysis for pain

Framework for using Bio-behavioral models for

Automated Pain detection Past Work and Available Databases

Feature extraction and tracking using Active

Appearance Model (AAM)

Rule-based Action Unit Recognition Results

Conclusion



Outline

Larger Context







Conclusion



Larger Context

Information technology and healthcare

enterprises.

Electronic Health Record (EHR) systems

EHR ± Data Treasure Trove Need for multidisciplinary skills

Nature of EHR data

Goal: Need training in IT and Health disciplines ± ³clinician¶s big picture" of the patient¶s condition based on its current

EHR data, assisted by the analysis, mining, and visualization of

historical EHR data securely accessed in a privacy-preserving manner

from disparate EHR systems.



Larger Context ± Excerpt from report of IEEE

Presidents http://dl.dropbox.com/u /2397114/IEEE_kam.pdf



Larger Context ± Report of IEEE

Presidents



Larger Context ± Report of IEEE

Presidents



Larger Context ± EHR Training & Research for

Advances in Informatics (EHR-Train)



Outline

Larger Context







Conclusion



Motivation for Automated Pain

Recognition

Behavioral observation ± important for assessing pain.

Many populations cannot communicate pain

± children,

± critically ill non-communicative patients,

± patients undergoing procedures,

± adults with dementia

Taskforce Position statement recognized use of Facial

Expression: ³Pain assessment in the nonverbal

patient: position statement with clinical practice

recommendations´.




Recognition

Facial expression is behavioral indicator of

pain - characterized by the Facial Action

Coding System (FACS)1 Ekman-Friesen

FACS: Objective assessment in expression

analysis - facial Action Units (AUs)

AUs - single or combination muscular activity that produces changes in facial

appearance ± 44 AUs

9 Pain-related AUs (example)

AUs detected by FACS experts

[1]Ekman P, Friesen WV, Hager JC. New Version of the Facial Action Coding System:

The Manual. Salt Lake City, UT, USA: Research Nexus Division of Network Research

Information; 2002.

Photo Source: Kim Komenich, with permission.

Pain AUs: Injured soldier.




Recognition

Facial expression is behavioral indicators of

pain - characterized by the Facial Action

Coding System (FACS)1 Ekman-Friesen

FACS: Objective assessment in expression

analysis - facial Action Units (AUs)

AUs - single or combination muscular activity that produces changes in facial

appearance ± 44 AUs

9 Pain-related AUs (example)

AUs detected by FACS experts

Facial expression coding using FACS - time

consuming, clinical use infeasible

[1]Ekman P, Friesen WV, Hager JC. New Version of the Facial Action Coding System:

The Manual. Salt Lake City, UT, USA: Research Nexus Division of Network Research

Information; 2002.

Photo Source: Kim Komenich, with permission.

Pain AUs: Injured soldier.



Outline

Larger Context







Conclusion



Framework for Using FACS - detecting

Pain-related Action Unit

Table 1. Description and Muscular Basis of Selected Action Units for Pain Facial Expression

Action Unit Description Muscular Basis

4 eye brow lowerer depressor glabellae, depressor supercilii; corrugator supercilii

6 cheek raiser orbicularis oculi; pars orbitalis

7 eye lid tightener orbicularis oculi; pars palebralis

9 nose wrinkler levator labii superioris alaeque nasi10 upper lip raiser levator labii superioris; caput infraorbitalis

20 lip stretcher risorius

26 jaw drop masetter; temporal and internal pterygoid relaxed

27 mouth stretch pterygoids, digastric

43 eyes closed relaxation of levator palpebrae superioris

AU Source: Ekman & Friesen, 1978; with permission;



Framework to use FACS ± feature

points to detect AUs (shape vertices)

66 vertices are first identified in the face image AUs are linked to these vertices or feature points



Framework to use FACS: Key

Ingredients for Pain Recognition

Behavioral model for assessing pain ± FACS

AUs are linked to key facial feature points

Approach to use FACS in Video Analysis:

± Mark features in key frames and train a model

± Use model to extract feature points and track them

± Use rules to map features to Pain AUs for

detection and recognition

The above 3 items define tasks in AU recognition



Framework to use FACS: Schematic of

Tasks for AU detection

Training Set

Labeling

Featureextraction and

tracking

Pain FacialExpression

Detection andRecognition



Framework to Use FACS: Task I -

Semi-automated labeling

Some Key frames are labeled with 66 vertices along the featurecues in the face.

Marking vertices by hand is tedious

Some key vertices are first identified and the location of remaining

vertices is constrained by their relationship to the key vertices.



Outline

Larger Context






Rule-based Action Unit Recognition

Results

Conclusion



Past work and available databases

Research on automated expression recognition uses geometricfeatures referred to as the shapes of the facial components (eyes,

mouth, etc.) and appearance features including textures, wrinkles,

bulges, and furrows.

Most of the expression recognition methods are classifier-based built

from large training sets.

A rule-based approach was developed by Pantic et al68 for AUdetection using temporal dynamics of the feature points.

Past work focused on general expression recognition

Only recently has some effort been directed at detecting pain-related

facial expression. Lucey et al use an Active Appearance Model (AAM)

based method to extract features and used support vector machines

(SVM) for pain recognition from video achieves of patients with

shoulder pain (acute pain)



Past work and available databases:

Existing Databases

Databases ± Cohn-Kanade facial expression database

most widely used database for facial expression

recognition.

± MMI facial expression database

contains both posed expressions and spontaneous

expressions of facial behavior.

± BU-3DFE database 3D range data of six prototypical facial expressions

± UNBC-McMaster Shoulder Pain Archive

Spontaneous expression caused by real pain



Our Video Database of Lung Cancer

Patients

Our database has videos of 43 lung cancer patients -- 10-minute

video with the camera focusing on the face

Each 600-sec video is partitioned into 30 equal-sized segments

with 20 seconds per segment

Segments are reviewed and scored by three trained codersindependently.

An AU is scored in a video time slot only if at least two coders

agree on its presence.



Outline

Larger Context







Results

Conclusion



Task II: Feature Extraction and Tracking -

Active Appearance Model (AAM)

AAM - parametric model of shape and texture, used in object

appearance modeling.

Shape model: n Eigen shapes Si plus the mean shape S0 .

Appearance model: m Eigen appearance Ai , plus the mean

Eigen texture A0

Goal of AAM: find parameters that minimize the error between

the observed image and synthesized image.



Task II: AAM Modification - Coefficient

Partitioning Method

AAM employs a set of parameters to control the shape and texturevariation, which define a registration between a target image and a

reference template.

In the original model, each coefficient controls certain motion

(represented by one Eigen shape) of all vertices simultaneously in the

shape model.

Convergence problems can occur when the algorithm tries to fit

different feature components on the face when the facial change is

localized.

In order to extract the expression information accurately, more flexibility

is necessary for the deformation of the shape model.



Task II: AAM Modification: Coefficient

Partitioning Method



Task II: AAM Fitting results

Labeled Training Images of patient P16.

Result of synthesizing patient face using AAM fitting for patient P16 in video segment 18.



Task II: AAM Modification: Coefficient

Partitioning Method

After processing all shape vertices are

available

Need to define rules to use vertices to

determine AU



Outline

Larger Context







Results

Conclusion



Task III: Rule-Based Action Unit Recognition

The feature extraction for rule-based method: needs shapeinformation from the AAM fitting result.

Main idea: Define rules from information of the feature point position to

synthesize AUs as described in the FACS manual.

Temporal information of the feature points can enhance the capacity

and robustness in AU recognition Distance Parameter extraction



Task III: Pain related Action Unit

(Recall)

Table 1. Description and Muscular Basis of Selected Action Units for Pain Facial

Expression

Action Unit Description Muscular Basis

4 eye brow lowerer depressor glabellae, depressor supercilii; corrugator

supercilii

6 cheek raiser orbicularis oculi; pars orbitalis7 eye lid tightener orbicularis oculi; pars palebralis

9 nose wrinkler levator labii superioris alaeque nasi

10 upper lip raiser levator labii superioris; caput infraorbitalis

20 lip stretcher risorius

26 jaw drop masetter; temporal and internal pterygoid relaxed

27 mouth stretch pterygoids, digastric

43 eyes closed relaxation of levator palpebrae superioris

AU Source: Ekman & Friesen, 1978; with permission;



Task III: Recognition Rules

The rules listed in Table 2 use AU descriptions in the FACS manual (scoring AU

with intensity B)



Task III: Visual Tracking Results

Plots of some tracked feature points in segment 18 of patient P16s video.



Task III: Recognition Procedure using

time evolution

A Complete AU occurrence has 3 stages: Onset, Apex, Offset

Two sets of thresholds are used: Lower

threshold for Onset and Offset and higher

threshold for Apex (the trajectory should have an

amplitude to score intensity B)





Schematic View of the system

Automated AU Recognition Method.



Outline

Larger Context







Conclusion



Results

Due to the sparse appearance of AUs in the patient

video, we evaluate performance by examining the

agreement between the AUs detected by the

algorithm and those scored by experts.

Tested several video segments for four patients

containing different AU combinations.

We compared AUs recognized by the computer with

those scored by at least two human coding experts

We found f ull agreement in the case of the

segments investigated so far .



Future Work

We will investigate all 43 patient videos. We areimproving the AAM fitting algorithm to make it more

robust.

Plan to new multi-view high-resolution patient video

dataset, so that we will get a better quality video tostudy the automated pain recognition system.

We will also extend our research to 3D accordingly to

better handle the rigid motion problem.



Conclusion

Use of Information Technology for Monitoring Pain inPatient Care.

Multi-disciplinary work on Video Analysis

Information technology and healthcare enterprises -

Electronic Health Record (EHR) Need for training a future workforce with

multidisciplinary skills.

Behavioral observation ± important for assessing

pain. Translational research to use FACS to develop an

automated pain AU recognition system



Thank You

Date post:	06-Apr-2018
Category:	Documents
Upload:	kashif-aziz-awan
View:	217 times
Download:	0 times

Ansari Automated Pain Detection FIT2011!12!20

Documents