A Dimensional Representation

of Emotion

James A. Russell

Boston College

Affective Computing & Intelligent Interaction

San Antonio, 24-26 Oct 2017.

Affectiva

Website says:

Emotion Recognition: Our technology

analyzes subtle facial expressions to identify

human emotions.

In our products we measure 7 emotion

metrics: anger, contempt, disgust, fear, joy,

sadness, and surprise.

Categories of Creatures

Biological

• Lions

• Tigers

• Bears

• Wolves

Mythical

• Werewolves

• Vampires

• Demons

• Zombies

Two Traditions

in the

Representation

of Emotional State

Category: discrete

A set of events with common properties

Dimension: continuous variation

A property on which events vary

Two Metaphors

Switch Metaphor:

7 patterns triggered:

Categories

Snowflake metaphor:

Infinite Number of

Unique Events:

Dimensions

“there is no limit to the

number of possible different

emotions”

“any classification of the

emotions is seen to be as true

and as 'natural' as any other “

Categories of Emotions?

William James

Outline

Categorical vs. Dimensional Representations of Emotion

Categorical View

Problems

Basic Emotion Theory

Dimensional View

Core Affect

Psychological Construction of Emotional Episodes

Categories can

Oversimplify

Sex: Man vs Woman

Race: Black vs White

Height: Tall vs Short

Shirt size: S M L XL

Color: Black vs Red

Categories

Scientific categories: necessary and

sufficient features

Everyday categories: fuzzy borders and

degree of membership

Game

Categorical Representation

Has Dominated

Aristotle

Darwin

Scientific Version

Basic Emotion Theory

Paul Ekman

Carroll Izard

Aristotle Ekman

Anger: II 2. Anger

Calmness: II 3. Joy

Friendship and enmity: II 4.

Fear: II 5. Fear

Shame and shamelessness: II 6.

Kindness: II 7.

Pity: II 8.

Indignation: II 9.

Envy: II 10.

Emulation: II 11.

Surprise

Sadness

Disgust

The Number of Categories of

Emotion

Varies with Language

Language

No word for “emotion” in many languages.

Some English words for basic emotions – “sadness,” “anger,” “fear,” “disgust” – have no translations in some languages.

Other languages have emotion words lacking in English: amae, fago, liget.

Even where translations exist:Closest Word to “anger” is not a good translation

– Anger for English speakers– Song among Ifaluk– Liget among Ilongot– Ningaq among Utku

Lost in Translation

Which Categories of Positive Emotion?

Darwin (1872)

Astonishment

Contemplation

Determination

Devotion

Happiness

High spirits

Cheerfulness

Joy

Laughter

Love

Maternal love

Pride

Tender (sympathy)

Ekman (1972)

Happiness

Subjective

Feeling

Nonverbal Signale.g., face, voice

Autonomic

Patterne.g., HR, SC

Instrumental

Action

Evente.g., danger

EmotionEvent

Theory of Basic Emotions

Basic Emotion Theory has

Problems

The theory based on categories:

Failure to find ANS patterns

Failure to find facial signals

Failure to find brain localizations

Failure to account for behavior

Lack of correlation among the components

Failure to specify the number of basic emotions

Subjective

Feeling

Nonverbal Signale.g., face, voice

Autonomic

Patterne.g., HR, SC

Instrumental

Action

Evente.g., danger

EmotionEvent

Theory of Basic Emotions

Scores of

Highly Fearful

Persons on 48

Mood Scales

High Arousal

Sleepiness

Displeasure Pleasure

Excitement

Serenity

Distress

Depression

Core Affect

Core Affect:

Two Dimensions

• A neurophysiological state accessible as

the simple feelings of:

• Pleasure-displeasure (valence; hedonic

tone; not moral valence, etc.)

• Activation (subjective sense of energy,

arousal)

High Arousal

Sleepiness

Displeasure Pleasure

Excitement

Serenity

Distress

Depression

Core Affect

• Always in some

state of core affect

• Basis of mood

• Part of, not all of,

emotional episode

High Arousal

Sleepiness

Displeasure Pleasure

Excitement

Serenity

Distress

Depression

Core Affect

• Dimensions are

bipolar

• Axes are

independent

• Intensity of category

is distance from

center

Subjective

Feeling

Nonverbal Signalfacial

expression

Autonomic

Patterne.g., HR, SC

Instrumental

Action

Evente.g., danger

EmotionEvent

Theory of Basic Emotions

Top Science NewsSaturday 21 October, 2017

Fear of Spiders and Snakes is Innate

Snakes and spiders evoke fear in many

people, even in developed countries where

hardly anybody comes into contact with

them.

Stefanie Hoehl, Kahl Hellmer, Maria

Johansson, Gustaf Gredebäck. Itsy Bitsy

Spider…: Infants React with Increased

Arousal to Spiders and Snakes. Frontiers in

Psychology, 2017; 8

DOI: 10.3389/fpsyg.2017.01710

Study6-month-old infants

Shown spiders (and/or flowers) or snakes

(and/or fish)

Study6-month-old infants

Shown spiders (and/or flowers) or snakes (and/or

fish)

Measured Pupil Dilation

A measure of arousal (not specific to fear)

So, spiders and snakes may be innately

arousing

Theory of Basic Emotion

Claim: Categorical Differences

happiness sadness anger surprise fear disgust

Charles Darwin’s The Expression of the Emotions in Man and Animal

Affectiva

Website says:

Emotion Recognition: Our technology

analyzes subtle facial expressions to identify

human emotions.

In our products we measure 7 emotion

metrics: anger, contempt, disgust, fear, joy,

sadness, and surprise.

Facial Expression in TV: Nose Scrunch

Is the Nose Scrunch a Universal

Facial Expression of Disgust?

Production:

Do disgusted people produce the nose

scrunch?

Recognition:

Do all human beings recognize disgust from

the nose scrunch?

Does the

Nose Scrunch

express

Disgust?

Nose Scrunch is Two Facial Changes

Production of the “Disgust Face”

Meta-Analysis

Duran, Reisenzein, & Fernandez-Dols (2017)

Found 9 published studies in which a specific basic

emotion was created in the laboratory and the face

was assessed.

By “disgust face” is meant the nose scrunch.

Some studies counted either nose wrinkle or upper

lip raise as a “hit.” Some required both.

Some studies scored a “hit” if nose scrunch

occurred for any one out of a set of stimuli.

Production

Did those who felt disgust produce the “facial

expression of disgust” = Nose Scrunch?

Disgust created by disgust-inducing film, memory of a

disgusting event, or presenting odor of feces. Self-

reported disgust above the middle of scale.

Correlation based on 4 studies (total N = 187): .24.

Proportion based on 5 studies (total N = 279): 32%

68 % of disgusted participants did not show the disgust

expression.

Other faces?Prototypical Surprise Face

Paul Ekman

in

Papua New Guinea

Recognition?

Expedition to Papua New Guinea

Carlos Crivelli, Sergio Jarillo,

Jose Miguel Fernandez Dols

Research Team

Carlos Crivelli, psychologist, De Montfort University, Leicester, UK

Sergio Jarillo, anthropologist, The Metropolitan Museum, NY; American Museum of Natural History, NY

Both speak the vernacular.

Conducted years of ethnographic work prior to experiments.

Know the local customs and live with adoptive families.

“Since these pictures show

universal facial expressions,

the message conveyed by

each face will usually be

quite obvious” (p. 11).

Prior work ConclusionCurrent study Results

Spontaneous Facial Expressions

Stimuli: Spontaneous Facial

Expressions of Papua New Guineans

photographed and labeled by Ekman

Studies in Papua New Guinea

1. Free labeling

32 children, aged 14-17 years, 17 male; 15 female

5 Faces from Ekman (1980)

2. Forced Choice with literate respondents

24 Children, aged 12-14 years, 13 male; 11 female

Same 5 faces

5 Ekman labels, plus 2 from free labeling

Results for Disgust Face

Free Labeling

Label as “disgust”: 6%

Label as “gibulwa”: 22%

“Gibulwa” means “wants to avoidsocial interaction”

Forced Choice

“Disgust” : 38%

“Gibulwa”: 33%

General Results

Free Labeling -- Matching Ekman’s prediction

Mean: 8%

Range: 0% - 16%

For no face was predicted label modal

Forced Choice – Matching Ekman’s prediction

Mean: 23%

Range: 13% - 38%

Insight

Even in USA and in Europe, these

spontaneous facial expressions were poorly

recognized (for the emotions predicted by

Ekman).

Most research offered in support of Basic

Emotion Theory used posed facial

expressions.

Ekman’s Proposed

Facial Signals

happiness sadness anger surprise fear disgust

Expressions are Posed

Charles Darwin’s The Expression of the Emotions in Man and Animal

Study 3: Standardized Faces

Faces: ADFES and Radboud sets

Response Format: Choice from array (5 faces)

Children aged 6-16 years:

Spanish N = 113

Papua New Guinea N = 68

Results for Disgust Face, Study 3

Point to the person who feels disgust

Children in Spain (N = 113)

Disgust Nose Scrunch: 83%

Children in Papua New Guinea (N = 68)

Disgust Nose Scrunch: 25%

Fear Gasping Face: 29% (modal choice)

Study 4

Replication in Mozambique

Control Group in Spain

Same team of researchers

Children (age 6 – 15 years)

Different sets of faces:

Static and Dynamic Faces from ADFES

Other faces we created

Disgust Results

Point to the person who feels disgust

in Mozambique

Static: 37% [32% picked “sad pout”]

Dynamic: 41% [24% picked “fear gasp”]

Study 5

Papua New Guinea and Spain

New Response Formats

Papua New Guinea: 40 children, mean age 12.24 y Spain: 40 children, mean age 12.23 y.

New Faces: 5 faces created in our lab: nose scrunch, plus “sick face” plus happy, angry, and sad.

Study 5

New Response Format

Free labeling: “How does this person feel?”

40 children in Papua New Guinea

Shown the standard disgust face,

“How does this person feel?”

3% said “disgust”

Study 5

Another New Response Format

Give a story: step in feces; finding a worm in

your food

40 children in Papua New Guinea

Pick the face for a story (choice from 5 faces):

27% picked “disgust face”

42% picked “sad face”

Is this person:

Angry

Afraid

Disgusted

Happy

Sad

Surprised

(none of the above)

Typical Method:

the “Disgust Face”

Standard Method

How does she feel?AngryDisgusted Embarrassed HappySadScaredSurprised

Pochedly, Widen, & Russell, 2012

Two Studies of the Disgust Face

• Study 1: N=120 children,

5-14 years

• Study 2: N=135 adults

How does she feel?AngryDisgusted Embarrassed HappySadScaredSurprised

3 “anchor” conditions

Anger anchorAnger

omittedSick faceanchor

0102030405060708090

100

Anger Anchor Anger Omitted Sick Face Anchor

% o

f R

esp

on

ses

Anchor Face ChangesInterpretation of “Disgust Face”

Angry DisgustedEmotion Label:

Summary

Basic Emotion Theory has

Problems

Failure to find predicted pattern in self-reported subjective experience

Failure to find facial signals in Papua New Guinea and Mozambique

Production

Recognition

Basic Emotion Theory has

Other Problems

Failure to find ANS patterns

Failure to find brain localizations

Failure to account for behavior

Lack of correlation among the components

Failure to specify the number of basic

emotions

Conclusion

The theory based on categories:

Pop psychology uses categories

No scientific version has been supported by evidence

Dimensional Account

Core Affect (part of, not all of, emotions)

Other Dimensions

Thank you

Questions?