Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Scalable Computational Models of Emotion for Virtual Characters

Joost Broekens, Doug DeGroot

{broekens, degroot}@liacs.nl

LIACS, Leiden University, The Netherlands.

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Contents

• What’s an emotion?

• Psychological theories of emotion.

• Appraisal Theory and BDI Agents.

• Why use emotions in agents?

• Problem definition.

• FeelMe systems

• Context Sensitive Appraisal Banks

• Experiment

• Conclusion

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

What’s an Emotion

• Common emotions: fearful, angry, happy, sad, surprised, disgusted

• Short episode triggered by an (internal/external) event composed of:– subjective feelings

– inclinations to act

– facial expressions

– cognitive evaluation,

– (and some other things)

• Heuristic relating events to goals, needs, desires, beliefs of an agent.– Evaluates personal relevance and helps decision-making (Neurological

evidence: Damasio)

• Communication medium.– Communicate internal state (Sociological evidence: Darwin, Ekman)

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

How is an emotion produced?

• James-Lange– Emotion results from the evaluation of the bodily reactions that are

provoked by events.

• Schacter-Singer (Two Factor Theory)– Emotion results from the cognitive evaluation labeling the arousal of the

organism. Arousal results directly from events.

Stimulus Response Feedback

Stimulus Arousal

EmotionEvaluation

Emotion

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Cognitive Appraisal Theory and BDI Agents

• Appraisal Theory (Frijda, Lazarus, Scherer, etc):– Emotion: is a result of the evaluation of the environment in relation to the

agent’s goals, needs, beliefs and desires (=appraisal).

– Evaluation in terms of appraisal dimensions: variables expressing a certain emotional aspect of a situation, e.g. valence or arousal.

– Appraisal Assumption: evaluation is both necessary and sufficient for an emotion to occur.

• BDI based agents:– Agent’s thinking based on beliefs, desires and intentions.

– Possess basics to which appraisal based emotions can be added.

Perception Appraisal Emotion

Beliefs/Desire/Goals/Etc.

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Why Use Emotions in Agents

• Virtual Agents (NPCs, Tutor agents) are enriched with emotions for, e.g., the following reasons:– Enhance sense of realism (VR Training).

– Entertainment (Games)

– Enhance communication between agent/robot and human (HCI).

• Examples: SIMS2, Mission Rehearsal Exercise (Marsella and Gratch), Kismet (Braezeal).

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Computational Models of Emotion in Virtual Agents

• Mostly Cognitive Appraisal Theory Based– Appraisal assumptions “evaluation in terms needs and goals of the agent

is necessary and sufficient” makes Cognitive Appraisal Theory suitable for Virtual Agents based on BDI architecture.

• Integrated into BDI structure / Architecture of the agent.– Because emotions result from cognitive evaluation, computational models

of emotion are integrated in BDI architecture.

– Not built as extendible module or add-on but often deeply integrated in the BDI architecture.

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Problem

• How to incrementally add sophistication to a computational model of emotion, while keeping it consistent with the emotions produced by the simpler version of the model? Why model-scalability:– Different mechanisms might give good emotion results in different

situations (e.g. event-based emotion encoding / appraisal-based emotion).

– Development benefits (selling upgrades/ debugging and evaluation).

• How to make scalable computational models of emotion? Why runtime-scalability:– Resources in games are limited, but very variable (high-end/low-end

PCs).

– Ideally users can trade-off the amount of emotional detail for e.g. frame-rate (analogous with e.g. Graphical detail).

– Other situations,e.g. viewing distance.

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

FeelMe system, a Dynamic Modular Approach

• Used FeelMe system (DeGroot) to work on this problem.• FeelMe system (limited overview).

• DSS: Decision Support System• AS: Appraisal System, emotionally evaluates the environment• EMS: Emotion Maintenance System, maintains emotional state• BMS: Emotional expression

AS

EMS BMS

DSS

Agent's environment/Agent's internal state

Perception Appraisal Emotion

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Appraisal System / Emotion Maintenance System.

• AS continuously emotionally evaluates the situation as constructed by the Decision Support System.

• AS interprets the situation in terms of appraisal dimension values.• AS sends a stream of n-dimensional vectors of these values to the EMS.

Such a vector is called an appraisal result.• EMS maintains the emotional state as a point in the n-dimensional space

of appraisal dimensions. (e.g. a point in the pleasure, novelty space)• EMS assumes appraisal results are changes, or “deltas” to the emotional

state. • EMS integrates these appraisal results with the existing emotional state.

– EMS is “pushed” towards a direction.

• I.E: Signal-based approach

AS

EMS BMS

DSS

Agent's environment/Agent's internal state

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Multiple “Appraisal Banks”

• What if multiple independent appraisal subsystems could send appraisal results?– Appraisal results can be produced by multiple appraisal mechanism,

including event encoding (e.g. events have a fixed emotional meaning).

– Modular and scalable approach to appraisal?

• Context Sensitive Appraisal Banks (i.e. appraisal modules)

• How to integrate the result of different concurrent “appraisal banks”?

AS

ASM EMS BMS

DSS

Agent's environment/Agent's internal state

AS

EMS BMS

DSSBank 1

Bank n

Agent's environment/Agent's internal state

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Constraints for the Integration of Results from Appraisal Banks

• EMS integrates appraisal results:– Appraisal-results defined at interval scale

– Banks together must produce non-zero positive and negative values.

• Context Sensitivity in Appraisal Banks– Sensitive to mutually exclusive situations.

• Either predefined (based on context), or

• Dynamic (based on appraisal of the current situation of the agent).

– Appraising on different detail level:• One bank: big picture

• Second bank: detailsAS

EMS BMS

DSSBank 1

Bank n

Agent's environment/Agent's internal state

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Experiment: PacMan

• To find out if such integration of appraisal bank information is feasible using the signal-based approach FeelMe system, and

• To find out if this integration permits the use of these banks as separate appraisal modules that add value to each other.– Allowing incremental design model-scalability

– Dynamic appraisal adaptation during run-time run-time scalability

• PacMan (Chow) simulation: compare the traces of the resulting emotional state while playing one level of PacMan (controlled by the human) of an Appraisal Systems with 1, or with 2 appraisal banks.

AS

EMS

Bank 1

AS

EMS

Bank 1

Bank 2

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

PacMan’s Appraisal System

• Appraisal Banks relate to PacMan’s following goals:– Points: all events related to gathering points.

– Survival: all events related to survival.

• The output of Appraisal Banks is dependent on the appraisal intensity of other banks but is independent from the appraisal mechanism of other banks.– Survival bank inhibits the effect of points bank.

– Rational: survival more basic and important than points

– Example of mutual exclusive banks, dynamically based on the current situation as evaluated by the appraisal banks..

survivalspospo 1' intintAS

EMS: Integrate

Survival

Points

Inhibition metric

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

PacMan’s Emotional State

• Based on three appraisal dimensions (Mehrabian):– Pleasure: related to goal

congruency.– Arousal: related to novelty

and attention needed for the event.

– Dominance: related to the influence PacMan has on its environment, i.e. PacMan’s power.

• EMS simply integrates the appraisal results from the Appraisal Banks

+P

+A

+D

-A

-P

-D

The following sample ratings illustrate definitions of various emotion terms when scores on each PAD scale range from -1 to +1:

angry (-.51, .59, .25) bored (-.65, -.62, -.33) curious (.22, .62, -.01) dignified (.55, .22, .61) elated (.50, .42, .23) hungry (-.44, .14, -.21) inhibited (-.54, -.04, -.41), loved (.87, .54, -.18) puzzled (-.41, .48, -.33) sleepy (.20, -.70, -.44) unconcerned (-.13, -.41, .08) violent (-.50, .62, .38).

The emotional state "angry" is a highly unpleasant, highly aroused, and moderately dominant emotional state. The "bored" state implies a highly unpleasant, highly unaroused, and moderately submissive state.

From: Albert Mehrabian’s (1980) PAD Scales.

n

ititt PADEE

01

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Results

• Emotional state (emotion) behaves more sophisticated to the conditions of the environment in the 2-bank (survival, points) configuration (e.g. eating a ghost is positive in the 2-bank case).

• Emotional state (emotion) is still consistent with the emotional state of those situations in which the 1-bank (survival) produced emotions that were meaningful.

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

224

4928

9632

14336

19040

23744

28448

33152

37856

42560

47264

51968

56672

61376

66080

70784

75488

80192

84896

89600

P

A

D

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

224

4928

9632

14336

19040

23744

28448

33152

37856

42560

47264

51968

56672

61376

66080

70784

75488

80192

84896

89600

P

A

D

Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands

Conclusions

• Context Sensitive Appraisal Banks in a dynamic, signal-based approach facilitate model-scalability (i.e. the ability to incrementally add sophistication to a computational model of emotions while staying consistent with the earlier version of that model)– If: appraisal banks are mutually exclusive either pre-programmed or

dynamic (experiment), and – If results from appraisal appraisal banks can be integrated in a meaningful

way.

• Runtime-scalability can be achieved using the appraisal bank setup.– Switching the “point” appraisal bank on/off would only add/remove

emotional sophistication, and not result in inconsistent emotional state behavior.

– Triggering the emotion system at a (slightly) different rates did not introduce differences in the resulting emotion (see paper).