Autonomous CharactersAutonomous Charactersfor Games and Animationfor Games and Animation

CraigCraig W. Reynolds W. ReynoldsSony Computer Entertainment AmericaSony Computer Entertainment America

May 1, 2000

Autonomous CharactersAutonomous Charactersfor Games and Animationfor Games and Animation Self-directingSelf-directing characters which operate autonomously characters which operate autonomously

("puppets that pull their own strings"("puppets that pull their own strings" -Ann Marion) -Ann Marion)

Applications in:Applications in: games and other interactive venuesgames and other interactive venues

animation for television and feature filmsanimation for television and feature films

History:History: first used experimentally in 1987first used experimentally in 1987

in wide commercial use todayin wide commercial use today

Autonomous CharactersAutonomous Characters Autonomous agents for simulated 3D worldsAutonomous agents for simulated 3D worlds

situatedsituated embodiedembodied

Intersection of several fieldsIntersection of several fields ethologyethology artificial lifeartificial life autonomous roboticsautonomous robotics dramatic charactersdramatic characters

Adjunct to physically-based modelingAdjunct to physically-based modeling dynamics versus volitiondynamics versus volition bouncing ball versus pursuing puppy bouncing ball versus pursuing puppy

Reactive BehaviorReactive Behavior Behavior driven by reaction to environmentBehavior driven by reaction to environment

both passive scenery and active charactersboth passive scenery and active characters

Simplifies complex animationSimplifies complex animation

many characters can be animated by a single many characters can be animated by a single behaviorbehavior

Allows user interactionAllows user interaction

improvisational style permits unscripted actionimprovisational style permits unscripted action

Creating Character BehaviorsCreating Character Behaviors By designBy design

programmingprogramming authoring authoring

(example: Motion Factory)(example: Motion Factory)

Through self-organizationThrough self-organization evolutionevolution and other forms of and other forms of machine learning:machine learning:

neural netsneural netsdecision treesdecision treesclassifier systemsclassifier systemssimulated annealingsimulated annealing

Ad hocAd hoc Behavioral Hierarchy Behavioral Hierarchy Action selectionAction selection

goals and strategiesgoals and strategies

Path selection / steeringPath selection / steering global motionglobal motion

Pose selection / locomotionPose selection / locomotion local motion (animation)local motion (animation)

Behavioral BlendingBehavioral Blending Discrete selectionDiscrete selection

One behavior at a timeOne behavior at a time

Behavioral blendingBehavioral blending

Summation / averagingSummation / averaging

Per "frame" selection (blend via inertia)Per "frame" selection (blend via inertia)

- First activeFirst active

- Stochastic (dithered) decision treeStochastic (dithered) decision tree

Behavioral AnimationBehavioral Animation

Behavioral AnimationBehavioral Animation Background actionBackground action

Autonomous charactersAutonomous characters

behavioral modelbehavioral model

graphical modelgraphical model

Improvised actionImprovised action

Behavioral Animation:Behavioral Animation:Group MotionGroup Motion IndividualIndividual

simple local behaviorsimple local behavior

interaction with:interaction with:

- nearby individualsnearby individuals

- local environmentlocal environment

Group:Group:

complex global behaviorcomplex global behavior

Behavioral Animation:Behavioral Animation:Examples of Group MotionExamples of Group Motion People People

crowds, mobs, passersbycrowds, mobs, passersby

Animal Animal

flocks, schools, herdsflocks, schools, herds

Vehicle Vehicle

traffictraffic

Applications of Behavioral Applications of Behavioral AnimationsAnimations 1987: 1987: Stanley and Stella in: Breaking the Ice, Stanley and Stella in: Breaking the Ice, (short)(short)Director: Larry Malone, Producer: Symbolics, Inc.Director: Larry Malone, Producer: Symbolics, Inc.

1988: 1988: BehaveBehave, (short), (short)Produced and directed by Rebecca AllenProduced and directed by Rebecca Allen

1989: 1989: The Little Death, The Little Death, (short)(short)Director: Matt Elson, Producer: Symbolics, Inc.Director: Matt Elson, Producer: Symbolics, Inc.

1992: 1992: Batman Returns, Batman Returns, (feature)(feature)Director: Tim Burton, Producer: Warner BrothersDirector: Tim Burton, Producer: Warner Brothers

1993: 1993: Cliffhanger, Cliffhanger, (feature)(feature)Director: Renny Harlin, Producer: Carolco.Director: Renny Harlin, Producer: Carolco.

1994: 1994: The Lion King, The Lion King, (feature)(feature)Director: Allers / Minkoff, Producer: Disney.Director: Allers / Minkoff, Producer: Disney.

Applications of Behavioral Applications of Behavioral AnimationsAnimations 1996: 1996: From Dusk Till Dawn, From Dusk Till Dawn, (feature)(feature)Director: Robert Rodriguez, Producer: MiramaxDirector: Robert Rodriguez, Producer: Miramax

1996: 1996: The Hunchback of Notre Dame, The Hunchback of Notre Dame, (feature)(feature)Director: Trousdale / Wise, Producer: Disney.Director: Trousdale / Wise, Producer: Disney.

1997: 1997: Hercules, Hercules, (feature)(feature)Director: Clements / Musker, Producer: Disney.Director: Clements / Musker, Producer: Disney.

1997: 1997: Spawn, Spawn, (feature)(feature)Director: DippDirector: Dipp₫₫, Producer: Disney., Producer: Disney.

1997: 1997: Starship Troopers, Starship Troopers, (feature)(feature)Director: Verhoeven, Producer: Tristar Pictures.Director: Verhoeven, Producer: Tristar Pictures.

1998: 1998: Mulan, Mulan, (feature)(feature)Director: Bancroft/Cook, Producer: Disney.Director: Bancroft/Cook, Producer: Disney.

Applications of Behavioral Applications of Behavioral AnimationsAnimations 1998: 1998: Antz, Antz, (feature)(feature)Director: Darnell/Guterman/Johnson, Producer: DreamWorks/PDI.Director: Darnell/Guterman/Johnson, Producer: DreamWorks/PDI.

1998: 1998: A Bugs Life, A Bugs Life, (feature)(feature)Director: Lasseter/Stanton, Producer: Disney/Pixar.Director: Lasseter/Stanton, Producer: Disney/Pixar.

1998: 1998: The Prince of Egypt, The Prince of Egypt, (feature)(feature)Director: Chapman/Hickner/Wells, Producer: DreamWorks.Director: Chapman/Hickner/Wells, Producer: DreamWorks.

1999: 1999: Star Wars: Episode I--Star Wars: Episode I--The Phantom MenaceThe Phantom Menace, , (feature)(feature)Director: Lucas, Producer: Lucasfilm. Director: Lucas, Producer: Lucasfilm.

2000: 2000: Lord of the Rings: Lord of the Rings: the Fellowship of the Ringthe Fellowship of the Ring (feature) (feature)Director: Jackson, Producer: New Line Cinema.Director: Jackson, Producer: New Line Cinema.

Steering BehaviorsSteering Behaviors

Steering BehaviorsSteering Behaviors seek or flee from a locationseek or flee from a location

pursuit and evasionpursuit and evasion

arrival (position / velocity / time constraints)arrival (position / velocity / time constraints)

obstacle avoidance / containmentobstacle avoidance / containment

path / wall / flow field followingpath / wall / flow field following

group behaviorsgroup behaviors

unaligned collision avoidanceunaligned collision avoidance

Leader followingLeader following

flocking (three components)flocking (three components)

Steering BehaviorsSteering Behaviors

steering behavior demossteering behavior demos

BoidsBoids

Boid Flocking Boid Flocking (three component steering behaviors)(three component steering behaviors)

SeparationSeparation

steer to move away from nearby flockmatessteer to move away from nearby flockmates

AlignmentAlignment

steer toward average heading of nearby flockmates steer toward average heading of nearby flockmates (accelerate to match average velocity of nearby flockmates)(accelerate to match average velocity of nearby flockmates)

CohesionCohesion

steer towards average position of nearby flockmatessteer towards average position of nearby flockmates

Boids:Boids:SeparationSeparation

Boids:Boids:AlignmentAlignment

Boids:Boids:AggregationAggregation

Boids Boids (full behavioral model)(full behavioral model)

Obstacle avoidanceObstacle avoidance

FlockingFlocking

separationseparation

alignmentalignment

cohesioncohesion

Migratory (attraction / repulsion)Migratory (attraction / repulsion)

Boids Web PageBoids Web Page

http://www.red.com/cwr/boids.htmlhttp://www.red.com/cwr/boids.html

Boids VideoBoids Video

boids video...boids video...

Boids Boids (real time (60Hz) and interactive)(real time (60Hz) and interactive)

Evolution of BehaviorEvolution of Behavior

Evolution of BehaviorEvolution of Behavior Agent in simulated worldAgent in simulated world

Evolution of Evolution of

behavioral controllerbehavioral controller

agent morphology agent morphology (see Sims SIGGRAPH 94)(see Sims SIGGRAPH 94)

Fitness based on agentFitness based on agent’’s performances performance

objective fitness metricobjective fitness metric

competitive fitnesscompetitive fitness

Corridor FollowingCorridor Following

Evolution of Corridor Following Evolution of Corridor Following Behavior in a Noisy WorldBehavior in a Noisy World Evolve controller for abstract vehicleEvolve controller for abstract vehicle

Task: corridor followingTask: corridor following

noisy range sensorsnoisy range sensors

noisy steering mechanismnoisy steering mechanism

Evolution of sensor morphologyEvolution of sensor morphology

Corridor Following: goalCorridor Following: goal

Corridor following: fitnessCorridor following: fitness

Corridor Following: ResultsCorridor Following: Results Works wellWorks well

Difficulty strongly related to the representation usedDifficulty strongly related to the representation used

"Competent" controllers easy to find"Competent" controllers easy to find

Reliability of controllers is difficult to measureReliability of controllers is difficult to measure

Coevolution of Tag PlayersCoevolution of Tag Players

Coevolution of Tag PlayersCoevolution of Tag Players The game of tagThe game of tag

symmetrical pursuit and evasionsymmetrical pursuit and evasion role reversalrole reversal

Goal: discover steering behavior for tagGoal: discover steering behavior for tag

Method: emergence of behaviorMethod: emergence of behavior coevolutioncoevolution competitive fitnesscompetitive fitness

Self-organization: no expert knowledge requiredSelf-organization: no expert knowledge required

Competition, Coevolution and Competition, Coevolution and the Game of Tag the Game of Tag (ALife IV, 1994)(ALife IV, 1994)

Coevolution of Taggers Coevolution of Taggers RevisitedRevisited December 1999 to presentDecember 1999 to present

Similar to 1994 work, but:Similar to 1994 work, but:

longer games longer games (25(25®®150)150)

steering angle limitssteering angle limits

obstacles and sensorsobstacles and sensors

demes and speciesdemes and species

improved performanceimproved performance

(faster computers, compilation of evolved programs)(faster computers, compilation of evolved programs)

Evolved Taggers: Evolved Taggers: Obstacles and SensorsObstacles and Sensors

Evolved TaggersEvolved TaggersDemes and SpeciesDemes and Species

Species

Deme

migration

competition

Evolved Taggers:Evolved Taggers:Quality of play over timeQuality of play over time

Evolved Taggers:Evolved Taggers:Handmade program in the openHandmade program in the open

Evolved Taggers:Evolved Taggers:Handmade among obstaclesHandmade among obstacles

Evolved Taggers:Evolved Taggers:Typical competitive fitness testTypical competitive fitness test

Evolved Taggers:Evolved Taggers:Typical competitive fitness test Typical competitive fitness test

Coevolution of Tag Players: Coevolution of Tag Players: ResultsResults It works! It works! (after a lot of tweaking)(after a lot of tweaking)

An ecology of competing behaviors did ariseAn ecology of competing behaviors did arise

Originally, evolved behaviors had been sub-optimalOriginally, evolved behaviors had been sub-optimal

(perhaps do to c(perhaps do to collusion:ollusion: "live and let live") "live and let live")

Finally (after demes, species, and harsh penalties) the Finally (after demes, species, and harsh penalties) the evolved tag players have exceeded the quality of play evolved tag players have exceeded the quality of play of my hand-crafted player.of my hand-crafted player.

ConclusionConclusion Autonomous characters:Autonomous characters:

add richness and complexity to virtual worldsadd richness and complexity to virtual worlds automate creation of groups and crowd scenesautomate creation of groups and crowd scenes allow life-like improvisational actionallow life-like improvisational action can react to unanticipated situations, like user inputcan react to unanticipated situations, like user input

Games and animation provide many applications of, Games and animation provide many applications of, testbeds for, and problems to be solved by research testbeds for, and problems to be solved by research in:in: artificial lifeartificial life artificial intelligenceartificial intelligence evolutionary computation evolutionary computation and other biologically-inspired methodsand other biologically-inspired methods

Slides temporarily removedSlides temporarily removed

Slides temporarily removedSlides temporarily removed

Applications of Applications of Autonomous CharactersAutonomous Characters Behavioral animation Behavioral animation (film and television)(film and television)

coordinated group motioncoordinated group motion extras / background actionextras / background action

Interactive multimedia Interactive multimedia (games / virtual reality)(games / virtual reality) opponents and alliesopponents and allies background charactersbackground characters

Autonomous roboticsAutonomous robotics search / exploration / mappingsearch / exploration / mapping prototyping for evolutionary roboticsprototyping for evolutionary robotics

Theoretical biologyTheoretical biology testing theories of emergent natural behaviortesting theories of emergent natural behavior

Combining Simultaneous Combining Simultaneous BehaviorsBehaviors CombinationCombination

discrete selectiondiscrete selection

behavioral blendingbehavioral blending

Low priority behavior should not be:Low priority behavior should not be:

completely locked outcompletely locked out

allowed to contradict (and perhaps cancel out) a allowed to contradict (and perhaps cancel out) a higher priority behaviorhigher priority behavior

Autonomous Character Autonomous Character Case StudiesCase Studies Hand programmedHand programmed

steering behavior librarysteering behavior library boidsboids hockey playershockey players

EvolutionEvolution

corridor following corridor following tag playerstag players

Steering-BasedSteering-BasedHockey SimulationHockey Simulation

Basic Hockey PlayerBasic Hockey Player Physical modelPhysical model

point masspoint mass limited force and velocitylimited force and velocity collision modeling (as cylinder)collision modeling (as cylinder)

Awareness ofAwareness of position and velocity of players and puckposition and velocity of players and puck position of rink and markingsposition of rink and markings

Behaviors:Behaviors: avoid rink walls and goal netsavoid rink walls and goal nets chase loose puck, skate towards locationchase loose puck, skate towards location

Assigned roleAssigned role(f(forward, wing, defenseman, goalie)orward, wing, defenseman, goalie)

Hockey Hockey Role Role ModelModel DefensemanDefenseman

if you have the puckif you have the puck

if your teammate has the puck...if your teammate has the puck...

if puck is within your zone:if puck is within your zone:- discourage shot on goal- discourage shot on goal- discourage pass to opponent- discourage pass to opponent- don- dont crowd goaliet crowd goalie

do basic hockey play stuffdo basic hockey play stuff

Hockey DemoHockey Demo

hockey demohockey demo

Corridor Following: Corridor Following: Experimental DesignExperimental Design Vehicle modelVehicle model

constant speed constant speed limited steering anglelimited steering angle noisy sensors (arbitrary number & direction)noisy sensors (arbitrary number & direction) noisy steering mechanismnoisy steering mechanism

Genetic ProgrammingGenetic Programming hybrid steady-state modelhybrid steady-state model worst of four noisy trialsworst of four noisy trials population: 2000population: 2000 size limit for evolved programs: 50size limit for evolved programs: 50