1
Professor Charles Rich Computer Science Department [email protected]
IMGD 4000 (D 17) 1
Basic Game AI
Technical Game Development II
With material from: Millington and Funge, Artificial Intelligence for Games, Morgan Kaufmann 2009. (Chapter 5)
IMGD 4000 (D 17) 2
Definitions?
§ What is artificial intelligence (AI) ? • subfield of computer science ? • subfield of cognitive science ?
§ What is “AI for Games” ? • versus “academic AI” ?
In games, everything (including the AI) is in service of the player’s experience (“fun”)
• What does it mean for a game AI to “cheat”? Resources: introduction to Buckland, www.gameai.com,
aigamedev.com, www.aiwisdom.com, www.ai4games.org
2
IMGD 4000 (D 17) 3
What’s the AI part of a game?
§ Everything that isn’t graphics (sound) or networking... J
• or physics (though sometimes lumped in)
• usually via the non-player characters
• but sometimes operates more broadly, e.g.,
– Civilization-style games (sophisticated simulations)
– interactive storytelling (drama control)
IMGD 4000 (D 17) 4
“Levels” of Game AI
§ Basic • decision-making techniques commonly used in
almost all games
§ Advanced
• used in practice, but in more sophisticated games
§ Future
• not yet used, but explored in research
3
IMGD 4000 (D 17) 5
This course
§ Basic game AI • decision-making techniques commonly used in
almost all games – basic pathfinding (A*) (IMGD 3000) – decision trees (today) – (hierarchical) state machines (today) – behavior trees (becoming basic) (today)
§ Advanced game AI • used in practice, but in more sophisticated games
– advanced pathfinding (last Thursday) – real-time camera control (last Monday)
IMGD 4000 (D 17) 6
Future Game AI ?
§ Take IMGD 4100 “AI for Interactive Media and Games” • fuzzy logic • other special topics
§ Take CS 4341 “Artificial Intelligence” • machine learning • planning
4
IMGD 4000 (D 17) 7
Two Fundamental Types of AI Algorithms
§ Non-Search vs. Search • Non-Search: amount of computation is predictable
– e.g., decision trees, state machines • Search: upper bound depends on size of search space
(often large) – e.g., minimax, planning – scary for real-time games – need to otherwise limit computation (e.g., threshold)
§ Where’s the “knowledge”? • Non-Search: in the code logic (or external tables) • Search: in state evaluation and search order functions
How about AI Middleware (“AI Engines”)?
§ Recent panel at GDC AI Summit: “Why so wary of AI middleware?”
§ Only one panelist reported completely positive experience • Steve Gargolinski, Blue Fang (Zoo Tycoon, etc.) • Used Havok Behavior (with Physics)
§ Most industry AI programmers still mostly write their own AI from scratch (or reuse their own code)
§ So we are going to look at coding details
IMGD 4000 (D 17) 8
5
AI Coding Theme (for Basic AI)
instead of...
§ a tangle of if-then-else statements
use... § object-oriented paradigm
IMGD 4000 (D 17) 9
First Basic AI Technique: Decision Trees
10 IMGD 4000 (D 17)
See code at:
https://github.com/idmillington/aicore src/dectree.cpp and src/demos/c05-dectree
6
IMGD 4000 (D 17) 11
Decision Trees
§ The most basic of the basic AI techniques
§ Easy to implement
§ Fast execution
§ Simple to understand
IMGD 4000 (D 17) 12
Deciding how to respond to an enemy
if visible? { if close? { attack; } else { if flank? { move; } else { attack; } } } else { if audible? { creep; } }
attack
move attack
creep
yes
visible?
flank?
close? audible? no yes
yes
yes
no
no
no
7
if visible? { if close? { attack; } else { if flank? { move; } else { attack; } } } else { if audible? { creep; }
if visible? { if close? { attack; } else if flank? { move; } else { attack; } } else if audible? { creep; }
IMGD 4000 (D 17) 13
Which would you rather modify?
attack creep
yes
visible?
close? audible? no yes
yes
no
no
move attack
flank?
yes no
??? ???
yes
IMGD 4000 (D 17) 14
O-O Decision Trees (Pseudo-Code)
class Node def decide() //return action
class Boolean : Decision yesNode noNode class MinMax : Boolean minValue maxValue testValue
def getBranch() if maxValue >= testValue >= minValue return yesNode else return noNode
yes
no yes
yes
yes
no
no
no
class Decision : Node
def getBranch() //return node
def decide() return getBranch().decide() class Action : Node def decide() return this
8
IMGD 4000 (D 17) 15
Building an O-O Decision Tree visible = new Boolean... audible = new Boolean... close = new MinMax... flank = new Boolean... attack = new Move... move = new Move... creep = new Move... visible.yesNode = close visible.noNode = audible audible.yesNode = creep close.yesNode = attack close.noNode = flank flank.yesNode = move flank.noNode = attack ...
attack
move attack
creep
yes
visible?
flank?
close? audible? no
yes yes
yes
no
no
no
...or a graphical editor
IMGD 4000 (D 17) 16
Modifying an O-O Decision Tree visible = new Boolean... audible = new Boolean... close = new MinMax... flank = new Boolean... ??? = new Boolean... attack = new Move... move = new Move... creep = new Creep... visible.yesNode = close visible.noNode = audible audible.yesNode = creep close.yesNode = attack close.noNode = ??? ???.yesNode = flank flank.yesNode = move flank.noNode = attack ...
attack creep
yes
visible?
close? audible? no yes
yes
no
no
move attack
flank?
yes no
???
yes
9
IMGD 4000 (D 17) 17
Decision Tree Performance Issues
§ individual node tests (getBranch) typically constant time (and fast)
§ worst case behavior depends on depth of tree • longest path from root to action
§ roughly “balance” tree (when possible) • not too deep, not too wide • make commonly used paths shorter • put most expensive decisions late
yes
no yes
yes
yes
no
no
no
Second Basic AI Technique: (Hierarchical) State Machines
18 IMGD 4000 (D 17)
10
Finite State Machines § Consider AI’s as “agents” that sense, think, then act § But many different rules for behaviors
• Ex: sensing, thinking and acting when fighting vs. running, vs. exploring…
• Can be difficult to keep rules consistent!
§ Try Finite State Machine • Natural correspondence between states and behaviors • Easy: to diagram, program, debug
§ Formally: • Set of states • A starting state • An input vocabulary • A transition function that maps inputs and current state to next state
IMGD 4000 (D 17) 20
Game Example: Soldier Agent
on guard
run away
fight small enemy
large enemy losing fight escaped
11
IMGD 4000 (D 17) 21
Hard-Coded Implementation
class Soldier enum State ON_GUARD FIGHT RUN_AWAY currentState
def update() if currentState == ON_GUARD { if small enemy { currentState = FIGHT start Fighting } else if big enemy { currentState = RUN_AWAY start RunningAway } } else if currentState == FIGHT { if losing fight { currentState = RUN_AWAY start RunningAway } } else if currentState == RUN_AWAY { if escaped { currentState = ON_GUARD start Guarding } }
on guard
run away
fight small enemy
large enemy losing fight
escaped
IMGD 4000 (D 17) 22
Hard-Coded State Machines
§ Easy to write (at the start)
§ Very efficient
§ Notoriously hard to maintain (e.g., debug)
12
IMGD 4000 (D 17) 23
Cleaner & More Flexible O-O Implementation class State def getAction() def getEntryAction() def getExitAction() def getTransitions() class Transition def isTriggered() def getTargetState() def getAction()
class StateMachine states initialState currentState = initialState def update() triggeredTransition = null for transition in currentState.getTransitions() { if transition.isTriggered() { triggeredTransition = transition break } } if triggeredTransition != null { targetState = triggeredTransition.getTargetState() actions = currentState.getExitAction() actions += triggeredTransition.getAction() actions += targetState.getEntryAction() currentState = targetState return actions } else return currentState.getAction()
...add tracing
on guard
run away
fight small enemy
large enemy losing
fight escaped
IMGD 4000 (D 17) 24
Combining Decision Trees & State Machines
§ Why? • to avoid duplicating expensive tests in state
machine:
alert
defend
alarm player in sight AND far
player in sight AND near
13
IMGD 4000 (D 17) 25
Combining Decision Trees & State Machines
alert
defend
alarm
player in sight?
far?
yes
yes
no
no
IMGD 4000 (D 17) 26
Hierarchical State Machines
§ Why?
search
goto disposal
goto trash
see trash
trash disposed
have trash
14
IMGD 4000 (D 17) 27
Interruptions (Alarms), e.g., Recharging?
search
goto disposal
goto trash
see trash
trash disposed
have trash
recharge
low power recharged
recharge
low power recharged
recharge
low power recharged
-search -trash
-disposal 6 - doubled the number of states!
IMGD 4000 (D 17) 28
Add Another Interruption Type?
12 - doubled the number of states again!
hide
battle all clear
-search-recharge
hide
hide
hide
hide hide
15
IMGD 4000 (D 17) 29
Hierarchical State Machine
search
goto disposal
goto trash
see trash
trash disposed
have trash
clean
recharge low power
recharged
• leave any state in (composite) ‘clean’ state when ‘low power’
• ‘clean’ remembers internal state and continues when returned to from “recharged’’
IMGD 4000 (D 17) 30
Add Another Interruption Type?
search
goto disposal
goto trash
see trash
trash disposed
have trash
clean
recharge low power
recharged
hide battle
all clear
7 states (including composite) vs. 12
battle all clear
hide -recharge
-clean
16
IMGD 4000 (D 17) 31
Add Another Interruption Type?
search
goto disposal
goto trash
see trash
trash disposed
have trash
clean
recharge low power
recharged
or could add another layer of nesting...
battle all clear
hide
clean-or-recharge
IMGD 4000 (D 17) 32
Cross-Hierarchy Transitions
§ Why? • suppose we want robot to “top off” battery (even if
it isn’t low) when it doesn’t see any trash
search
goto disposal
goto trash
see trash
trash disposed
have trash
clean
recharge low power
recharged
17
IMGD 4000 (D 17) 33
Cross-Hierarchy Transitions
search
goto disposal
goto trash
see trash
trash disposed
have trash
clean
recharge low power
recharged
no trash and less than 75% power
IMGD 4000 (D 17) 34
HFSM Implementation Sketch
class State // stack of return states def getStates() return [this] // recursive update def update() // rest same as flat machine class Transition // how deep this transition is def getLevel() // rest same as flat machine struct UpdateResult // returned from update transition level actions // same as flat machine
class HierarchicalStateMachine // same state variables as flat machine // complicated recursive algorithm* def update () class SubMachine : HierarchicalStateMachine, State def getStates() push this onto currentState.getStates()
*See full pseudo-code at http://www.cs.wpi.edu/~rich/courses/imgd4000-d17/slides/hsm.pdf Add tracing/debug code!!
18
A “Becoming Basic” AI Technique: Behavior Trees
35 IMGD 4000 (D 17)
What is a Behavior Tree?
§ Drawn as a “tree” like a decision tree
§ Hierarchical like HFSM (hierarchical finite state machine)
§ Real-time version of HTN (hierarchical task network)
§ Builtin to UE4
§ Many plugins for Unity: Behaviour Machine, RAIN, Behave, DecisionFlex, ...
19
What is a Behavior Tree?
§ Specifies a real-time model of task execution
• Switch between tasks in modular fashion
§ Similar to HFSM, but blocks are tasks (“behavior”) not state
§ Early use for NPCs (Halo, Bioshock, Spore)
§ Tree nodes are root, control flow, execution
Search and grasp plan of two-armed robot
https://upload.wikimedia.org/wikipedia/commons/1/1b/BT_search_and_grasp.png
“Behavior” in Behavior Tree
§ Sense, Think, Act § Repeat
20
Sense
Think
21
Act
Behavior Tree with Memory
In UE4, the “Memory” is called “Blackboard”
22
C++ Implementation of Behavior Trees
§ See Chapter 9 of IMGD 4100 textbook: Programming Game AI by Example, Mat Buckland, Wordware, 2005
UE4 Behavior Trees vs. “Standard”
§ UE4 Event Driven • Do not poll for changes, but listen for events that
trigger changes
§ UE4 “conditionals” not at leaf • Allows easier to distinguish control versus task • Allows them to be passive (event driven)
§ UE4 simplifies parallel nodes (typically confusing) • Simple parallel for concurrent tasks • Services for periodic tasks
https://docs.unrealengine.com/latest/INT/Engine/AI/BehaviorTrees/HowUE4BehaviorTreesDiffer/index.html
23
UE4 Behavior Tree
(Describe each next)
UE4 Behavior Tree - Root
24
UE4 Behavior Tree - Composite
(Describe each next)
Composite - Sequence
“And”
25
Composite - Selector
“Or”
Composite – Simple Parallel
26
Service
Decorators
27
Task
Blackboard (Memory)
28
Project 2 - UE4 Behavior Tree
“The Behavior Tree Quick Start Guide walks you through the process of creating a NavMesh, creating an AI Controller, creating a Character that will be controlled by that AI Controller, and creating all the parts necessary for a simple Behavior
Tree.”
https://youtu.be/q6vTg2roI6k