Date post: | 29-Nov-2014 |
Category: |
Technology |
Upload: | andrea-tucci |
View: | 813 times |
Download: | 3 times |
Hierarchical Planning in Video Games
by Andrea Tucci
Outline
OutlineIntroduction
Hierarchical Task Networks
Overview
Definition
Expressivity
Complexity
Applications
Video Games AI : A Hierarchical Task Networks approach
Planning in games
HTNs in games
Example of Study : Killzone
Conclusions
References
OutlineIntroductionHTNsVideo Games Ai : HTNsConclusionsReferences
Introduction
Introduction
Hierarchical Planning in a real-time environment application : video games
Overview of Hierarchical Task Networks
Video Games AI, focusing on planning
Killzone : a commercial game with HTNs
Show issues and advantages of Hierarchical Planning in dynamic worlds
IndexIntroductionHTNsVideo Games Ai : HTNsConclusionsReferences
Hierarchical Task Networks
Overview - Classical planning vs HTNs
Classical AI planning : how reach a goal? → actions
an action is represented by an operator < c, e>, c is a precondition, e is the effect of the action
states : configurations of the world generated by applying actions
planner aim is to find the (minimal) path from initial state to the goal state, the path is composed of actions
Hierarchical Task Network (HTN) planning : actions to reach a (composite) task
task : composite or primitive
composite tasks have to be decomposed in subtasks
primitive tasks can be executed directly by applying an operator
planner decomposes compound tasks of a given task network until only primitive tasks are found
OutlineIntroductionHTNs>>Overview>Definition>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
DefinitionGiven a first order HTN language L , we have a vocabulary < V, C, P, F, T, N > where
V = { v1, v2, ... } is an infinite set of variables
C is a finite set of constant symbols
P is a finite set of predicate symbols
F is a finite set of primitive tasks
T is a finite set of compound tasks
N = {n1, n2 ... } is an infinite set of task labels
Goals replaced by tasks. A task can be:
goal task, tasks to achieve
primitive task, obtained by executing the corresponding action
compound task, require decomposition
e.g. “travelByPlane” may require subtasks like buyTicket, goToAirport, fly, goToDestination
OutlineIntroductionHTNs>Overview>>Definition>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
DefinitionA Task Network is a collection of tasks and constraints : (( n1 : α1), ... , (nm : αm), Φ)
Each αi is a task labeled ni
Φ is a boolean formula; can contain :
variable constraints (e.g x = y, x,y ∈ V or x ∈ V, y ∈ C )
temporal ordering (e.g n ≺ n’, n,n’ ∈ N)
truth constraints (e.g (n,l) i.e.“n must be true immediately after n, n ∈ N and l is a literal)
Operator is of the form (f(v1, ..., vk),l1, ..., lk) where f ∈ F, v ∈ V and li is a literal
in this representation there are no preconditions in operators : they are expressed as goal tasks in task networks
A method is a pair (α, d) where α is not a primitive task and d is a task network
This means : in order to achieve α, we use d (i.e. all its subtasks)
Example : (on(v1,v2),d) where d is a task network.
OutlineIntroductionHTNs>Overview>>Definition>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
DefinitionA planning problem is composed of:
an initial state
a task network, that encode the problem to be solved
a planning domain, that contains methods and operators
OutlineIntroductionHTNs>Overview>>Definition>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
DefinitionGiven a planning problem with task network d, a rough Algorithm is :
1.Perform a compound task by decomposing it in subtask (use a method); replace the compound task with the subtasks provided by the used method
2.Are there other compound tasks? If yes goto (1.);
3.Find a totally-ordered ground instantiation of d that satisfies all the constraints.
Example (from blocks world domain) : put v1 on v2
OutlineIntroductionHTNs>Overview>>Definition>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
Syntax used :do precedes a primitive task
perform precedes a compound taskachieve precedes a goal task
ExpressivityHTNs provides all the STRIPS concepts (states, actions, goals)
translation of a STRIPS encoded domain to HTNs domain done in polynomial time
Is the opposite true?
NO!
HTNs can express situations impossible to express using STRIPS operators
HTNs provide a constraint language to express many type of interactions
STRIPS does not have the same level of abstraction
There exist HTN planning domains that can not be represented by any finite number of STRIPS-style operators1
OutlineIntroductionHTNs>Overview>Definition>>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
ComplexityA planning problem P can be
propositional : no variables are allowed
totally ordered : all tasks in any task network are totally ordered
regular : all the task networks contain max one non-primitive task and is the first or the last task
primitive : the task network we need to plan for contains only primitive tasks
Plan existence is
strictly semi-decidable2;
decidable if methods are restricted to be acyclic3
is EXPSPACE-hard and DOUBLE-EXPTIME if the planning probem(P) is restricted to be totally ordered; PSPACE-hard and EXP-TIME if it is propositional4
EXPSPACE-complete if P is regular5
PSPACE-complete if P is regular and propositional6
is NP-complete if P is primitive7
OutlineIntroductionHTNs>Overview>Definition>Expressivity>>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
ComplexityOutlineIntroductionHTNs>Overview>Definition>Expressivity>>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
ApplicationsScheduling
Video Games
Automated Composition of Web Services
Evaluating Terrorist Threats
Evacuation
Military Tasks
Others
OutlineIntroductionHTNs>Overview>Definition>Expressivity>Complexity>>ApplicationsVideo Games Ai : HTNsConclusionsReferences
Applications - SHOPSimple Hierarchical Ordered Planner 2 (SHOP 2) is a
hierarchical planner developed in 2001, written in LISP.
OutlineIntroductionHTNs>Overview>Definition>Expressivity>Complexity>>ApplicationsVideo Games Ai : HTNsConclusionsReferences
(defdomain basic-example ( (:operator (!pickup ?a) () () ((have ?a))) (:operator (!drop ?a) ((have ?a)) ((have ?a)) ())
(:method (swap ?x ?y) ((have ?x)) ((!drop ?x) (!pickup ?y)) ((have ?y)) ((!drop ?y) (!pickup ?x)))))
(defproblem problem1 basic-example ((have banjo)) ((swap banjo kiwi)))
Plans:(((!DROP BANJO) (!PICKUP KIWI)))
Video Games AI : A Hierarchical Task Networks
Approach
Planning in GamesPlanning in games arises in the last years
Planning approach makes video games agents (bots,NPCs) more dynamic w.r.t. old approaches
Planning approach can split action planning from action execution
Planning increases autonomy of NPCs : they have alternate solutions to fulfill a goal
Planning can handle new situations
First Encounter Assault Recon (F.E.A.R.) developed by Monolith was the first commercial game that used planning in its AI
in 2005 won “best AI award” from GameSpot
still considered as one of the best games for AI
GOAP : Goal Oriented Action Planning,decide what to do AND how to do it
Hierarchical planning : more natural in some situations (reasoning on tasks rather than actions)
OutlineIntroductionHTNsVideo Games Ai : HTNs>>Planning in Games>HTNs in Games>Example of Study>KillzoneConclusionsReferences
Planning in GamesOutlineIntroductionHTNsVideo Games Ai : HTNs>>Planning in Games>HTNs in Games>Example of Study>KillzoneConclusionsReferences
Planning in GamesOutlineIntroductionHTNsVideo Games Ai : HTNs>>Planning in Games>HTNs in Games>Example of Study>KillzoneConclusionsReferences
Hierarchical Task Networks in Games
Used in both individual and coordination decisions
In video games AI, actions can be dictated also by
OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>>HTNs in Games>Example of Study>KillzoneConclusionsReferences- error prone +little CPU overhead
- human effort- case specific
Script
HTNs can be used as script generators
offline planning, then the output actions are converted to scripts
Other approach : real-time planning
Examples of StudyUnderstand how HTNs work in practice
Real-Time planning
How planners are integrated with the entire system
Design of HTN based agents
Differences with other implementations
OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>>Example of Study>KillzoneConclusionsReferences
Killzone 2
Killzone 2 First Person Shooter, developed by Guerrilla Games published in 2009
Need for a new multiplayer bot system...why?
In its predecessor, Killzone, coordination between bots of the same team were missing
Goals :
Extensible bot architecture
Improve Killzone AI
Include cooperative behavior
Outperform old AI
Entertain and challenge players
OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
Killzone 2 Quick overview of AI Outline
IntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
Strategy
Bot
Squad
•Goal-based•Defines what to achieve in the mission by using heuristics•Creates squads•Execution of mission tasks
•Coordinate bots (e.g. targets) •Action synchronization
•Execution of Squad task• Inform their squad
Killzone 2 vs KillzoneKillzone 2
Killzone
OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
Killzone 2 vs Killzone (2)OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
Killzone Killzone 2
Approach
Goal selection
Goal satisfaction
Atom part
Front end
Goal-based HTNs
Heuristics High-level HTN choices
Behavior Plan
Action Primitive task
Body goals Body goals
Killzone 2 - AIGoal behavior system of Killzone, replaced by task planning mechanism
Improve flexibility
Dynamic : decomposition of a compound task may lead to different plans (i.e. different behavior)
HTN planner based on SHOP syntax
Specialized primitive tasks : is possible to add facts in the HTN database for a certain amount of time and delete them in any moment(!remember ?period ?fact) (!forget ?fact)
HTN planner is used for evaluate the relevance of goals and find how to reach themKnowledge domains provides how tasks are performed and also tasks priorityAgents use a combination of generalized and specialized knowledge rules
OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
Killzone 2 - AI (2)What should AI do?
Decided by high-level HTNs
Compound task(s) as output
How agent should perform decisions?
Low-level HTNs decisions
Each agent has a set of daemons that provide info about the world
The planner requires those facts before it starts
HTNs used in squad
There is an invisible leader that uses planner for coordination purposes
Informations shared
Squad plans are series of orders for team members
OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
Killzone 2 - Squad AIOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
Strategy Team member messages
Facts
Daemons
Planner Plan
Task execution
Order to members
Killzone 2 - AI (2)(:method (order_member_defend ?inp_member ?inp_id ?inp_level ?inp_marker ?inp_context_hint)
… (branch_advance () ( (!forget member_status ?inp_member **) (!remember - member_status ?inp_member ordered_to_defend ?inp_id) (!start_command_sequence ?inp_member ?inp_level 1) (do_announce_destination_waypoint_to_member ?inp_member) (!order ?inp_member clear_area_filter) (!order ?inp_member set_area_restrictions (call find_areas_to_wp ?inp_member (call get_entity_wp ?inp_marker))) (!order_custom ?inp_member move_to_defend ?inp_marker ?inp_context_hint) (!order_custom ?inp_member send_member_message_custom completed_defend ?inp_id) (order_set_defend_area_restriction ?inp_member (call get_entity_area ?inp_marker)) (!order_custom ?inp_member defend_marker ?inp_marker ?inp_context_hint) (!end_command_sequence ?inp_member) ) ))
OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
Killzone 2 - AI (2)HTN used in individual bot’s decisions
combat behavior
mission specific tasks
other (heal, repair ... )
OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences(:method (select_weapon_and_attack_as_turret ?inp_threat)
(branch_use_bullets // Only use bullets against humanoids and turrets. (and
(or (threat ?inp_threat humanoid) (threat ?inp_threat turret) ) (distance_to_threat ?inp_threat ?threat_distance) (call lt ?threat_distance @weapon_bullet_max_range) ) ((attack_as_turret_using_weapon_pref ?inp_threat wp_bullets))
)
( branch_use_rockets // Don't use rockets against humanoids and turrets. (and
(not (threat ?inp_threat humanoid)) (not (threat ?inp_threat turret)) (distance_to_threat ?inp_threat ?threat_distance) (call lt ?threat_distance @weapon_rocket_max_range) ) ((attack_as_turret_using_weapon_pref ?inp_threat wp_rockets)) ))
Killzone 2 - agent designOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
Killzone 2 - real time environment
Planning ahead is not so good, we are in a real-time environment
Long plans tend to invalidate
☛ Local plans are made
Planning done 5 times per second, 2 times per second for squad
Invalidation of a decided plan of actions : better plan or changing environment
Problem : re-planning so often is
OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
HEAVYHow to deal with this?
Killzone 2 - real time environment (2)
While re-planning, if we come up with a current task, just update it.
Only decompose the part of the plan which we are interested at the moment, delaying the other.
Re-plan only if a certain percentage of preconditions is not valid anymore (i.e. the world changed too much)
Strategies for optimize re-planning
tasks updating
subtasks priority order changes
partial decomposition of tasks
delayed decomposition of tasks
OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
Killzone 2 - conclusionsOld AI (bots from Domination game mode) vs new AI (Group Strategy) : 174 tests
battles in a multiplayer modality
OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
Conclusions
ConclusionsOverview of HTNs
Use of HTNs in video games
offline planning (e.g. Script generation)
online planning
Real-Time (HTN) planning
much more interesting
deal with re-planning
OutlineIntroductionHTNsVideo Games Ai : HTNs>>ConclusionsReferences
Conclusions - HTN in real-time environment
Make planners for real-time environments
build connection to the problem to be solved ... but is like “hard-code” the planner
make full logic representation, gaining expressive power and more generality ... but loosing efficiency
How can we maintain a “generic” planner with efficiency?
Some optimizations:
Use critics to prevent backtracking in an unfeasible plan
critics are used to remove conflicts within a plan as soon as possible
Partial re-planning
why re-plan an entire plan? We can adapt it :(example) agent A needs an object O, so a plan is made. A can’t find O, but he can replace it with another object that has the same function, no need to re-plan for the whole task (go to the shop etc..)
OutlineIntroductionHTNsVideo Games Ai : HTNs>>ConclusionsReferences
Conclusions - HTN in real-time environment
Domain compilation
Introduced for JSHOP planner
unique instance of the planner, generated compiling both the domain and the problem
☛ Expert planner for that problem
In real-time environment we only can compile the domain
Time slicing
Allows to pause and restart the planner
Spread time of planning in more frame
☛ Characters are able to perceive events
Other optimizations based on the problem
Make precondition checking light : heavy computations (e.g. pathfinding) are done by sensors of the agent and stored in a working memory
☛ Sensors provide up-to-date data to the planner
Caching perceptions as working memory facts, containing important informations
Garbage collector : a sensor should delete a fact added that is no relevant anymore
OutlineIntroductionHTNsVideo Games Ai : HTNs>>ConclusionsReferences
Conclusions - HTN in real-time environment
So...are HTNs good or not? Can be used well in a real time environment?
In general, planning in more than one layer is better
at any given level of the HTN, the number of applicable operators is smaller than the total number (reduction of a search space)
invalid plan can be detected at a top level and never be considered for expansion
more suitable for dynamic world : support of re-planning is good
OutlineIntroductionHTNsVideo Games Ai : HTNs>>ConclusionsReferences
ReferencesPresentation based on
HTN Planning : Complexity and Expressivity (K. Erol, J. Hendler, D. Nau)
Ms thesis : A hierarchically-layered multiplayer bot system for a first-person shooter (T. Verweij)
Papers consulted
SquadSmart Hierarchical Planning and Coordinated Plan Execution for Squads of Characters (P. Gorniak, I. Davis)
Agent Architecture Considerations for Real-Time Planning in Games (J. Orkin)
Hierarchical Plan Representations for Encoding Strategic Game AI (H. Hoang, S. Lee-Urban, H. Muñoz-Avila
Planning with Hierarchical Task Networks in Video Games (J.Kelly, A. Botea, S. Koening)
Other material
Hierarchical Planning in Dynamic Worlds (N.Wallace, Ai Programming Wisdom vol. 2)
Killzone 2 Multiplayer Bots slides (Slides by A.Champandard, T.Verweij, R.Straatman)
Images of STRIPS-based and HTNs-based games, are taken from AiGameDev.com
OutlineIntroductionHTNsVideo Games Ai : HTNsConclusions>>References
ReferencesNotes (from HTN Planning : Complexity and Expressivity)
1. Theorem 7
2. Theorem 1
3. Theorem 2
4. Theorem 3
5. Theorem 4
6. Theorem 5
7. Theorem 6
OutlineIntroductionHTNsVideo Games Ai : HTNsConclusions>>References
ThanksThank you for your attention!
@andreatux
OutlineIntroductionHTNsVideo Games Ai : HTNsConclusionsReferences