Artificial Intelligence For

Game Developers

e-Institute Publishing, Inc.

©Copyright 2004 e-Institute, Inc. All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system without prior written permission from e-Institute Inc., except for the inclusion of brief quotations in a review. Editor: Susan Nguyen Cover Design: Adam Hoult E-INSTITUTE PUBLISHING INC www.gameinstitute.com Brian Hall. Artificial Intelligence for Game Developers All brand names and product names mentioned in this book are trademarks or service marks of their respective companies. Any omission or misuse of any kind of service marks or trademarks should not be regarded as intent to infringe on the property of others. The publisher recognizes and respects all marks used by companies, manufacturers, and developers as a means to distinguish their products. E-INSTITUTE PUBLISHING titles are available for site license or bulk purchase by institutions, user groups, corporations, etc. For additional information, please contact the Sales Department at sales@gameinstitute.com

i

Table of Contents

CHAPTER 1:PATHFINDING I ..............................................................................................................................................1 INTRODUCTION .....................................................................................................................................................................2 1.1 A FEW GUIDELINES ........................................................................................................................................................4

1.1.1 LOVE AND KISSES...........................................................................................................................................................4 1.1.2 HARD DOES NOT EQUAL FUN.........................................................................................................................................4 1.1.3 PLAY FAIR ......................................................................................................................................................................5

1.2 FUNDAMENTAL ARTIFICIAL INTELLIGENCE .......................................................................................................5 1.2.1 DECISION MAKING..........................................................................................................................................................6 1.2.2 PATHFINDING..................................................................................................................................................................6

1.3 GETTING STARTED.........................................................................................................................................................7 1.4 INTRODUCTION TO PATHFINDING ...........................................................................................................................7 1.4.1 GRAPHS AND PATHFINDING.....................................................................................................................................8 1.5 GRAPH TRAVERSALS...................................................................................................................................................10

1.5.1 NON-LOOK-AHEAD ITERATIVE TRAVERSALS ...............................................................................................................11 1.5.1.1 Random Backstepping..........................................................................................................................................12 1.5.1.2 Obstacle Tracing..................................................................................................................................................12

1.5.2 LOOK-AHEAD ITERATIVE TRAVERSALS........................................................................................................................13 1.5.2.1 Breadth First Search ............................................................................................................................................13 1.5.2.2 Best First Search ..................................................................................................................................................14 1.5.2.3 Dijkstra’s Method.................................................................................................................................................14 1.5.2.4 A* Method ............................................................................................................................................................15

1.5.3 LOOK-AHEAD RECURSIVE TRAVERSALS ......................................................................................................................15 1.6 NON-LOOK-AHEAD ITERATIVE METHODS, IN DEPTH......................................................................................16

1.6.1 RANDOM BACKSTEPPING..............................................................................................................................................16 The Algorithm ..................................................................................................................................................................16

1.6.2 OBSTACLE TRACING .....................................................................................................................................................17 The Algorithm ..................................................................................................................................................................18

1.7 LOOK-AHEAD ITERATIVE METHODS, IN DEPTH................................................................................................19 1.7.1 A NOTE ON IMPLEMENTATION EXAMPLES....................................................................................................................19 1.7.2 BREADTH FIRST SEARCH ..............................................................................................................................................21 1.7.3 BEST FIRST SEARCH......................................................................................................................................................26

1.7.3.1 Max (dx, dy) .........................................................................................................................................................27 1.7.3.2 Euclidean Distance ..............................................................................................................................................27 1.7.3.3 Manhattan (dx + dy) ............................................................................................................................................28

1.8 EDSGER W. DIJKSTRA AND HIS ALGORITHM......................................................................................................30 1.8.1 THREE COMMON VERSIONS OF DIJKSTRA’S..................................................................................................................31

1.8.1.1 Version One..........................................................................................................................................................31 1.8.1.2 Version One Example...........................................................................................................................................32 1.8.1.3 Version Two .........................................................................................................................................................35 1.8.1.4 Version Two Example ..........................................................................................................................................36 1.8.1.5 Version Three.......................................................................................................................................................39 1.8.1.6 Version Three Example ........................................................................................................................................40

1.8.2 OUR VERSION OF THE ALGORITHM...............................................................................................................................44 1.8.3 THE IMPLEMENTATION OF OUR VERSION .....................................................................................................................46

ii

1.9 LOOK-AHEAD RECURSIVE METHODS ....................................................................................................................50 1.9.1 DEPTH FIRST SEARCH ...................................................................................................................................................50

CONCLUSION ........................................................................................................................................................................53 CHAPTER 2: PATHFINDING II ..........................................................................................................................................55 OVERVIEW.............................................................................................................................................................................56 2.1 A*: THE NEW STAR IN PATHFINDING .....................................................................................................................56

2.1.1 HOW A* WORKS...........................................................................................................................................................57 2.1.2 LIMITATIONS OF A* ......................................................................................................................................................60 2.1.3 MAKING A* MORE EFFICIENT.......................................................................................................................................60

2.2 OUR VERSION OF THE ALGORITHM.......................................................................................................................61 2.4.1 TERRAIN TYPES.............................................................................................................................................................69

Jungle ...............................................................................................................................................................................69 Forest ...............................................................................................................................................................................69 Plains................................................................................................................................................................................69 Desert ...............................................................................................................................................................................69 Foothills ...........................................................................................................................................................................70 Mountains.........................................................................................................................................................................70 Roadway...........................................................................................................................................................................70 Trail..................................................................................................................................................................................70 Swamp ..............................................................................................................................................................................70 Water ................................................................................................................................................................................70

2.4.2 UNITS............................................................................................................................................................................71 Infantry.............................................................................................................................................................................71 Wheeled Vehicles..............................................................................................................................................................71 Tracked Vehicles ..............................................................................................................................................................71 Hovercraft ........................................................................................................................................................................71

2.4.3 TERRAIN TYPE VS. UNIT TYPE WEIGHTING HEURISTIC.................................................................................................72 2.4.4 DEFINING THE MAP.......................................................................................................................................................73

2.5 SIMPLIFYING THE SEARCH: HIERARCHICAL PATHFINDING ........................................................................73 2.5.1 A MAP OF THE US.........................................................................................................................................................74 2.5.2 A DUNGEON..................................................................................................................................................................75 2.5.3 A REAL TIME STRATEGY MAP ......................................................................................................................................76

2.6 PATHFINDING ON NON-GRIDDED MAPS................................................................................................................77 2.6.1 SUPERIMPOSED GRIDS...................................................................................................................................................77 2.6.2 VISIBILITY POINTS / WAYPOINT NETWORKS.................................................................................................................78 2.6.3 RADIAL BASIS ...............................................................................................................................................................79 2.6.4 COST FIELDS .................................................................................................................................................................79 2.6.5 QUAD-TREES ................................................................................................................................................................80 2.6.6 MESH-BASED NAVIGATION ..........................................................................................................................................80

2.7 ALGORITHM DESIGN STRATEGY.............................................................................................................................81 2.7.1 CLASS HIERARCHY .......................................................................................................................................................81 2.7.2 MAPGRIDWALKER INTERFACE .....................................................................................................................................81

2.8 GRID DESIGN STRATEGY............................................................................................................................................84 2.8.1 MAPGRID INTERFACE ...................................................................................................................................................84 2.8.2 MAPGRIDNODE CLASS .................................................................................................................................................86 2.8.3 MAPGRIDPRIORITYQUEUE CLASS ................................................................................................................................88

2.9 MFC DOCUMENT/VIEW ARCHITECTURE AND OUR DEMO .............................................................................91

iii

2.9.1 THE FORM VIEW PANEL ...............................................................................................................................................92 2.10 CONCLUSION ................................................................................................................................................................93 CHAPTER 3: DECISION MAKING I..................................................................................................................................95 FLOCKING .............................................................................................................................................................................95 OVERVIEW ............................................................................................................................................................................96 3.1 INTRODUCTION TO FLOCKING................................................................................................................................96

3.1.1 BEHAVIOR BASED MOVEMENT .....................................................................................................................................96 3.1.2 SEPARATION .................................................................................................................................................................98 3.1.3 COHESION.....................................................................................................................................................................99 3.1.4 AVOIDANCE ..................................................................................................................................................................99 3.1.5 ALIGNMENT ................................................................................................................................................................100 3.1.6 OTHER POSSIBLE BEHAVIORS .....................................................................................................................................100

3.2 THE FLOCKING DEMO...............................................................................................................................................101 3.2.1 DESIGN STRATEGIES ...................................................................................................................................................101 3.2.2 MFC AND OUR DEMO .................................................................................................................................................102 3.2.3 OUR IMPLEMENTATION...............................................................................................................................................103 3.2.4 SEPARATION ...............................................................................................................................................................115 3.2.5 AVOIDANCE ................................................................................................................................................................118 3.2.6 COHESION...................................................................................................................................................................120 3.2.7 ALIGNMENT ................................................................................................................................................................122 3.2.8 CRUISING....................................................................................................................................................................124 3.2.9 STAY WITHIN SPHERE ................................................................................................................................................127

CONCLUSION ......................................................................................................................................................................129 CHAPTER 4: DECISION MAKING II: STATE MACHINES ........................................................................................131 OVERVIEW ..........................................................................................................................................................................132

DECISION TREES..................................................................................................................................................................133 STATE MACHINES................................................................................................................................................................134 RULE BASE..........................................................................................................................................................................134 SQUAD BEHAVIORS .............................................................................................................................................................135

4.1 INTRODUCTION TO FINITE STATE MACHINES .................................................................................................137 4.1.1 TRANSITION DIAGRAMS..............................................................................................................................................137

Some Examples ..............................................................................................................................................................137 4.1.2 USES OF FINITE STATE MACHINES..............................................................................................................................139

Animation.......................................................................................................................................................................139 Game State .....................................................................................................................................................................141 Save File System.............................................................................................................................................................142 Artificial Intelligence .....................................................................................................................................................143

4.2 THE STATE MACHINE DEMO...................................................................................................................................143 4.2.1 THE IMPLEMENTATION ...............................................................................................................................................144

The State Machine Class................................................................................................................................................145 The State Class...............................................................................................................................................................148 The Action Classes .........................................................................................................................................................151 The Transition Classes...................................................................................................................................................153

4.3 SCRIPTING IN GAMES................................................................................................................................................157 4.4 INTRODUCTION TO PYTHON...................................................................................................................................158

iv

4.4.1 SCOPE AND WHITESPACE ............................................................................................................................................158 4.4.2 DEFAULT TYPES AND BUILT-INS.................................................................................................................................159 4.4.3 CLASSES......................................................................................................................................................................160 4.4.4 FUNCTIONS .................................................................................................................................................................161 4.4.5 CONTROL STATEMENTS ..............................................................................................................................................162 4.4.6 IMPORTING PACKAGES................................................................................................................................................164 4.4.7 EMBEDDING PYTHON ..................................................................................................................................................164

Boost.Python: Embedding Python using Templates.......................................................................................................165 Making a Module ...........................................................................................................................................................165 Exposing a Function.......................................................................................................................................................165 Exposing a Class ............................................................................................................................................................166

4.5 OUR SCRIPTING ENGINE...........................................................................................................................................167 The Script Engine Class .................................................................................................................................................169 The Scripted Action Class ..............................................................................................................................................176 The Scripted Transition Class ........................................................................................................................................178 Some Examples...............................................................................................................................................................180

CONCLUSION ......................................................................................................................................................................182 CHAPTER 5: WAYPOINT NETWORKS..........................................................................................................................185 OVERVIEW...........................................................................................................................................................................186 5.1 WAYPOINT NETWORKS.............................................................................................................................................187

5.1.1 WAYPOINTS ................................................................................................................................................................187 Discrete Simulations in Continuous Worlds...................................................................................................................188 The Waypoint Class........................................................................................................................................................189

5.1.2 NETWORK EDGES........................................................................................................................................................193 The Network Edge Class ................................................................................................................................................193

5.1.3 THE WAYPOINT NETWORK .........................................................................................................................................195 The Waypoint Network Class .........................................................................................................................................195

5.2 NAVIGATING THE WAYPOINT NETWORK ..........................................................................................................201 5.3 FLOCKING AND WAYPOINT NETWORKS ............................................................................................................210

5.3.1 THE PATHFIND BEHAVIOR ..........................................................................................................................................210 Getting Stuck ..................................................................................................................................................................216

5.3.2 A WORD ON AVOIDANCE ............................................................................................................................................217 5.4 SQUADS AND STATE MACHINES.............................................................................................................................220

5.4.1 METHODS OF SQUAD COMMUNICATION......................................................................................................................220 Direct Control ................................................................................................................................................................220 Poll the Leader ...............................................................................................................................................................221 Events .............................................................................................................................................................................221

5.4.2 THE SQUAD MEMBER..................................................................................................................................................221 The Squad Entity Class...................................................................................................................................................221 The Squad Member State Machine.................................................................................................................................226

5.4.3 THE SQUAD LEADER ...................................................................................................................................................228 The Squad Leader Class.................................................................................................................................................228 The Squad Leader State Machine...................................................................................................................................232

5.5 SETTING UP THE DEMO.............................................................................................................................................234 CONCLUSION ......................................................................................................................................................................241

1

Chapter 1

Pathfinding I

2

Introduction

Artificial intelligence (AI) is one of the critical components in the modern game development project. With the exception of graphics and sound, there are very few elements that are as vitally important when it comes to establishing engaging gameplay. The AI breathes the life of the development team and the designers into the game, and presents the player with the challenges that keep the game interesting and fun to play. In many ways, artificial intelligence is the game. In fact, it is not uncommon for modern game engines to devote as much as 20% or more of their processing time solely to artificial intelligence. Artificial intelligence can be an awkward subject to define because many game developers hold a different set of ideas about what it means and what exactly it constitutes. So let us begin by trying to establish a working definition and then we will move on to what components it encompasses. We all have a pretty good understanding about what the term “artificial” means. It is a man-made substitute for something natural (i.e., a simulation). But “intelligence” immediately begs the question, “What do we mean when we say something is intelligent?” Since this is not a Psychology course, we need not delve too deep to arrive at a useful answer. Certainly there are many ways that people intuitively characterize intelligence. We often think of intelligence as a measure of one’s ability to acquire knowledge and learn from experience. A more utilitarian definition might focus on the use of reasoning faculties to solve problems. By combining and simplifying these various concepts we can arrive at a fairly good standard definition: artificial intelligence is the application of simulated reasoning for the purposes of making informed decisions and solving problems. This seems like a fair enough way to characterize AI and it probably sits well with what most of you had in mind when pondering the nature of the terminology. In the relatively young field of artificial intelligence, the definition we have constructed here is quite applicable. Much research has gone, and continues to go, into creating machines that simulate human intelligence. Some people might place specific methods of solving problems in different AI categories while others bind it all up into a single unified AI concept, but whether taken collectively or not, this is probably a good overall means for understanding AI conceptually. But an important question to ask is, “is this what we mean when we talk about AI in games?” Well, in a manner of speaking, yes. But for our purposes as game developers, we will simplify even further and define artificial intelligence as the means by which a system approximates the appearance of intelligent decision processes. This concept of the “appearance” of intelligence is a very critical point. It is significantly important because there is obviously a distinct difference between artificial intelligence being intelligent and artificial intelligence appearing intelligent. A quick story will illustrate this point. While working on the game Psi Ops: The Mindgate Conspiracy™, our team spent a good deal of time making the various enemies behave more intelligently. We programmed them to crouch and duck, roll out and fire, hide behind cover, throw grenades from cover, dodge objects thrown at them, chase a fleeing player, and pull alarms when they needed help. Despite all this effort, the game designers did not think that the enemies were intelligent enough. They wanted them to be “smarter” and exhibit even more complex and intelligent behaviors.

3

The AI programming team’s initial response to this request was simply to double the hit points of the minions. The results of this small modification may surprise you -- the designers were thrilled with the “intelligence enhancement.” But in reality, it is clear that the characters were no more intelligent than they were before the hit-point increase. Essentially what we learned was that the enemies were dying too quickly for the designers to fully appreciate their intelligence. While it may not immediately jump out at you, this story illustrates an important point: when it comes to artificial intelligence in games, more often than not, it is all about end user perception. That is, for game development, it is fair to say that if it looks smart, it is smart. It is ultimately irrelevant to the player how an AI system makes the decisions it does. They care only that the system seems to produce behaviors that give at least the outward appearance of having been thoughtfully considered. In other words, game AI is essentially a results-oriented concept. How the AI was able to arrive at the decision it did (and we will explore some of the methods for handling decision making in this course) is not remotely as important as the action that took place when the decision was finally made. This is not a new concept of course. Alan Turing’s famous test of machine intelligence is a good example. The Turing Test conceived of locking away a human interrogator in one room while a human and a computer were situated in another room. The means of communication between the two rooms would be text only. The central question was, could the interrogator tell the difference between the human and the computer based on an interactive exchange of questions and dialog? Turing suggested that the measure of the machine’s intelligence was its ability to convince the human interrogator that it was interacting with another human. This is not that far from where we find ourselves today. After all, our goal is to design an AI that makes the player believe that the entities in the game world are behaving the way one would expect an intelligent being to behave. To be sure, this was not always the case in the game field. Indeed the earliest games used virtually no AI at all. Games like Space Invaders, Centipede, Galaga, and Donkey Kong made little effort to convince the player that they were interacting with truly intelligent beings. Hardware limitations resulted in gameplay that relied almost exclusively on pattern-driven events with some varying degree of randomness. Eventually, a good player would recognize and memorize those patterns (e.g., where the next enemy would appear at the top of the screen) and use that knowledge to advance in the game. Today’s games exhibit a considerably more sophisticated set of artificial intelligence than those early titles could provide. As the rendering of more realistic scenery and in-game characters continues the steady march forward, the AI programmer will feel the pressure of having to maintain pace. Physically realistic looking game characters are expected to be paired with realistic looking behavioral traits. However, in modern game systems, graphics and sound have their own dedicated hardware, while AI remains CPU bound. So our job is to build AI systems that can provide that added realism without consuming all of the processing load needed for other important game tasks that aid in player immersion (like realistic physics models for example).

4

1.1 A Few Guidelines

Before we begin discussion about the different types of artificial intelligence we are going to examine in this course, let us first take a moment to establish a few helpful guidelines that will come in handy when designing artificial intelligence for games.

1.1.1 Love and Kisses

One of the most important things to remember is the “KISS” method. KISS is an acronym that stands for “Keep It Simple Stupid”. As games become more advanced and hardware becomes ever faster, the software simulations are becoming more advanced to follow suit. As AI developers, not only do we not have the luxury of infinite processing time, but we will soon learn that we do not need the systems to be overly complicated. This is where our second and related acronym comes in: “LOVE”. This is short for “Leave Out Virtually Everything”. In most cases, the simplicity of the artificial intelligence used in many games would probably shock you. Remember the mantra from earlier: artificial intelligence approximates the appearance of intelligent decision processes. In other words, the AI only needs to convince the player that it is doing something smart. To be sure, there is a fine line that the AI developer must always be aware of. While simplicity can be a beautiful thing, we must be careful to make sure that the simplicity of our implementation does not come at the cost of the player experience. The last thing we want is for our game AI to become predictable and boring.

1.1.2 Hard Does Not Equal Fun

It is probably fair to say that most people do not want to play a game where it takes fourteen hours to complete a small level because the puzzles are too hard or they keep dying over and over again. Indeed it is not complicated to code artificial intelligence that is so difficult to beat that the game is no longer any fun to play. Real time strategy (RTS) games can easily fall into this trap. Since an RTS game is deeply mathematical, it is very easy for a developer, who has advance knowledge of all of the inner workings of the simulation, to build an artificial intelligence system that makes optimal use of its resources, buildings, and units all of the time. The problem with this approach is that a human player could never be as efficient as the computer driven artificial intelligence. Apart from the obvious advantage of pure calculation ability that the computer maintains, on a more practical level, at any given time a player can only interact with so many units and can only view a subset of the entire map. This places him at an impossible disadvantage versus the AI opponent. You should always be aware of these types of imbalances which the game interface imposes on the player. Certainly in this case it is obvious that the player will never be able to play as effectively as the artificial intelligence and allowances will need to be made.

5

1.1.3 Play Fair

Taking a cue from our last rule, as much as possible, you should make the artificial intelligence play by the same rules that the player must abide by. It is supremely frustrating to play a game where the artificial intelligence “cheats”. Once again, real time strategy games often exhibit a tendency to cheat in this fashion. For example, in many cases, the game AI knows where the player units are located and often it does not need to pay the same costs for unit production. This is grossly unfair to the player. But RTS games are not the only culprits. In many first person shooters, the artificial intelligence also knows where the player is, and when alerted, relentlessly chases him down, regardless of where he runs. Additionally, enemies in many shooters do not have to worry about ammunition resources. All of this adds up to an unfair advantage and potentially, a very disenchanted player.

1.2 Fundamental Artificial Intelligence

There are many subcategories of artificial intelligence, each of which has its own usage scenario. Some of these types tend towards the complex and as such, remain more in the domain of academic research than in game development (although there is often some crossover). Classification, for example, is a type of artificial intelligence that typically takes some input data and classifies it as something else. For instance, there may be a system that looks at a small bitmap and determines what letter of the alphabet or number it is. Types of classification systems include neural networks and fuzzy systems. These systems require “training” in order to produce the classifications desired. This training typically involves showing the system examples of each of the things which need to be classified, along with the expected result. The training algorithm then adjusts the internals of the classification system to attempt to produce the desired output on the fly. Here we see the concept of an AI actually learning and getting smarter as a result. These systems can be very difficult to build and perfect and are not widely used in games. However, the idea of simulated “learning” makes these systems very popular in the wider AI research field. We will not be discussing classification much in this course since it tends to be a more esoteric type of AI which is often more complicated than we will need in the typical commercial game. Life Systems are another type of artificial intelligence that is popular in AI research, but less so in games. Genetic algorithms are a popular example. Life Systems work by creating a set of artificial intelligence systems, letting them perform, and then rating them. The ones with the highest rated performances survive and evolve, while the ones with the lowest rated performances are killed off. Clearly we see a relationship to the study of evolution at work here. What is most interesting about these systems is the concept of emergent behaviors. Rather than scripted sequences that are hardwired into the AI, such systems will often produce completely unexpected behaviors that emerge as a result of the AI adapting and maturing over time. This kind of AI can be a lot of fun to observe and study, but there is a major downside. Emergent behaviors tend to lead to systems that are very idiosyncratic. Sometimes you will get the behavior you expect, and other times you will not. The problem is, when you cannot control the outcomes, scenario design becomes very difficult. While there are some games that make use of this AI (e.g., SimCity™), most games do not. The Adaptable AI seminar here at the Game Institute explores

6

an interesting way to approach Life Systems by utilizing concepts from biology, evolution and genetic science. It is a great way to follow up the material we will study in this course if you are interested in investigating this area further. It is worth noting upfront that in this course, we are going to adopt a practical approach to studying AI. Our goal is not to learn a little bit about everything, but rather to zero in on the key areas of study that the typical game AI programmer will need to master if he wants to join a professional programming team. As such, we are going to spend almost all of our time focusing on two of the most fundamental artificial intelligence systems that game developers need to learn and understand: decision making and environment navigation (or “pathfinding”).

1.2.1 Decision Making

Decision making is the core component of all artificial intelligence systems. It is a compilation of routines which help the AI entities decide what they want to do next. This system typically determines decisions such as what to build, when to attack, when to look for cover, when to shoot, when to run away, when to get health, etc. Decision making is the chief means by which the artificial intelligence appears intelligent. State machines, decision trees, and squad behaviors will be examples of decision making that we will talk about in this course.

1.2.2 Pathfinding

Pathfinding is the aspect of artificial intelligence systems which assists the AI driven entities with navigating in the game environment. At its simplest, this means moving the entity from one location to the next without running into things. Pathfinding is arguably the most fundamental type of artificial intelligence for games because without it, entities will remain unable to take on any convincing physical presence. Indeed there are very few genres of games where these algorithms will not be needed. The combination of just these two AI types can lead to the production of virtually any game scenario you desire. You can create NPCs that range from the simplest of life-forms to emotionally complex entities that exhibit sophisticated reasoning ability. Of course, we know that appearances can be deceiving and that under the hood things may not be nearly as complicated as what the behaviors would indicate. But as we discussed earlier, perception is everything and results are what matters. The decision making and environment navigation systems we develop together in this course will serve as a solid foundation for your AI engine and will provide you with the ability to really express yourself creatively in future projects. If you are also taking the Graphics Programming series here at the Game Institute, then combined with this course, you will have a very impressive set of tools that you can leverage to build tech demos for your next interview or even complex games for your own enjoyment. While our focus in this course will be on the AI fundamentals, please feel free to drop by the live discussion sessions if you would like to talk about other types of AI that we will not cover in the text.

7

1.3 Getting Started

Now that we have had a quick overview of the various types of artificial intelligence and some ground rules have been established, we will waste no time in getting started. We will begin our AI studies together with one of the most important areas of artificial intelligence: pathfinding. This topic will serve as a good lead-in for Decision Making because in a sense, pathfinding represents the simplest decision making process of all. The decision centers around the question: how do I get from point A to point B? The decisions themselves are ultimately a choice between directions of travel. That is, if I am ‘here’ and I want to go ‘there’, what is the next step I should take? Should I go left, right, up, down, etc.? What step should I then take after that? And so on. At first you might think that this does not really seem to be artificial intelligence. But recall our earlier emphasis on the appearance of intelligence. Even if your game engine simply selected random points in the world at random times and said to an NPC, “go there”, from the player’s perspective the NPC would appear to have some particular purpose as he wandered by. So by providing the means to get from A to B, we have instilled the NPC with some very basic, but still very important game AI capability. In remainder of this chapter the following questions will be addressed:

What is pathfinding? Why is pathfinding useful or necessary? What is a graph? What is a weighted graph? What is a directed graph? What are the traditional types of pathfinding methods? How are the traditional types of pathfinding methods implemented? Who is E. Dijkstra? What is Dijkstra’s Algorithm? What are some traditional implementations of Dijkstra’s Algorithm?

1.4 Introduction to Pathfinding

Pathfinding is a critical component in just about every game on the shelves. Without pathfinding, autonomous entities would not be able to get from place to place. Think of a simple case where you have a large field and a tractor that needs to go from one side to the other. The tractor starts on one side and proceeds in a straight line to the other side. When the tractor reaches the other side, it stops. Did it use a pathfinding algorithm? Of course! It may be very simple and rudimentary, but it found a straight-line path from one side of the field to the other. If the field had ditches to avoid, this particular algorithm would not have been the best choice. How would the tractor get across in that case? That question is one of many we will learn the answer to as we progress in this course.

8

1.4.1 Graphs and Pathfinding

Pathfinding is ultimately about the traversal of a graph. For our purposes, a graph is simply a set of points connected by paths between them (see Figure 1.1).

A

B

C

D

E

F

G

Figure 1.1 A graph can contain any number of points (also called ‘nodes’) as well as any number of connections between those points. The interesting thing about a graph is that there are actually an infinite number of paths from any point on the graph to any other point on the graph. Pathfinding in the gaming world typically means finding the shortest path. It would not make sense to have an avatar running back and forth between points, or going in circles multiple times before reaching the destination. Graphs can also have costs associated with traveling a particular path between points. A cost is a value that indicates an implied relative expense for choosing one path over another. Depending on how we choose to interpret this value, one path will be deemed more cost-effective to traverse than another, so we will generally want to choose that path over the alternative(s). This type of graph is referred to as a weighted graph (see Figure 1.2).

9

A

B

C

D

E

F

G

1

2

3

5

1

1

1

1

Figure 1.2 Weighted graphs are identical to un-weighted graphs in all respects, except that there are weights/costs associated with the paths between points. This weight might be fixed or it might even be a function. Lastly, there is the concept of a directional graph. In directional graphs, each path between the points can be considered to be one-way (see Figure 1.3).

A

B

C

D

E

F

G

1

2

3

5

1

1

1

1

Figure 1.3 Like a weighted graph, a directional graph can contain weights on the paths between its nodes. These types of graphs can be more complex, as the entity cannot always go back from the direction it came.

10

1.5 Graph Traversals

Pathfinding is simply a shortest distance graph traversal. Let us imagine that we are traversing the un-weighted graph in Figure 1.1, and we want to get from A to E. We would want to travel from A to B to D to E because this path is the most direct route. However, in the case of the weighted graph (Figure 1.2), we would choose the path, A to B to D to F to G to E, as it is the least expensive path in terms of cost. In the case of the directed graph (Figure 1.3), we would choose the path A to B to D to F to E. How we determined that those paths were the least expensive is covered in the next topic. But first we need to redefine our graph. The graphs we have already seen are a bit contrived, so let us change our design to something more like a map, as this is a fairly common graph layout in games. For now, we will define our graph as a regularly spaced grid of points where one can travel N, NE, E, SE, S, SW, W, or NW (Figure 1.4).

Figure 1.4 A graph such as the one in Figure 1.4 represents a map which can be found in many top-down real time strategy games because such games typically take place over vast expanses of terrain. In most games, terrain geometry is built using a uniform grid of polygons, and this lends itself well to such a representation. Units can move in any of the cardinal directions. To prevent movement to a particular node requires only the removal of the node from the graph, simply marking it as impassable. To make it more expensive to travel to a given node, a cost can be associated either with the node itself or the path to the node.

11

Figure 1.5

Although the graph displayed in Figure 1.4 is the primary type of graph we will be examining in this course, it is very difficult to read in that form. From now on, we will look at graphs as shown in Figure 1.5. The green dot represents where we start (our origin) and the red dot represents where we wish to go (our destination). Valid paths of travel are in all of the cardinal directions. As grid squares become increasingly more expensive to travel through, they will become darker gray. Impassable grid squares will be black.

1.5.1 Non-Look-Ahead Iterative Traversals

With this new graph to navigate, and an easier way to look at it, let us briefly talk about some of the methods that might be used to get from origin to destination, but which typically have pitfalls. These methods all have one thing in common; they do not look ahead to find a good path to the goal. They will make their decision based solely on their current position and the position/direction of the goal. The concept itself is simple: take one step at a time towards the goal. The system becomes more complicated when obstacles in the environment must be navigated around. So first, let us examine the most common methods of avoiding obstacles in non-look-ahead iterative traversals.

12

1.5.1.1 Random Backstepping

Figure 1.6

The simplest method is to take one step at a time in the direction of the goal. If an obstacle is encountered, try to step around it. If the obstacle is too large/long (i.e. 3 or more squares long centered on the current location), take a step back in a random direction, and try again. This method encounters serious problems if a cul-de-sac is encountered as it only takes a single step back (see Figure 1.6).

1.5.1.2 Obstacle Tracing

Figure 1.7

13

Another method is to move one step at a time in the direction of the goal, and if an obstacle is encountered, trace around it to the right. This method encounters problems in complicated graphs, as it can get caught in a cycle where it repeats (see Figure 1.7). To prevent this method from entering infinite loops, a common solution is to detect if the path taken traces across the path again. However, this does not make the method any more successful. Another method is to trace to the right until the path is crossed, then trace to the left until the path is crossed. In the case of this graph, tracing to the left would have succeeded.

1.5.2 Look-Ahead Iterative Traversals

Now that we have seen some of the methods that can be used to traverse a graph without looking ahead, let us take a look at some methods that plan the entire path before taking a single step.

1.5.2.1 Breadth First Search

Figure 1.8

One of the most fundamental graph traversal methods is Breadth First Search. This method finds the shortest path in an un-weighted graph by iteratively searching the neighbors of the start position until it reaches the end position (see Figure 1.8). This is a robust method which will always find the shortest path, but it can require much CPU time doing it.

14

1.5.2.2 Best First Search

Figure 1.9

Another method which is very similar to the Breadth First Search is the Best First Search. This method iteratively searches all the neighbors of the start node of an un-weighted graph, but it chooses the neighbor with the perceived best chance of having a path first (see Figure 1.9). This method will always find a path if there is a path to be found, but it may not be the shortest. It sacrifices the shortest path for the speed in which it finds a path using a heuristic.

1.5.2.3 Dijkstra’s Method

Figure 1.10

15

Another method, created by E. Dijkstra and now called Dijkstra’s method, is a very robust method of traversing graphs. This method finds the shortest path of a weighted graph by keeping track of the cost to every node (see Figure 1.10). This is a useful method but it is not the fastest method when dealing with large graphs.

1.5.2.4 A* Method

Figure 1.11

One of the most efficient pathfinding methods available is known as A* (A-star). This method is a very robust weighted graph traversal that makes use of heuristics to find the goal in a timely manner (see Figure 1.11). This method is very powerful as it allows extra knowledge about the graph to be leveraged in the heuristic. This method will be the center of discussion in the next chapter.

1.5.3 Look-Ahead Recursive Traversals

Some graph traversal methods are most easily implemented via recursion. The most popular of these methods is the Depth First Search traversal. Instead of searching all the neighbors as in other methods, it searches deep into the graph first. This method can have problems if the depth of the search is not limited. Many times this is handled by limiting the depth by guessing the distance to the goal, via a heuristic, and increasing the depth until the goal is found. This can be a very time consuming traversal, and dangerous in large graphs, due to recursive depth.

16

1.6 Non-Look-Ahead Iterative Methods, In Depth

We have discussed performing pathfinding one step at a time and mentioned some methods for dealing with navigating around obstacles when they are encountered. Now let us take a closer look at these obstacle avoidance strategies to better understand them.

1.6.1 Random Backstepping

The Random Backstepping (or Random Bounce) method is simple in its execution; it moves a step at a time towards the goal. If it runs into an obstacle, however, it chooses a random direction in which to move and tries moving toward the goal again. Though simple and elegant, this method will fail to get out of deep cul-de-sacs.

The Algorithm

bool RandomBounce(Node start, Node goal) { Node n = start; Node next; while(true) { next = n.getNodeInClosestDirectionToGoal(goal); if (next == goal) return true; while (next.blocked) next = n.getRandomNeighbor(); n = next; } return false; }

Listing 1.1 The method outlined in Listing 1.1 is straightforward. View it in its entirety, and then read on for more detailed discussion. bool RandomBounce(Node start, Node goal)

Let us start with the declaration itself. We will provide a start node and the goal node at which we are attempting to arrive. When we are done, we will return true if we arrived at the goal node, and false if we fail. Node n = start; Node next;

17

First, some locals are defined to keep track of the starting node and the next node to which we plan to go. The node is initialized to be the starting node which was passed in. while(true)

This method will run until we find a solution. Presumably, if we fail at finding a solution, we will give up after some number of iterations rather than cycling forever as this particular loop does. next = n.getNodeInClosestDirectionToGoal(goal);

The method getNodeInClosestDirectionToGoal() is graph specific, but it will always return the best neighbor node to this node which will put us closer to the goal. if (next == goal) return true;

If the next node returned is the goal node, then we successfully made it to the goal. while (next.blocked) next = n.getRandomNeighbor();

Here is where the obstacle avoidance is applied. If the next best node that leads us towards the goal is blocked, a randomly selected neighbor to this node will be selected and tested. This is done until a neighbor node is found which is not blocked. Presumably, we would exit if it was discovered that all of our neighbor nodes are blocked. n = next;

After a valid next node is returned, it will be set to our current node and iteration continues. The idea is to take a step at a time towards the goal, and if the step we wish to take is blocked, pick a random direction and go in that direction instead. This process is continued until we reach our goal. This is very different from any of the algorithms that we are going to implement in this course, as it does not keep track of the path it took; it just knows where it is, and where it wants to go. It also never gives up in its search for the goal. An enhancement to this algorithm might be to add some kind of maximum iteration count which is checked periodically so that it does not continue to fail forever. In many cases this is adequate for some games, even if it may be a little boring.

1.6.2 Obstacle Tracing

Obstacle Tracing is exactly like the Random Bounce method in its means for getting to the goal. The difference is that it attempts to trace around an encountered obstacle until it can head toward its goal again. A more robust method would change the direction in which it traces, or wait until it crosses a line from the start to the goal again before attempting to approach the goal.

18

The Algorithm

bool Trace(Node start, Node goal) { Node n = start; Node next; while(true) { next = n.getNodeInClosestDirectionToGoal(); if (next == goal) return true; while (next.blocked) next = n.getLeftNeighbor(next); n = next; } return false; }

Listing 1.2 The method outlined in Listing 1.2 is identical to the random bounce method with the exception of what it does when its desired node is blocked. Review the code in its entirety and read on for more in-depth discussion. bool Trace(Node start, Node goal)

Let us start with the declaration itself. Just like Random Bounce, a start node and the goal node at which we are trying to arrive will be given. We will return true if we arrived at the goal node, and false if we fail. Node n = start; Node next;

Some local variables are defined to keep track of the current node, and the next node we plan to go to. The start node which was passed in will be initialized as our starting node. while(true)

Just as with RandomBounce, this method will run until a solution is found. Presumably, we will give up after some number of iterations rather than continuing forever as this loop does. next = n.getNodeInClosestDirectionToGoal(goal);

Just as with the Random Bounce method, getNodeInClosestDirectionToGoal() is graph specific, but it will always return the best neighbor node to this node, which will put us closer to the goal. if (next == goal) return true;

19

If the next node returned is the goal node, then we successfully made it to the goal. while (next.blocked) next = n.getLeftNeighbor(next);

This is where the Obstacle Tracing method differs from the Random Bounce method. Rather than grabbing a random neighbor, the neighbor of this node which will take us to the left of the node passed in will be selected. The method getLeftNeighbor is graph specific, but it will always return the node which will take you to the left of the input node. We will keep looking to the left until we get a node that is not blocked. Presumably, we would exit if we found ourselves in a condition where there was no way out. n = next;

After a valid next node is returned, it is set to our current node and iteration is continued. We could make this routine a little more robust by checking to see where we started tracing and trace all the way around until we returned to where we started. If we did return to our start position, we can try to trace the other way, or just give up. We could also calculate the line between our start point and our end point, and if we passed that line twice while tracing, we could give up or try something other than tracing (maybe resorting to a little random bouncing).

1.7 Look-Ahead Iterative Methods, In Depth

We mentioned a variety of look-ahead iterative methods which plan the entire route from the starting point to the goal in advance. This ensures the path chosen will be effective, and in most cases, the shortest. Let us look at these methods in more detail, as well as an implementation example for each.

1.7.1 A Note on Implementation Examples

Both implementation examples we are going to discuss are taken from the code provided in the course projects. typedef std::vector<std::string> stringvec; class MapGridWalker { public: typedef enum WALKSTATE { STILLLOOKING, REACHEDGOAL, UNABLETOREACHGOAL } WALKSTATETYPE;

20

MapGridWalker(); MapGridWalker(MapGrid* grid) { m_grid = grid; } virtual ~MapGridWalker(); virtual void drawState(CDC* dc, CRect gridBounds) = 0; virtual WALKSTATETYPE iterate() = 0; virtual void reset() = 0; virtual bool weightedGraphSupported() { return false; }; virtual bool heuristicsSupported() { return false; } virtual stringvec heuristicTypesSupported() { stringvec empty; return empty; } virtual std::string getClassDescription() = 0; void setMapGrid(MapGrid *grid) { m_grid = grid; } MapGrid *getMapGrid() { return m_grid; } protected: MapGrid *m_grid; };

Listing 1.3 In order to make the chapter demo display the path as it was being discovered, objects derived from MapGridWalker will do their traversals one step at a time during the iterate() method. After they iterate each step of the traversal, drawState() is called to draw the current state of the traversal. For the sake of brevity, we will only discuss the contents of the iterate() method and any heuristic functions that apply to the pathfinding algorithm. Of course, in most games, you would want to find the entire path in one pass rather than iterating repeatedly. Let us discuss a few of the elements of this class in more detail. typedef enum WALKSTATE { STILLLOOKING, REACHEDGOAL, UNABLETOREACHGOAL } WALKSTATETYPE;

The WALKSTATE enumeration will provide information on the progress of the algorithm in its search for the goal. STILLLOOKING represents that the algorithm is still searching for the goal, and has not encountered any problems as yet. REACHEDGOAL means the algorithm has reached the goal and a path has been created. UNABLETOREACHGOAL is returned when the algorithm cannot build a path from the start and goal nodes given. MapGridWalker(); MapGridWalker(MapGrid* grid) { m_grid = grid; }

The class supports a default constructor as well as a constructor, both of which take the grid upon which it will walk as a parameter. If the default constructor is used, the grid must be set separately.

21

virtual ~MapGridWalker();

The class has a virtual destructor so that derived classes can properly clean up their resources if delete is called on a MapGridWalker pointer. virtual void drawState(CDC* dc, CRect gridBounds) = 0;

The drawState method allows the class to draw its current progress into the given device context within the bounds given. This allows the UI to visualize the progress of the algorithm. virtual WALKSTATETYPE iterate() = 0;

The iterate method is the primary interface to the class. This method will perform one iteration of the graph traversal and return its state. virtual void reset() = 0;

This method resets the algorithm so it can start again. virtual bool weightedGraphSupported() { return false; }; virtual bool heuristicsSupported() { return false; } virtual stringvec heuristicTypesSupported() { stringvec empty; return empty; }

These methods inform us if the given class instantiation can support weighted graphs or heuristics. Additionally, the interface for which heuristics are supported are given as strings for the UI. virtual std::string getClassDescription() = 0;

This method returns a description of the class for the UI. void setMapGrid(MapGrid *grid) { m_grid = grid; } MapGrid *getMapGrid() { return m_grid; }

These accessors provide access to the map grid which the walker is traversing.

1.7.2 Breadth First Search

The Breadth First Search algorithm is a simple traversal of the graph, where each neighbor is visited before its siblings are. This method does not care about weighted graphs, as it finds the shortest path in steps from start to finish. The largest problem with this algorithm is encountered with large graphs -- this traversal can take a very long time. bool BreadthFirstSearch(Node start, Node goal) {

22

Queue open; Node n, child; start.parent = NULL; open.enqueue(start); while(!open.isEmpty()) { n = open.dequeue(); n.setVisited(true); if (n == goal) { makePath(); return true; } while (n.hasMoreChildren()) { child = n.getNextChild(); if (child.visited()) continue; child.parent = n; open.enqueue(child); } } return false; }

Listing 1.4 Take a moment and examine the algorithm in Listing 1.4. Notice how it ends in the event that it fails to find a path, unlike the non-look-ahead methods. As mentioned before, this method, as well as all of the other methods we will discuss henceforth, builds the entire path before it takes a single step. Let us look at the method in more detail. bool BreadthFirstSearch(Node start, Node goal)

First, the method expects a starting node and goal node. It will return true if it finds a path to the goal, and false if it does not. This algorithm will find the entire path before returning. Our implementation of this algorithm (which we will address later) takes place on the inside of the while loop so that we can inspect each iteration. Queue open; Node n, child; start.parent = NULL;

A queue is needed to hold the nodes which we plan to visit, as well as a couple of nodes to keep track of where we are currently, and which child we are about to visit. We make sure to set the parent pointer of the starting node to NULL since this is where we started. open.enqueue(start);

23

Our queue is primed by adding the start node to it. This is the first node which we will visit since we are starting at this node. while(!open.isEmpty())

We will iterate through every node in the queue until we find the goal, at which point we will abort the loop. If the queue becomes empty and the goal was not found, we cannot reach the goal node from the start node. n = open.dequeue();

Once inside the loop, the next node from the queue is returned and set as our current node. n.setVisited(true);

We will mark this child as visited so that we do not visit it again. Remember, a neighbor of this node has this node as a neighbor. Thus, it will try to visit this node unless it is specified that it has already been visited before. if (n == goal) { makePath(); return true; }

If the current node is the goal node, we will make the path and return success. while (n.hasMoreChildren())

Next, we will iterate across all the children of the current node. child = n.getNextChild();

The current child is set as the next child of the current node. if (child.visited()) continue;

If this child has been visited already, we will skip it. child.parent = n; open.enqueue(child);

This child’s parent is set as the current node so that we know how we reached it. Then, the child node will be added to the queue to be visited later.

24

To summarize, first a queue is built and our starting position placed onto it. Iteration occurs until our queue is empty or until a path is found. During each step of the iteration, a node is removed from our queue, marked as visited, and tested to see if it is our goal. Then, each of our children is added to the queue if they have not been visited before. This last step is of utmost importance. If the children are not tested for prior visitation, we will never leave the first node, as each neighbor of the first node also has the first node as its neighbor. So basically they would go about adding each other to the queue ad infinitum! Also, by using a queue, we are guaranteeing that each node we add to the queue will be checked in the order they were visited, thereby enforcing the breadth first traversal. Using a stack would make it depth first (with some other modifications, as we will see later). As each node is added to the queue, we also ensure that we set the child’s parent to be the node we grabbed from the queue. This allows a path to be built and also shows how we arrived to our current location. MapGridWalker::WALKSTATETYPE BreadthFirstSearchMapGridWalker::iterate() { if(m_open.size() > 0) { m_n = (MapGridNode*)m_open.front(); m_open.pop(); m_n->setVisited(true); if(m_n->equals(*m_end)) return REACHEDGOAL; // we found our path... int x, y; // add all adjacent nodes to this node // add the east node... x = m_n->m_x+1; y = m_n->m_y; if(m_n->m_x < (m_grid->getGridSize() - 1)) visitGridNode(x, y); // The other directional checks go here, // but that would take a tremendous amount of space // add the north-east node... x = m_n->m_x+1; y = m_n->m_y-1; if(m_n->m_y > 0 && m_n->m_x < (m_grid->getGridSize() - 1)) visitGridNode(x, y); return STILLLOOKING; } return UNABLETOREACHGOAL; // no path could be found }

Listing 1.5 void BreadthFirstSearchMapGridWalker::visitGridNode(int x, int y) { // if the node is blocked or has been visited, early out if(m_grid->getCost(x, y) == MapGridNode::BLOCKED || m_nodegrid[x][y].getVisited()) return;

25

// we are visitable m_open.push(&m_nodegrid[x][y]); m_nodegrid[x][y].m_parent = m_n; }

Listing 1.6 Listing 1.5 and Listing 1.6 contain the important parts of the implementation of the breadth first search as found in our demo. Let us go over this implementation. MapGridWalker::WALKSTATETYPE BreadthFirstSearchMapGridWalker::iterate()

The iterate() method begins on the inside of the while loop from our algorithm snippet. It will return the state of the current iteration in order to inform the application whether the algorithm is still looking for a path, has found a path, or has failed to find a path. if(m_open.size() > 0)

First we will check to see if the queue is empty. If it is, there is no valid path from the start node to the goal node. m_n = (MapGridNode*)m_open.front(); m_open.pop(); m_n->setVisited(true);

The next node from the queue is returned and set as visited. The setVisited method on the node simply sets a Boolean flag. if(m_n->equals(*m_end)) return REACHEDGOAL; // we found our path...

If the new node returned from the queue is the goal node, successful status is returned. This means a path to the goal node has been found. // add all adjacent nodes to this node // add the east node... x = m_n->m_x+1; y = m_n->m_y; if(m_n->m_x < (m_grid->getGridSize() - 1)) visitGridNode(x, y);

Next we check each of our neighbors. In the demo code, a grid represents our graph, so we do some border checking on the grid to ensure we have not overstepped the edge, and if this is true, we visit the node. void BreadthFirstSearchMapGridWalker::visitGridNode(int x, int y)

The method visitGridNode visits the grid node specified at x and y.

26

if(m_grid->getCost(x, y) == MapGridNode::BLOCKED || m_nodegrid[x][y].getVisited()) return;

First, check to see if the node is blocked or visited. If the node is either blocked or visited, it returns without visiting the node. m_open.push(&m_nodegrid[x][y]); m_nodegrid[x][y].m_parent = m_n;

If the node can be visited, it is added to the queue, and the parent of the visited node is set to be the current node in order to track how we arrived there. return STILLLOOKING;

After all of the neighbor nodes are visited, we return STILLLOOKING to indicate further iteration is required to find the goal. To summarize, the queue is checked first to determine whether it is empty. If it is, we cannot reach the goal from our current location. Otherwise, we remove the first node in line from our queue, and test to see if we are at the goal. If so, we return and iteration stops. If the goal has not been reached, we add each of our neighbor nodes, provided that they exist (we live on a 2D grid and therefore edges occur), they are accessible, and we have not visited them yet. We then return STILLLOOKING so that iteration will continue in the next time-slice. The visitGridNode() method wraps up the parts of the algorithm that take care of all the things which need to happen when a node is visited. It checks to see if the node is blocked or visited, and if so, aborts early. If the node is not blocked or visited, the method pushes the node onto the queue, and sets the parent so we can see how we arrived.

1.7.3 Best First Search

The Best First Search is an optimized Breadth First Search in that is uses a heuristic to choose which nodes to traverse next instead of just traversing them in a sequential order. This method also has the same disregard for weighted graphs, as it only cares about the number of nodes needed to traverse from start to finish. This is a good method, as it is much faster than the Breadth First Search, but it might not always find the shortest path to the goal. Path length will be primarily dependent on the appropriateness of the heuristic chosen. bool BestFirstSearch(Node start, Node goal) { PriorityQueue open; Node n, child; start.parent = NULL; open.enqueue (start); while(!open.isEmpty()) {

27

n = open.dequeue(); if (n == goal) { makePath(); return true; }

while (n.hasMoreChildren()) { child = n.getNextChild(); if (child.visited()) continue; child.parent = n; child.setCost = findCost(child, goal); open.enqueue(child); } } return false; }

Listing 1.7 At a first glance, you are probably wondering how this method (Listing 1.7) is any different than the one we just discussed. The magic is in the queue type we use. The best first search uses a priority queue that is keyed on the perceived cost to the goal. This allows the method to start traversing in a direction towards the goal before it would start investigating nodes that take us away from the goal. Aside from the use of a priority queue, the only other difference is the cost heuristic. Let us take a moment to discuss a few common heuristics.

1.7.3.1 Max (dx, dy)

The Max(dx, dy) method uses the maximum of the x distance and the y distance to the goal. Often, this heuristic underestimates the distance to the goal. If the goal is directly above, below, left of, or right of the node (in a grid environment such as ours), the estimate is reasonably accurate. If the node position is diagonal to the goal, the estimate becomes less accurate.

1.7.3.2 Euclidean Distance

The Euclidean distance method uses the standard Euclidean formula to determine the length of the

vector from the node to the goal. This formula is: ( ) ( )22ngng yyxxd −+−= An important point to

remember when dealing with square roots is that, not only are they expensive, they require floating point precision. If your costs are integers, you will lose precision and your estimate will be more inaccurate.

28

1.7.3.3 Manhattan (dx + dy)

The Manhattan (dx + dy) method uses the x distance added to the y distance to the goal. Often this method overestimates the distance to the goal. Like the Max(dx, dy) method, if the goal is directly above, below, left of, or right of the node (in a grid environment such as ours), the estimate is reasonably accurate. If the node position is diagonal to the goal, the estimate becomes less accurate. MapGridWalker::WALKSTATETYPE BestFirstSearchMapGridWalker::iterate() { if(!m_open.isEmpty()) { m_n = m_open.dequeue(); m_n->setVisited(true); if(m_n->equals(*m_end)) { // we found our path... return REACHEDGOAL; } int x, y; // add all adjacent nodes to this node // add the east node... x = m_n->m_x+1; y = m_n->m_y; if(m_n->m_x < (m_grid->getGridSize() - 1)) visitGridNode(x, y); // // The other directional checks go here, // but that would take a tremendous amount of space // // add the north-east node... x = m_n->m_x+1; y = m_n->m_y-1; if(m_n->m_y > 0 && m_n->m_x < (m_grid->getGridSize() - 1)) { visitGridNode(x, y); } return STILLLOOKING; } return UNABLETOREACHGOAL; // no path could be found }

Listing 1.8 void BestFirstSearchMapGridWalker::visitGridNode(int x, int y) { // if the node is blocked or has been visited, early out if(m_grid->getCost(x, y) == MapGridNode::BLOCKED || m_nodegrid[x][y].getVisited())

29

return; // we are visitable m_nodegrid[x][y].m_parent = m_n; m_nodegrid[x][y].m_cost = goalEstimate(&m_nodegrid[x][y]); m_open.enqueue(&m_nodegrid[x][y]); }

Listing 1.9 The above implementation is nearly identical to the prior algorithm with the exception of m_open being a priority queue keyed on the heuristic goal estimate. The node’s cost is calculated only if it is added to the queue, in which case it is set via the goalEstimate() function. This function implements one of the heuristic methods we discussed above. Let us walk through the code and discuss it in more detail. MapGridWalker::WALKSTATETYPE BestFirstSearchMapGridWalker::iterate()

Like all of our implementations, the iterate method does the work, and returns information telling us whether it needs to be called again because it is still searching, whether it found the goal, or whether it cannot find the goal. if(!m_open.isEmpty())

Just like the Breadth First Search, the first thing to check for is an empty queue. If it is empty and we have not found the goal, we cannot get to the goal from the start position. m_n = m_open.dequeue(); m_n->setVisited(true);

Next we take the first item off the priority queue and use that as our current node. We also mark it as visited so we do not try to visit it again. if(m_n->equals(*m_end)) { // we found our path... return REACHEDGOAL; }

Next we determine if our current node is, in fact, the goal. If it is, we return that we have reached our goal. // add all adjacent nodes to this node // add the east node... x = m_n->m_x+1; y = m_n->m_y; if(m_n->m_x < (m_grid->getGridSize() - 1)) visitGridNode(x, y);

We then visit all of our neighbors, just as we did in the Breadth First Search method. Again, we have a visitGridNode method that does the work of visiting the node for us.

30

void BestFirstSearchMapGridWalker::visitGridNode(int x, int y)

As before, it takes an (x, y) coordinate of the node it is to visit on our grid. // if the node is blocked or has been visited, early out if(m_grid->getCost(x, y) == MapGridNode::BLOCKED || m_nodegrid[x][y].getVisited()) return;

It checks to see if the node is blocked or already visited, and if either condition is true, it returns without visiting the node. // we are visitable m_nodegrid[x][y].m_parent = m_n; m_nodegrid[x][y].m_cost = goalEstimate(&m_nodegrid[x][y]); m_open.enqueue(&m_nodegrid[x][y]);

If this node can be visited, it sets the parent of the child node as the current node, determines the cost of this node per our heuristic estimate as discussed above, and adds the node to the priority queue. The priority queue automatically sorts the node into its proper place in the queue. return STILLLOOKING;

After we visit all of our neighbor nodes, we return that we need more iteration to find the goal.

1.8 Edsger W. Dijkstra and his Algorithm

Edsger W. Dijkstra was born in 1930 in The Netherlands. He was one of the first to think of programming as a science in itself and actually called himself a programmer by profession in 1957. The Dutch government did not recognize programming as a real profession, however, so he had to re-file his taxes as “theoretical physicist.” He won the Turing Award from the Association for Computing Machinery in 1972, and was appointed to the Schlumberger Centennial Chair in Computer Science at the University of Texas in 1984. He also is responsible for developing the prized “shortest-path” algorithm that has been integral to many computer games. E. Dijkstra’s shortest path algorithm is so useful and well-known, that it has simply been dubbed the “shortest path algorithm.” It is so popular that if you were to mention pathfinding to most programmers, they would assume you were speaking of this particular algorithm. Interestingly enough, E. Dijkstra’s algorithm varies a bit depending on where you look it up.

31

1.8.1 Three Common Versions of Dijkstra’s

Let us analyze three common versions of the Dijkstra’s shortest path algorithm in a little more detail. First the algorithm will be shown, and then an example will be walked through for each of the versions.

1.8.1.1 Version One

procedure dijkstra(w, a, z, L) L(a) := 0 for all vertices x ≠ a do L(x) := ∞ T := set of all vertices // T is the set of vertices whose shortest distance // from a has not been found while z ∈ T do begin choose v ∈ T with minimum L(v) T := T – {v} for each x ∈ T adjacent to v do L(x) := min{L(x), L(v) + w(v, x)} end end dijkstra

Listing 1.10

Listing 1.10 shows a version of Dijkstra’s algorithm where it finds the shortest path from a to z. In this algorithm, w denotes the set of weights where w(i, j) is the weight of the edge between point i and j, L(v) denotes the current minimum length from a to v. This particular algorithm does not track the actual path from a to z, just the length of the path. The algorithm works by first initializing L for all vertices, except a, to a very large value. It then chooses a vertex with the shortest length, and removes it from the set of all vertices. Then for each adjacent vertex, it calculates the new minimum distance.

32

1.8.1.2 Version One Example

a

b c

d e

f g

z

1

23

4

5

7

2

2

1

3

4

6

Figure 1.12 For the walkthrough of this algorithm, let us use this simple graph (Fig 1.12) as our example. The vertices are marked a through z, and the cost for a given edge is labeled nearest the edge center.

a

b c

d e

f g

z

1

23

4

5

7

2

2

1

3

4

6

∞

0 ∞

∞

∞

∞∞

∞

Figure 1.13 When the algorithm begins, it initializes the lengths from a to all the other vertices to a very large value. It also places each of the vertices in a list.

33

a

b c

d e

f g

z

1

23

4

5

7

2

2

1

3

4

6

2

0 ∞

∞

∞

∞1

∞

Figure 1.14 In the first iteration, the algorithm naturally selects the a vertex, as it was initialized to 0 during initialization and is lowest. It is removed from the vertex list, and all of the vertices adjacent to a (b and f) have their L values calculated.

a

b c

d e

f g

z

1

23

4

5

7

2

2

1

3

4

6

2

0 4

∞

∞

61

∞

Figure 1.15 In the second iteration, the algorithm chooses vertex f as it has the lowest cost, and it is removed from the vertex list. The adjacent vertices (d and g) have their L values calculated, and the algorithm moves on.

34

a

b c

d e

f g

z

1

23

4

5

7

2

2

1

3

4

6

2

0 4

4

6

61

∞

Figure 1.16 In the third iteration, the algorithm chooses vertex b, as it has the lowest cost, and it is removed from the vertex list. The adjacent vertices (d, e, and c) then have their L values calculated. The z vertex is now the only vertex that has not had an L value calculated.

a

b c

d e

f g

z

1

23

4

5

7

2

2

1

3

4

6

2

0 4

4

6

61

5

Figure 1.17 In the fourth iteration, the algorithm chooses the c vertex, and it is removed from the vertex list. The adjacent vertices (z and e) have their L values calculated, and we now have all of the vertices in the graph with an L value. The algorithm would next pick d, and then finally z. When z is removed from the vertex list, the algorithm stops and it is seen that the shortest path from a to z is 5 units long. Of course, without going back and looking, we have no way of knowing that path to take is a-b-c-z, so it would be a good idea to keep track of this. We do that in our algorithm, as you will see later.

35

1.8.1.3 Version Two

Given the arrays distance, path, weight and included, initialize included[source] to true and included[j] to false for all other j. Initialize the distance array via the rule if j = source distance[j] = 0 else if weight[source][j] != 0 distance[j] = edge[source][j] else if j is not connected to source by a direct edge distance[j] = Infinity for all j Initialize the path array via the rule if edge[source][j] != 0 path[j] = source else path[j] = Undefined Do Find the node J that has the minimal distance among those nodes not yet included Mark J as now included For each R not yet included If there is an edge from J to R If distance[j] + edge[J][R] < distance[R] distance[R] = distance[J] + edge[J][R] path[R] = J While all nodes are not included

Listing 1.11 The algorithm in Listing 2.2 utilizes 3 arrays to do its work. It is similar to the former version in that the distance array is the L value, but it differs in that it tracks the actual path to the goal as well. It also uses an array to mark vertices that have been chosen rather than removing them from the list. The algorithm runs to completion when all nodes have been included. We could shorten the algorithm easily by changing the while loop to “while the goal node is not included” since once the goal node is included, we have the shortest path.

36

1.8.1.4 Version Two Example

5

4

1

3

2

410

1421

800

310

200

400

612

2985

Figure 1.18 Let us use the graph in Figure 1.18 for this version of the algorithm. We will walk through the iteration of the algorithm and examine the contents of the various arrays along the way. The walk-through for this graph will use our algorithm starting at vertex 1.

5

4

1

3

2

410

1421

800

310

200

400

612

2985

distance[2] = 800 path[2] = 1 included[2] = falsedistance[3] = 2985 path[3] = 1 included[3] = falsedistance[4] = 310 path[4] = 1 included[4] = falsedistance[5] = 200 path[5] = 1 included[5] = false

Figure 1.19 First we initialize our distance, path, and included arrays. All of the path array locations are set to 1 (where we started) and none of the other vertices are marked as included.

37

5

4

1

3

2

410

1421

800

310

200

400

612

2985

distance[2] = 800 path[2] = 1 included[2] = falsedistance[3] = 2985 path[3] = 1 included[3] = falsedistance[4] = 310 path[4] = 1 included[4] = falsedistance[5] = 200 path[5] = 1 included[5] = true

Figure 1.20 In the first iteration, we find the vertex with the smallest distance, which is vertex 5. We then mark that vertex as included, and check to see if the distances from vertex 1 to vertex 5’s neighbors are smaller than the one already stored, which they are not.

5

4

1

3

2

410

1421

800

310

200

400

612

2985

distance[2] = 800 path[2] = 1 included[2] = falsedistance[3] = 1731 path[3] = 4 included[3] = falsedistance[4] = 310 path[4] = 1 included[4] = truedistance[5] = 200 path[5] = 1 included[5] = true

Figure 1.21

38

In the second iteration, we see that vertex 4 has the shortest distance and is not included. We mark it as included, and then update our distances. This is because the distance from vertex 1 to vertex 4 to vertex 3 is shorter than the distance from vertex 1 directly to vertex 3. We also update the path to vertex 3 to indicate that travel through vertex 4 from the source is the shortest path.

5

4

1

3

2

410

1421

800

310

200

400

612

2985

distance[2] = 800 path[2] = 1 included[2] = truedistance[3] = 1210 path[3] = 2 included[3] = falsedistance[4] = 310 path[4] = 1 included[4] = truedistance[5] = 200 path[5] = 1 included[5] = true

Figure 1.22 In the third iteration, we see that vertex 2 has the shortest distance, so we mark it included. We also see that by traveling through vertex 2 to vertex 3, it is shorter than traveling through vertex 4, so we update the distance for vertex 3 as well as the path. In the fourth iteration, all that is left is vertex 3, so we mark it as included. Nothing changes in terms of path or distance, so we now have the shortest distance as well as the path to all vertices from vertex 1.

39

1.8.1.5 Version Three

void Dijkstra( Table T ) { Vertex V, W; while( true ) { V = Smallest Unknown Distance Vertex; if( V == Not A Vertex ) break; T[ V ] .Known = true; for Each W Adjacent To V if( !T[ V ].Known ) { // Update W. decrease ( T[ W ].Dist To T[ V ].Dist + C ( V, W ); T[ W ].Path = V; } } }

Listing 1.12

This algorithm is very similar to the previous algorithm. It maintains a list of vertices that are known, the distance to each vertex, as well as the path to each vertex. The biggest difference is more of an architectural change. Rather than keeping data in arrays, a table is used to manage the weights, and Vertex structures are used to store the related path data. This method also finds the shortest path to each vertex in the graph.

40

1.8.1.6 Version Three Example

v3

v1 v2

v4

v6 v7

v5

5

4 3

1

48

2

10

2

1

6

Figure 1.23 For this last example, we will take a look at how things change when using a directed graph rather than a non-directed graph. The graph above is a directed graph where travel is only allowed in the direction of the arrows. Let us traverse this graph starting at v1.

v3

v1 v2

v4

v6 v7

v5

5

4 3

1

48

2

10

2

1

6

∞ ∞

∞

∞

∞ ∞

0

Figure 1.24 First we initialize all of the nodes to unknown, and the distances to infinity. We also set the parent vertex, for each vertex, to 0.

41

v3

v1 v2

v4

v6 v7

v5

5

4 3

1

48

2

10

2

1

6

∞ 1

2

∞

∞ ∞

0

Figure 1.25 In the first iteration, we mark the starting vertex as known, and update the distance members of v2, and v4. We also set both parents to v1.

v3

v1 v2

v4

v6 v7

v5

5

4 3

1

48

2

10

2

1

6

13 1

2

3

9 5

0

Figure 1.26

In the second iteration, we mark v4 as known, as it has the shortest distance so far, and update all of its neighbor’s distances. For those neighbor vertices it does set the distance for, v4 makes itself their parent vertex as well.

42

v3

v1 v2

v4

v6 v7

v5

5

4 3

1

48

2

10

2

1

6

3 1

2

3

9 5

0

Figure 1.27 In the third iteration, we mark v2 as known, and update all of its neighbor’s distances. In this case, there are no neighbors that need to be updated.

v3

v1 v2

v4

v6 v7

v5

5

4 3

1

48

2

10

2

1

6

3 1

2

3

9 5

0

Figure 1.28 In the fourth iteration, we mark v5 as known, and try to update neighbors. Again, no updates are needed.

43

v3

v1 v2

v4

v6 v7

v5

5

4 3

1

48

2

10

2

1

6

3 1

2

3

8 5

0

Figure 1.29 In the fifth iteration, we mark v3 as known, and update neighbors. This time, we actually find a shorter route to v6, and update its distance and make v3 its parent vertex.

v3

v1 v2

v4

v6 v7

v5

5

4 3

1

48

2

10

2

1

6

3 1

2

3

6 5

0

Figure 1.30 In the sixth iteration, we mark v7 as known, and update its neighbors. Again we find a shorter path to v6, so it is updated and v7 is made its parent vertex. The last iteration, we mark v6 as known, and no updates are needed. Now we are done.

44

1.8.2 Our Version of the Algorithm

bool DijkstraSearch(Node start, Node goal) { PriorityQueue open; Node n, child; start.parent = NULL; start.cost = 0; open.enqueue(start); while(!open.isEmpty()) { n = open.dequeue(); n.setVisited(true); if (n == goal) { makePath(); return true; } while (n.hasMoreChildren()) { child = n.getNextChild(); COSTVAL newcost = n.cost + cost(n, child); if (child.visited()) continue; if (open.contains(child) && child.cost <= newcost) continue; child.parent = n; child.cost = newcost; if (!open.contains(child)) open.enqueue(child); else open.reenqueue(child); } } return false; }

Listing 1.13

Our version of the algorithm is very much like the last two versions we studied. That is, we will keep track of the shortest distance we have found thus far at each node and also keep track of which nodes we have visited. The biggest difference is how we pick which node to next traverse through. We use a priority queue to sort our unvisited nodes in order of their cost. We then grab the top one off of the queue and do our traversal. Let us discuss this particular version in more detail since it is the version we will be using in our demo.

45

bool DijkstraSearch(Node start, Node goal)

Like the other algorithms, this one expects a start node and a goal node, and returns if it was capable of finding a path. PriorityQueue open; Node n, child;

As in Best First Search, a priority queue is used to keep track of the nodes we need to visit, and we will have a current node as well as the current child we are visiting of the current node. start.parent = NULL; start.cost = 0;

We will start out by setting the parent of our starting node to NULL to denote it is indeed the start. We also set the cost to 0. open.enqueue(start);

Next we will initialize the queue by adding our start node to it since we will want to visit it first. while(!open.isEmpty())

While the queue is not empty, we will iterate through all the children of each node in the queue. If the queue empties before we find the goal, there is no path from the start node to the goal. n = open.dequeue(); n.setVisited(true); if (n == goal) { makePath(); return true; }

For each iteration, we will grab a node off the queue and make it our current node. We also mark this node as visited so we do not visit it again. If this node is the goal node, we found the path, so we make it and return success. while (n.hasMoreChildren())

We then iterate across each of the current nodes’ children. child = n.getNextChild(); COSTVAL newcost = n.cost + cost(n, child);

46

For each child, we compute the cost from this node to the child and add it to the cost which this node has stored as the computed cost from the start node to it. This allows us to keep track of the total cost it takes to get from the start node to every other node as we visit it. if (child.visited()) continue; if (open.contains(child) && child.cost <= newcost) continue;

Here is where the algorithm starts to differ from the other algorithms we have discussed so far. Like the other algorithms, if we have visited this child node, we do not visit it again. But if we have not visited this child, but we have already determined that we need to visit it, we check to see if the cost we’ve computed previously for this child is less than the cost we just computed. This allows us to update the cost to this particular child if we found a shorter path to this child. If we have a cost for this child computed already and it is shorter than the cost we just found, we ignore this path to the child node since we have a better one already. child.parent = n; child.cost = newcost; if (!open.contains(child)) open.enqueue(child); else open.reenqueue(child);

If we determine that we want to visit this child, we set its parent to be our current node, set its cost to be our computed cost, and if the queue does not already contain the child, we add it. If the queue does contain the node, we inform the queue that it needs to reinsert the child into its proper position now that its cost has changed.

1.8.3 The Implementation of Our Version

MapGridWalker::WALKSTATETYPE DijkstrasMapGridWalker::iterate() { if(!m_open.isEmpty()) { m_n = m_open.dequeue(); m_n->setVisited(true); if(m_n->equals(*m_end)) { // we found our path... return REACHEDGOAL; } int x, y; // add all adjacent nodes to this node x = m_n->m_x + 1; y = m_n->m_y; if(m_n->m_x < (m_grid->getGridSize() - 1))

47

visitGridNode(x, y); // All other directions here, but that takes up too much space // add the north-east node... x = m_n->m_x + 1; y = m_n->m_y - 1; if(m_n->m_y > 0 && m_n->m_x < (m_grid->getGridSize() - 1)) visitGridNode(x, y); return STILLLOOKING; } return UNABLETOREACHGOAL; }

Listing 1.14 void DijkstrasMapGridWalker::visitGridNode(int x, int y) { int newcost; bool inqueue; if(m_grid->getCost(x, y) == MapGridNode::BLOCKED || m_nodegrid[x][y].getVisited()) return; newcost = m_n->m_cost + m_grid->getCost(x, y); inqueue = m_open.contains(&m_nodegrid[x][y]); if(inqueue && m_nodegrid[x][y].m_cost <= newcost) { // do nothing... we are already in the queue // and we have a cheaper way to get there... } else { m_nodegrid[x][y].m_parent = m_n; m_nodegrid[x][y].m_cost = newcost; if(!inqueue) { m_open.enqueue(&m_nodegrid[x][y]); } else { m_open.remove(&m_nodegrid[x][y]); m_open.enqueue(&m_nodegrid[x][y]); } } }

Listing 1.15 Here is the actual implementation from our demo. Similar to the algorithms we discussed already, it makes use of the priority queue to keep our nodes sorted in order of cost. We grab the top node off our

48

queue, see if it is traversable, update all of its neighbors, and continue on until we find the goal node. Let us go over our implementation of the algorithm in more detail. MapGridWalker::WALKSTATETYPE DijkstrasMapGridWalker::iterate()

As in all our implementations, the iterate method starts inside the while loop of our algorithm snippet. It returns a status of needing more iteration because it is still looking, whether it found the goal, or if it cannot find the goal. if(!m_open.isEmpty())

We begin by checking to see if the queue is empty. If it is, we cannot find a path from the start to the goal. Otherwise we begin another iteration. m_n = m_open.dequeue(); m_n->setVisited(true);

We grab the next node off the queue and make it our current node. We also mark that node as visited so that we do not visit it again. if(m_n->equals(*m_end)) { // we found our path... return REACHEDGOAL; }

If the current node is, in fact, the goal node, we have reached our goal and return success. // add all adjacent nodes to this node // add the east node... x = m_n->m_x + 1; y = m_n->m_y; if(m_n->m_x < (m_grid->getGridSize() - 1)) { visitGridNode(x, y); }

We then check each of the current node’s neighbors. Again we make sure to stay within the bounds of our grid and let the visitGridNode method do the work. void DijkstrasMapGridWalker::visitGridNode(int x, int y)

This method takes an (x, y) coordinate and visits the corresponding grid node. if(m_grid->getCost(x, y) == MapGridNode::BLOCKED || m_nodegrid[x][y].getVisited()) return;

49

First it checks to see if the node in question is blocked or already visited. If it is, it returns and does not visit the node. newcost = m_n->m_cost + m_grid->getCost(x, y); inqueue = m_open.contains(&m_nodegrid[x][y]);

Next it computes the cost to this child node via the current node. Also, we check to see if the node in question is already in our queue. if(inqueue && m_nodegrid[x][y].m_cost <= newcost) { // do nothing... we are already in the queue // and we have a cheaper way to get there... }

If we are already in the queue, and the new cost we computed is greater than the cost the child node already has, we ignore the node since we already have a cheaper way to get there. m_nodegrid[x][y].m_parent = m_n; m_nodegrid[x][y].m_cost = newcost;

If we determine we have a cheaper way to get to the child node, we set its parent to the current node, and its cost to the cost we computed for it. if(!inqueue) { m_open.enqueue(&m_nodegrid[x][y]); } else { m_open.remove(&m_nodegrid[x][y]); m_open.enqueue(&m_nodegrid[x][y]); }

If the child node is not in the queue, we simply add it. If it is in the queue, we remove it and add it again so it can be put in its proper place. return STILLLOOKING;

After we visit all of the neighbor nodes of the current node, we return STILLLOOKING to indicate that we need more iterations to find the goal.

50

1.9 Look-Ahead Recursive Methods

As discussed earlier, there are some look-ahead pathfinding methods that are most easily implemented with recursion. The prime example we discussed is the Depth First Search. Let us take a look at this algorithm, and discuss it in detail.

1.9.1 Depth First Search

The Depth First Search algorithm is a simple traversal for weighted or non-weighted graphs, in which siblings are visited before neighbors. The method has a few caveats. The Depth First Search method is recursive in nature, and unless the depth to which it searches is constrained, it will search to an infinite depth in an attempt to find its goal. This method also has a tendency to wrap around unless we constrain it to moving towards the goal, if at all possible. bool DepthFirstSearch(Node node, Node goal, int depth, int length) { int d; if (node == goal) { makePath(); return true; } if (depth < MAXDEPTH) { while (node.hasMoreChildren()) { child = node.getNextChild(); d = node.dist + node.getCost(child); if (!isTowardsGoal(node, child, goal)) continue; if (child.visited() || d > child.cost) continue; child.parent = node; child.visited = true; child.cost = d; if (DepthFirstSearch(child, goal, depth+1, child.cost)) return true; child.visited = false; } } return false; }

Listing 1.16

51

After looking over the algorithm in Listing 1.16, you should notice it is recursive in nature rather than iterative. Use of recursion allows us to leverage the call stack rather than maintaining our own stack. We might have implemented this method using iteration, and it would have looked much like the others except for its use of a stack rather than a queue. However, using recursion for this method is much more elegant. Let us go over this algorithm in a little more detail. bool DepthFirstSearch(Node node, Node goal, int depth, int length)

The recursive method DepthFirstSearch takes a node to search from, a goal to get to, the depth to search to, and the current cost. Each call to this method will change the node, depth, and length parameters while the goal will remain the same. if (node == goal) { makePath(); return true; }

If the node passed in is the goal, we make the path and return our success. The true return value will trigger a full recursive unroll to get us out and back to the initial caller of the method. if (depth < MAXDEPTH)

If we have not exceeded our depth, we search further, otherwise we will return false to say we did not find the goal. while (node.hasMoreChildren())

If we have not exceeded our depth, we will iterate across all the passed in node’s children. child = node.getNextChild(); d = node.dist + node.getCost(child);

For each child, we will compute the distance to this child by adding our passed in node’s pre-computed cost to the cost of getting to the child node. if (!isTowardsGoal(node, child, gloal)) continue;

Here we do a little trickery to keep the algorithm from doing loops. We check to see if the child helps us to get towards the goal. The implementation of isTowardsGoal is graph specific, but it will return true if the passed-in node is closer to the goal and false if it is not. If the child does not take us closer to the goal, we do not traverse it since it might take us on a crazy, winding path. if (child.visited() || d > child.cost) continue;

52

If the child has been visited already or the child’s cost is cheaper than the computed cost, we also skip this child. child.parent = node; child.visited = true; child.cost = d;

Next we set the child’s parent to be the node passed in, mark the child as visited, and set its cost to be the cost we computed. if (DepthFirstSearch(child, goal, depth+1, child.cost)) return true;

We then recursively call the method again using the child node as the node to pass in, increment the depth, and pass in the child’s cost as the length. If this returns true, we found the goal and return immediately. This will unroll the recursive stack back to the initial caller. child.visited = false;

Here is another tricky bit. After the recursive call, we mark the child as unvisited again, since we might need to go through it via another depth traversal. To summarize, we start by calling DepthFirstSearch() and pass the start node, the end node, a depth of 1, and a length of 0. The algorithm would first check the node passed in to see if it is the goal. If so, we make the path and return all the way out of the recursive stack. Otherwise, if our depth is less than the max depth we wish to search to, we iterate through each child. If the child is in the direction of the goal, the child has not been visited, and the cost to the child is less than the child’s current remembered cost, we recursively call DepthFirstSearch on that child, incrementing depth and passing the cost to the child. It is important to be sure that the cost to the child is better than the last cost, and that we are moving in the direction of the goal. Otherwise, the algorithm will create curly, winding paths that lead nowhere. Also, it is important to mark nodes as visited as we traverse into the graph, and unmark them on our way back out. This is so that we do not visit the same node more than once on the way into the graph, but we are sure to try them again if a search to a given depth fails. An added improvement that could be made is to iteratively increase the MAX_DEPTH value to enable searching deeper into the graph until we find a goal. One might also attempt to calculate a beginning MAX_DEPTH using a heuristic goal estimate and implement the iterative deepening from that starting point so as to reduce the number of deepening iterations.

53

Conclusion

In this chapter we have discussed pathfinding at its most basic. We talked about graphs, what they are, and why they are important in pathfinding. We also examined single step path traversals, as well as iterative and recursive methods of pathfinding. These latter methods determine optimal paths through the graph to the goal. Finally, we looked at some specific implementations for some of the common algorithms used in pathfinding. In the next chapter we will expand our understanding of pathfinding by looking at more complex pathfinding methods such as A* and hierarchical pathfinding. One thing you hopefully recognized is that even the simplest pathfinder requires decision making; even when that decision was as simple as “we hit a barrier, so try moving in some random direction to get around it”. As we progressed, we saw that the means for improving the efficiency of the search and the ability to circumnavigate obstacles involved more complex decision making criteria (such as the various heuristics we mentioned). Again, while this may not be the pure Decision Making AI that we will learn about later on, you can probably understand why some programmers tend to lump everything together into a single catch-all AI category (which we made efforts to define at the outset) while others might consider this just a branch on a larger tree. In a sense, they are both right. After all, the job of the pathfinder is to make an entity move from point A to point B in a manner such that, on screen, it looks like the entity “figured out” how to get there in the shortest or quickest way possible. To the player, the entity certainly looks like it knows what it is doing and is therefore exhibiting some manner of intelligence. According to our original definition of artificial intelligence, this certainly fits the bill. Keep these thoughts in mind as you work your way through the rest of the course.

55

Chapter 2

Pathfinding II

56

Overview

In the last chapter we learned about some of the important AI subcategories that game developers will encounter in their projects. We mentioned that the two subcategories that we were going to focus on in this course are decision making and pathfinding since they are the two most common and critical AI components in the majority of games. Along the way learned that even these two seemingly very different concepts are somewhat related to one another. At the very least, we know that the goal in both cases is to use algorithms to produce behaviors that appear intelligent to the end user. After all, this was how we defined artificial intelligence. In the case of pathfinding we know that the idea is to use our knowledge of the environment to create algorithms that determine the best way to travel from place to place. From the player’s perspective, the end result will be entities that maneuver around obstacles in the game world as they attempt to reach a given destination. Since those destination points will be updated quite frequently in real-time (based on decision making techniques we will learn about later in the course), the illusion of autonomous intelligent entities is fostered and maintained. So far, we have examined some fundamental types of pathfinding and how it relates to games. In this chapter, we will discuss more advanced pathfinding techniques used in games and see how to apply them. Primarily we will discuss A* and its common advantages and disadvantages as well as how heuristics can be used to produce better results. Additionally, we will discuss ways of simplifying the pathfinding problem with hierarchical pathfinding. We will also discuss methodologies for pathfinding in non-gridded environments such as we find in many 3D games (although our chosen implementation will not come until later in the course, after we have discussed decision making in detail). Finally, we will discuss the chapter demo in detail and the design stratagems employed in its development. In this chapter we will answer the following questions:

What is A*? What are some of the advantages and disadvantages of A*? What are heuristics and how can they be used to assist A*? How can A*’s behavior be modified with different heuristics? What is hierarchical pathfinding and how can it be used? What are some of the methodologies for extending pathfinding systems for use in non-gridded

environments? What is the Algorithm Design Strategy? What is the Grid Design Strategy?

2.1 A*: The New Star in Pathfinding

A* is a more recent development in the arena of pathfinding algorithms. It combines the power of heuristics from Best First Search to limit its search time, with the ability to deal with weighted graphs from Dijkstra’s. It is an extremely versatile algorithm which many of today’s games use for their core pathfinding needs.

57

2.1.1 How A* Works

bool AStarSearch(Node start, Node goal) { PriorityQueue open; List closed; Node n, child; start.parent = NULL; open.enqueue(start); while(!open.isEmpty()) { n = open.dequeue(); if (n == goal) { makePath(); return true; } while (n.hasMoreChildren()) { child = n.getNextChild(); COSTVAL newg = n.g + child.cost; if ((open.contains(child) || closed.contains(child)) && child.g <= newg) continue; child.parent = n; child.g = newg; child.h = GoalEstimate(child); child.f = child.g + child.h; if(closed.contains(child)) closed.remove(child) if(!open.contains(child)) open.enqueue(child); else open.requeue(child); } closed.add(n); } return false; }

Above we have a fairly generic implementation of A*. It makes some assumptions about the type of container classes to use, but otherwise it is fundamentally how A* works. After reviewing it, you should see that it is very similar to Dijkstra’s and Best First Search combined into one algorithm. Note that it searches like Dijkstra’s, but uses the heuristic estimate to limit the search as in Best First Search. A* calculates three values for each node: f, g, and h. The g value is the current true cost to get to the node. The h value is the heuristic estimate value, which is typically the estimated cost from the node to the goal. The f value is the sum of h and g, and is the value by which A* sorts the nodes it will search, with lowest f values being searched first. Let us go over this algorithm in a little more detail.

58

bool AStarSearch(Node start, Node goal)

Like the algorithms we discussed in the previous chapter, we get a start node and a goal node, and return whether we found a path or not. PriorityQueue open; List closed;

Unlike our previous algorithms, A* has two lists to keep track of. The open list is a priority queue just like in Djikstra’s and Best First Search. This will allow us to go through our candidate nodes in the order that makes the most sense. The closed list is a list of nodes we have already searched, but might need to examine again at a later time. Node n, child; start.parent = NULL; open.enqueue(start);

Like the other algorithms, we will want a current node, and a current child node. We will set the parent of the start node to NULL since we know it is the start, and we prime the open priority queue by adding the start node to it since that is where we begin. while(!open.isEmpty())

Like the other algorithms, we also iterate through the queue until we find the goal or the queue empties. If we do not find the goal before the queue empties, there is no path from the start node to the end node. n = open.dequeue(); if (n == goal) { makePath(); return true; }

For each iteration, we will grab a node off the queue. We then check to see if it is the goal node, and if it is, we make the path and return our success. while (n.hasMoreChildren())

We then iterate across all the children of our current node. child = n.getNextChild(); COSTVAL newg = n.g + child.cost;

For each child, we compute a new actual cost based on the current node’s actual cost and the cost to the child.

59

if ((open.contains(child) || closed.contains(child)) && child.g <= newg) continue;

If either the queue or the closed list contains the child, and the child’s actual cost is less than the newly computed cost, we skip this child since we already have the shortest path to this child in the queue. child.parent = n; child.g = newg; child.h = GoalEstimate(child); child.f = child.g + child.h;

If we determine that we need to visit this child, we set the child’s parent to the current node, set the computed actual cost to the child, set the child’s estimated distance to the goal using our heuristic (just like in Best First Search), and set our total cost value to be the sum of the actual cost to this node plus the estimated cost to the goal. if(closed.contains(child)) closed.remove(child) if(!open.contains(child)) open.enqueue(child); else open.requeue(child);

Next, if the closed list contains the child, we need to remove it since we want to visit it again. Also, if the child is not in the open queue already, we add it. If it is, we requeue it so that it gets placed in the right spot in the queue with its new total cost. closed.add(n);

After we visit all of the children of the current node, we place the current node in the closed list so we do not visit it again, unless we find a cheaper way to get there. To summarize, we start off with two lists, open and closed. The open list is the list of nodes which needs to be searched and the closed list is the list of nodes which have already been searched. If there is a node in the closed list to which a shorter path is found, it is updated and moved to the open list again. The algorithm starts by putting the start node in the open list. Then, while the open list is not empty, the node with the lowest f value is checked to see if it is the goal. If it is, we make the path and return. If it is not, we iterate through all of its children. For each child, we determine a new g value, and check to see if the child is already in a list, as well as whether its cost is less expensive. If so, we ignore this child. Otherwise, the parent node is set, the g value is set to the new g value we calculated, the h value is set to the heuristic estimate, and the f value is calculated. Then, if the child is in the closed list, we remove it from the list. If the child is not in the open list, it is added. If it is in the open list, its position in that list is updated. After iteration through all of the node’s children, the node to the closed list is added.

60

2.1.2 Limitations of A*

A* is a wonderful pathfinding algorithm but it is not without limitations. A* will always find the shortest path to the goal, provided the heuristic estimate from any given child to the goal is never greater than the real distance to the goal. If the estimate is greater than the true distance to the goal, the algorithm will produce results which are not optimal. Moreover, the open and closed lists in A* can be inefficient when dealing with very large graphs. If the methods used to locate nodes in the lists, add nodes to the lists, and remove nodes from the lists are inefficient, the algorithm will be very slow. In addition, the memory requirements to store the nodes in these lists would increase dramatically. A* is a strong contender to keep in mind for your pathfinding needs, but it is always better to use the simplest method when the simplest method works well. Remember our KISS principle from the first chapter!

2.1.3 Making A* More Efficient

A* can be made more efficient so that it becomes a solid choice for our game needs. The first approach to optimizing A* is to improve the storage method for the open and closed lists. Some versions use queues, others use stacks, some use heaps, while still others use hash maps. The important point is to select a container class which can quickly locate nodes, insert nodes, remove nodes, and in the case where the list is not ordered, sort the list. Priority queues are a good choice because they keep the selection of the next best node simple because it is always at the front of the list. Stacks provide easy insertion and removal, and hash maps provide quick node location. They all have their benefits, but we cannot have the best of all worlds. So you should definitely experiment with different containers to find out which one works best in your particular implementation. Another optimization is to consider the search from a higher level and perform smaller searches for each step along the way. In a dungeon you might go room by room, or on a large outdoor map you might divide it into larger squares and go from region to region. In this case you will begin with a search using the larger regions, and then determine how to get across each region in a subsequent step. This is very much akin to the spatial partitioning techniques you learn about in the Graphics Programming training series here at the Game Institute. Indeed, you should be able to reuse many of the data structures and algorithms (quad-trees, kd-trees, etc.) that you learn about in Graphics Programming Module II in order to accomplish this objective. Some final optimizations to consider relate to the open and closed lists in particular. There are ways (using recursion) to totally eliminate both the open and closed lists. We will still encounter problems as we did with the Depth First Search, but we do not use the significant amount of memory that the open and closed lists use. Another method is to limit the number of nodes we will store in the open list, dropping the candidates with the highest f values if we reach our max open node count. By doing this, we do not need a closed list and this will save some processing time and memory. However, it is not guaranteed to find the best path.

61

2.2 Our Version of the Algorithm

MapGridWalker::WALKSTATETYPE AStarMapGridWalker::iterate() { if(!m_open.isEmpty()) { m_n = (AStarMapGridNode*)m_open.dequeue(); if(m_n->equals(*m_end)) { return REACHEDGOAL; } int x, y; // add all adjacent nodes to this node // add the east node... x = m_n->m_x+1; y = m_n->m_y; if(m_n->m_x < (m_grid->getGridSize() - 1)) { visitGridNode(x, y); } // Check the rest of the directions here // see the code for details // add the north-east node... x = m_n->m_x+1; y = m_n->m_y-1; if(m_n->m_y > 0 && m_n->m_x < (m_grid->getGridSize() - 1)) { visitGridNode(x, y); } m_closed.enqueue(m_n); return STILLLOOKING; } return UNABLETOREACHGOAL; }

void AStarMapGridWalker::visitGridNode(int x, int y) { int newg; // if the node is blocked or has been visited, early out if(m_grid->getCost(x, y) == MapGridNode::BLOCKED) return; // we are visitable newg = m_n->m_g + m_grid->getCost(x, y);

62

if( (m_open.contains(&m_nodegrid[x][y]) || m_closed.contains(&m_nodegrid[x][y])) && m_nodegrid[x][y].m_g <= newg) { // do nothing... we are already in the queue // and we have a cheaper way to get there... } else { m_nodegrid[x][y].m_parent = m_n; m_nodegrid[x][y].m_g = newg; m_nodegrid[x][y].m_h = goalEstimate( &m_nodegrid[x][y] ); m_nodegrid[x][y].m_f = m_nodegrid[x][y].m_g + m_nodegrid[x][y].m_h; if(m_closed.contains(&m_nodegrid[x][y])) m_closed.remove(&m_nodegrid[x][y]); // remove it if(!m_open.contains(&m_nodegrid[x][y])) m_open.enqueue(&m_nodegrid[x][y]); else { // update this item's position in the // queue as its cost has changed // and the queue needs to know about it m_open.remove(&m_nodegrid[x][y]); m_open.enqueue(&m_nodegrid[x][y]); } } }

Here is the actual implementation from our demo. Similar to Dijkstra’s Method, it makes use of the priority queue to keep our nodes sorted in order of cost. We grab the top node off our queue, see if it is traversable, update all its neighbors, and continue on until we find the goal node. Let us take a closer look at this implementation. MapGridWalker::WALKSTATETYPE AStarMapGridWalker::iterate()

As with our other implementations, our iterate interface begins inside the outer while loop of our algorithm snippet. It also returns the state of the graph traversal, informing the caller if it has found the goal, is unable to find the goal, or requires more iterations. if(!m_open.isEmpty())

We begin by checking to see if our open queue is empty. If the queue is empty, we cannot find a path from the given start node to the goal node. m_n = (AStarMapGridNode*)m_open.dequeue(); if(m_n->equals(*m_end)) { return REACHEDGOAL; }

63

For each iteration, we grab the node with the lowest perceived cost to the goal and make it our current node. If this node is the goal, we return success. // add all adjacent nodes to this node // add the east node... x = m_n->m_x+1; y = m_n->m_y; if(m_n->m_x < (m_grid->getGridSize() - 1)) { visitGridNode(x, y); }

We then visit each neighbor of the current node. Again we check to ensure that the node we want to visit is within the bounds of our grid. Also, we call upon visitGridNode to do the work of visiting the neighbor node. void AStarMapGridWalker::visitGridNode(int x, int y)

As in previous implementations, visitGridNode does the work of visiting the grid node at the given (x, y) coordinate. // if the node is blocked or has been visited, early out if(m_grid->getCost(x, y) == MapGridNode::BLOCKED) return;

First, it determines if the grid node to be visited is blocked, and if it is, it returns without visiting the node. newg = m_n->m_g + m_grid->getCost(x, y);

If the grid node is not blocked, we compute the new actual cost to the node via the current node. if( (m_open.contains(&m_nodegrid[x][y]) || m_closed.contains(&m_nodegrid[x][y])) && m_nodegrid[x][y].m_g <= newg)

If the open queue or the closed list contains the node already, and the new cost is higher than the cost already computed for this child node, we skip this node since we already have a path to this child node which is shorter. Otherwise, we will want to visit this node. m_nodegrid[x][y].m_parent = m_n; m_nodegrid[x][y].m_g = newg; m_nodegrid[x][y].m_h = goalEstimate( &m_nodegrid[x][y] ); m_nodegrid[x][y].m_f = m_nodegrid[x][y].m_g + m_nodegrid[x][y].m_h;

Once it is determined that the child node needs visiting, its parent is set to be the current parent so that we know how we got here. We also set its actual cost to be the newly computed actual cost. Next, we estimate the distance to the goal using our goal estimate (exactly like we did in Best First Search).

64

Lastly, we compute the total cost for this node by summing the actual cost with the estimate to the goal cost. if(m_closed.contains(&m_nodegrid[x][y])) m_closed.remove(&m_nodegrid[x][y]); // remove it

After we have computed our costs, we check to see if the closed list contains this node. If it does, we remove it since we plan to visit it again. if(!m_open.contains(&m_nodegrid[x][y])) m_open.enqueue(&m_nodegrid[x][y]); else { // update this item's position in the // queue as its cost has changed // and the queue needs to know about it m_open.remove(&m_nodegrid[x][y]); m_open.enqueue(&m_nodegrid[x][y]); }

Last, we check to see if the open queue already contains this node. If the queue does not contain the node, we add it. If it does contain the node, we requeue it into its proper position by removing it and adding it back. m_closed.enqueue(m_n);

After we have visited all of the current node’s neighbors, we add the current node to the closed list. This will keep us from visiting it again unless we find a shorter path to it. return STILLLOOKING;

Finally, we return STILLLOOKING to ensure we get more iterations so we can find the goal node. return UNABLETOREACHGOAL;

If our open queue was empty, we return UNABLETOREACHGOAL to ensure we cease iterating since we will never find the goal. Now that we have obtained a deeper understanding of how A* and our implementation of A* works, let us take a moment to walk through our implementation of the algorithm with an example graph from our demo. Now that we have obtained a deeper understanding of how A* and our implementation of A* works, let us take a moment to walk through our implementation of the algorithm with an example graph from our demo. Now that we have obtained a deeper understanding of how A* and our implementation of A* works, let us take a moment to walk through our implementation of the algorithm with an example graph from our demo.

65

Graph Open List Closed List Notes

(1,2) (2,2) (2,1)

(1,1) Here we have an example map where we want to get from corner to corner. The path is clear-cut, but pathfinding algorithms do not know that. Note that A* will arrive at the goal in 15 iterations rather than spend lots of time searching the expensive areas. By making the path free, you should see how the heuristic will provide for the solution sooner than Dijkstra’s would.

(1, 3) (2, 3) (2, 2) (2, 1)

(1,1) (1,2)

In the first iteration, the choice is simple. (1,2) was on the top of the list because it is free. The heuristic estimates were the same for all three possible nodes using Max(dx, dy). In the second iteration, the heuristic shows that going to (1,3) is cheapest (it is also free).

(2, 4) (1, 4) (2, 3) (2, 2) (2, 1)

(1, 1) (1, 2) (1, 3)

(1.3)’s neighbors are put on the open list, and it is placed in the closed list. We see from our estimate that, regardless of which node we choose, the estimate is the same, but the cost is zero for (2, 4) and (1, 4). Since (2, 4) was added last it goes on top in this implementation.

(3, 5) (2, 5) (1, 4) (3, 3) (3, 4) (1, 5)

(1, 1) (1, 2) (1, 3) (2, 4)

As we search (2, 4)’s neighbors, we see that either (3, 5) or (2, 5) is our best bet (as they are free). (3, 5) becomes the clear winner as it is added after (2, 5). Each step has still taken us closer to the goal so our heuristic has not done much for us yet.

66

(4, 4) (4, 5) (2, 5) (1, 4) (3, 3) (3, 4) (4, 6) (2, 6) (3, 6) (1, 5)

(1, 1) (1, 2) (1, 3) (2, 4) (3, 5)

As we search (3, 5)’s neighbors, we see that we again have a tie between (4, 4) and (4, 5). (4, 4) gets added on top because it was searched last. Notice our open list is getting very long now.

(4, 5), (2, 5), (1, 4), (5, 3), (4, 3), (5, 5), (5, 4), (3, 3), (3, 4), (4, 6), (2, 6), (3, 6),

(1, 5)

(1, 1) (1, 2) (1, 3) (2, 4) (3, 5) (4, 4)

Here we see that our heuristic is going to make us backtrack a bit. We are now potentially headed upward, and “away” from our goal. We see we have nodes in our open list which do not go away from our goal so we search them first.

(2, 5), (1, 4), (5, 3), (4, 3), (5, 6), (5, 5), (5, 4), (3, 3), (3, 4), (4, 6), (2, 6), (3, 6),

(1, 5)

(1, 1) (1, 2) (1, 3) (2, 4) (3, 5) (4, 4) (4, 5)

We backtrack a bit and see that (4, 5) did not have any better ways to go. Again, we have some nodes in our open list with better heuristic values so we try them first.

(1, 4), (5, 3), (4, 3), (5, 6), (5, 5), (5, 4), (3, 3), (3, 4), (1, 6), (4, 6), (2, 6), (3, 6),

(1, 5)

(1, 1) (1, 2) (1, 3) (2, 4) (3, 5) (4, 4) (4, 5) (2, 5)

We still find no better paths here. There is one more node to check in our open list before we can get back to where we were before, so we check it next.

67

(5, 3), (4, 3), (5, 6), (5, 5), (5, 4), (3, 3), (3, 4), (1, 6), (4, 6), (2, 6), (3, 6), (1, 5)

(1, 1) (1, 2) (1, 3) (2, 4) (3, 5) (4, 4) (4, 5) (2, 5) (1, 4)

There is still nothing better here. We go back to (5, 3) in the next iteration in order to see if we can get moving towards the goal again.

(6, 4), (6, 3) (4, 3), (5, 6), (5, 5), (5, 4), (3, 3), (3, 4), (4, 2), (6, 2), (5, 2), (1, 6), (4, 6), (2, 6), (3, 6), (1, 5)

(1, 1), (1, 2), (1, 3), (2, 4), (3, 5), (4, 4), (4, 5), (2, 5), (1, 4), (5, 3)

Here we are headed back along our path again. We have (6, 4) and (6, 3) which we can try. (6, 4) was added last so we try it next.

(6, 5), (6, 3) (4, 3), (5, 6), (5, 5), (5, 4), (3, 3), (3, 4), (7, 3), (7, 5), (7, 4), (4, 2), (6, 2), (5, 2), (1, 6), (4, 6), (2, 6), (3, 6),

(1, 5)

(1, 1), (1, 2), (1, 3), (2, 4), (3, 5), (4, 4), (4, 5), (2, 5), (1, 4), (5, 3),

(6, 4)

Our heuristic helps by declaring that (6, 5) is a better choice than (6, 3) as it is closer to the goal. We search there next.

(6, 6), (6, 3) (4, 3), (5, 6), (5, 5), (5, 4), (3, 3), (3, 4), (7, 6), (7, 3), (7, 5), (7, 4), (4, 2), (6, 2), (5, 2), (1, 6), (4, 6), (2, 6), (3, 6), (1, 5)

(1, 1), (1, 2), (1, 3), (2, 4), (3, 5), (4, 4), (4, 5), (2, 5), (1, 4), (5, 3), (6, 4), (6, 5)

Now we are moving nicely towards the goal. We see that our next best step is (6, 6) so we move in that direction.

68

(7, 7), (6, 7), (6, 3), (4, 3), (5, 7), (5, 6), (5, 5), (5, 4), (3, 3), (3, 4), (7, 6), (7, 3), (7, 5), (7, 4), (4, 2), (6, 2), (5, 2), (1, 6), (4, 6), (2, 6), (3, 6), (1, 5)

(1, 1), (1, 2), (1, 3), (2, 4), (3, 5), (4, 4), (4, 5), (2, 5), (1, 4), (5, 3), (6, 4), (6, 5),

(6, 6)

We now have a choice between (7, 7) and (6, 7). (7, 7) is chosen as it was added last via the search. Remember that Max(dx, dy) will not pick a node that is both closer in x and y; only the one that is closer of the two. This can result in a lot of ties as you can see from this example.

(8, 8), (7, 8), (6, 7), (6, 3), (4, 3), (5, 7), (5, 6), (5, 5), (5, 4), (3, 3), (3, 4), (8, 6), (8, 7), (6, 8) (7, 6), (7, 3), (7, 5), (7, 4), (4, 2), (6, 2), (5, 2), (1, 6), (4, 6), (2, 6), (3, 6), (1, 5)

(1, 1), (1, 2), (1, 3), (2, 4), (3, 5), (4, 4), (4, 5), (2, 5), (1, 4), (5, 3), (6, 4), (6, 5), (6, 6), (7, 7)

We are nearing the end. We see that (8, 8) is closest to the goal (and is the goal), so we go there next. The next iteration ends the search.

(8, 8), (7, 8), (6, 7), (6, 3), (4, 3), (5, 7), (5, 6), (5, 5), (5, 4), (3, 3), (3, 4), (8, 6), (8, 7), (6, 8) (7, 6), (7, 3), (7, 5), (7, 4), (4, 2), (6, 2), (5, 2), (1, 6), (4, 6), (2, 6), (3, 6), (1, 5)

(1, 1), (1, 2), (1, 3), (2, 4), (3, 5), (4, 4), (4, 5), (2, 5), (1, 4), (5, 3), (6, 4), (6, 5), (6, 6), (7, 7)

Our implementation does not remove the last node from the open list after we arrive at the goal. We simply quit and make the path. Notice how many nodes are still in the open list. Even with a small grid, the list becomes very large very quickly. An efficient container class for this list is highly recommended for optimal use on larger graphs.

2.3 Heuristics: A Few Examples

Let us take a moment to discuss heuristic estimates with A* and the multitude of ways they can be used. The greatest advantage of A* is its heuristic estimate, which is used to modify and optimize its search. At its heart, A* is simply a Breadth First Search without the heuristic. But with the heuristic, you can control how A* performs its search, and which nodes it chooses to search first. For example, the heuristic can be as simple as a cost estimate to the goal, or it can be the cost estimate to the goal coupled with performance penalties or bonuses for traveling on a specific type of terrain. You can use the state of the object (velocity, momentum, accelerations) to determine the quality or suitability of a node. With the appropriate heuristic, A* may even be used to solve the little puzzle game where you push the numbers around until they are in sequence. A* has even been used with performance heuristics to

69

determine the shortest yet safest way to do load balancing on multi-server networked systems. The power of the heuristic is not to be underestimated.

2.4 A Simple Real Time Strategy Game Design

In order to examine how A*’s behavior can be modified by the heuristic, let us consider a very simple real time strategy game design. To start, some units and some basic terrain types will be defined. We will then define a heuristic which will determine a cost multiplier for the standard heuristic estimate (such as max(dx, dy)) and define how each unit is effected by a given terrain type.

2.4.1 Terrain Types

Jungle

Jungle terrain consists of varied elevations mixed with dense tropical vegetation. It is very demanding and nearly impossible to traverse other than by foot.

Forest

Forest terrain consists of regions of land lightly-to-densely covered with coniferous and deciduous trees, as well as various types of underbrush. While not as demanding as jungle terrain, larger vehicles are unable to traverse this terrain due to tree spacing and underbrush.

Plains

Plains terrain consists of flat land to gently rolling hills covered with rowed crops and grasses. This terrain is fairly forgiving and is easily traversed by most modes of transportation.

Desert

Desert terrain can be anything from steppes to rolling dunes. This terrain is typically not restrictive to vehicles but may be a bit more costly for those on foot due to heat.

70

Foothills

Foothill terrain consists of rolling hills through small canyons. The terrain is typically very rocky with uneven ground making travel difficult for most, and nearly impossible for large vehicles.

Mountains

Mountain terrain consists of very steep slopes and rocky uneven ground. This terrain is considered impassible to all but those on foot and, even so, is still very difficult to traverse.

Roadway

Roadways are dirt, gravel, or paved surfaces which are wide enough for large vehicles to travel along, though possibly only one lane at a time. They provide for very easy travel, though they may not go in the direction desired.

Trail

Trails are dirt paths which are sufficiently cleared for those on foot or small vehicles to use for traveling more quickly. They are typically winding in nature which may make them unsuitable for exclusive use.

Swamp

Swamp terrain consists of wetlands and fairly dense undergrowth. The ground itself is soft which mires vehicles, especially heavy ones. Travel through this terrain is slow but possible.

Water

Water terrain is any type of body of water, whether a river, lake, or ocean. Streams and small bodies of water contained within a terrain are not included as they are typically easily fordable or avoidable without much added cost.

71

2.4.2 Units

For now, let us define four separate types of units, defined by their primary mode of motion. We will have Infantry, Wheeled Vehicles, Tracked Vehicles, and Hovercraft. We then have two more specific unit types for each generic type of unit, with exception of the Hovercraft type.

Infantry

Infantry are soldiers traveling on foot. Being on foot gives them excellent maneuverability. They can traverse all types of terrain aside from water with little impediment, although trails, roadways and other non-varied terrain are preferred. Light infantry carries light backpacks and arms, thereby making them capable of moving around easily. Heavy infantry carries large backpacks, shoulder mounted rocket launchers, or other similar encumbrances. Due to such heavy equipment, they are incapable of traversing jungles, mountains, and swampy terrain.

Wheeled Vehicles

Wheeled vehicles are vehicles that use wheels on axels. Being mostly lighter and smaller vehicles, they are capable of traversing terrains such as lightly forested areas, plains, deserts, foothills, and using normal roadways, as well as general footpaths and trails. Our two types of units are Jeeps and Armored Personnel Carriers. Jeeps are the smaller and lighter of the two and are more capable of traversing dense terrain. APC’s, on the other hand, are heavier and larger vehicles, rendering them incapable of using trails, and limiting their ability to function in the varied terrain of foothills.

Tracked Vehicles

Tracked vehicles are vehicles that maneuver by use of linked treads running over many sets of wheels. This gives them tremendous amounts of traction and surface area allowing heavier chassis. This also limits their maneuverability, thereby restricting them from swamps, jungles, and the use of trails. The two tracked vehicles we have defined are tanks and mobile base units. Tanks are capable of traversing forested areas, plains, deserts, and foothills, as well as using roadways. Mobile base units are very large in size rendering them incapable of traversing forested areas or foothills.

Hovercraft

Hovercraft are vehicles that move around by riding a cushion of air produced by a large fan or turbine blowing down toward the ground. The air is trapped by a skirt around the vehicle which makes close

72

contact with the ground and forms a seal. This seal keeps the air trapped under the vehicle, thereby allowing it to move about. If the seal is broken, the vehicle is immobilized until the seal can be reformed, which limits its movement to areas without dense vegetation or uneven terrain. Below we have a table showing which units are capable of traversing which terrain types. A check means the unit is capable of traversing the terrain (although possibly at great cost). An x means the vehicle is incapable of traversing the terrain at any cost. Unit\Terrain Jungle Forest Plains Desert Foothills Mountains Roadway Trail Swamp Water

Light Infantry

Heavy Infantry

Jeep APC Tank Mobile Base Hovercraft

2.4.3 Terrain Type vs. Unit Type Weighting Heuristic

In the example real time strategy game we are designing, we defined seven different types of units and ten different types of terrain. We will now define a Terrain Type versus Unit Type weighting heuristic. For each unit, we will store the cost weight of traversing each type of terrain. This cost weight will be used as a multiplier of the heuristic estimate. Any single node cost over a limit will be considered blocked and impassible terrain. If we wanted to be even more advanced, we could specify a unit task and the task would choose the type of terrain best suited for the task. For instance, if a unit were in reconnaissance mode, it would prefer terrain types that are more suited to stealth. Below we have a table of proposed unit type terrain modifiers. This is completely arbitrary but should give you an idea of how it would work. Ideally, you would adjust these numbers to make the units move in the fashion desired. Unit\Terrain Jungle Forest Plains Desert Foothills Mountains Roadway Trail Swamp WaterLight Infantry

3.0 1.5 1.2 1.8 1.5 3.0 1.0 1.0 1.5 100.0

Heavy Infantry

100.0 2.0 1.3 2.0 2.0 100.0 1.0 1.0 100.0 100.0

Jeep 100.0 1.5 1.1 1.2 1.5 100.0 1.0 1.0 100.0 100.0 APC 100.0 1.8 1.1 1.2 100.0 100.0 1.0 100.0 100.0 100.0 Tank 100.0 2.0 1.0 1.1 1.3 100.0 1.0 100.0 100.0 100.0 Mobile Base

100.0 100.0 1.2 1.2 100.0 100.0 1.0 100.0 100.0 100.0

Hovercraft 100.0 100.0 1.3 1.3 100.0 100.0 1.0 100.0 1.2 1.1

73

Using our newly defined set of weights, we can define our heuristic estimate function. We will use whichever heuristic estimate function we choose (Max, Manhattan, Euclidean), and multiply it by our weight for this unit on this terrain. This gives us the function:

tuWnhnh ,)()( ⋅′= , where h’(n) is our duly appointed heuristic estimate, and W is the matrix of weights for each unit on each terrain. Bear in mind that modifying only the heuristic will not make the system work as if by magic. We will need to apply similar weights to the actual costs of the nodes for the terrain and for the given units, or else the heuristic estimates will be significantly overestimated.

2.4.4 Defining the Map

At this point, if you were going to start assembling your game, you have a set of units, a set of terrains, weights for each unit for each terrain type, and a heuristic to use those weights. You now need to define your map. The best choice in this case is probably going to be a grid-oriented set of points that allow travel in all of the cardinal directions as well as diagonally. A 2D bitmap texture might serve you nicely here, where each texel equals a node. Start by setting the weights for travel between each node to be 1 for all nodes, and let the heuristic decide the nodes on which to travel. You will have to apply the weights to the nodes themselves as we traverse them because the actual cost to the node is what drives A* to search for a short path. You will then assign a terrain type to each node, perhaps using a paint program if a texture is your map method of choice. Storing the weights can be done using a text file (e.g., an .ini file) or you can hard-code the values directly into your application. Finally, you need to create several units and instruct them to wander from place to place. Once done, you will have the beginnings of a real time strategy game of your very own! The artificial intelligence required to make our units perform intelligently is a topic to be addressed later in this course. But for now, you should be able to put together something simple, resembling the features discussed here.

2.5 Simplifying the Search: Hierarchical Pathfinding

Hierarchical pathfinding is an approach to pathfinding which attempts to reduce the number of nodes the pathfinding algorithm has to consider when building a path. The concept is simple in theory, and not much more difficult in practice. Start by breaking down the gaming area into sub-areas. For each sub-area, break it up into further sub-areas. Repeat this process until it does not make sense to break up the sub-areas any further. The algorithm is then modified to find its way from the source to the goal via the lower resolution sets, then via the higher resolution sets included in the path found across the lower resolution sets. This does not always provide the absolute best path, but it helps reduce a larger problem into several smaller and more manageable ones. Let us look at a few examples to see how this works.

74

2.5.1 A Map of the US

Figure 2.1

Let us say we have a map of the United States, and we want to find a path from one city to another. We will break up the United States first into states and then into counties. We first find the state our starting and ending cities are in, and find our way from state to state. We then determine our path across each of the states we know need to be crossed, one at a time, going from county to county. Suppose we wanted to take a trip from Chicago to Las Vegas. Figure 2.1 shows the need to go from Illinois, through Iowa, through Nebraska, through Colorado, through Utah, and into Nevada. It would be terribly inefficient to calculate a path using all of the roads of all of the states. We begin by determining the best way to get to Iowa first, and from there to Nebraska, and from there to Colorado, and from there to Utah, and finally to Nevada. You can see how this limits the scope of our search, and also allows us to spread the processing of the search across many game cycles, as we do not need to worry about getting across the next state until we have already traversed the prior state.

75

2.5.2 A Dungeon

Figure 2.2

Another example is a dungeon with multiple rooms and levels. The dungeon could be broken up into levels first, and then broken down further into rooms on each level. If we wanted to get from one room in one level to another room in another level, we would first find out which levels we would need to get through. Then we would determine which rooms we needed to pass through for each of those levels. Finally, we find out how to get across each of those rooms. Take, for example, Figure 2.2. This dungeon has many rooms. If each room were to have various obstacles and pillars around which we needed to navigate, you might think that it would make sense to figure out how to exit the room before worrying about how to get all the way across the dungeon. But before you can concentrate on finding a path from your current location to the door, you first have to figure out which door is the most appropriate one to start from. In some implementations, you might decide to simply let the pathfinder work its way back to the correct door and then use only the game engine’s collision system to navigate from the current position to the selected door. In most cases however, a pathfinder is be used even for this task (albeit in conjunction with the collision engine). Regardless of how you implement the close distance navigation, the larger point here is the breakdown of navigation tasks from low resolution to high resolution.

76

2.5.3 A Real Time Strategy Map

For a real time strategy game map, we could divide the map into sixteen squares. We would then divide each of the sixteen squares into sixteen squares again, and repeat this breakdown until we had squares of small enough size that we could quickly traverse them. We would then traverse the largest squares to determine which ones needed to be crossed, and then determine which smaller squares would need to be crossed from that point, and so on. This method is very much like the quad-tree method you will study in the Graphics Programming course series here at the Game Institute, so you might wish to apply some of that knowledge to tackling this problem.

77

2.6 Pathfinding on Non-Gridded Maps

In the world of gaming, it cannot always be assumed that we are dealing with maps that are based on grid systems. Yet we still must find our way around the maps. While there are many ways to accomplish this, let us cover some of the most common methodologies for traveling from one place to another in worlds where there are no pre-defined natural grids. Usually in non-gridded environments, next step closer techniques are utilized to move around locally while dodging around and fighting, and the higher level pathfinding is only used when trying to go longer distances. In these cases, it is typical to “acquire” the larger pathfinding graph by getting to the closest node, and then pathfinding to the closest node to your destination. Once you acquire the closest point to the goal on your larger pathfinding graph, you fall back to next step closer techniques to access the actual goal position.

2.6.1 Superimposed Grids

One solution is to make the non-gridded world a gridded world. To do this, a grid is superimposed over the gaming area, which is where our pathfinding will be done. For multi-level systems, we can create grids for each level, and define entry and exit points to move from one level to another.

78

2.6.2 Visibility Points / Waypoint Networks

Visibility points are a common way of determining where obstacles are in order to avoid them. The idea is to place points around the obstacles, and draw lines from each point to every other point such that the lines do not cross through any obstacles. These points are then used by the pathfinding algorithm to determine where you can walk. These systems are also referred to as waypoint networks. The finer the network of points, the finer the movements your entities will have. Coarse networks result in jagged paths and zig-zagging. We will look at waypoint networks in more detail a little later in the course as they will be our method of choice for 3D world navigation.

79

2.6.3 Radial Basis

Radial Basis functions are functions that look similar to a normal distribution curve. Centering one of these functions on each of the obstacles allows the pathfinding algorithm to determine distances to obstacles and incur higher cost as it gets closer to an obstacle. The algorithm can then travel in the direction that induces the least amount of cost. These types of systems tend to be more expensive since the radial basis function contains an ex (exponential) expression.

2.6.4 Cost Fields

Similar to the radial basis method, this method surrounds obstacles with cost fields. Cost fields are typically implemented using continuous functions, and the pathfinding method simply uses gradient descent or the Newton-Rhapson method to find the lowest cost and travel in that direction. The problem with this method is that it can get caught in local minima, requiring some sort of agitation method to get back out. Moreover, like the Radial Basis method, this method can be computationally expensive.

80

2.6.5 Quad-Trees

This method is a combination of hierarchical pathfinding and the grid method. The area is cut into quads, and then each of those quads is cut into quads. The largest quad that can be formed without crossing a boundary of an obstacle is then created and stored. Recursion occurs to some depth which limits the number of nodes, but also provides fine pathfinding near obstacles. The centers and corners of the squares are used as route points. For more information about creating quad-trees, please consult the Graphics Programming Module II course available here at the Game Institute.

2.6.6 Mesh-Based Navigation

For 3D worlds consisting of polygonal data, graphs can be built from the mesh data itself. This is a tricky method, and requires a fair amount of work on the part of both the artist/level designer and the programmer, but it allows the use of world geometry as your pathfinding graph. The idea is to define the polygons in your world that are ‘floor-walkable’, and build an adjacency list for each one. In this way, it can be determined which floor polygon can be traversed to reach another traversable floor polygon. Walls and other obstacle polygons will not be included in the adjacency list. Additionally, as other moving entities cross the polygons, the larger polygons can be dynamically tesselated and the ones they are standing on removed so that they can be circumnavigated. Once the entity moves off the polygon, it can be re-added into the adjacency lists and remerged if all of its pieces are available again. This is a fairly complex system, and will not be demonstrated in this course. Our preference for 3D world navigation in this course will be waypoint networks and we will discuss those techniques a little later in our studies.

81

2.7 Algorithm Design Strategy

In order to allow the application to show each step of the pathfinding algorithm’s decision making process, the algorithms need to be designed so that an iteration function is repeatedly called which updates the display between steps, rather than finding the entire path in a loop. This method is useful because it also allows the update rate of the algorithm to be changed dynamically, thereby allowing quicker or slower playback of the graph search. Of course, your actual game implementation may not have this one step at a time design requirement, but it is easy enough to modify the approach to generate either the complete path in one pass or to do n iterations before returning.

2.7.1 Class Hierarchy

2.7.2 MapGridWalker Interface

The class hierarchy is designed around the base class MapGridWalker. This class provides the basic interface common to all MapGridWalker types. It tells us if weighted graphs are supported, whether or not heuristics are supported, what types of heuristics are supported, and so on. In our demo, there are two classes which support weighted graphs, and two classes which support heuristics. This architecture provides for the ability to add heuristics, as well as add new types of MapGridWalkers, simply and easily. typedef std::vector<std::string> stringvec; class MapGridWalker { public: typedef enum WALKSTATE { STILLLOOKING, REACHEDGOAL, UNABLETOREACHGOAL } WALKSTATETYPE;

MapGridWalker

BreadthFirstSearch MapGridWalker

BestFirstSearch MapGridWalker

AStar MapGridWalker

Dijkstras MapGridWalker

82

MapGridWalker(); MapGridWalker(MapGrid* grid) { m_grid = grid; } virtual ~MapGridWalker(); virtual void drawState(CDC* dc, CRect gridBounds) = 0; virtual WALKSTATETYPE iterate() = 0; virtual void reset() = 0; virtual bool weightedGraphSupported() { return false; }; virtual bool heuristicsSupported() { return false; } virtual stringvec heuristicTypesSupported() { stringvec empty; return empty; } virtual std::string getClassDescription() = 0; void setMapGrid(MapGrid *grid) { m_grid = grid; } MapGrid *getMapGrid() { return m_grid; } protected: virtual void visitGridNode(int x, int y) = 0; MapGrid *m_grid; };

The MapGridWalker interface is fairly straightforward. Let us go over it a bit at a time. typedef enum WALKSTATE { STILLLOOKING, REACHEDGOAL, UNABLETOREACHGOAL } WALKSTATETYPE;

MapGridWalker defines the WALKSTATE enumeration, which provides the application with an understanding of the walker’s progress. STILLLOOKING informs the application whether it must call iterate again to keep looking for the goal. REACHEDGOAL informs the application that the goal has been reached and iterate need not be called again. UNABLETOREACHGOAL informs the application that the walker has failed to find a path to the goal and further calls to iterate will not make any progress. MapGridWalker(); MapGridWalker(MapGrid* grid) { m_grid = grid; }

MapGridWalker supports a default constructor as well as a constructor which supplies a MapGrid upon which to walk. Accessors to set the MapGrid are also provided so the default constructor can be used. virtual void drawState(CDC* dc, CRect gridBounds) = 0;

The virtual drawState() function allows the specific implementation of the MapGridWalker to draw its current state into a Windows Device Context (CDC) using the bounds of the window provided in the rect gridBounds. This allows walker specific drawing to be done in an object oriented fashion. virtual WALKSTATETYPE iterate() = 0;

83

The virtual iterate() method makes one iteration of the walker’s algorithm before returning a value corresponding to its current state (using the enum). virtual void reset() = 0;

The virtual reset() method resets the walker’s state to the start node and reinitializes its map grid to mark all nodes as not visited. virtual bool weightedGraphSupported() { return false; };

The virtual weightedGraphSupported() method defaults to false, but the specific implementation may return true if the walker supports navigation of weighted graphs. virtual bool heuristicsSupported() { return false; }

The virtual heuristicsSupported() method also defaults to false, but may be overloaded in the specific class to return true if the walker supports the use of heuristics. virtual stringvec heuristicTypesSupported()

The heuristicTypesSupported() method returns an empty vector of strings in the default implementation, but may be overloaded by a walker that supports heuristics to provide a vector of strings that contain the descriptions of the heuristics supported. This vector is used by the application to populate the heuristics dropdown. void setMapGrid(MapGrid *grid) { m_grid = grid; } MapGrid *getMapGrid() { return m_grid; }

There are some accessor methods to obtain the specific walker class’s name (to populate the pathfinding method dropdown), and set or get the MapGrid object which the walker will navigate. virtual void visitGridNode(int x, int y) = 0;

The virtual method visitGridNode allows derived MapGridWalkers to visit the grid node at the given coordinate in its own specific fashion.

84

2.8 Grid Design Strategy

The MapGrid is simply a two-dimensional array of MapGridCell objects, which is a nested class of MapGrid. Each of these cells has a cost associated with it for moving into them. The MapGrid class also keeps track of the start and end indices into the grid for the walkers to use in their search. The walkers themselves build a PriorityQueue or a STL queue which contains MapGridNode objects. MapGridNode objects keep track of which MapGridCell they represent by storing the row and column index into the MapGrid. MapGridNode objects also keep track of state information for the walker class such as current traversal cost, visited state, and so forth. Walker specific MapGridNodes can easily be subclassed from MapGridNode as in the instance of the AStarMapGridNode, since A* requires more cost state variables to be stored than the other methods discussed. These special-case MapGridNodes may also be used for other special-case walkers to allow for extensibility. The MapGridNode is segregated from the MapGrid itself so that the grid can be discarded and replaced with another type of map environment. Note however, that the walkers themselves must be rewritten to use different map types; hence, their derivation from MapGridWalker, which is used to walk MapGrids.

2.8.1 MapGrid Interface

class MapGrid { public: class GridCell { public: GridCell(); GridCell(int cost); GridCell(const GridCell& copy); GridCell &operator=(const GridCell& rhs); inline int getCost() const { return m_cost; } void setCost(const int cost) { m_cost = cost; } private: int m_cost;

MapGrid

MapGridCell

MapGridNode MapGridPriorityQueue

AStarMapGridNode std::queue <MapGridNode*>

85

}; MapGrid(int gridsize); virtual ~MapGrid(); int getGridSize() const { return m_gridsize; } int getCost(int x, int y) const; void setCost(int x, int y, const int cost); void setStart(int x, int y) { m_startx = x; m_starty = y; } void getStart(int &x, int &y) const { x = m_startx; y = m_starty; } void setEnd (int x, int y) { m_endx = x; m_endy = y; } void getEnd (int &x, int &y) const { x = m_endx; y = m_endy; } private: GridCell **m_grid; int m_gridsize; int m_startx, m_starty, m_endx, m_endy; };

The MapGrid class is fairly simple. It contains a nested class GridCell, which contains only the cost needed to enter that cell during a traversal. The MapGrid itself consists of a two-dimensional array of GridCell objects, and the row and column indices to the start and end nodes. The MapGrid class has various accessors for setting and getting the cost of a given GridCell, the start node, and the end node. Let us talk about it in a little more depth. GridCell(); GridCell(int cost); GridCell(const GridCell& copy);

The contained class GridCell has a default constructor which initializes the m_cost variable to 1, as well as a constructor which assigns the cost passed in. There is also a copy constructor for deep copies. GridCell &operator=(const GridCell& rhs);

The assignment operator helps prevent copy constructs and is also useful for general assignment. inline int getCost() const { return m_cost; } void setCost(int cost) { m_cost = cost; }

Accessors provide access to the m_cost variable of the GridCell. private: int m_cost;

The cost value is encapsulated and requires the accessors to gain access. MapGrid(int gridsize); virtual ~MapGrid();

86

The MapGrid itself provides only a constructor, which expects a grid size used to construct a two dimensional array of GridCell objects. The destructor is virtual in order to allow potential inheritance and proper polymorphic destruction. int getGridSize() const { return m_gridsize; } int getCost(int x, int y) const; void setCost(int x, int y, const int cost); void setStart(int x, int y) { m_startx = x; m_starty = y; } void getStart(int &x, int &y) const { x = m_startx; y = m_starty; } void setEnd (int x, int y) { m_endx = x; m_endy = y; } void getEnd (int &x, int &y) const { x = m_endx; y = m_endy; }

The class provides various accessors to obtain the data contained within the class. The size of the grid, the cost of a given node, the start position, and the goal position can all be obtained and modified via these accessors. private: GridCell **m_grid; int m_gridsize; int m_startx, m_starty, m_endx, m_endy;

The class owns a two dimensional array of GridCell objects which it allocates during construction. It is aware of the size of this array, and knows about the start and end positions for the pathfinding system.

2.8.2 MapGridNode Class

class MapGridNode { public: // constructors MapGridNode() { m_cost = m_x = m_y = 0; m_parent = NULL; m_visited = false;} MapGridNode(int x, int y, MapGridNode *parent, bool visited, int cost) { m_x = x; m_y = y; m_parent = parent; m_visited = visited; m_cost = cost; } MapGridNode(const MapGridNode &copy); // destructor virtual ~MapGridNode() { m_parent = NULL; } virtual MapGridNode &operator=(const MapGridNode &rhs); virtual bool operator==(const MapGridNode &rhs); virtual bool operator<(const MapGridNode &rhs); virtual bool operator>(const MapGridNode &rhs); // accessors void setParent(MapGridNode* parent) { m_parent = parent;} void setVisited(bool visited) { m_visited = visited; }

87

bool getVisited() const { return m_visited; } virtual void setCost(int cost); virtual int getCost() const; // helpers bool equals(const MapGridNode &rhs) const { return ((m_x == rhs.m_x) && (m_y == rhs.m_y)); } // members int m_x, m_y; // the coord of the grid cell int m_cost; bool m_visited; const static int BLOCKED; MapGridNode *m_parent; };

The MapGridNode class is the interface via which the walkers navigate the MapGrid object. These classes contain the intermediate state information needed by the walker classes to properly navigate the MapGrid. These classes are created and placed into Queue objects which the walkers use to decide which nodes are to be traversed next, and also store the current traversal costs. The MapGridNode base class provides indices into the MapGrid for their location, a cost value (which may be interpreted by the specific walker class however it needs), a visited flag, and a pointer to the parent node. The pointer to the parent node is extremely important as this node is the only way we can know how the walker class got to this node. By traversing these pointers in a linked list fashion, we are able to find our way back from the ending node to the starting node to build our path. Notice that get and setCost() are virtual, allowing subclasses (such as AStarMapGridNode) to provide their own cost metrics. Let us take a closer look at this class. MapGridNode() { m_cost = m_x = m_y = 0; m_parent = NULL; m_visited = false;} MapGridNode(int x, int y, MapGridNode *parent, bool visited, int cost) { m_x = x; m_y = y; m_parent = parent; m_visited = visited; m_cost = cost; } MapGridNode(const MapGridNode &copy);

The MapGridNode class provides a default constructor which initializes all of the values to null defaults. There is also a constructor which takes all the parameters necessary to build the node in place and a copy constructor for deep copies. virtual ~MapGridNode() { m_parent = NULL; }

The destructor is virtual to allow for correct polymorphic destruction of derived classes. virtual MapGridNode &operator=(const MapGridNode &rhs);

An assignment operator is provided to help prevent overuse of copy construction, as well as for general assignment usage. virtual bool operator==(const MapGridNode &rhs);

88

virtual bool operator<(const MapGridNode &rhs); virtual bool operator>(const MapGridNode &rhs);

Various comparators are defined for allowing comparisons to be made. STL requires the < operator to use this object in sorted containers. The other operators are for convenience. void setParent(MapGridNode* parent) { m_parent = parent;} void setVisited(bool visited) { m_visited = visited; } bool getVisited() const { return m_visited; } virtual void setCost(int cost); virtual int getCost() const;

Various accessors are provided to obtain and manipulate the class data such as current parent, visited state, and cost metrics. The cost metrics are virtual to allow derived classes to have their own cost metrics. bool equals(const MapGridNode &rhs) const { return ((m_x == rhs.m_x) && (m_y == rhs.m_y)); }

The equals method helps derived classes perform default comparisons while adding their own in derived comparator operators. int m_x, m_y; // the coord of the grid cell int m_cost; bool m_visited; const static int BLOCKED; MapGridNode *m_parent;

The node’s data consists of its coordinate in the MapGrid, its cost, its visited status, a constant to represent blocked cost, and a pointer to its parent for completed path traversal.

2.8.3 MapGridPriorityQueue Class

class MapGridPriorityQueue { public: MapGridPriorityQueue(); ~MapGridPriorityQueue() { makeEmpty(); delete m_head->m_node; delete m_tail->m_node; delete m_head; delete m_tail; } void makeEmpty(); bool isEmpty() { return m_size == 0; } void enqueue( MapGridNode *node ); MapGridNode* dequeue(); void remove(MapGridNode *node); bool contains(MapGridNode *node) const;

89

private: class QueueNode { public: QueueNode() { m_node = NULL; m_next = m_back = NULL; } QueueNode(MapGridNode *node) { m_node = node; m_next = m_back = NULL; } MapGridNode *m_node; QueueNode *m_next; QueueNode *m_back; }; unsigned int m_size; QueueNode *m_head; QueueNode *m_tail; };

The MapGridPriorityQueue class is a linked list of QueueNode objects, which contains a pointer to a MapGridNode. It keeps the list sorted in order of the MapGridNode’s cost via the getCost() method of MapGridNode, and keeps the nodes with the cheapest cost on the top of the list. It is critically important to note that this class does not re-sort the list if a node pointed to by this list has its cost changed. In these instances, the node must be removed from the list, and reinserted. The MapGridPriorityQueue class provides a few methods which are important to note. The first is the enqueue( ) method. This method inserts a MapGridNode into the list and orders it by its cost. The second method is the contains( ) method. This method searches the list for a MapGridNode and returns true if the node is in the list. The next method is the remove( ) method. This method removes a MapGridNode from the list if it is in the list. Also, there are the isEmpty( ) and makeEmpty( ) methods. These methods determine if the list is empty and empty the list, respectively. Let us take a look at this class in more depth. MapGridPriorityQueue();

The MapGridPriorityQueue provides only the default constructor which initializes the list as empty. ~MapGridPriorityQueue() { makeEmpty(); delete m_head->m_node; delete m_tail->m_node; delete m_head; delete m_tail; }

The destructor empties the queue, and frees the head and tail nodes. void makeEmpty();

90

The makeEmpty method empties the queue and properly frees the memory as necessary. bool isEmpty() { return m_size == 0; }

The isEmpty method returns if the queue is empty. void enqueue( MapGridNode *node );

The enqueue method adds the node to the queue and places it in sorted order based on its cost. MapGridNode* dequeue();

The dequeue method removes and returns the node which has the lowest cost in the queue. void remove(MapGridNode *node);

The remove method simply removes the node from the queue. bool contains(MapGridNode *node) const;

The contains method searches the queue for the node and returns if the node is contained in the queue. QueueNode() { m_node = NULL; m_next = m_back = NULL; } QueueNode(MapGridNode *node) { m_node = node; m_next = m_back = NULL; }

The contained class QueueNode is the actual data which the MapGridPriorityQueue contains. It has a default constructor which assigns its data to null, and a copy constructor which copies the node to which it points, but it does not place it in the queue. MapGridNode *m_node; QueueNode *m_next; QueueNode *m_back;

The QueueNode contains a pointer to the node it is holding, as well as pointers to the next queue node and the previous queue node. unsigned int m_size; QueueNode *m_head; QueueNode *m_tail;

The MapGridPriorityQueue stores a size, a head pointer to the front of the queue, and a tail pointer to the back of the queue. You will see all of these members in action when you explore the source code to the pathfinding lab project.

91

2.9 MFC Document/View Architecture and Our Demo

MFC has an interesting architecture called Document/View. The core idea is that you have a single Document object containing all information relating to a single instance of the application’s data (such as a Word document, Excel spreadsheet, or in our case, a MapGrid) and any number of View classes meant to display that data in some fashion. In our application we have a CPathfindingApp which is the controlling entity. It contains a CMainFrame which is the window frame itself that contains the CPathfindingFormView, the C3DView, and the CMapGridDoc. It associates the view and the document via a DocumentTemplate object.

Note: We will not discuss in too much detail how MFC does its work in this fashion, as it is outside the scope of this course. However, those of you who have taken the C++ Programming courses here at Game Institute are in a good position to begin your investigations into MFC. It is a large API, but not difficult to learn, given your current level of Windows programming experience. Perhaps exposure to MFC here in this course will inspire you to look further. It is a powerful system that allows you to quickly assemble all sorts of useful Windows applications.

Every View has a pointer to its Document so that it can get data from the document to display itself. In our case the document contains a MapGrid and one of each of the MapGridWalker objects. We then let the view update the MapGrid and instruct the MapGridWalker to do its iteration via the CMapGridDoc. The 3DView and CPathfindingFormView can then be updated to display the correct results based on what information is in the CMapGridDoc.

CPathfindingApp

CMainFrame

CMapGridDoc

CPathfindingFormView MapGrid

MapGridWalker

C3DView

92

2.9.1 The Form View Panel

Our panel is designed so that we can click on the grid and make our map, select a pathfinding method, any heuristics that might be appropriate, the update rate, the start, the finish, and make it go. We capture mouse clicks via the CPathfindingFormView (which can be done using MSVC’s class wizard), and do a hit test against the grid. Whichever grid square we click is the one we modify (provided the user clicked one). The buttons for the grid costs are enabled and disabled based upon which pathfinding method is selected in the Pathfinding Method dropdown. When the selection changes, we check to see if the method supports weighted graphs and enable or disable the grid costs buttons as appropriate. The Heuristic Method drop down is also disabled or enabled and populated based on the selected method. We check if the method selected supports heuristics and, if so, we enable the drop down and populate the list from the specific walker. The Weight textbox and spinner are only enabled if heuristics are enabled. The update rate scroller updates the frequency of the timer which drives the iterate calls when the app is in Find Path mode. The set start and end buttons enable setting the start and end points of the grid when selected. The Find Path button starts the app searching for a path using the currently selected method and heuristic (if any). Lastly, the Generate Terrain button takes the current grid, and generates a small terrain for the 3DView. It uses A* to find the shortest path to the goal and make a sandy road along the path. We will not discuss in detail how it does all of that, as it is outside the scope of this course. If you would like to know more about the 3D aspects of this course, it is suggested that you explore the Graphics Programming courses offered here at the Game Institute (Module I, Chapter Seven in this case).

93

2.10 Conclusion

This brings us to the end of our core pathfinding discussions. In this chapter we were able to introduce one of the most powerful algorithms available to us: A*. We talked about how it works and how we can improve it if we find performance becoming a concern. We also talked about a number of hierarchical pathfinding methods that can prove to be useful when dealing with large maps or non-gridded worlds. It is well worth your time to try implementing some of the hierarchical methods we discussed using the source code you obtained during your studies in the Graphics Programming series. There is much there that can be applied; both with respect to spatial partitioning and to rendering on the whole. You are encouraged to start bringing some of those tools to bear as you work your way through this course. For example, try to get your animated characters (Graphics Programming Module II) to traverse various gridded scenes (which you can create easily in GILES™). Or perhaps try your hand at implementing something using the RTS design we presented earlier in the lesson. Of course, before you try any of these projects, make sure that you understand the demonstration that accompanies this chapter. In the next chapter we will begin to transition between pathfinding methods and decision making. You will see that during the development of a behavioral system called flocking, we will begin to blur the line between these disciplines and bring elements of ideas from both camps to bear. While our flocking system will not directly implement graph-oriented pathfinding as we have done in these last two chapters (although it will take advantage of it later in the course), it will still provide a form of environment navigation, mostly within a more localized area. More on this in the next lesson.

95

Chapter 3

Decision Making I:

Flocking

96

Overview

In the previous chapters, we looked at how to implement some of the more common pathfinding methods used in today’s games. We also touched on the different types of artificial intelligence, which we mentioned included pathfinding, decision making, classification, and life systems. The next type of artificial intelligence we will discuss in the course is decision making. In transitioning from pathfinding systems to decision making systems, we have decided to begin with a method known as flocking. Flocking is actually an interesting mix of the two concepts whereby decisions are made about how to perform pathfinding based on what the rest of the flock is doing. This discussion will set the stage for other forms of decision making that we will look at in the next chapter. In this chapter we answer the following questions:

What is flocking? What are the common components of flocking? How are the common components of flocking implemented?

3.1 Introduction to Flocking

Flocking is a very popular and commonly used AI grouping concept that has existed in computing for a long time. Flocking systems are often used to simulate the behaviors of schools of fish or flocks of birds. While the individual entities participating in a flock are also known as boids in some algorithms, they will be referred to as entities in this text. We can also refer to a flock as a group. This is done in order to make it easier to extend the terminology to group/squad based behavior later in the course. Group/squad behavior is based on a similar concept, where groups of entities cooperate with one another to make decisions and navigate the environment.

3.1.1 Behavior Based Movement

Flocking is classified as “behavior based movement.” That is, there is a set of behaviors which each entity in the group applies to determine its movement. Each behavior may or may not be influenced by entities that are nearby, and each will determine the movement for an entity which is appropriate for the particular behavior. For instance, a “move up” behavior would simply apply a movement vector that takes the entity straight up. There are four key movement behaviors typically applied in flocking systems. Using these four behaviors will result in fairly realistic representations of flocks of birds or schools of fish. They are also helpful when simulating movement for other entity groupings, as we will see later. We will examine each of these behaviors in detail shortly. First, let us set forth a few standards which we will be using in our examples.

97

Figure 3.1

Figure 3.1 shows a group of entities represented as colored triangles. The yellow filled triangle in the center represents the entity of interest. Its movement choices will be examined and it will be referred to as “our entity” in this section. The empty blue triangles are other entities which our entity can perceive. The green entities are outside our entity’s perception range, and will be ignored by our entity, even though they belong to its group.

Figure 3.2

The radial lines on the underlying polar grid will show the relative positions of all other entities with respect to our entity. Note that if desired we can limit the perception of our entity to a cone rather than a full 360 degree area. For instance, if we wanted to reduce our entity’s field of view to 144 degrees (the top two arcs), then the four blue entities outside the cone range would become green (Figure 3.2). In Figure 3.2, the red section of the polar grid indicates the area in which our entity can perceive the other members of its group.

98

Note the concentric rings in the polar grid. These can be used to easily visualize the distance from our entity to the other entities. The separation behavior has a particular interest in this information since it is influenced by distance. With these standards set, let us now take a more detailed look at the behaviors.

3.1.2 Separation

The separation behavior strives to keep the entities in a group from clumping too closely together without letting them drift too far apart. In the figure on the right, we have a group of entities. The blue entities are within our bounds of knowledge, so we pay attention to them, while the green entities are too far away to concern us, so they are ignored. Let us say that in our separation example we want a distance of two grid spaces between each entity. It follows that any entity within the bounds of the red circle is too close. The entity above and to the left (connected by the red line) is too close according to our specifications, so we calculate a vector that takes us away from this entity, as shown by the red arrow.

In the figure on the left, there are no entities which fall within the distance specification to our entity. In fact, all of the entities are further away than the second circle. In this case, we choose the closest entity to our entity, and move towards it (as indicated by the red arrow). Thus, the separation behavior strives to keep the separation between entities a certain fixed distance. In both of these cases, the desired move vector is summed together with the rest of the desired move vectors from the other behaviors. Let us continue on and look at those next.

99

3.1.3 Cohesion

The cohesion behavior tries to emulate the behavior in flocks of birds and schools of fish where they tend to “stick together”. The idea is that while we do not want the entities to run into each other, we want them to be in close proximity because this provides a degree of safety. In real schools of fish for example, they stay together because it presents a larger combined front to predators, and conveys the impression that the group as a whole is a larger entity than the predator. In the ocean, size mostly determines who will be eaten by whom. If that concept fails, there is a secondary safety measure which is based on lowering the probability of being eaten if the entity is not alone. For example, if a shark is chasing after two fish, the fish that will likely survive need only swim faster than his brethren. Take another look at our group of entities. Again, we are concerned with the yellow filled entity in the center of the polar grid; the blue entities are those in the group that are in our range of interest, and the green entities are those outside that range. The cohesion algorithm collects all of the entities within our interest range and computes the collection of entities’ average position. We then have our entity move towards this position. As in the case of the separation behavior, this desired move vector is summed up with the other behaviors’ desired moves.

3.1.4 Avoidance

The avoidance behavior is the behavior that directs the group to move away from things they do not want to be in proximity to or come into contact with. Examples for schools of fish include predator fish that will eat them, boats and their whirling propeller blades, and other obstacles which they cannot swim through. The idea is that the entities would sense things other than their group mates, and if the thing they sense is not something they want to get too close to, they move away from it. In the figure on the right, the large red triangle might represent a predator fish. Our entity finds that it is too close to this fish (in this case, sensing a predator fish at all would be considered too close), so the behavior determines a desired movement vector in the direction of the red arrow. This vector will be added in with the rest of the desired movement vectors from all the other behaviors. This behavior is quite interesting because it differs from the others in that it includes concepts like observation and environmental awareness. That is, the entity is paying attention to objects and/or events beyond just itself and its group.

100

3.1.5 Alignment

The alignment behavior attempts to keep all of the entities in the group aligned in approximately the same direction. As in the real world, all of the fish in a school tend to swim in the same direction. Consider the figure on the right. Most of the entities are headed in the general same direction, but not exactly. The alignment behavior gathers up the entities in our range, averages their heading, and sets our entity’s desired heading to be more like the average heading. This should not be done instantly, since it is not realistic that such a change occurs immediately. Over time, this method will gradually adjust our entity’s direction so that it will be in line with the alignment of the rest of the group.

Another way the alignment behavior could work would be to take the nearest entity’s heading rather than an average heading of all the nearby group mates (see figure on left). This is computationally less expensive, since we do not have to search through all of our nearby group mates and average up their headings. But it can also result in parts of the group veering off in much different directions since we only pay attention to the entity closest to us.

3.1.6 Other Possible Behaviors

While the four behaviors discussed previously are the mainstay of flocking implementations, there are other behaviors that can be helpful. For instance, a cruising behavior is a useful behavior. A cruising behavior decides what direction the entity would go if that entity were alone. In most cases these behaviors simply maintain the last heading the entity had and add some random variation. There are times where this behavior actually would get called upon even when there are group mates, because sometimes (mostly due to the avoidance behavior making them run away from something) an entity may be separated from its group and forced to fend for itself.

101

Another example of an extra behavior is the “keep in sphere” behavior used in our demo. This behavior will be covered in more detail shortly, but the basic premise is that, as long as the entity is within the bounds of the sphere, the behavior has no input. However, as soon as the entity leaves the sphere, the behavior puts in a request for a movement towards the center of the sphere.

3.2 The Flocking Demo

In keeping with the spirit of traditional flocking, the demo provided for this chapter consists of several schools of fish. The player takes on the role of a northern pike, hungry and ready to eat. There are schools of blue gill and large mouth bass swimming about, and they will react to the player if he gets too close. If the fish are touched, they will not disappear as if they have been “eaten” (that is left to you to add!). The main goal is to provide a means by which a flocking implementation can be seen, and provide an interface through which the various parameters of each of the behaviors can be adjusted to see the results.

3.2.1 Design Strategies

Figure 3.9

The first thing to understand is how everything is laid out in the Flocking Demo. Figure 3.9 shows the collaboration diagram for the cEntity class. The cEntity class is basically a single thing in a group. Other implementations might call it the ‘boid’ in a flock. Every cEntity is in a cGroup, even if it is the only one in the group. All of the cGroups are managed by the cWorld object, of which there is only one. Each cEntity also maintains a connection to its rendered representation as well, but we do not need to go into details about that.

102

Figure 3.10

Every cEntity has a list of cBehavior objects which it uses to determine how it wants to move around. It does not own these behaviors but shares them with all of the other entities. The behavior itself gets a reference to the current entity when it is applied, to make the behavior shareable. The algorithm does not need to maintain state since the entity can do that. As you will notice in Figure 3.10, not all of the behaviors are group related. Only the Alignment behavior, the Cohesion behavior, and the Separation behavior make use of the group information. The rest of the behaviors solely depend upon the entity being acted upon. We will go over each of these classes in detail shortly.

3.2.2 MFC and our Demo

Figure 3.11

103

As in the Pathfinding Demo, the Flocking Demo makes use of MFC. There is a CFlockingDemoApp which has a current document (CFlockingDemoDoc) and some associated views (CFlockingDemoView and C3DView). In this particular application, the document does not hold anything; the C3DView holds the rendering engine which keeps track of the cWorld object and its associated entities. The CFlockingDemoView, however, does have some things of interest. This view contains a tab control which has all the property panels for each of the behaviors. If you desired to create your own behaviors, this is where you will want to link them up to the interface to have your own property panel. As you can see in Figure 3.11, the CFlockingDemoView keeps track of a collection of pages, one for each of the behaviors. Later we will see that the cEntity objects do not own the behaviors. They all share the same set because the behavior itself does not change. This allows the property panels to modify the behaviors globally for all cEntity objects. Now let us examine the actual implementation of our demo.

3.2.3 Our Implementation

class cWorld { public: cWorld(void); virtual ~cWorld(void); void Add(cGroup &group); void Remove(cGroup &group); tGroupList &Groups() { return(mGroups); } virtual void Iterate(float timeDelta); virtual void Render(LPDIRECT3DDEVICE9 pDevice); protected: tGroupList mGroups; };

Above we have the declaration for the cWorld object. There is only one of these in existence at any given time. It holds all of the groups and is responsible for iterating them during each time step. Additionally, the world takes on the responsibility for ensuring that the groups render themselves. Let us go over this in a little more detail. cWorld(void); virtual ~cWorld(void);

A default constructor is provided which initializes the group list properly. The group list at construction time is empty. The destructor is virtual to allow for more specific derived classes to be polymorphically destructed properly. The destructor ensures that all of the groups are properly freed. void Add(cGroup &group); void Remove(cGroup &group);

104

The cWorld object provides the ability to add and remove groups from its list. When a new group is added to the cWorld object, it takes over responsibility for its lifetime, so the group added need not be destroyed externally. If the group is removed, however, responsibility is relinquished and the group must be freed manually. tGroupList &Groups() { return(mGroups); }

The cWorld also provides an accessor to the list of groups it contains. This allows external objects to iterate across all of the items that exist in the world if necessary. The avoidance behavior makes use of this method. virtual void Iterate(float timeDelta);

The iterate method advances time by the time delta for each of the groups. Since this is done in an iterative fashion, the items that are last in the list gain the most benefit as they have watched others go before themselves and know more about the true state of the world at the end of a time slice. However, this comes with a potential penalty as they could be eaten before they get their turn. virtual void Render(LPDIRECT3DDEVICE9 pDevice);

The cWorld is responsible for ensuring that each of the groups is rendered when the time comes. It passes responsibility to each of the individual groups to make sure their contents are rendered correctly. tGroupList mGroups;

The world owns the list of groups that it holds. It will delete any of the groups in this list, so once a group is added to the world, you do not need to externally destroy it as the world assumes that responsibility. class cGroup { public: cGroup(cWorld &world); virtual ~cGroup(void); void Add(cEntity &entity); void Remove(cEntity &entity); tEntityList &Entities() { return(mEntities); } virtual void Iterate(float timeDelta); virtual void Render(LPDIRECT3DDEVICE9 pDevice); protected: tEntityList mEntities; cWorld &mWorld; }; typedef vector<cGroup*> tGroupList;

105

Here we have the definition of the cGroup class. This class holds one collection of cEntities that will be acting together. It also holds a reference to the cWorld which owns it so that it can gain access to the list of all of the other cGroup objects. cGroup(cWorld &world); virtual ~cGroup(void);

The cGroup object provides no default constructor, as it needs a reference to the cWorld object that owns it to build its held reference. The destructor is virtual to allow for correct polymorphic destruction in the event that a new type of cGroup is derived. The cGroup object is responsible for its list of cEntity objects and will destroy them when the destructor is called. void Add(cEntity &entity); void Remove(cEntity &entity);

Similar to the cWorld, the cGroup takes ownership of the cEntity objects added to its list. If the cEntity is removed from the list, the responsibility of releasing the cEntity is relinquished and must be done externally. tEntityList &Entities() { return(mEntities); }

The group provides access to its list of entities so that the other entities may know of their group mates. virtual void Iterate(float timeDelta);

The iterate method passes time for all of the entities in the group’s list. This method will be called by the cWorld object that holds the cGroup. virtual void Render(LPDIRECT3DDEVICE9 pDevice);

The cGroup is responsible for ensuring that all of the entities owned by it are rendered when the time comes. This method will be called by the cWorld that owns the cGroup. tEntityList mEntities; cWorld &mWorld;

The cEntity objects contained in the mEntities vector are all owned by the cGroup object, and will be destroyed upon destruction of the cGroup object. The cWorld reference is made available so the cGroup can gain access to all of the other cGroup objects contained in the cWorld object. class cEntity { public: cEntity ( cWorld &world, unsigned type,

106

float senseRange, float maxVelocityChange, float maxSpeed, float desiredSpeed, float moveXScalar, float moveYScalar, float moveZScalar ); virtual ~cEntity(void); virtual void Iterate(float timeDelta); virtual void Render(LPDIRECT3DDEVICE9 pDevice); void SetFriendMask(unsigned mask); unsigned FriendMask(void); unsigned EnemyMask(void); unsigned EntityType(void); tEntityDistList &VisibleGroupMembers(void); tEntityDistList &VisibleEnemies(void); void AddBehavior(cBehavior &beh); void RemoveBehavior(cBehavior &beh); void Set3DRepresentation(RenderLib::CObject*object); void SetCurrentGroup(cGroup *group); D3DXVECTOR3 Position(void); void SetPosition(const D3DXVECTOR3 &pos); D3DXVECTOR3 Velocity(void); void SetVelocity(const D3DXVECTOR3 &vel); D3DXQUATERNION Orientation(void); void SetVelocity(const D3DXQUATERNION &orient); D3DXVECTOR3 DesiredMove(void); void SetDesiredMove(const D3DXVECTOR3 &move); float MaxSpeed(void); float DesiredSpeed(void); protected: void UpdateGroupVisibility(void); void UpdateEnemyVisibility(void); bool VisibilityTest(cEntity &otherEntity, float &dist); cGroup* mCurrentGroup; tBehaviorList mBehaviors; cWorld &mWorld; unsigned mFriendMask; unsigned mEnemyMask; unsigned mEntityType; RenderLib::CObject *mObject; D3DXVECTOR3 mPosition; D3DXVECTOR3 mVelocity;

107

D3DXQUATERNION mOrientation; D3DXVECTOR3 mDesiredMoveVector; float mSenseRange; float mMaxVelocityChange; float mMaxSpeed; float mDesiredSpeed; float mMoveXScalar; float mMoveYScalar; float mMoveZScalar; tEntityDistList mVisibleGroupMembers; tEntityDistList mVisibleEnemies; };

Here we have the cEntity class. All of the inline implementations were stripped because they are trivial and take up unnecessary space. Refer to the code for the full version. If the cBehaviors are the work horses of the flocking system, the cEntity is the coach driver. This is ultimately the ‘thing’ that the behaviors will actually be moving around. In our demo, each cEntity is one fish. cEntity ( cWorld &world, unsigned type, float senseRange, float maxVelocityChange, float maxSpeed, float desiredSpeed, float moveXScalar, float moveYScalar, float moveZScalar );

The cEntity constructor takes quite a few parameters. First, it takes a reference to the world so that it can gain access to the list of all of the other groups in the world. Next, it takes a bitmask type to identify what it is. In our demo, there is a player type and a non-player type. This is so that the non-player fish know to run away from the player fish. Next there is a sense range. This value represents how far the entity can see other entities. Any entity farther away than this will be completely ignored. Next there is a max velocity change. This value clamps how quickly the entity can change speeds. This helps prevent instant directional changes and jumps, and keeps the entity moving smoothly. Next is max speed and desired speed. The max speed is the absolute maximum speed the entity can travel, while the desired speed is the speed the entity prefers to travel. The last three scalar values adjust the amount of movement in each of the cardinal directions. This is useful for clamping specific types of movement; for instance, vertical change. Fish tend to swim left and right more than up and down, so the y scalar gets decreased to clamp the amount of vertical movement.

108

virtual ~cEntity(void);

The destructor for cEntity is virtual in order that derived types can be correctly destructed. This is mentioned for every destructor to emphasize its importance. If you do not understand the reason for this, it is highly recommended that you take the Game Institute course C++ Programming for Game Developers Module I. The cEntity does not own the behaviors it uses, so it does not free them. It also does not own the render object it uses since the scene graph owns that. Thus, the base class cEntity does not free anything, because it does not own anything. virtual void Iterate(float timeDelta);

The Iterate method applies each of the behaviors which the cEntity has and is the core of the implementation. There are a few other notable methods which the cEntity class exposes, but they are actually called during the course of iteration so we will talk about them as we come across them. Let us take a look at the actual implementation of this method. void cEntity::Iterate(float timeDelta) { mPosition += mVelocity * timeDelta; mVisibleGroupMembers.clear(); mVisibleEnemies.clear(); UpdateGroupVisibility(); UpdateEnemyVisibility(); tBehaviorList::iterator it; for (it = mBehaviors.begin(); it != mBehaviors.end(); it++) { cBehavior *beh = *it; beh->Iterate(timeDelta, *this); } float velChange = D3DXVec3Length(&mDesiredMoveVector); if (velChange > mMaxVelocityChange) { D3DXVec3Normalize(&mDesiredMoveVector, &mDesiredMoveVector); mDesiredMoveVector *= mMaxVelocityChange; } mVelocity += mDesiredMoveVector; mVelocity.x *= mMoveXScalar; mVelocity.y *= mMoveYScalar; mVelocity.z *= mMoveZScalar; float speed = D3DXVec3Length(&mVelocity); if (speed > mMaxSpeed) { D3DXVec3Normalize(&mVelocity, &mVelocity); mVelocity *= mMaxSpeed; } D3DXVECTOR3 vec;

109

D3DXVec3Normalize(&vec, &mVelocity); float pitch = atan2f(-vec.y, sqrtf(vec.z*vec.z + vec.x*vec.x)); float yaw = atan2f(vec.x, vec.z); D3DXQuaternionRotationYawPitchRoll(&mOrientation, yaw, pitch, 0.0f); }

Again, comments were stripped from this listing to compact it as much as possible. As we examine each line of code, a general picture will appear. Refer to the code for the unedited version. mPosition += mVelocity * timeDelta;

The first action of this method is to update our position using the current velocity. This would have to be either the first or absolute last action. In our case, it will be first. What we are doing here is numerically integrating velocity to get our position using standard Euler integration. We use the time delta passed in to avoid frame dependence. mVisibleGroupMembers.clear(); mVisibleEnemies.clear();

Next, we clear our visibility lists. The cEntity class keeps track of which other entities it can see in its group as well as any enemies it can see. We clear these lists every frame and build them anew. UpdateGroupVisibility();

The next thing we do is update our group visibility list. Let us look at how that is done. void cEntity::UpdateGroupVisibility() { if (mCurrentGroup){ tEntityList &entities = mCurrentGroup->Entities(); tEntityList::iterator it; for (it = entities.begin(); it != entities.end(); ++it){ cEntity *e = *it; // skip ourselves if (e == this) continue; float dist; if (VisibilityTest(*e, dist)) { // keep this list sorted pair<float, cEntity*> thepair(dist, e); tEntityDistList::iterator pos = upper_bound(mVisibleGroupMembers.begin(), mVisibleGroupMembers.end(), thepair); mVisibleGroupMembers.insert(pos, thepair); } } } }

110

Here we have the implementation of UpdateGroupVisibility. This method iterates across each of the members of this entity’s group, and determines if they can be seen by this entity. if (mCurrentGroup)

First we confirm that we have a group. While all entities should be in a group, checking pointers is always a good idea. tEntityList &entities = mCurrentGroup->Entities(); tEntityList::iterator it; for (it = entities.begin(); it != entities.end(); ++it)

Next, we obtain the list of entities in our group, and begin iterating through them. cEntity *e = *it; // skip ourselves if (e == this) continue;

If the entity we are currently iterating across is this entity, we skip it, since we do not consider it for visibility. float dist; if (VisibilityTest(*e, dist))

If the current entity is not this entity, we perform a visibility test. bool cEntity::VisibilityTest(cEntity &otherEntity, float &dist) { // simple test for now, are they close enough that we can "sense them" D3DXVECTOR3 distVec = otherEntity.Position() - Position(); dist = D3DXVec3Length(&distVec); if (dist < mSenseRange) return(true); return(false); }

The visibility test is very straightforward. We simply determine if the entity in question’s position is within our sense range. If it is, we can see it, otherwise, we cannot. This method could easily be modified to provide a cone of vision rather than a simple distance check. // keep this list sorted pair<float, cEntity*> thepair(dist, e); tEntityDistList::iterator pos = upper_bound(mVisibleGroupMembers.begin(), mVisibleGroupMembers.end(), thepair); mVisibleGroupMembers.insert(pos, thepair);

111

After we determine an entity to be visible, we keep track of its actual distance to us in a pair, and add it to the visible members vector. Notice that the vector is kept sorted. This allows us to easily acquire the closest or farthest entity in the group. We can also perform O log2 n searches on the list to find a specific entity if we desire. UpdateEnemyVisibility();

After we have updated our group’s visibility list, we update our enemy visibility list. Let us take a look at how that is done, since it has a notable difference. void cEntity::UpdateEnemyVisibility() { tGroupList &groups = mWorld.Groups(); tGroupList::iterator git; for (git = groups.begin(); git != groups.end(); ++git) { cGroup *group = *git; tEntityList &entities = group->Entities(); tEntityList::iterator it; for (it = entities.begin(); it != entities.end(); ++it) { cEntity *e = *it; // skip friendly groups if ((e->EntityType() & EnemyMask()) == 0) break; float dist; if (VisibilityTest(*e, dist)) { // keep this list sorted pair<float, cEntity*> thepair(dist, e); tEntityDistList::iterator pos = upper_bound(mVisibleEnemies.begin(), mVisibleEnemies.end(), thepair); mVisibleEnemies.insert(pos, thepair); } } } }

The UpdateEnemyVisibility method obtains the list of groups from the world, and iterates over all of the entities in all of the groups in search of enemy entities. tGroupList &groups = mWorld.Groups(); tGroupList::iterator git; for (git = groups.begin(); git != groups.end(); ++git)

First, the list of groups is obtained from the world, and iteration begins. cGroup *group = *git; tEntityList &entities = group->Entities();

112

tEntityList::iterator it; for (it = entities.begin(); it != entities.end(); ++it)

For each group, the list of entities within the group is obtained, and that list is iterated over. if ((e->EntityType() & EnemyMask()) == 0) break;

Here is the important part. On the assumption that all of the entities within the same group are of the same type (which is safe for this demo, but is not necessarily always the case), if the first entity in the group is not an enemy, the entire group is not an enemy and can be ignored. This is purely an optimization, but it is important to note that the masks of the entities in question are compared with this entity’s mask to determine whether it should be considered an enemy. float dist; if (VisibilityTest(*e, dist)) { pair<float, cEntity*> thepair(dist, e); tEntityDistList::iterator pos = upper_bound(mVisibleEnemies.begin(), mVisibleEnemies.end(), thepair); mVisibleEnemies.insert(pos, thepair); }

For all those entities determined to be enemies, we do the same visibility test we did for the group mates, and keep track of the distances to the enemy in a sorted list (just as we did with the visible group members list). tBehaviorList::iterator it; for (it = mBehaviors.begin(); it != mBehaviors.end(); it++) { cBehavior *beh = *it; beh->Iterate(timeDelta, *this); }

Once we have updated our visible friends and enemies lists, we iterate across our list of behaviors and iterate each one. We will go over the implementations of the individual behaviors shortly. float velChange = D3DXVec3Length(&mDesiredMoveVector); if (velChange > mMaxVelocityChange) { D3DXVec3Normalize(&mDesiredMoveVector, &mDesiredMoveVector); mDesiredMoveVector *= mMaxVelocityChange; }

After we have applied all of our behaviors, we begin some post processing on our desired move vector to bring it in line. First, we determine the length of the desired move vector and ensure it is not bigger than our max velocity change. If it is, we normalize our desired move vector, and set its length (by scaling it) to be the maximum velocity change. In effect, we clamp the vector’s length to be that of our max velocity change.

113

mVelocity += mDesiredMoveVector;

Next, we add our desired move vector to our current velocity vector. This will nudge our current velocity vector in the direction of the new desired move. mVelocity.x *= mMoveXScalar; mVelocity.y *= mMoveYScalar; mVelocity.z *= mMoveZScalar;

We then apply our Cartesian move scalars. In the case of our demo, we reduce the amount of vertical movement (y axis) so as to produce more lifelike fish movement. float speed = D3DXVec3Length(&mVelocity); if (speed > mMaxSpeed) { D3DXVec3Normalize(&mVelocity, &mVelocity); mVelocity *= mMaxSpeed; }

Next, we effectively clamp our velocity to be within the bounds of our maximum speed. We do this by first getting the length of our velocity vector, and if that length is greater than the max speed, we normalize the velocity vector and scale the result by our max speed. D3DXVECTOR3 vec; D3DXVec3Normalize(&vec, &mVelocity); float pitch = atan2f(-vec.y, sqrtf(vec.z*vec.z + vec.x*vec.x)); float yaw = atan2f(vec.x, vec.z); D3DXQuaternionRotationYawPitchRoll(&mOrientation, yaw, pitch, 0.0f);

Lastly, we perform some trigonometric calculations to convert our velocity vector into a quaternion to hold our orientation. Afterwards, we have a new velocity and orientation with which to apply for the next iteration to get a new position. Before we discuss the movement behaviors, there is one last class definition to investigate. class cBehavior { public: cBehavior(void) : mGain(1.0f) {} virtual ~cBehavior(void) {} virtual void Iterate(float timeDelta, cEntity &entity) = 0; float Gain(void) { return(mGain); } void SetGain(float gain) { mGain = gain; } virtual string Name(void) { return("Base Behavior"); } private: float mGain; };

114

Here we have the base class cBehavior. This is the interface through which all of the behaviors do their work. Let us go over it in detail. cBehavior(void) : mGain(1.0f) {} virtual ~cBehavior(void) {}

The default constructor initializes the gain of the behavior to 1.0. The gain value is used to determine the weight upon which the result of the behavior is applied to the entity in question’s desired move. The destructor is virtual to allow for correct polymorphic destruction of derived classes. virtual void Iterate(float timeDelta, cEntity &entity) = 0;

The pure virtual iterate method takes a time delta for the amount of time that has passed since the last call to this method, and the entity to which to apply the behavior’s movement decisions. float Gain(void) { return(mGain); } void SetGain(float gain) { mGain = gain; }

The class provides accessors to allow the gain to be modified post construction. virtual string Name(void) { return("Base Behavior"); }

The Name method is used by the user interface to properly name the tab of the tab control. float mGain;

The mGain variable is used to modify the degree to which the behavior will modify the desired behavior of the entity in question. The default is 1.0 which is full effect, while 0.5 would be half effect, and 2.0 would be double effect. The user interface only allows fractional gains. Now that we have a good understanding of the framework of the application, let us take a look at the implementation of the behaviors implemented in the demo.

115

3.2.4 Separation

class cSeparationBehavior : public cGroupBehavior { public: cSeparationBehavior(float sepDist, float minPercent, float maxPercent); virtual ~cSeparationBehavior(void); virtual void Iterate(float timeDelta, cEntity &entity); protected: float mSeparationDistance; float mMinSeparationPercentage; float mMaxSeparationPercentage; };

Here we have the declaration for the Separation Behavior as it exists in our demo. For the sake of brevity, the accessors have been removed from this listing. See the code for the unabridged version. cSeparationBehavior(float sepDist, float minPercent, float maxPercent);

The separation behavior takes three parameters upon construction. The first is the separation distance that the behavior will try to maintain between entities in the group. This applies to entities too close as well as too far away. The goal of the behavior is to ensure all of the entities in the group always stay exactly this distance away from each other. The min percent and max percent values are the minimum and maximum separation percentages that will be applied to limit large changes. The algorithm will determine the actual separation percentage, and then clamp it to these values. virtual void Iterate(float timeDelta, cEntity &entity);

As with the base class, the Iterate method takes the time passed since the last iteration, and the entity to which to apply the movement. Let us take a closer look. void cSeparationBehavior::Iterate(float timeDelta, cEntity &entity) { tEntityDistList &groupMembers = entity.VisibleGroupMembers(); if (groupMembers.empty()) return; cEntity &nearestGroupMember = *groupMembers.front().second; float distanceToClosestGroupMember = groupMembers.front().first; float separationPercentage = distanceToClosestGroupMember / mSeparationDistance; D3DXVECTOR3 desiredMoveAdj = nearestGroupMember.Position() – entity.Position(); if (separationPercentage < mMinSeparationPercentage) separationPercentage = mMinSeparationPercentage;

116

if (separationPercentage > mMaxSeparationPercentage) separationPercentage = mMaxSeparationPercentage; D3DXVECTOR3 currentDesiredMove = entity.DesiredMove(); if (distanceToClosestGroupMember < mSeparationDistance) { D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= -separationPercentage; currentDesiredMove += desiredMoveAdj * Gain(); } else if (distanceToClosestGroupMember > mSeparationDistance) { D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= separationPercentage; currentDesiredMove += desiredMoveAdj * Gain(); } entity.SetDesiredMove(currentDesiredMove); }

The comments have been stripped out for brevity. See the code for the unabridged version. Let us go over this in detail. tEntityDistList &groupMembers = entity.VisibleGroupMembers(); if (groupMembers.empty()) return;

First, we grab the list of group members. Since this is a group based behavior, if the group is empty, we cannot do anything. Thus, we return. cEntity &nearestGroupMember = *groupMembers.front().second; float distanceToClosestGroupMember = groupMembers.front().first;

Next, we get the nearest group member which, on account of sorting, should be at the front of the group members list. We then get the pre-computed distance to that group member. float separationPercentage = distanceToClosestGroupMember / mSeparationDistance;

Next we compute the separation percentage as the ratio between the actual distance to the closest group member, and the desired separation distance. D3DXVECTOR3 desiredMoveAdj = nearestGroupMember.Position() – entity.Position();

We also compute a vector from this entity’s position to the closest group member. if (separationPercentage < mMinSeparationPercentage) separationPercentage = mMinSeparationPercentage; if (separationPercentage > mMaxSeparationPercentage) separationPercentage = mMaxSeparationPercentage;

117

We clamp the computed separation percentage to our minimum and maximum separation percentages. D3DXVECTOR3 currentDesiredMove = entity.DesiredMove();

We grab the entity’s current desired move. if (distanceToClosestGroupMember < mSeparationDistance) { D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= -separationPercentage; currentDesiredMove += desiredMoveAdj * Gain(); }

If the distance to the closest member is less than our desired separation distance, we are too close. Thus, we normalize the vector from this member to our closest member, and scale it by our negated separation percentage. Why negated? The vector we computed was from this entity to the closest member, and we want a vector going the other way, so we negate it. Lastly, we scale our desired move adjustment by our gain, and add the result to the current desired move. else if (distanceToClosestGroupMember > mSeparationDistance) { D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= separationPercentage; currentDesiredMove += desiredMoveAdj * Gain(); }

If the distance to the closest member was greater than the separation distance, then we are too far away. Thus, we normalize the vector from this entity to the closest member, and scale it by our separation percentage. We then scale the desired move adjustment vector by our gain, and add the result to the current desired move for the entity. Why do we scale our desired movement vector by our separation percentage? The idea is based on the law of diminishing returns. If you are a great distance from the desired separation distance, you move faster to get there. If you are very close, you move very slowly. This helps settle us into the range we want rather than overshooting all the time. entity.SetDesiredMove(currentDesiredMove);

Finally, we set our entity’s desired move to be the newly computed desired move. If we were accurate about the separation distance, then both of the if statements would fail. We would set the desired move to the local value we had cached so nothing would change.

118

3.2.5 Avoidance

class cAvoidanceBehavior : public cBehavior { public: cAvoidanceBehavior(float avoidDist, float speed); virtual ~cAvoidanceBehavior(void); virtual void Iterate(float timeDelta, cEntity &entity); protected: float mAvoidanceDistance; float mAvoidanceSpeed; };

The comments have been removed from this listing for brevity. See the code for the full listing. Let us go over this behavior in detail. cAvoidanceBehavior(float avoidDist, float speed);

The avoidance behavior needs only two parameters; the avoid distance, and a speed. The avoid distance is the distance at which the threat to be avoided will actively be avoided, while the speed is the rate at which the entity will flee from the threat. virtual void Iterate(float timeDelta, cEntity &entity);

As with all of the behaviors, the Iterate method takes the time since the last iteration, and a reference to the entity to which the movement is to be applied. Let us go over this implementation. void cAvoidanceBehavior::Iterate(float timeDelta, cEntity &entity) { tEntityDistList &enemies = entity.VisibleEnemies(); if (enemies.empty()) return; cEntity &nearestEnemy = *enemies.front().second; float nearestEnemyDist = enemies.front().first; // head away from the enemy if (nearestEnemyDist < mAvoidanceDistance) { D3DXVECTOR3 desiredMoveAdj = entity.Position() – nearestEnemy.Position(); D3DXVECTOR3 currentDesiredMove = entity.DesiredMove(); D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= mAvoidanceSpeed; // move away currentDesiredMove += desiredMoveAdj * Gain(); entity.SetDesiredMove(currentDesiredMove); } }

The implementation is fairly straightforward -- we find the closest visible enemy, and run the other way.

119

tEntityDistList &enemies = entity.VisibleEnemies(); if (enemies.empty()) return;

First and foremost, we get the list of visible enemies. If there are no visible enemies, we do nothing. cEntity &nearestEnemy = *enemies.front().second; float nearestEnemyDist = enemies.front().first;

If there are visible enemies, then we get the first one in the list which has been sorted, and we fetch the distance to that enemy. if (nearestEnemyDist < mAvoidanceDistance)

If that distance is less than our avoidance distance, we run away. D3DXVECTOR3 desiredMoveAdj = entity.Position() – nearestEnemy.Position();

First we compute a vector from the enemy to us. D3DXVECTOR3 currentDesiredMove = entity.DesiredMove();

Then we grab our current desired move vector. D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= mAvoidanceSpeed; // move away currentDesiredMove += desiredMoveAdj * Gain();

We then normalize our enemy-to-us vector, scale it by our movement speed, apply our gain, and add the result to the current desired move. entity.SetDesiredMove(currentDesiredMove);

Lastly, we set the current desired move to our newly computed desired move.

120

3.2.6 Cohesion

class cCohesionBehavior : public cGroupBehavior { public: cCohesionBehavior(float turnRate); virtual ~cCohesionBehavior(void); virtual void Iterate(float timeDelta, cEntity &entity); protected: float mTurnRate; };

As usual, the comments have been removed from this listing for brevity. See the code for the full listing. Let us go over this in detail. cCohesionBehavior(float turnRate);

The cohesion behavior takes a single parameter -- the turn rate. The turn rate is the maximum rate at which the entity will change direction in order to head towards the average center of the group. virtual void Iterate(float timeDelta, cEntity &entity);

The Iterate method takes the time passed since the last iteration, and a reference to the entity for which to generate a movement. Let us look at this implementation. void cCohesionBehavior::Iterate(float timeDelta, cEntity &entity) { tEntityDistList &groupMembers = entity.VisibleGroupMembers(); if (groupMembers.empty()) return; // compute center of mass of the group D3DXVECTOR3 groupCenterOfMass(0.0f, 0.0f, 0.0f); tEntityDistList::iterator it; for (it = groupMembers.begin(); it != groupMembers.end(); ++it) { cEntity *e = (*it).second; groupCenterOfMass += e->Position(); } groupCenterOfMass /= (float)groupMembers.size(); // move towards the center of the group D3DXVECTOR3 desiredMoveAdj = groupCenterOfMass - entity.Position(); D3DXVECTOR3 currentDesiredMove = entity.DesiredMove(); D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= mTurnRate; currentDesiredMove += desiredMoveAdj * Gain(); entity.SetDesiredMove(currentDesiredMove); }

121

The cohesion behavior iterates across the list of visible group members, and computes the group’s average center of mass. It then generates a vector towards this center. tEntityDistList &groupMembers = entity.VisibleGroupMembers(); if (groupMembers.empty()) return;

First, the list of visible group members is obtained. If there are no visible group members, we return, as there is no group center of mass. D3DXVECTOR3 groupCenterOfMass(0.0f, 0.0f, 0.0f); tEntityDistList::iterator it; for (it = groupMembers.begin(); it != groupMembers.end(); ++it) { cEntity *e = (*it).second; groupCenterOfMass += e->Position(); } groupCenterOfMass /= (float)groupMembers.size();

Next, we iterate across the visible group members list, and sum up the positions of each member. We then divide out the number of group members that were visible to obtain the average group position, or center of perceived mass. D3DXVECTOR3 desiredMoveAdj = groupCenterOfMass - entity.Position();

We then compute a vector towards that center. D3DXVECTOR3 currentDesiredMove = entity.DesiredMove();

And obtain our current desired move. D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= mTurnRate; currentDesiredMove += desiredMoveAdj * Gain();

We normalize the us-to-center vector, and scale it by our maximum turn rate. Next we apply our gain and add the result to our current desired move. entity.SetDesiredMove(currentDesiredMove);

Finally, we set our entity’s desired move to our newly computed desired move.

122

3.2.7 Alignment

class cAlignmentBehavior : public cGroupBehavior { public: cAlignmentBehavior(float turnRate); virtual ~cAlignmentBehavior(void); virtual void Iterate(float timeDelta, cEntity &entity); protected: float mTurnRate; };

The comments have been removed from this listing for brevity. Look to the code for the full listing. Let us go over this behavior in detail. cAlignmentBehavior(float turnRate);

Like the cohesion behavior, the alignment behavior takes only a single parameter, the turn rate. The turn rate parameter limits the rate at which the entity will change direction in an effort to match heading with its visible group mates. virtual void Iterate(float timeDelta, cEntity &entity);

As in all the other behaviors, the Iterate method takes the time passed since the last iteration and the entity upon which to add desired move. Let us examine this implementation. void cAlignmentBehavior::Iterate(float timeDelta, cEntity &entity) { tEntityDistList &groupMembers = entity.VisibleGroupMembers(); if (groupMembers.empty()) return; cEntity &nearestGroupMember = *groupMembers.front().second; // match the heading of our closest group member D3DXVECTOR3 desiredMoveAdj = nearestGroupMember.Velocity(); D3DXVECTOR3 currentDesiredMove = entity.DesiredMove(); D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= mTurnRate; currentDesiredMove += desiredMoveAdj * Gain(); entity.SetDesiredMove(currentDesiredMove); }

In this implementation, we elected to use the heading of the nearest group member to align an entity, rather than an average of all of the visible group members. It would be simple enough to make the modification to average all of the headings of all of the visible group members and use that rather than only the closest member. You can try this as an exercise if you wish.

123

tEntityDistList &groupMembers = entity.VisibleGroupMembers(); if (groupMembers.empty()) return;

First, we get the list of visible group members. If that list is empty, we return. We cannot align without other members to align with. cEntity &nearestGroupMember = *groupMembers.front().second;

Next we obtain the closest group member by virtue of our sorted list. D3DXVECTOR3 desiredMoveAdj = nearestGroupMember.Velocity();

We then obtain the velocity of our nearest group member. The velocity embodies the direction which the member is facing. D3DXVECTOR3 currentDesiredMove = entity.DesiredMove();

We then obtain our current desired move. D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= mTurnRate; currentDesiredMove += desiredMoveAdj * Gain();

Next we normalize the velocity obtained from our closest group member, and scale it by our turn rate. This is now our desired move adjustment. We apply our gain, and add the result to the current desired move. entity.SetDesiredMove(currentDesiredMove);

Finally, we set this entity’s desired move to be our newly computed desired move.

124

3.2.8 Cruising

We have now covered all of the normal behaviors used in flocking applications, but there are more behaviors yet to investigate in our demo. The first is the Cruising behavior which, as mentioned earlier, is the behavior which decides where the entity would go if the decision were based solely on that entity. This is useful when it cannot see any of its group mates and needs to decide where to go. class cCruisingBehavior : public cBehavior { public: cCruisingBehavior ( float randMoveXChance, float randMoveYChance, float randMoveZChance, float minRandomMove, float maxRateChange, float minRateChange ); virtual ~cCruisingBehavior(void); virtual void Iterate(float timeDelta, cEntity &entity); protected: float mRandMoveXChance; float mRandMoveYChance; float mRandMoveZChance; float mMinRandomMove; float mMaxRateChange; float mMinRateChange; };

The comments have been removed from this listing for brevity. Look to the code for the full listing. Let us go over this behavior in detail. cCruisingBehavior ( float randMoveXChance, float randMoveYChance, float randMoveZChance, float minRandomMove, float maxRateChange, float minRateChange );

The cruising behavior takes quite a few parameters. The first three are percent chances the entity will decide to move in one of the cardinal directions. Next, we have the min random move, which is the minimum amount the entity will decide to move in the direction chosen. Last, we have the max and min rate changes, which limit the amount the entity is allowed to change direction.

125

virtual void Iterate(float timeDelta, cEntity &entity);

The Iterate method takes the time passed since the last iteration, and a reference to the entity upon which the cruising shall occur. Let us have a look at the implementation. void cCruisingBehavior::Iterate(float timeDelta, cEntity &entity) { // determine how fast we are going vs how fast // we would like to be going float currentSpeed = D3DXVec3Length(&entity.Velocity()); float percentDesiredSpeed = fabs((currentSpeed - entity.DesiredSpeed()) / entity.MaxSpeed()); float signum = (currentSpeed - entity.DesiredSpeed()) > 0? -1.0f : 1.0f; // clamp rate changes if (percentDesiredSpeed < mMinRateChange) percentDesiredSpeed = mMinRateChange; if (percentDesiredSpeed > mMaxRateChange) percentDesiredSpeed = mMaxRateChange; // add some random movement D3DXVECTOR3 desiredMoveAdj(0.0f, 0.0f, 0.0f); float randmove = (float)rand() / (float)RAND_MAX; if (randmove < mRandMoveXChance) desiredMoveAdj.x += mMinRandomMove * signum; else if (randmove < mRandMoveYChance) desiredMoveAdj.y += mMinRandomMove * signum; else if (randmove < mRandMoveZChance) desiredMoveAdj.z += mMinRandomMove * signum; D3DXVECTOR3 currentDesiredMove = entity.DesiredMove(); D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= mMinRateChange * signum; currentDesiredMove += desiredMoveAdj * Gain(); entity.SetDesiredMove(currentDesiredMove); }

The algorithm performs straightforward random picking of which direction to go next, and applies it. float currentSpeed = D3DXVec3Length(&entity.Velocity());

First, we determine how fast we are going at the moment. float percentDesiredSpeed = fabs((currentSpeed - entity.DesiredSpeed()) / entity.MaxSpeed());

We then determine what percentage of the entity’s desired speed we have achieved, as a ratio of its max speed. float signum = (currentSpeed - entity.DesiredSpeed()) > 0? -1.0f : 1.0f;

126

Next we establish whether we are going faster or slower than our desired speed, and store off a signum multiplier. By doing this, we can easily flip the direction of any directional vectors we create. if (percentDesiredSpeed < mMinRateChange) percentDesiredSpeed = mMinRateChange; if (percentDesiredSpeed > mMaxRateChange) percentDesiredSpeed = mMaxRateChange;

Next, we clamp the percent desired speed to our rate limits. This will prevent us from stopping instantly or jumping to full tilt from a standstill. float randmove = (float)rand() / (float)RAND_MAX; if (randmove < mRandMoveXChance) desiredMoveAdj.x += mMinRandomMove * signum; else if (randmove < mRandMoveYChance) desiredMoveAdj.y += mMinRandomMove * signum; else if (randmove < mRandMoveZChance) desiredMoveAdj.z += mMinRandomMove * signum;

Now we decide which direction we want to move by generating a random number, and comparing it against our percent chances per cardinal direction. In the case of the demo, the default is to limit the chance to start going up or down in the y dimension to keep the motion more realistic. D3DXVECTOR3 currentDesiredMove = entity.DesiredMove();

We grab our current desired move. D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= mMinRateChange * signum; currentDesiredMove += desiredMoveAdj * Gain();

We then normalize our randomly generated move vector, and scale it by our minimum rate change to get us going in the direction desired. We multiply by our signum for some extra randomness, apply our gain, and add the result to the current desired move. entity.SetDesiredMove(currentDesiredMove);

Finally, we set our entity’s desired move using our newly calculated desired move.

127

3.2.9 Stay Within Sphere

The last behavior implemented in this demo is the Stay Within Sphere behavior. This behavior was written to prevent the necessity of having to teleport the fish to keep them around the player’s location. Thus, they will just turn around when they get too far away. Let us take a look at the implementation. class cStayWithinSphereBehavior : public cBehavior { public: cStayWithinSphereBehavior(const D3DXVECTOR3 &center, float radius); virtual ~cStayWithinSphereBehavior(void); virtual void Iterate(float timeDelta, cEntity &entity); protected: D3DXVECTOR3 mCenter; float mRadius; };

The comments have been removed from this listing for brevity. Look to the code for the full listing. Let us go over this in detail. cStayWithinSphereBehavior(const D3DXVECTOR3 &center, float radius);

The stay within sphere behavior takes a center for the sphere, and a radius for the sphere’s radius. virtual void Iterate(float timeDelta, cEntity &entity);

This behavior uses the Iterate method, which takes the time from the last iteration and the entity to keep within the sphere. Let us see how it does this. void cStayWithinSphereBehavior::Iterate(float timeDelta, cEntity &entity) { D3DXVECTOR3 toCenter = mCenter - entity.Position(); float dist = D3DXVec3Length(&toCenter); if (dist > mRadius) { D3DXVECTOR3 desiredMoveAdj = toCenter / dist; D3DXVECTOR3 currentDesiredMove = entity.DesiredMove(); D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= entity.MaxSpeed(); currentDesiredMove += desiredMoveAdj * Gain(); entity.SetDesiredMove(currentDesiredMove); } }

128

The behavior is pretty simple. It determines the distance from the center of the sphere to the entity. If that distance puts the entity outside the sphere, it moves the entity towards the center of the sphere. Let us take a closer look. D3DXVECTOR3 toCenter = mCenter - entity.Position(); float dist = D3DXVec3Length(&toCenter);

First, the distance from the entity to the center of the sphere is computed. if (dist > mRadius)

If that distance is greater than the radius of the sphere, we are outside the bounds of the sphere. D3DXVECTOR3 desiredMoveAdj = toCenter / dist;

If we are outside the sphere, we first compute a desired move adjustment by taking the vector from the entity to the center of the sphere, and divide out the distance to the center. D3DXVECTOR3 currentDesiredMove = entity.DesiredMove();

We then get the current desired movement vector. D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= entity.MaxSpeed(); currentDesiredMove += desiredMoveAdj * Gain();

Then we normalize the desired movement adjustment, and scale it by the maximum speed of the entity. We want to be within the sphere. We apply our gain and add the result to the current desired move. entity.SetDesiredMove(currentDesiredMove);

Finally, we set our entity’s desired move to be the newly computed desired move. At this stage we now have a fairly complete flocking demonstration. There is plenty to learn here so it would be best if you took the needed to time really examine the source code for the demo in detail.

129

Conclusion

In this chapter we made the transition from pathfinding to decision making. We have found that flocking can be a useful means to simulate real life behavior of groups which move together in a fashion where they are directly influenced by other nearby entities. This type of behavior is found in real life in flocks of birds, schools of fish, herds of cows and other livestock, as well as crowds of people. You can probably imagine lots of scenarios where the behaviors we studied could be applied in game situations. Even at a simple level you could imagine a flock consisting of only two members – the player and his buddy. That buddy could use any number of these behaviors to stay within a certain distance and travel in roughly the same direction as the player. If attacked, your buddy might have to break away to defend himself, but if he has to stay within a given sphere around the player, ultimately flee if the player decides to flee. Lots of interesting ideas abound here and the results on screen can be very compelling. Obviously you could continue to extend even this simple simulation just by adding other buddies to your flock. Pretty soon you find yourself with some crude but convincing squad-like behavior. The squad-like behavior can be made much more effective with the addition of a robust decision making architecture for each entity (and even for the group itself) and some proper environmental awareness and navigation. And of course, you can apply these same ideas to groups of enemies as well (e.g. a group of Orcs in an RPG game). But even just for getting groups of entities to navigate around in the environment and maintain some semblance of cohesion (e.g., in a real-time strategy game) this system can be very helpful. Your flock can be loose and somewhat randomly distributed over a given area or you can have it maintain tight unit formation. Just create a new behavior (e.g., a grid-based concept for more rigid formations) and you are ready to go. As our fish demo demonstrates, flocking behaviors are useful in any environment. You can use it on land to gather your armies, at sea to manage your naval forces, or even in outer space to manage your armada of various spacecraft. To summarize, flocking makes use of Behavior Based Movement, which effectively sums up the desired movements from several behavioral algorithms to produce a final desired movement. The methods we discussed were:

Separation – Keeping the entity at a given distance from all of its neighboring group mates. Cohesion – Keeping the entity in line with the center of mass of the overall group, thereby giving

the appearance that the entity has a preference to be near the group. Alignment – Keeping the entity aligned in its orientation with the rest of the group (or at least its

closest group mate) so as to keep it traveling in the same direction as the group as a whole. Avoidance – Keeping the entity out of harm from dangerous entities. Cruising – Giving the entity a will of its own when it has no nearby group mates to guide it.

Flocking sets the stage for further exploration of decision making in that it is ultimately all about deciding where an entity wants to go. Decision making itself is the core of artificial intelligence, as it is the means for making something look or feel intelligent. It binds the other types of artificial intelligence together, as it makes use of the classification systems to make its decisions, and also makes decisions to move to desired destinations, which of course requires pathfinding.

130

In the next chapter, we will discuss one of the most flexible decision making systems available to the game AI developer: the finite state machine. Moreover, we will examine scripting and how its use can extend our finite state machines. Finally, we will look at our finite state machine application, which allows us to create our own custom state machines, simulate them, and save them for use in our games.

131

Chapter 4

Decision Making II:

State Machines

132

Overview

Having covered the topic of flocking in the last chapter, it will be easy for us to extend the concepts of making decisions about which way an entity wants to move into the more general concept of Decision Making. As mentioned early in the course, generalized decision making is the aspect of artificial intelligence systems which provides entities with the appearance of intelligence. In the case of a game like Psi Ops: The Mindgate Conspiracy©, the decision making system allowed the enemies and bosses to determine which attacks to make and when. It also determined where the characters wanted to move, when to duck, and when to roll out of the way. In a game like chess, the decision making system determines which move to play next, how to get itself out of check, or keep you in check. In a real time strategy game, the decision making system determines which buildings to build, which units to construct, how many units should collect resources, and what types of resources to collect. It also determines which military units with which to attack you, and how to mount that attack. The decision making system is the part of the artificial intelligence that takes all of the inputs and produces some action. There are quite a few types of decision making systems. The most common are:

Decision Trees State Machines Rule Base Squad Behaviors

We will begin our current chapter by briefly discussing each of these types of decision making systems. Once done, we will move on to examine in detail the system that we will choose for our decision making needs: state machines. State machines are very powerful forms of decision making systems with many different uses. We will talk about the most common uses of state machines and then delve into the state machine demo and its implementation. In this chapter we answer the following questions:

What is a finite state machine? What are finite state machines used for? What is a transition diagram? How is the finite state machine system implemented in our demo? What is scripting and how is it used in games? What is Python? How do I embed Python? What can I do with my newly embedded Python?

Before we start answering these questions, let us start with some brief discussion of general decision making systems.

133

Decision Trees

Decision Trees are one of the oldest forms of decision making techniques in games. A decision tree is basically a large set of nested if-then-else statements. Conditions are continually evaluated as you move down the tree. Eventually you end up in a leaf node of the tree where you have arrived at the decision concerning what you plan to do. An important thing to remember is that decision trees are completely stateless. Thus, during every iteration, the entire tree is evaluated again to come up with what the entity need to do. This can cause state flapping, which is, essentially, entities appearing to exhibit indecision.

See Player?

Attacking?

On PatrolPath?

Go To Patrol Path Patrol Path

No Yes

No Yes

Tired ofChasing?

Chase Player Stop AttackingStop Chasing

Attack Player

YesNo

No Yes

Figure 4.1

In Figure 4.1 we have an enemy that first checks to see if he is currently attacking. If so, he checks if he can see the player. If he is able to see the player, he attacks. If he cannot, he checks to see if he is tired of chasing the player. If so, he stops attacking. Otherwise, he chases the player. If he was not attacking the player, he checks to see if he is on his patrol path. If so, he patrols on his path, and if he is not, he returns to his patrol path.

134

State Machines

State machines, also known as finite state machines (FSM), take a different approach from decision trees. Rather than evaluating the entire decision process every time, the state machine remembers the last thing it was doing, and only evaluates decisions to see if it should leave the current state.

AttackingPatroling

Going toPatrol Path Chasing

Reached Patrol Path?

See Player?

Unable to See Player?

Tired of Chasing?

Able to See Player?See Player?

Figure 4.2

Figure 4.2 illustrates the state machine system under the equivalent circumstances as the decision tree example we just discussed in Figure 4.1. The red circle represents the initial condition. The character begins by walking his patrol. When he sees the player, he immediately attacks him and continues to do so until he cannot see the player anymore. At this point, he starts chasing the player. He will chase the player until he either sees the player again, in which case he goes back to attacking, or until he gets tired of chasing, in which case he returns to his path. When he reaches his patrol path, he will start patrolling again, unless he sees the player first, in which case he will start attacking him.

Rule Base

A rule base is similar to a decision tree, except it can contain some additional branching or conditions. In that sense, it is somewhat like a jump table. Each rule is evaluated, and the rule that receives the highest score (or the first one that evaluates to true, however it may be implemented) determines the decision. Rule 1: Conditions: Player in View Decision: Attack Rule 2: Conditions: Player not in view, not chasing, on patrol path Decision: Patrol Path Rule 3:

135

Conditions: Player not in view, not chasing, not on patrol path Decision: Go to Patrol Path Rule 4: Conditions: Player not in view, chasing, chase time limit has not expired Decision: Chase Player Rule 5: Conditions: Player not in view, chasing, chase time limit has expired Decision: Go to Patrol Path

Above we have a simple rule base that describes the conditions for each rule and the decision of the rule when it gets the highest score from the scoring system. The scoring system awards the most points to the most specific rule which matches, doles out fewer points for the lesser rules that match, and doles out even fewer points for the rules that partially match.

Squad Behaviors

Squad behavior is a hot topic in games, although it tends to be blown a bit out of proportion. Most people think of squad behavior and associate it with the extensive amount of communication and cooperation required of people acting in squads in the military. This is really only somewhat true with squad behaviors in games. A squad behavior, at its heart, is really just a more complex flocking system. One of the entities is picked as the squad leader, and that leader makes decisions that control the rest of the group. Each member of the group then acts on the orders of the squad leader (or not, depending on circumstances in the game of course).

Player Squad >50%?

Player Squad >25%?

No Yes

No Yes

Supported Attack Open Attack Supported Attack Open Attack

My Squad >50%?

My Squad >50%?

Player Squad >75%? Yes

My Squad >50%?

My Squad >50%?

Covered Attack Supported Attack Flee Flanking Attack

No Yes No Yes No Yes No Yes

Figure 4.3

Figure 4.3 presents an example of how a squad leader might decide to command his squad based on the relative strength between his squad and the player’s squad. If both his squad and the player’s squad are near full strength, he would decide to attempt a flanking attack, whereas if his squad is weaker than the player’s, he would decide to flee. Alternatively he might decide to try a supported attack, using his

136

snipers to target entities of interest while his machine gunners lay down a blanket of suppressing fire. Under other circumstances, he might find conditions to be such that his men should find cover and attack from behind. Or perhaps he would command his men to attack out in the open.

Attack

Open Attack? Yes

SupportedAttack?No Yes

Cover Attack?No YesSniper?

Snipe from CoverSuppressive Fire

from knee orprone.

No Yes

Attack from CoverSniper?

Snipe from CoverAttack from Flank

No Yes

No

Figure 4.4

Consider Figure 4.4. This is an example of the individual squad member’s decision tree. If the squad leader commands an open attack, every squad member obeys. If the squad leader commands a supported attack and the squad member is a sniper, he snipes from cover. Otherwise he will lay down suppressive fire. If the squad leader commands a cover attack, everyone would attack from a covered position. In all other cases, if the squad member is a sniper, he snipes from cover, while the machine gunners attack from the flanks. This is a pretty general overview of the concept, but you can probably already imagine various scenarios that involve squad behaviors. We still have a bit of ground to cover in this chapter before we can really begin to implement very interesting squad behavior, but the general groundwork is visible here. We will revisit this topic a bit later in the course when we assemble our final demonstrations and bring all of our concepts together.

137

4.1 Introduction to Finite State Machines

At their core, finite state machines are a type of graph. Each node in the graph is a state, while the connections between them represent transitions. The machine starts in some state, and while in that state, performs some pre-determined actions. After the actions are performed, all of the possible transitions from the current state into other states are evaluated. When one of the transitions evaluates to true, the state machine changes state into the new state specified by the transition. Unlike the decision tree we discussed earlier, it is not possible to go from one decision to any other decision. It is only possible to go from a given decision to the subset of all decisions for which this decision is applicable. For instance, in a decision tree, we might have the final decisions of walking on hands, walking, and climbing a ladder. It does not make much sense to be able to transition from climbing a ladder to walking on hands, but the decision tree would not prevent it. A state machine, however, is able to restrict the possible states into which the climbing ladder state could enter to only the walk state.

4.1.1 Transition Diagrams

While state machines can be described by tables or a simple text description, the most effective means to display the possible state transitions is via a transition diagram. A transition diagram displays all of the states that belong to the state machine, and all of the transitions between the states. It also can specify the conditions under which a transition would evaluate to true, and thereby cause a state change. Throughout the rest of this chapter, we will be using transition diagrams to explain our examples. Let us take a quick look at some example transition diagrams in order to acquire a firm understanding of them before moving on to some example uses of state machines.

Some Examples

Figure 4.5

138

The transition diagram in Figure 4.5 is very simple. The ovals are the states and the lines with arrows are the transitions, with the arrow pointing towards the destination state if the transition should evaluate to true. In the case of Figure 4.5, we start out in the Adder state, which adds 5 to the x value of the system. If the x value evaluates to greater than 5, then the system transitions to the subtractor state. The subtractor state then subtracts 1 from the value each time it is executed until the value is less than 0, in which case the system transitions back to the Adder state.

Figure 4.6

The next example (shown in Figure 4.6) represents a more complicated state machine. This state machine implements a system where, given a value to start at x and a value to get to g, it ultimately resolves x to g. It starts in the Init state, which sets up the variables for tracking the desired start value and goal value. From there, the machine is started. If x is equal to g, we move to the Done state. If x is less than g, we go to the Adder state where we add 5 at a time. Otherwise, if x is greater than g, we move to the subtractor state which subtracts 5 at a time. In either of the large accumulator states, if x becomes equal to g we move to the Done state. If we passed g with x, we start backing up one at a time. Once x equals g, we move to the Done state. Now that we have seen how transition diagrams can help us visualize our state machines, let us move on to some actual uses of state machines as they pertain to games.

139

4.1.2 Uses of Finite State Machines

In games, state machines are used all the time. In fact, just about any place where you have a switch statement in C++, there is probably some semblance of a state machine in action. State machines can be used for all sorts of things including artificial intelligence, animation control, game state, save file systems, networking engines, screen layout systems, and much more. Indeed, state machines are such useful systems specifically because they can be configured to handle a multitude of tasks. Let us take a look at some of the more common uses of finite state machines in game development, and talk about some examples of each application.

Animation

One of the most common uses of state machines in games is animation control. Animation sets in games typically include all types of sub-animations, such as blends between running and walking, running jumps, walking jumps, walking forward to strafing left or right, and so on. State machines provide a convenient means to define the appropriate transitions between animations as well as to keep track of the animation currently being played. Let us take a look at an example of how a state machine could be used to control the transitions between the movements of a character that can stand still, turn, walk, and strafe. Consider the transition diagram on the next page. Typically, animation-controlling state machines are complicated by virtue of the number of animations and blends that games typically have for characters. The default state of this state machine is the idle state. That is the state where the player is giving no input to the system, and the character is simply standing around. The transition lines specify the keyboard command which the player would give to move the character around, using the WSAD style of control. The number of states in this example is pretty high because every animation has a blend in and out state which may or may not have an animation associated with it. Also note that while it might appear that merely pushing A might drive you right through the transition state, the blending states might also have the condition that their blend be finished before the transition goes through. However, there is no need to overcomplicate the already complex machine with this information. Spend some time reviewing the machine.

140

141

Game State

Another very common use of state machines is for preserving game state information. In fact, the reason why it is called game state has a background in state machine terminology. Consider a mission-based scenario. Some goals need to be set, typically in a specific order but not necessarily, and the mission progress is advanced as the goals are satisfied.

Figure 4.7

For example, note Figure 4.7. In this mission, there is a critical teammate who must survive the mission, and there are three separate waypoints which need to be visited while defeating all enemies. At each waypoint, you must kill all the enemies before you can continue on. If at any point along the way you or your critical teammate perishes, you fail the mission. This is a very common way for games to use state machines.

142

Save File System

Save file systems are another common use of state machines. In many cases, saving or restoring a file from disk can involve many steps which may or may not occur depending upon the situation of the save file.

Figure 4.8

Figure 4.8 illustrates a possible save flow for a memory card based save system. Initially, there is a check to see if there is a memory card, and if not, a request is made for one to be inserted. If there is a memory card, a slot must be chosen to which we want to save. If that slot is empty, we check for sufficient space, and if available, save to the card. If there is not space, we try to free up space, or insert a new card if we want to use a different memory card. If the slot we picked has a save in it already, we can overwrite it, or pick another slot. We can see how state machines can keep the number of potential options down to a minimum at each state, while maintaining some memory about what was going on beforehand.

143

Artificial Intelligence

The last of the most common uses of state machine systems is for artificial intelligence. State machines are incredibly useful for artificial intelligence because of their inherent ability to “remember” information. What the AI decides to do is dependent upon what he is doing at the moment as well as what is going on around him. This is a very important distinction because other systems require that the current actions of the system be fed into the system externally or otherwise bolted on. As discussed earlier in the chapter, we can use the state machines to drive the individual characters as well as group leaders. A demo state machine system has been implemented for this chapter which will allow us to play with state machines in some depth. Let us take a look at it and then put together some examples of state machines that could drive AI in different games.

4.2 The State Machine Demo

The State Machine Demo provides both an interface through which you see a state machine working, as well as a means by which you can create and save state machines for use in your game. There is definitely room for improvement on the feature set of the tool, but it is a solid start for a generalized system where you can build custom state machines for any purpose in your game. Let us take a look at the interface:

Figure 4.9

144

The pane on the right in Figure 4.9 shows a representation of the state machine. The red state is the current state. On the tool bar, there are the options for creating a new state machine, opening a saved state machine, saving the current state machine, resetting the current state machine, and iterating the simulation for one iteration for the current state machine. The left pane shows a tree view of the current state machine. There is always a single machine at the top level, and it can contain any number of states. Each state can have a list of actions that it performs when the state is entered, as well as when it is exited. There is also a list of actions it can perform for each simulation iteration. Each state also has a list of transitions that it will evaluate after its simulation iteration to see if it should transition out.

4.2.1 The Implementation

Like all of the other demos included in this course, the State Machine Demo makes use of MFC. The CStateMachineDoc class holds onto the cStateMachine object and is responsible for serializing it. The CStateTreeView class is responsible for ensuring the tree view is up to date, and the CStateMachineView is responsible for drawing the state transition diagram. There are various dialogs responsible for aiding in adding/removing/updating states, actions, and transitions.

cStateMachine

cActioncTransition

CStateMachineDoc

CStateMachineApp

CStateMachineView CStateTreeView

cState

Figure 4.10

145

The cStateMachine class maintains a vector of pointers to cState objects. Each of the cState objects maintains a vector of pointers to the cAction objects used in the enter, exit, and iteration phases of the state. Additionally, the cState objects maintain a vector of cTransition object pointers. The cAction class has derived versions for actions that act on a single value, as well as scripted actions which we will discuss later. Similarly, the cTransition class has some derived types for scripted transitions and transitions that do simple comparison operations on a value held by the state. Let us take some time to delve into the implementation of the state machine in detail.

The State Machine Class

class cStateMachine : public cObject { public: typedef vector< shared_ptr<cState> > tStateList; cStateMachine(void); virtual ~cStateMachine(void); void Iterate(void); void AddState(shared_ptr<cState> state); void RemoveState(shared_ptr<cState> state); shared_ptr<cState> GetStatePtr(cState *state); shared_ptr<cState> GetStatePtrByName(string &name); void Reset(void); tStateList &States(void) { return(mStates); } void ToDot(string &dotString); int Serialize(ofstream &ar); int UnSerialize(ifstream &ar); protected: tStateList mStates; shared_ptr<cState> mCurrentState; };

Above we have the class declaration for the cStateMachine class. One thing you might notice is the use of shared_ptr<> template classes. The shared_ptr<> template class is an automatic reference counting class provided by the Boost template libraries (www.boost.org). The shared_ptr<> template smart pointer is not intrusive, so it uses additional storage for the ref counts and use counts. This is important to note, since creating a new shared_ptr<> will create a new instance of the ref and use count storage. It is absolutely critical that the original or a copy of the shared_ptr<> initially created be used when making new shared_ptr<> objects pointing to the same memory, or the reference counting will break. Let us now go over the details of this class.

146

cStateMachine(void); virtual ~cStateMachine(void);

The cStateMachine class provides a default constructor which builds an empty state list and has a NULL current state. The destructor is virtual to provide for correct deletion of derived types. void cStateMachine::Iterate(void) { if (mCurrentState) { mCurrentState->Iterate(); for ( cState::tTransitionList::iterator it = mCurrentState->Transitions().begin(); it != mCurrentState->Transitions().end(); ++it ) { cTransitionPtr trans = *it; if (trans->ShouldTransition()) { mCurrentState->Exit(); mCurrentState = trans->TargetPtr(); mCurrentState->Enter(); break; } } } }

The iterate method iterates the current state’s simulation, and checks to see if the state should transition afterwards. Let us walk through this method. if (mCurrentState)

First, we check to see if we have a current state pointer to ensure we can iterate its simulation. mCurrentState->Iterate();

We then iterate the current state. In the future, we might pass a time delta in to provide for frame independent AI code. for( cState::tTransitionList::iterator it = mCurrentState->Transitions().begin(); it != mCurrentState->Transitions().end(); ++it )

We then loop over all of the transitions for the current state.

147

cTransitionPtr trans = *it; if (trans->ShouldTransition())

We check each transition to see if the criteria for that transition have been met. mCurrentState->Exit();

If we determine we should transition, we first exit the existing state, allowing any exit time actions to take place. mCurrentState = trans->TargetPtr();

We then set our current state to be the target state of the transition. mCurrentState->Enter();

Now we enter the new current state, and execute all of the actions in the new state’s enter list. break;

We break out of the loop, since we have transitioned, and we do not want to try to transition again. void AddState(shared_ptr<cState> state); void RemoveState(shared_ptr<cState> state);

The AddState and RemoveState methods provide for a means to add and remove states from the machine. The machine takes responsibility for deleting the states, but the shared_ptr<> classes should do that for us when the last reference to the state goes away. shared_ptr<cState> GetStatePtr(cState *state); shared_ptr<cState> GetStatePtrByName(string &name);

These methods provide for a means to get the actual shared_ptr<> object that the machine is holding onto. This is mostly necessary for the GUI code, but it also helps so we do not miscalculate our ref counting. void cStateMachine::Reset(void) { if (!mStates.empty()) mCurrentState = mStates.front(); if (mCurrentState) mCurrentState->Reset(); }

The Reset method makes the first state in the state list the current state, and resets its value back to its initial value. This is used to reset the machine. tStateList &States(void) { return(mStates); }

148

Here we have an accessor to gain access to the list of states the machine has. void ToDot(string &dotString);

The ToDot() method builds a string which contains an AT&T Graphviz™ (http://www.research.att.com/sw/tools/graphviz/) DOT format representation of the state machine. This is used by the transition view to draw the diagram. int Serialize(ofstream &ar); int UnSerialize(ifstream &ar);

The Serialize and UnSerialize methods are the methods responsible for writing and reading the state machine to/from disk.

The State Class

class cState : public cObject { public: typedef vector<shared_ptr<cTransition> > tTransitionList; typedef vector<shared_ptr<cAction> > tActionList; cState(string name) : mName(name), mValue(0), mInitialValue(0) {} cState(float initialValue, string name) : mValue(initialValue), mInitialValue(initialValue), mName(name) {} virtual ~cState(void); float Value(void) const { return(mValue); } float InitialValue(void) const {return(mInitialValue);} void SetValue(float value) { mValue = value; } void SetInitialValue(float value) { mInitialValue = value; } void Reset(void); const string &Name(void) const { return(mName); } void SetName(const string &name) { mName = name; } void Enter(void); void Iterate(void); void Exit(void); tTransitionList &Transitions(void) { return(mTransitions); } tActionList &Actions(void) { return(mActions); }

149

tActionList &EnterActions(void) { return(mEnterActions); } tActionList &ExitActions(void) { return(mExitActions); } shared_ptr<cAction> GetActionPtr(cAction *action, tActionList &actions); shared_ptr<cTransition> GetTransitionPtr(cTransition *trans); int Serialize(ofstream &ar); int UnSerialize(ifstream &ar, cStateMachine &sM); protected: int SerializeActions(ofstream &ar, tActionList &aL, string actionListType); int UnSerializeActions(ifstream &ar, tActionList &aL, shared_ptr<cState> &state); float mValue; float mInitialValue; tTransitionList mTransitions; tActionList mActions; tActionList mEnterActions; tActionList mExitActions; string mName; };

The cState class provides the list of actions to be run at iteration time, as well as the time when the state is entered or exited. The state class also provides the transitions which determine which other states can be entered from this state. Let us take a look at the state class in greater depth. cState(string name) : mName(name), mValue(0), mInitialValue(0) {} cState(float initialValue, string name) : mValue(initialValue), mInitialValue(initialValue), mName(name) {} virtual ~cState(void);

The cState class provides no default constructor, but does provide a constructor to initialize the name of the state, as well as a means to set an initial value and the name of the state. The destructor is virtual to provide for correct polymorphic destruction of derived classes. float Value(void) const { return(mValue); } float InitialValue(void) const {return(mInitialValue);} void SetValue(float value) { mValue = value; } void SetInitialValue(float value) { mInitialValue = value; }

cState provides accessors to its initial value and current value members.

150

void cState::Reset(void) { mValue = mInitialValue; Enter(); }

The Reset() method resets the current value in the state to the initial value, and executes all of the actions in its enter list. This is used for resetting the state machine. const string &Name(void) const { return(mName); } void SetName(const string &name) { mName = name; }

The class provides a means by which to access the name, or update it. void Enter(void); void Iterate(void); void Exit(void);

The Enter(), Iterate(), and Exit() methods execute the list of actions appropriate to the method by iterating over the list and calling Execute() on each action therein. tTransitionList &Transitions(void) { return(mTransitions); } tActionList &Actions(void) { return(mActions); } tActionList &EnterActions(void) { return(mEnterActions); } tActionList &ExitActions(void) { return(mExitActions); }

The class provides accessors to the lists of transitions and actions it contains. shared_ptr<cAction> GetActionPtr(cAction *action, tActionList &actions); shared_ptr<cTransition> GetTransitionPtr(cTransition *trans);

Similar to the cStateMachine class, the cState class provides a means to get access to the actual shared_ptr<> objects contained within the class to keep the reference counting up to date. int Serialize(ofstream &ar); int UnSerialize(ifstream &ar, cStateMachine &sM);

The class also provides methods to serialize itself to/from a file.

151

The Action Classes

class cAction : public cObject { public: cAction(cStatePtr state) : mState(state) {} virtual ~cAction(void) {} virtual void Execute(void) {}; virtual string Label(void) { return(string("Base Action")); } cState &State(void) const { return(*StatePtr()); } cStatePtr StatePtr(void) const { return(mState.lock()); } protected: weak_ptr<cState> mState; };

The action class does the work in a state. Notice the Action class has a weak_ptr<> to the state it belongs in. This is another Boost template library smart pointer. A weak_ptr<> maintains a pointer to the object, but does not increment its ref count until it is locked. This is to prevent cyclic ref counted pointer connections which would prevent the memory from being freed. A shared_ptr<> object is typically used for something that is a HAS A relationship whereas a weak_ptr<> object is typically used for something that is a USES A relationship. The cState owns the cAction, so the cState gets the shared_ptr to the cAction, and the cAction gets a weak_ptr<> back to the cState. Let us examine this class a little more closely. cAction(cStatePtr state) : mState(state) {} virtual ~cAction(void) {}

The constructor takes a shared_ptr<> to the state the action belongs to. This ensures proper ref counting, and weak_ptr<> objects need a shared_ptr<> object to be constructed. The destructor is virtual to allow for proper polymorphic destruction of derived types. virtual void Execute(void) {};

The execute method does nothing in the base class. virtual string Label(void) { return(string("Base Action")); }

The label is for the UI. It is overloaded in the derived types. cState &State(void) const { return(*StatePtr()); } cStatePtr StatePtr(void) const { return(mState.lock()); }

The StatePtr() method locks the weak_ptr<> object, which returns a shared_ptr<> object. The State() method calls StatePtr() to get the shared_ptr<> object, and then dereferences it to hand back a reference.

152

class cValueBasedAction : public cAction { public: cValueBasedAction(cStatePtr state, float value) : cAction(state), mValue(value) {} virtual ~cValueBasedAction(void) {} virtual void Execute(void) {}; virtual string Label(void) { return(string("Value based Action")); } float Parameter(void) const { return(mValue); } protected: float mValue; };

The cValueBasedAction is a base class for actions that act on the cState object’s value data member using a parameter. Let us look at what is different from the cAction base class. cValueBasedAction(cStatePtr state, float value) : cAction(state), mValue(value) {}

The constructor now also takes a value to initialize the cValueBasedAction’s parameter. virtual void Execute(void) {};

The execute method still does nothing. The derived types will take care of that. float Parameter(void) const { return(mValue); }

We now have a parameter member accessor. Let us take a look at the derived types’ Execute methods, since that is all that changes from now on: void cAddAction::Execute(void) { State().SetValue(State().Value() + mValue); }

The cAddAction adds its value to its cState object’s value. void cSubtractAction::Execute(void) { State().SetValue(State().Value() - mValue); }

The cSubtractAction subtracts its value from its cState object’s value. void cInverseSubtractAction::Execute(void) { State().SetValue(mValue - State().Value()); }

The cInverseSubtractAction subtracts its cState object’s value from its value. void cMultiplyAction::Execute(void) { State().SetValue(State().Value() * mValue); }

153

The cMultiplyAction multiplies its value by its cState object’s value. void cDivideAction::Execute(void) { State().SetValue(State().Value() / mValue); }

The cDivideAction divides its cState object’s value by its value. void cInverseDivideAction::Execute(void) { State().SetValue(mValue / State().Value()); }

The cInverseDivideAction divides its value by its cState object’s value. There is one other type of cAction derived class, the cScriptedAction, but we will address it later after we have discussed scripting.

The Transition Classes

class cTransition : public cObject { public: cTransition(cStatePtr source, cStatePtr target) : mSource(source), mTarget(target) {} virtual ~cTransition(void) {} virtual bool ShouldTransition(void) { return(false); } cState &Source(void) { return(*SourcePtr()); } cState &Target(void) { return(*TargetPtr()); } cStatePtr SourcePtr(void) { return(mSource.lock()); } cStatePtr TargetPtr(void) { return(mTarget.lock()); } virtual string Label(void) { return(string("Base Transition")); } protected: weak_ptr<cState> mSource; weak_ptr<cState> mTarget; };

The cTransition class determines if a given cState should transition into the target state given by the transition. Like the cAction class, the cTransition class uses weak_ptr<> objects for its cState pointers. Let us take a deeper look at this class. cTransition(cStatePtr source, cStatePtr target) : mSource(source), mTarget(target) {} virtual ~cTransition(void) {}

The cTransition class constructor takes a source cState pointer and a target cState pointer. This will define a transition from source to target. The destructor, as always, is virtual to provide for correct polymorphic destruction of derived types.

154

virtual bool ShouldTransition(void) { return(false); }

The base class ShouldTransition method simply returns false. The derived types will do some work here to determine if the source state should transition to the target state. cState &Source(void) { return(*SourcePtr()); } cState &Target(void) { return(*TargetPtr()); } cStatePtr SourcePtr(void) { return(mSource.lock()); } cStatePtr TargetPtr(void) { return(mTarget.lock()); }

Like the cAction class, we have accessors to both references to the source and target states as well as shared_ptr<> objects. virtual string Label(void) { return(string("Base Transition")); }

The label is overridden by the derived classes to provide the UI with a descriptive label. class cComparitorTransition : public cTransition { public: cComparitorTransition ( cStatePtr source, cStatePtr target, float threshold, cComparitor *func ) : cTransition(source, target), mThreshold(threshold), mFunc(func) { } virtual ~cComparitorTransition(void) { if (mFunc) delete mFunc; mFunc = NULL; } virtual bool ShouldTransition(void) { return((*mFunc)(Source().Value(), mThreshold)); } float Threshold(void) const { return(mThreshold); } void SetThreshold(float threshold) { mThreshold = threshold; } virtual string Label(void) { string label; char number[16]; label += (*mFunc).Label(); label += " "; sprintf(number, "%0.3f", mThreshold); label += number; return(label); }

155

protected: float mThreshold; cComparitor *mFunc; };

The cComparitorTransition class is a simple transition class which uses a comparitor which will make a comparison between the value of the state and the parameter given to the transition. It will use this comparison to determine if the state should transition to the source state. Let us go over this class. cComparitorTransition ( cStatePtr source, cStatePtr target, float threshold, cComparitor *func ) : cTransition(source, target), mThreshold(threshold), mFunc(func) { }

The constructor takes the source and target states, as the base class does, but it also takes a threshold, and a pointer to a comparison function. virtual ~cComparitorTransition(void) { if (mFunc) delete mFunc; mFunc = NULL; }

The destructor frees the comparison function if it has one. virtual bool ShouldTransition(void) { return((*mFunc)(Source().Value(), mThreshold)); }

The ShouldTransition method simply returns the result of the comparison function operation. It uses the value of the state and the threshold of the transition for the comparison operands. float Threshold(void) const { return(mThreshold); } void SetThreshold(float threshold) { mThreshold = threshold; }

We also have some accessors for the transition’s threshold. virtual string Label(void) { string label; char number[16]; label += (*mFunc).Label(); label += " "; sprintf(number, "%0.3f", mThreshold);

156

label += number; return(label); }

Our Label method makes a descriptive label for our UI. Note the use of the comparison function’s label method to supplement the label of the transition. Now that we have seen the cComparitorTransition implementation, let us take a look at the cComparitor object itself, as well as the different comparisons we have already implemented. class cComparitor { public: virtual bool operator()(float lhs, float rhs) = 0; virtual string Label(void) = 0; };

The cComparitor class itself is very simple. It provides a binary operator, which is pure virtual in the base class, as well as a label for the UI (again pure virtual). Now let us see some actual implementations of the cComparitor class. bool cGreaterThanComparitor::operator()(float lhs, float rhs) { return(lhs > rhs); }

The cGreaterThanComparitor checks to see if the left hand side value is greater than the right hand side value. bool cGreaterThanOrEqualComparitor::operator()(float lhs, float rhs) { return(lhs >= rhs); }

The cGreaterThanOrEqualComparitor checks to see if the left hand side value is greater than or equal to the right hand side value. bool cLessThanComparitor::operator()(float lhs, float rhs) { return(lhs < rhs); }

The cLessThanComparitor checks to see if the left hand side value is less than the right hand side value. bool cLessThanOrEqualComparitor::operator()(float lhs, float rhs) { return(lhs <= rhs); }

The cLessThanOrEqualThanComparitor checks to see if the left hand side value is less than or equal to the right hand side value. bool cEqualComparitor::operator()(float lhs, float rhs) { return(lhs == rhs); }

157

The cEqualComparitor checks to see if the left hand side value is equal to the right hand side value. bool cNotEqualComparitor::operator()(float lhs, float rhs) { return(lhs != rhs); }

The cNotEqualComparitor checks to see if the left hand side value is not equal to the right hand side value. The only other type of cTransition we will discuss is the cScriptedTransition, which we will address later after we have talked about scripting. We now have a good understanding of state machines and how they function, and we have gone through a large portion of the source code used in our chapter demonstration. Please refer back to the diagrams and descriptions in this section if you are unsure of how these pieces all fit together. At the moment our state machine implementation is fairly robust and can certainly be extended to handle lots of different needs. But we can make it even more powerful and flexible by adding scripting to our system. This will be the subject of the next section of this chapter.

4.3 Scripting in Games

Not everything in our game has to be implemented using a heavyweight language such as C++. C++ is very powerful, but it does have some pitfalls in game development. Perhaps the greatest flaw is the inability to easily and quickly update the system; recompiling a game can take a very long time. Besides, game designers, many of whom will not always be accomplished computer scientists with extensive C++ training, do not necessarily need access to the actual code of the game. In many cases, it is argued that they should not have such access. Indeed, you might even think it fairly dangerous to grant that level of access to team members who are not qualified C++ developers. An alternative approach that allows game designers to make various updates and modifications to game behavior and functionality without changing the core source code (and without the need for repeated compilations for every change) would certainly be useful. This is where scripting and scripting languages come into play. Scripting is a methodology for making code data driven. It can be used to make a variety of things happen in your game. For example, your AI controlled NPC can be scripted to take a specific action when interacting with the player. Scripting can also be used to control simple or complex sequences of events such as which movies play when, or which sound effects and particle effects are triggered to occur at a certain time during a non-interactive in-game cinema sequence, etc. Scripting is typically used in places where a lot of iteration is required because it can be done faster, given its ability to iterate by re-running the script rather than recompiling and linking the game, and launching the scenario again. Scripts are generally written using a lightweight programming language that is much easier for less technical team members to learn. The language commands are most often interpreted at runtime by the scripting system, although some systems provide just-in-time compiling to speed things up. Thus our main game source code does not need to be recompiled every time changes to action sequences or other events are made. Scripting is also typically memory management friendly, at least from a user’s point of

158

view. In most scripting systems you can simply define a variable and it will be cleaned up whenever the garbage collection system determines it is no longer in use. This can be good and bad, since it can cause strange memory usage patterns if you are working on embedded platforms and not on a PC where you have virtual memory. There are numerous scripting languages available such as Perl, Python, Lua, VBScript, JavaScript and its variants, UnrealScript, etc. Some game development shops, such as Bioware (makers of Neverwinter Nights©) prefer to create their own scripting system, while others, like our team at Midway Games (makers of Psi Ops: The Mindgate Conspiracy©) prefer to use existing systems like Python. Developing a proprietary scripting system requires a significant amount of time and effort. With the maturation of existing scripting languages and systems and their current flexibility and stability, many game shops are making the decision to go with technologies that are freely available and ready for use. In this course we will follow that trend and use a scripting language that is powerful and easily integrated into our existing C++ source code. We have chosen Python as the scripting language for this course, but as we saw a moment ago, there are other options available should you decide that a different choice better suits your game project. Even if you do decide to adopt an alternative language, the concepts and techniques we will study in this chapter should help prepare you for what you will need to do to get up and running with another system. In the next section, we will take a crash course in Python. Fortunately, using the Boost library (which we will discuss shortly) we will have a very simple task of integrating scripting into our state machine. Boost provides a wonderful C++ to Python interface library that will make our life very easy.

4.4 Introduction to Python

Python is an interpreted objected oriented programming language. It include features like classes, exceptions, high level dynamic data types, modules, dynamic typing, and a large library of both built-in functionality as well as extended functionality via modules implemented in C or C++. While you will receive a very quick introduction to Python here in this course, it is highly recommended that you visit the Python website (www.python.org) and look at their tutorials and other materials to learn more and get more comfortable with the language.

4.4.1 Scope and Whitespace

An important thing to be aware of right upfront is that Python determines scope based on leading whitespace and does not use the familiar curly braces { } or similar character driven means. Thus, if you define a method at leading whitespace count 0, every line that is part of that method must be at leading whitespace count 1 or more. You will better understand this when we get to the examples, but it is advised that you download a decent text editor that allows visualization of whitespace. Fortunately, the Scintilla (http://www.scintilla.org/) editor has been embedded into the script editor dialogs in the State Machine Demo, so in the demo you will see whitespace.

159

4.4.2 Default Types and Built-Ins

Python has a standard assortment of data types. You do not have to specify the type when a variable is created; it is determined from context. Boolean types can be set to 0 or 1, or False or True. Comparisons are done very much like C/C++:

< Less Than

<= Less Than or Equal

> Greater Than

>= Greater Than or Equal

== Equal

<> or != Not Equal

Is Object Identity

is not Negated Object Identity

And Logical AND

Or Logical OR

Not Negation Operator

The only major differences are in the “is”, “is not”, “and”, “or”, and “not” constructs. There are four distinct types of numbers: integers, long integers, floating point numbers, and complex numbers. Integers, long integers, and floating point numbers are equivalent to the C++ counterparts, only bigger than you would think. Integers are longs in C++ (32 bits), and floating point numbers in Python are double precision in C++ (64 bits). Boolean numbers are a special type of integer. Complex numbers are numbers with i conjugate parts. There is probably not a major need to work with those because complex numbers are not used much in games. All of the numerical operations are the same as C++ (including % for modulo), and there is also a power operator ** (i.e. x ** y is x to the yth power). Python also supports iterator types for iterating across sequences, but refer to the Python documentation for proper descriptions. Sequence types are also supported in Python. Strings are considered a sequence type, as are lists, tuples, buffers, and xrange objects. The important thing to remember is that strings can be written using single or double quotes. Python does not care as long as the start and end of the literal string is done in the same way. Most sequence objects support the “in” and “not in” operations which are used in iterating

160

across items in the container. The built-in method len() can also be used to determine the number of items in the sequence. Slices can be performed on most sequence types using [i:j:k] and a single item can be obtained with [i], and the built-ins min() and max() provide the smallest and largest item in the container. There is a plethora of string operations which allow for various things such as changing case, stripping whitespace, and determining if a character is a digit or alpha-numeric. See the documentation for more details. There are also File objects and Class objects, as well as mapping types such as Dictionaries and Mappings. Please refer to the official documentation for these concepts.

4.4.3 Classes

Python has a concept of classes, just like C++. In fact, as you will see later, we can even derive Python classes from our C++ classes! Python does not have a concept of public versus private data, however, so every data member and method will be public. Let us take a look at an example Python class to see some of these concepts just discussed in action: class HelloWorld: def say_hi(self): print “Hello World!”

Here we have a definition of the HelloWorld class. This class provides a single method, say_hi(). All class methods must have the first parameter be self. The self value is basically the this pointer in C++. While C++ will explicitly pass the this pointer on the stack to the method, Python requires we specify it manually. Notice the indenting that was mentioned before. The class scope is everything greater than 0 whitespace, and the say_hi is everything greater than 1. Let us look at a more complex example. class HelloWorldCount: “document string for hello world count” hi_count = 0 initialized = False __init__(self): self.hi_count = 0 self.initialized = True def say_hi(self): print “Hello World!” self.hi_count = self.hi_count + 1

Here we have a more complex class. Note the string under the class declaration which is a document string. You can put documentation strings in the first line after a class or method declaration which Python can tell you about. This could be used in your game to describe the event you are working on or you could ignore it altogether. Pay special attention to the indention, as this is how Python knows which lines belong to which methods, and what belongs to the class itself.

161

hi_count = 0 initialized = False

These are members which are stored in the class and they have initial values assigned. Note that we do not have to specifically declare the variable type. Python understands how the variables are used based on context. In this case we are assigning particular values that make it pretty clear what types of variables these should be. This is actually a fairly common feature in scripting languages. __init__(self): self.hi_count = 0 self.initialized = True

This method is similar to a constructor. It is not identical to a constructor, but it is as close as we get in Python. The __init__ method can have additional parameters as well, so you can initialize a class with external data. In this example, it initializes the hi_count to 0 (not really necessary, just to illustrate what the __init__ method does), as well as sets the initialized member to True. Notice the member variables are scoped using the self member. This is required. If desired, we could call other methods using the self member as well. def say_hi(self): print “Hello World!” self.hi_count = self.hi_count + 1

In the say_hi method, we again print “Hello World!” but we also increment our hi_count. Python also supports inheritance and multiple inheritance like so: class SpecialHelloWorld(HelloWorld): def say_hi(self): HelloWorld.say_hi(self) print “Special Hello World!”

If multiple inheritance were desired, we would list the classes separated by commas in the parentheses after the new class name. Our base class implementation can be called by simply using the base class name as if it were an object, and passing in our self member. Python class methods are always virtual. Thus, the most specialized derived type’s method will be called.

4.4.4 Functions

Freestanding functions in Python can be defined outside the scope of any class. They can be called from within any class that includes the package in which the function is defined. It is also possible to define freestanding functions that are implemented in C or C++ using the methods we will describe shortly. It is important to note that functions always pass their parameters by object reference, so if the object is mutable, any changes made to it will be reflected after the function call.

162

A few examples: def pythagorean(a, b): “Find the length of the hypotenuse c as given by sides a and b” c = a*a + b*b c = sqrt(c) return c

Here we have the Pythagorean Theorem 222 cba =+ . This method finds the value of c given a and b. Clearly, 22 bac += , and this function does exactly that. def toFahrenheit(celsius): “Find equivalent temperature in Celsius degrees from Fahrenheit degrees” fahrenheit = (9.0/5.0) * celsius + 32.0 return fahrenheit

Here we have a function which converts degrees Celsius to degrees Fahrenheit. def toCelsius(fahrenheit): “Find equivalent temperature in Fahrenheit degrees from Celsius degrees” celsius = (5.0/9.0) * (fahrenheit - 32.0) return celsius

Here we have a function which converts degrees Fahrenheit to degrees Celsius. def doNothing(): “Do a whole lot of nothing” pass

Here we have a function which does a whole lot of nothing. The pass keyword is required so that the language parser can know where the scope of the function begins and ends. Again, this is due to the leading whitespace rules. def jump(timesToJump = 3): “Jump however many times” pass

Here we have a function with default parameters. This is just as how C++ does it.

4.4.5 Control Statements

Like any other programming language, Python has its share of control statements, although it does not have quite the same number of control statements as C++. Python supports if-then-else statements, and for loops. If-then-else statements look like the following:

163

if x < 0: print “x is less than zero” elif x > 0: print “x is greater than zero” else: print “x is zero”

Notice that in C++, we would use “else if” and not “elif”. Again, the leading whitespace rules dictate which statements belong in which branch. if x < 0: print “x is less than zero” if x < -10: print “x is less than -10” else: print “x is less than zero but greater than -10”

Note how the leading whitespace can make statements which would require braces in C++ easier to write. The leading whitespace can cause problems in other places, but here it is useful to resolve a common ambiguity. For loop statements are somewhat different than what we are familiar with in C++. They iterate over a range specified, or over all of the elements in a list. hats = [‘top hat’, ‘cowbow hat’, ‘baseball cap’] for hat in hats print “I like my”, hat

Above we have a list of hats, and we iterate across every element in the list. We also can iterate over a range in numbers. Print “Counting to 10” for i in range(1, 10) print i

Here we are counting from 1 to 10 and printing it out. Print “Counting to 10 by 2” for i in range(2, 10, 2) print i

The range function also allows us to modify our increment. Here we are counting from 2 to 10 by 2s. hats = [‘top hat’, ‘cowbow hat’, ‘baseball cap’] for hat in hats[:]: print “I like my”, hat hats.insert(0, hat)

164

Normally, adding to the container you are iterating over is a bad idea since it can invalidate your iteration. The slice operator [:]: however, makes it easy to make a copy of the list in place, and iterate over the copy, while inserting into the original. Print “Counting to 100 printing only by 5” for i in range(0, 200) if i % 5 == 0: print i else: continue if i == 100: break else: pass

Here we are counting up to 200 one at a time. If our count is not a multiple of 5, we continue. If we reach 100, we break out of the loop. If our count is not 100, we do nothing. Again we have a use of the pass keyword. It was not strictly required here since an else statement was not needed, but it is used to demonstrate that pass can be used anywhere, not just in an empty function declaration.

4.4.6 Importing Packages

The only remaining bits about Python which are necessary to know at this stage concern the packages. Packages are collections of functionality that can be brought into the application via an import statement (very much like a C++ library). All of the functionality exposed to Python in our case is contained in a package called the GI_AISDK package. It is imported using the following line at the top of the file. from GI_AISDK import *

This line imports everything from the GI_AISDK package. It is possible to replace * with specific things you wish to import, but for the purposes of this demo, you will want to import everything.

4.4.7 Embedding Python

Python provides a C API for embedding the Python interpreter into your own applications. This is not as difficult as it sounds, and much of the work of abstracting it into a system you can use has already been done. In the State Machine Demo, Python has been encapsulated such that you can define a C++ class, expose it to Python, derive a Python type from it, create an instance of the Python derived type from within the C++ application, and then call its methods polymorphically from C++. This was no mean feat incidentally; it took many long nights and a good deal of assistance from the developers at Boost.Python to get the system working. But at the end of the day, we basically have a seamless C++ to Python integration ready for use in your own applications.

165

Boost.Python: Embedding Python using Templates

Boost.Python provides a software layer which allows us to do two primary things. 1. Expose our C++ classes to Python so that Python classes can be derived from them, as well as call methods on C++ types that have been exposed. 2. Write and use Python-like language in C++ to do work; including instantiating Python types from C++, as well as extracting values from those types. While you will likely have to do little to expose your own classes if you are just using the framework provided, it is highly recommend that you visit the Boost.Python website (http://www.boost.org/libs/python/doc/index.html) and read over the tutorials and documentation. In the next section we will take a look at how some of the most common things can be accomplished using Boost.Python.

Making a Module

#include <boost/python.hpp> #include <boost/python/class.hpp> #include <boost/python/module.hpp> #include <boost/python/def.hpp> BOOST_PYTHON_MODULE(GI_AISDK) { // class and function exposures go here }

A module is a set of one or more classes or functions. It is essentially our core Python compilation unit (much like the combination of an .h file and a .cpp file in C++). As seen above, making a module is very straightforward. First, the Boost.Python library headers must be included. Second, the BOOST_PYTHON_MODULE will build a new module of the name provided. Notice that there are no quotes around the name; this must be a fully qualified valid variable name.

Exposing a Function

Exposing a function from C++ to Python using Boost.Python is fairly easy if the function you are exposing is not terribly complex. int someFunction(int);

Assume we have some function as described above.

166

def(“some_function”, someFunction);

All we need do is add the line above in our Python module’s scope. This exposes the C++ function someFunction as the Python function some_function. Although this may seem quite simple, there will be complications once you start using references and pointers. Boost.Python has concepts for internal references, and custodians and wards for ref counting, and copying const references, and a multitude of actions which occur once you start passing around addresses to memory. Please take a look at the Boost.Python documentation for the full scope of how to perform complex function exposure, since they have covered these features in great depth.

Exposing a Class

Exposing simple C++ classes is also very straightforward until you start working with complex virtual functions and their ilk. We will get you started with some simple classes, but again, it is highly advised that you seek out the Boost.Python documentation and read through it to get a better understanding of the more complex cases. class HelloWorld { public: HelloWorld(std::string personToGreet); void setPersonToGreet(std::string person) { mPersonToGreet = person; } void greetPerson(void) { cout << “Hello World and “ << mPersonToGreet << endl; } protected: std::string mPersonToGreet; };

Let us assume we have the simple class above. class_<HelloWorld>(“HelloWorld”, init(std::string)) .def(“set_person_to_greet”, &HelloWorld::setPersonToGreet) .def(“greet_person”, &HelloWorld::greetPerson) ;

We could expose this class as shown. The class_<> template exposes the type given as the template parameter. Its constructor parameters are the name of the class exposed to Python, as well as the type of initialization function which is required to initialize the object properly. We then list each method in the class we wish to expose, and give the def() method the name we would like to expose the method as. The def() method returns a reference to the class_<> object again so we can chain the def calls. This becomes complex once virtual methods and so on are added, so please refer to the Boost.Python documentation.

167

4.5 Our Scripting Engine

cActioncTransition

cScriptedTransition cScriptedAction

cPythonScriptedTransition cPythonScriptedActionWrapcPythonScriptedTransitionWrap cPythonScriptedAction

cScriptEngine

cPythonScriptEngine

Figure 4.11

As shown in Figure 4.11, our scripting engine implements specialized Python scripted types of the cAction and cTransition classes. The “Wrap” suffixed classes are classes that are necessary for Boost.Python to properly handle virtual methods on the C++ side. For more information on why that is necessary, please refer to the Boost.Python documentation. The scripting engine also has a cScriptEngine interface which is implemented for Python by the cPythonScriptEngine. Let us delve into the implementations of these classes in more detail. First, we will discuss how we actually exposed the classes, and then we will talk about the classes themselves. BOOST_PYTHON_MODULE(GI_AISDK) { // Expose the State class to Python class_<cState, cStatePtr >("State", init<float, std::string>()) .add_property("value", &cState::Value, &cState::SetValue) .add_property("initial_value", &cState::InitialValue, &cState::SetInitialValue) ; class_<cAction>("Action", init<cStatePtr >()); // Expose a base class new scripted actions should derive from class_<cScriptedAction, cPythonScriptedActionWrap, boost::noncopyable >("PythonScriptedAction", init<cStatePtr >()) .def("state", &cAction::State, return_value_policy<reference_existing_object>()) ; class_<cTransition>("Transition", init<cStatePtr, cStatePtr >()); // Expose a base class new scripted transitions should derive from class_<cScriptedTransition, cPythonScriptedTransitionWrap,

168

boost::noncopyable >("PythonScriptedTransition", init<cStatePtr, cStatePtr >()) .def("source", &cTransition::Source, return_value_policy<reference_existing_object>()) .def("target", &cTransition::Target, return_value_policy<reference_existing_object>()) ; }

Here we have the actual exposure of the C++ types to Python for our module using Boost.Python. BOOST_PYTHON_MODULE(GI_AISDK)

First we declare a new module called the GI_AISDK. // Expose the State class to Python class_<cState, cStatePtr >("State", init<float, std::string>())

Next we expose the cState class, and inform the class that it will be storing the cState objects internally as smart pointers. See the Boost.Python documentation for more details. We specify we want the class exposed as type “State” to Python, and that it has an initialization requiring a float and an STL string. .add_property("value", &cState::Value, &cState::SetValue)

We then add a property for the Value of the state to the exposure, and expose it as “value” on the Python side. We supply the accessors to Python for getting and setting this property. This allows us to use the value member as an actual data member rather than having to use accessors on the Python side. .add_property("initial_value", &cState::InitialValue, &cState::SetInitialValue)

We also add a property for the Initial Value of the state, and expose it as “initial_value” to Python using the same mechanism. class_<cAction>("Action", init<cStatePtr >());

Next we expose the cAction class as “Action”, and specify that it has an initialization that requires a cState smart pointer. // Expose a base class new scripted actions should derive from class_<cScriptedAction, cPythonScriptedActionWrap, boost::noncopyable >("PythonScriptedAction", init<cStatePtr >())

Now we expose the cScriptedAction class, but specify that it will contain the cPythonScriptedActionWrap class to allow for the virtual functions. See the Boost.Python documentation for why this step is required. We also specify this that class is not able to be copied, since it is a pure virtual base class. We expose this type as “PythonScriptedAction” to Python, and specify that it requires a cState smart pointer for its initialization method.

169

.def("state", &cAction::State, return_value_policy<reference_existing_object>())

We then provide an accessor to the contained cState object exposed as “state” to Python. We inform the system to use the existing object reference rather than any other tricky business. Again, see the Boost.Python documentation for further details on why we do this. class_<cTransition>("Transition", init<cStatePtr, cStatePtr >());

Next, we expose the cTransition class. We expose it as “Transition” to Python, and inform Python that the init method will require two cState smart pointers. // Expose a base class new scripted transitions should derive from class_<cScriptedTransition, cPythonScriptedTransitionWrap, boost::noncopyable >("PythonScriptedTransition", init<cStatePtr, cStatePtr >())

Lastly, we expose cScriptedTransition, and specify it will store a cPythonScriptedTransitionWrap class to allow for the virtual methods. This class is non-copyable since it is a pure virtual class. We expose it as “PythonScriptedTransition” to Python, and specify that it requires two cState smart pointers for its initialization. .def("source", &cTransition::Source, return_value_policy<reference_existing_object>())

We expose the source cState object as “source” to Python using the accessor, and indicate that it is to return the existing object. .def("target", &cTransition::Target, return_value_policy<reference_existing_object>())

Finally, we provide an accessor to the target state, exposed as “target” to Python. Again, we reference the existing object. It is highly recommended that you look through the Boost.Python documentation to get a firm grip on what we just reviewed and to help fill in any missing pieces that you are not yet comfortable with. Otherwise, if you try to expose your own classes, you run the risk of becoming lost and confused, not to mention frustrated.

The Script Engine Class

class cScriptEngine { public: cScriptEngine(void) {} virtual ~cScriptEngine(void) {}

170

virtual bool Initialize(void) = 0; virtual bool Finalize(void) = 0; virtual bool CompileScript(const string &scriptName, const string &script) = 0; };

The cScriptEngine class provides the interface to the scripting engine. It properly initializes the script engine, as well as properly shuts it down, providing a time for final cleanup. It also provides a means by which to parse and compile a script for use in the runtime. cScriptEngine(void) {} virtual ~cScriptEngine(void) {}

The constructor provides default initialization for the script engine, while the destructor is virtual to allow for correct polymorphic destruction of derived script engine types. virtual bool Initialize(void) = 0;

The Initialize method provides a means for derived script engines to properly initialize their subsystems. virtual bool Finalize(void) = 0;

The Finalize method provides a means for derived script engines to properly shut down their subsystems and free their resources. virtual bool CompileScript(const string &scriptName, const string &script) = 0;

The CompileScript method provides a means to parse and compile the script on the derived script engine, and make it usable for runtime processing. Returning true means successful compilation occurred while false indicates a script error of some sort. class cPythonScriptEngine : public cScriptEngine { public: virtual bool Initialize(void); virtual bool Finalize(void); virtual bool CompileScript(const string &scriptName, const string &script); object GetObject(const string &typeName); string GetErrString(void); static cPythonScriptEngine &Instance(void); protected: cPythonScriptEngine(void); virtual ~cPythonScriptEngine(void);

171

object GetLineNumber(object traceBack); handle<> mMainModule; handle<> mMainNamespace; handle<> mBuiltinsNamespace; };

The cPythonScriptEngine implements the cScriptEngine interface, and encapsulates the acts of starting up and shutting down the scripting engine. It also provides access for obtaining error messages from runtime exceptions as well as compiles the script for use at runtime. Lastly, it provides a means by which to obtain an object of a specified type from the Python namespace. Let us look at this class in more depth. cPythonScriptEngine(void); virtual ~cPythonScriptEngine(void);

The constructor and destructors do nothing more than the base class, but they are protected, since the cPythonScriptEngine is a singleton instance class. bool cPythonScriptEngine::Initialize(void) { // add custom module PyImport_AppendInittab("GI_AISDK", initGI_AISDK); // initialize the python interpretor Py_Initialize(); // obtain the __main__ namespace mMainModule = handle<>(borrowed( PyImport_AddModule("__main__") )); mMainNamespace = handle<>(borrowed(PyModule_GetDict(mMainModule.get()) )); // evaluate the python builtins builtins mBuiltinsNamespace = handle<>(borrowed( PyEval_GetBuiltins() )); return true; }

The Initialize method initializes the Python scripting engine, and properly sets up the built-ins and interpreter. Let us take a closer look. // add custom module PyImport_AppendInittab("GI_AISDK", initGI_AISDK);

This adds our custom module to the initialization phase of the Python engine. This is needed in order for us to use this module at runtime. // initialize the python interpretor Py_Initialize();

The Py_Initialize method does the primary initialization of Python’s main dictionary and modules.

172

// obtain the __main__ namespace mMainModule = handle<>(borrowed( PyImport_AddModule("__main__") ));

Here we obtain a handle to the main module. This is equivalent to the main() entry point in a C or C++ program. mMainNamespace = handle<>(borrowed(PyModule_GetDict(mMainModule.get()) ));

Here we obtain the namespace for this module. All of our newly created objects will be created in this namespace. // evaluate the python builtins builtins mBuiltinsNamespace = handle<>(borrowed( PyEval_GetBuiltins() ));

Here we evaluate the built-ins namespace; this allows us to use built-in methods and types. return true;

We return success. The Boost.Python handle<> containers throw exceptions if they get NULL pointers, which happens if initialization fails. bool cPythonScriptEngine::Finalize(void) { // reset our handles mMainModule.reset(); mMainNamespace.reset(); mBuiltinsNamespace.reset(); // shutdown the python interpretor Py_Finalize(); return(true); }

The Finalize method shuts down the Python interpreter, which frees up all its resources. Let us look at this more closely. // reset our handles mMainModule.reset();

The handle<> objects from Boost.Python are smart pointers, so resetting them relinquishes their reference. This will release the memory if this is the last object holding a reference. Thus, we free our main module. mMainNamespace.reset();

We also free our main namespace. mBuiltinsNamespace.reset();

173

Lastly, we free our built-ins. Py_Finalize();

After freeing the namespaces, we call Py_Finalize, which shuts down the interpreter, and does a final garbage collection. return(true);

We return success. bool cPythonScriptEngine::CompileScript(const string &scriptName, const string &script) { // Parse the script node *pythonNode = PyParser_SimpleParseString(script.c_str(), Py_file_input); // Dump any errors that may have occurred if (PyErr_Occurred()) return(false); // if the script parsed properly, compile the code // and inject it into the dictionary PyCodeObject* codeObject = PyNode_CompileFlags(pythonNode, const_cast<char *>(scriptName.c_str()), NULL); // free our parsed script node PyNode_Free(pythonNode); if (!codeObject) { if (PyErr_Occurred()) return(false); } // evaluate the code so the new class can find its way into the dictionary PyObject *evaluateResult = PyEval_EvalCode(codeObject, mMainNamespace.get(), mMainNamespace.get()); Py_DECREF(codeObject); if (!evaluateResult) { if (PyErr_Occurred()) return(false); } Py_DECREF(evaluateResult); return(true); }

The CompileScript method parses and compiles a Python script, and inserts the object code into the main dictionary for runtime use. Let us take a look at what it is doing.

174

node *pythonNode = PyParser_SimpleParseString(script.c_str(), Py_file_input);

First, we have Python parse the script string using PyParser_SimpleParseString(). This will return a node pointer, which we will use to compile the script. if (PyErr_Occurred()) return(false);

If any errors occurred, the script contains bad syntax and we return an error to that effect. PyCodeObject* codeObject = PyNode_CompileFlags(pythonNode, const_cast<char *>(scriptName.c_str()), NULL);

Next, we compile the script into object code using PyNode_CompileFlags. This will return a PyCodeObject pointer if successful, which can be evaluated to insert it into the namespace. PyNode_Free(pythonNode);

Now that we have a PyCodeObject pointer to our compiled script, we do not need our parse node anymore, so we free it. if (!codeObject) { if (PyErr_Occurred()) return(false); }

If we do not have a code object, we check if an error occurred. If so, we return failure. Something was wrong with the code. PyObject *evaluateResult = PyEval_EvalCode(codeObject, mMainNamespace.get(), mMainNamespace.get());

If we have a valid PyCodeObject pointer, we evaluate it, which inserts it into the namespace. It is effectively executing the code, which in Python will insert object declarations into the namespace if they exist. This returns a success result. Py_DECREF(codeObject);

Now that we are done with our PyCodeObject pointer, we remove our reference to free it. if (!evaluateResult) { if (PyErr_Occurred()) return(false); }

175

If our evaluation result is NULL, something went wrong. We check for an error code and return if an error occurred. Py_DECREF(evaluateResult);

Now that we are done with our evaluation result, we remove our reference, and it should free itself. return(true);

Finally we return success. object cPythonScriptEngine::GetObject(const string &typeName) { dict main_namespace(mMainNamespace); return main_namespace[typeName]; }

GetObject is a convenience method for obtaining an object from the Python namespace. Let us take a look at it. dict main_namespace(mMainNamespace);

First we obtain the main namespace as a dictionary object. This lets us look up objects like a map. return main_namespace[typeName];

We then lookup our object by name in the dictionary, and return it. string GetErrString(void); object GetLineNumber(object traceBack);

The GetErrString and GetLineNumber methods look at the exception stack, and parse out the required information to give a useful message about the runtime exception that just occurred. This is about 100 lines of very unattractive Python innards code which is not necessary to know but, if you do feel so inclined to delve into the exception handling, feel free to study the source code. static cPythonScriptEngine &Instance(void);

The last method of note is the Instance method, which simply returns a static instance of the class.

176

The Scripted Action Class

class cPythonScriptedActionWrap : public cScriptedAction { public: cPythonScriptedActionWrap(PyObject *self, cStatePtr state) : cScriptedAction(state), mSelf(self) {} virtual ~cPythonScriptedActionWrap(void) { mSelf = NULL; } virtual string Label(void) { return("Python Scripted Action"); }; void Execute(void); protected: PyObject *mSelf; };

The PythonScriptedActionWrap class is a wrapper class needed for Boost.Python to allow polymorphic calling of Python derived virtual functions. It is a bit tricky, so look at the Boost.Python documentation for more details on why this is necessary. Let us take a look at it. cPythonScriptedActionWrap(PyObject *self, cStatePtr state) : cScriptedAction(state), mSelf(self) {}

The constructor takes a PyObject pointer which Boost.Python will provide when building the Python object, as well as the standard cState smart pointer that the base class needs. virtual ~cPythonScriptedActionWrap(void) { mSelf = NULL; }

The destructor simply NULLs its self pointer. void cPythonScriptedActionWrap::Execute(void) { call_method<void>(mSelf, "execute"); }

The Execute method uses the Boost.Python call_method<> template function, which performs all of the magic of calling a specific method on a Python object. Note that it is calling the “execute” method, which should define the scripted behavior for the derived type’s execution. class cPythonScriptedAction : public cScriptedAction { public: cPythonScriptedAction ( cStatePtr state, const string &scriptName, const string &script ); virtual ~cPythonScriptedAction(void) { }

177

virtual string Label(void) { return("Python Scripted Action"); }; void Execute(void); const string &ScriptName(void) const { return(mScriptName) ; } const string &Script(void) const { return(mScript) ; } protected: object mPythonInstance; string mScriptName; string mScript; };

The cPythonScriptedAction is our wrapper for Boost.Python’s wrapper. This object builds a new derived Python type given by the script name, and passes along the function calls as required. Let us take a closer look. cPythonScriptedAction::cPythonScriptedAction ( cStatePtr state, const string &scriptName, const string &script ) : cScriptedAction(state), mScriptName(scriptName), mScript(script) { object scriptType = cPythonScriptEngine::Instance().GetObject(scriptName); mPythonInstance = scriptType(state); }

The constructor takes the state pointer required by the base class, as well as the name of the script, and the script source itself. It then builds the Python object and stores off its instance. object scriptType = cPythonScriptEngine::Instance().GetObject(scriptName);

Here, we use our GetObject method to obtain the object from the namespace. mPythonInstance = scriptType(state);

We then construct a new object of that type, passing in the state parameter it needs to build itself. We store off this new object as our instance so we can make calls to it later. void cPythonScriptedAction::Execute(void) { call_method<void>(mPythonInstance.ptr(), "execute"); }

The Execute method simply uses the call_method<> template function, exactly as the Wrapper object did, only on the python instance object.

178

The Scripted Transition Class

class cPythonScriptedTransitionWrap : public cScriptedTransition { public: cPythonScriptedTransitionWrap ( PyObject *self, cStatePtr source, cStatePtr target ) : cScriptedTransition(source, target), mSelf(self) {} virtual ~cPythonScriptedTransitionWrap(void) { mSelf = NULL; } virtual string Label(void) { return("Python Scripted Transition"); }; bool ShouldTransition(void); protected: PyObject *mSelf; };

Just as with the derived scripted action class, the derived scripted transition class requires a wrapper class to provide for polymorphic calling of the derived types in Python. The cPythonScriptedTransitionWrap does exactly that. cPythonScriptedTransitionWrap ( PyObject *self, cStatePtr source, cStatePtr target ) : cScriptedTransition(source, target), mSelf(self) {}

The class requires a PyObject, which is provided by Boost.Python when the object is constructed, as well as two cState smart pointers required by the base class. virtual ~cPythonScriptedTransitionWrap(void) { mSelf = NULL; }

The destructor simply NULLs the mSelf PyObject pointer. bool cPythonScriptedTransitionWrap::ShouldTransition(void) { return(call_method<bool>(mSelf, "should_transition")); }

Like the cPythonScriptedActionWrap class, this class uses the call_method<> template function to call the method on the Python object and automatically extracts the return value. This value is then returned to the caller.

179

class cPythonScriptedTransition : public cScriptedTransition { public: cPythonScriptedTransition ( cStatePtr source, cStatePtr target, const string &scriptName, const string &script ); virtual ~cPythonScriptedTransition(void) { } virtual string Label(void) { return("Python Scripted Transition"); }; bool ShouldTransition(void); const string &ScriptName(void) const { return(mScriptName) ; } const string &Script(void) const { return(mScript) ; } protected: object mPythonInstance; string mScriptName; string mScript; };

Like cPythonScriptedAction, cPythonScriptedTransition provides a wrapper for the Wrap class. It allows us to store the script name and script itself for ease of editing, and handles passing through the function calls. cPythonScriptedTransition::cPythonScriptedTransition ( cStatePtr source, cStatePtr target, const string &scriptName, const string &script ) : cScriptedTransition(source, target), mScriptName(scriptName), mScript(script) { object scriptType = cPythonScriptEngine::Instance().GetObject(scriptName); mPythonInstance = scriptType(source, target); }

The constructor takes a cState smart pointer for the source and the target, and passes them to the base class. It also takes the name of the script, and the script string itself, and stores it off for later use. It then gets the object type from the namespace, and builds a new one. object scriptType = cPythonScriptEngine::Instance().GetObject(scriptName);

Here we get the object type from the main namespace using the script name. mPythonInstance = scriptType(source, target);

180

We then build a new object of the given type, passing in the required arguments, and store off the instance it returns. bool cPythonScriptedTransition::ShouldTransition(void) { return(call_method<bool>(mPythonInstance.ptr(), "should_transition")); }

Just like the wrapper class, we use the call_method<> template function to call our method on our derived Python class, extract the result, and return it to the caller.

Some Examples

Now that we have looked at the internals of Python integration and the classes we will use to accomplish this in our project, let us review some examples of Scripted Actions and Scripted Transitions to get a better feel for how everything fits together. from GI_AISDK import * class AddFibonacciAction(PythonScriptedAction): n1 = 1 n2 = 0 def execute(self): # add up last time, and time before self.state().value = self.n2 + self.n1 self.n2 = self.n1 self.n1 = self.state().value

Here we have a scripted action that increases the state’s value via the Fibonacci sequence (defined as 12 −− += nnn FFF ). Let us take a closer look at the script. from GI_AISDK import *

First, we import our module. class AddFibonacciAction(PythonScriptedAction):

We then define a new action class as AddFibonacciAction which is derived from PythonScriptedAction. n1 = 1 n2 = 0

We create two data members, n1 and n2, which will store local data for our calculations. def execute(self):

181

We define our execute method which will be called by the C++ code. # add up last time, and time before self.state().value = self.n2 + self.n1

First we get our state’s current value, and make it n2 + n1. self.n2 = self.n1

We then update n2 to be what n1 was. self.n1 = self.state().value

Finally we set n1 to be what the new state’s value is, so that it can be added in the next iteration. Now let us try a scripted transition. from GI_AISDK import * from random import * class RandomTransition(PythonScriptedTransition): def should_transition(self): if uniform(0.0, 10.0) > 5: return True return False

Here we have a transition that decides to transition if a random number generated is greater than 5. Let us take a closer look. from GI_AISDK import *

First we import our GI_AISDK module fully. from random import *

We then import the random module fully. class RandomTransition(PythonScriptedTransition):

Here we define a new transition class called RandomTransition which is derived from PythonScriptedTransition. def should_transition(self):

We define the should_transition method, which is called by our C++ class. if uniform(0.0, 10.0) > 5:

182

return True return False

If the random number generated by the uniform distribution random generator is greater than 5, we transition, otherwise, we do not. At this point you should have a pretty good high level idea of how Python integrates into an application. The demo for this chapter will hopefully make all of this much clearer to you when you see everything in one place. Once again, we strongly recommend that you spend some time browsing the Python and Boost.Python documentation, look at some tutorials on the web, and try to establish a comfort level with the concepts introduced here in this chapter. Scripting adds a lot of flexibility to your application and ultimately allows you to build some very sophisticated AI. Whether you choose to use Python or some other language, hopefully you now have a better insight into how these systems work and how you might use them to accomplish your AI objectives.

Conclusion

The most common types of decision making are:

Decision Trees – Decision trees are nested if-then-else statements, which are evaluated for every time step in order to reach a decision. They contain no state information, and rely upon external sources to make their decisions.

State Machines – State machines, or finite state machines, keep track of their current decision, and only evaluate the questions necessary to make a different decision.

Rule Base – Rule base decision systems evaluate a series of rule criteria and score them. The rule which is given the highest score determines the decision.

Squad Behaviors – Squad behaviors are basically a complex form of flocking mixed with decision making, where one entity tells the rest of the group what to do. The entities themselves can use state machines, or decision trees or rule base decision systems to determine what they want to do. The squad behavior system essentially employs the concept of one entity telling fellow entities how to behave.

In this chapter, we discussed state machines and their power to handle decision making tasks under various conditions. We found that state machines can be used for such things as:

Animation Systems Game State Save File Systems Artificial Intelligence

We also discovered how such a system could be implemented, and examined a specific implementation of one of these systems in our chapter demonstration.

183

We also discussed the topic of Scripting Engines, and how they are useful in games. We discussed the types of scripting systems commonly used, and delved into embedding such a system in our demo. We learned about Python, and had a quick and dirty crash course in how to write it. We learned about Boost.Python, and how it can be used to embed Python in our games for easy extensibility. We talked about how we embedded Python in our chapter demo, and the implementations we used for that system. Lastly, we went over a couple of examples of a scripted action and a scripted transition, so we could get a better understanding for how we might want to extend our demo using scripting. In the next chapter, we are going to begin pulling together all of our AI systems into a single SDK that you can use in your games. As part of that discussion we will revisit a pathfinding concept introduced very briefly at the start of the course – waypoint networks. We chose waypoint networks as the navigation dataset in this case because they are very easy to work with in any 3D environment since they are not constrained by grids (making them very popular in game development shops). We will explore these networks in more detail. One of our main goals will be to integrate our waypoint data with the decision making concepts we learned in this chapter. That is, we will use our waypoints to store information that can trigger behaviors in our AI entities upon reaching the waypoint. A very handy GILES™ Waypoint Network Generator plug-in has been included with this course to facilitate all of this. So before moving on, please make sure that you understand the material and source code introduced in this chapter. We will be using it all again throughout the remainder of this course.

185

Chapter 5

Waypoint Networks

186

Overview

So far, we have talked about pathfinding algorithms, flocking algorithms, decision systems, and scripting. In this chapter we will start bringing all of these ideas together to demonstrate how you might actually use each of these systems in an integrated environment. We will start by talking about waypoint networks, a specific implementation of a pathfinding graph we briefly touched on earlier. We will talk about how we can attach data to these networks such that the decision making system, a state machine such as one we discussed previously, can make decisions based on the data in the network. We will then talk about squads and squad leaders, and how we can implement their state machines so they cooperate with one another. In this chapter we answer the following questions:

• What are waypoint networks? • How can we attach data to waypoints, so we can use it to make decisions?

• What are the common methods of implementing squad communication? • How can we bring all of this together?

187

5.1 Waypoint Networks

Figure 5.1

In Chapter Two we talked very briefly about pathfinding on non-gridded maps. We mentioned that one of the more popular methods of dealing with these sorts of worlds is called waypoint networks (or visibility points). Figure 5.1 should look familiar, since it is the same one we talked about previously when we introduced waypoint networks. The red polygons are obstacles, the small blue dots are waypoints, and the blue lines between them are the edges of the network. In this chapter, we are going to discuss this method of dealing with continuous worlds in depth. We will talk about the architecture of the network, methods to traverse the network, and some additional considerations that come up when using such a system for games.

5.1.1 Waypoints

Let us begin by talking about the waypoints themselves since they are obviously the fundamental element in a waypoint network. Ultimately, a waypoint network is just a collection of waypoints and the edges between them. If you recall some of the terminology we used during the early part of this course, a waypoint network is a graph, and the waypoints are just nodes in the graph. So what are some important characteristics of a waypoint? Well, first of all, it has to have a position in space. It also needs to have a collection of edges to other waypoints that can be reached from it. A waypoint probably should have some sort of identifier (an simple integer ID, a GUID, etc.), although it is not strictly necessary. It should also have a radius and we will talk about why this is important in just a bit. Other useful information would be an orientation. The orientation is handy if you want to hint to the entities traversing the network that something useful might be found in the direction of this

188

waypoint. For example, you might set up a waypoint that is at the top of a hill, with an excellent vantage over an enemy base. You could set the orientation of this waypoint to face towards the base, so the entities traversing the network could see that it is a good sniping position. The last thing you might want to attach to a waypoint is general blind data. General blind data is basically game related data that is attached to the waypoint, but the waypoint does not necessarily know, nor care about, what that data is. It is just holding onto it for the game entities to process when they come across it. Then, when the entity runs across the waypoint, it can peer into that blind data (with full knowledge of what is in there), and make some decisions based on it. In our chapter demo, we store a color as blind data. We use this color to set the color of the entity’s arrow when we render it. Some other examples of things you might want to use as blind data on waypoints are:

• Animation trigger data – tells the entity to play a specific animation upon reaching the waypoint • Wait Signal – tells the entity to pause briefly upon reaching the tagged waypoint • Look around signal – tells the entity to pause and look around for enemies • Cover – tells the entity that crouching here would provide them with cover from the direction

indicated by the orientation of the waypoint • Defend – tells the entity that this position is a good defensive position • Danger – tells the entity that this position is particularly dangerous • Posture – tells the entity that movement from this waypoint should be done using a given posture

(crouch, run, walk)

Discrete Simulations in Continuous Worlds

Let us now talk a little about why the radius data member of a waypoint can be helpful. The important thing to remember about continuous worlds in games is that the game is a discrete simulation, not a continuous one. You have a certain amount of time pass each frame, and you typically update the position of an entity by integrating the velocity of the entity using that time delta (using standard Euler or other numerical integration techniques).

Figure 5.2

Let us assume that your entity is moving at say, 10 meters/sec, but your game only updates every 33 milliseconds (30Hz, or 30 frames per second). That means the smallest distance your entity can travel in

189

a given frame is 3.3 meters (10 m/s * 0.033 s). So if your waypoint is a single point in space, the likelihood that your entity will land right on it is pretty slim. But if you give your waypoint a radius, and if your entity is “close enough” to the waypoint, then the entity has reached the goal. Take a look at Figure 5.2. If the entity started at the green circle, and he wanted to get to the red star, he could repeatedly jump over the star, every frame, and never reach it, resulting in the series of red circles. However, if the radius of the black circle around the star was used, the entity would have been found to have reached the waypoint after the first iteration.

The Waypoint Class

Now that we have a good understanding of the principle of the waypoint, let us take a look at the actual class we used to represent the waypoint in our demo. class cWaypoint { public: cWaypoint(const D3DXVECTOR3 &pos, const D3DXQUATERNION &orient,

float radius); virtual ~cWaypoint(); const tWaypointID &GetID() const { return mID; } D3DXVECTOR3 &GetPosition() { return mPosition; } const D3DXVECTOR3 &GetPosition() const { return mPosition; } D3DXQUATERNION &GetOrientation() { return mOrientation; } const D3DXQUATERNION &GetOrientation() const { return mOrientation; } float GetRadius() const { return mRadius; } void SetPosition(const D3DXVECTOR3 &position)

{ mPosition = position; } void SetOrientation(const D3DXQUATERNION &orientation)

{ mOrientation = orientation; } void SetRadius(float radius) { mRadius = radius; } void AllocBlindData(UINT size); void FreeBlindData(); template<class T> void GetBlindData(UINT offset, T &data) const { assert(offset <= (mBlindDataSize - sizeof(T))); T *dataPtr = (T*)((char*)mBlindData + offset); data = *dataPtr; } template<class T> void SetBlindData(UINT offset, T data) { assert(offset <= mBlindDataSize - sizeof(T)); T *dataPtr = (T*)((char*)mBlindData + offset); *dataPtr = data; }

190

bool AddEdge(const cNetworkEdge &edge); bool RemoveEdge(const cNetworkEdge &edge); void ClearEdges(); float GetCostForEdge(const cNetworkEdge &edge,

const cWaypointNetwork &network) const; tEdgeList &GetEdges() { return mOutgoingEdges; } const tEdgeList &GetEdges() const { return mOutgoingEdges; } int Serialize(ofstream &ar); int UnSerialize(ifstream &ar); protected: friend class cWaypointNetwork; cWaypoint(); private: tWaypointID mID; tEdgeList mOutgoingEdges; D3DXVECTOR3 mPosition; D3DXQUATERNION mOrientation; float mRadius; int mBlindDataSize; void *mBlindData; };

There is a lot to take in there, but most of it is accessors. First we will cover the class data members. tWaypointID mID;

Each waypoint has an ID. A tWaypointID class in our demo is actually a tGUID class, which is a wrapper class for the Microsoft Windows GUID structure. GUID stands for “Globally Unique Identifier”, and the GUID structure stores a 128-bit integer in a specific fashion to serve that purpose. It is used by Windows for COM object registration, among other things. GUIDs look like {D5FEE50A-625B-4b6b-B10B-FAD046F0A729}, and can be generated for us using the GuidGen tool provided with Microsoft Visual Studio, as well as the ::CoCreateGuid() method provided by the Windows API. We will talk more about what these IDs are used for when we discuss the waypoint network class. tEdgeList mOutgoingEdges;

A waypoint also has a list of edges. The tEdgeList type is really just a typedef for an STL vector of network edge classes, which we will discuss shortly. This list of edges is used during the traversal to see which waypoints can reach which other waypoints. D3DXVECTOR3 mPosition; D3DXQUATERNION mOrientation;

Waypoints also have a position in space, along with an orientation. We chose quaternions for the orientations in our demo. Quaternions are a useful way to represent rotation data because they have low memory footprint, offer smooth interpolation, and solve the problem of gimble lock. A discussion on

191

how quaternions work is beyond the scope of this course however, but if it interests you, further discussion on the topic can be found in the 3D Graphics Programming series and the Game Mathematics course here at Game Institute. float mRadius;

Waypoints also have the aforementioned radius. This value is used to determine if an entity is “close enough” to be considered as having arrived at the waypoint. int mBlindDataSize; void *mBlindData;

Lastly we have some data members for our blind data. In this implementation, blind data is stored as a block of bytes, which can be accessed using template functions provided. We will discuss those shortly. Now that we know what the basic data of the class is, let us discuss the implementations of the non-trivial methods, starting with the edge management methods. bool cWaypoint::AddEdge(const cNetworkEdge &edge) { // look for the edge, if we find it, don't add it again for (tEdgeList::iterator it = mOutgoingEdges.begin();

it != mOutgoingEdges.end(); ++it) { cNetworkEdge &e = *it; if (e == edge) return false; } mOutgoingEdges.push_back(edge); return false; }

The AddEdge method iterates through its list of edges, and if the edge to be added is not found, it is added to the list. bool cWaypoint::RemoveEdge(const cNetworkEdge &edge) { for (tEdgeList::iterator it = mOutgoingEdges.begin();

it != mOutgoingEdges.end(); ++it) { cNetworkEdge &e = *it; if (e == edge) { mOutgoingEdges.erase(it); return true; } } return false; }

192

The RemoveEdge method iterates through its list of edges, and if the edge is found, removes it from the list. float cWaypoint::GetCostForEdge(const cNetworkEdge &edge, const cWaypointNetwork &network) const { cWaypoint *dest = network.FindWaypoint(edge.GetDestination()); if (!dest) return 0.0f; D3DXVECTOR3 vec = GetPosition() - dest->GetPosition(); float distance = D3DXVec3Length(&vec); return distance * edge.GetCostModifier(); }

This method returns the cost for a given edge. While we have not yet discussed the implementation details of the edge class, this should still be fairly self-explanatory. First, the waypoint obtains the destination waypoint of the edge from the network. It then computes the distance from itself to the destination waypoint. This distance is the base cost of the edge. The edge also has a cost modifier associated with it, which is used to scale the base cost of the edge. template<class T> void GetBlindData(UINT offset, T &data) const { assert(offset <= (mBlindDataSize - sizeof(T))); T *dataPtr = (T*)((char*)mBlindData + offset); data = *dataPtr; }

This template method retrieves a value from the blind data block. It requires the caller know the byte offset into the data block so that you can locate your data, as well as the type of data you want to extract. It uses this information to get the data out of the block for you by adding the byte offset to the data block pointer and casting it to your data type for you. The usage pattern looks like this: COLORREF color; someWaypoint->GetBlindData(0, color);

Here, the template method deduced the type of data you wanted by the type of the reference you passed in. The byte offset in this case is 0. void SetBlindData(UINT offset, T data) { assert(offset <= mBlindDataSize - sizeof(T)); T *dataPtr = (T*)((char*)mBlindData + offset); *dataPtr = data; }

This template method works in a similar fashion to the GetBlindData method. Again, it requires the caller to know the byte offset into the data block where the desired data is to be stored, as well as the

193

type of data to store there. It then gets the pointer to the data block plus the offset, casts it to the type it needs, and sets the data for you. The usage pattern looks pretty much identical to the last one: COLORREF color; someWaypoint->SetBlindData(0, color);

Here the template method deduced the type of the data you wanted to set based on the parameter you passed in, and the offset again is 0. The remaining methods of the Waypoint class are either inline accessors for data, are too trivial for remark, or are outside the scope of this conversation (namely, serialization of the data).

5.1.2 Network Edges

Now that we have discussed the waypoints in a waypoint network, let us go over the details of the edges between the waypoints. Edges are unidirectional in our chapter demo implementation, though this need not be the case. To make a bidirectional edge in our implementation, we simply create two edges, one from the source to the destination, and one from the destination to the source. As such, edges in our implementation have only a destination waypoint, and not a source. In our implementation, a waypoint owns an edge, and it has a destination that it leads to. Edges have two other important characteristics: an open flag, and a cost modifier. The open flag determines if the edge can be traversed. This is useful in game situations that have certain environmental conditions, such as a drawbridge. If the bridge is up, the edge is closed, if the bridge is down, the edge is open, and can be traversed. The same logic would apply to doors or areas in the game world that might be temporarily off limits (perhaps a chemical weapon was used in the area). The cost modifier allows us to control how often an edge will be traversed. In our demo, the cost of an edge is the Euclidean distance from the owner waypoint to the destination waypoint, multiplied by the cost modifier. So if we want the edge to be traversed more often, we make the modifier less than one but greater than or equal to zero. To traverse less often, we can make the cost modifier greater than one. Before we look at the implementation of the network edge class in our demo, a final note on network edges is in order. If you were so inclined, you could have blind data on the edges, just like on the waypoints. You could use that information as you traversed the network to make additional decisions for your entity. We did not do it in our demo, but it is worth remembering. The nice thing about a system like this is that it is very flexible and you can really let your creativity carry you almost as far as you want to go.

The Network Edge Class

Now that we know what is involved with the network edges, let us look at the actual implementation of the network edge class in our demo.

194

// // cNetworkEdge - an edge from one waypoint to another // class cNetworkEdge { public: cNetworkEdge(const tWaypointID &destination, float cost = 1.0f,

bool open = true); virtual ~cNetworkEdge(); tWaypointID &GetDestination() { return mDestination; } const tWaypointID &GetDestination() const { return mDestination; } float GetCostModifier() const { return mCostModifier; } bool IsOpen() const { return mOpen; } void SetDestination(const tWaypointID &destination)

{ mDestination = destination; } void SetCostModifier(float costmod) { mCostModifier = costmod; } void SetIsOpen(bool open) { mOpen = open; } bool operator==(const cNetworkEdge &rhs); int Serialize(ofstream &ar); int UnSerialize(ifstream &ar); protected: friend class cWaypoint; cNetworkEdge() {} private: tWaypointID mDestination; float mCostModifier; bool mOpen; };

This is not quite as complicated as the waypoint class, and again, the bulk of the interface is accessors. Let us take a closer look, starting once again with the data members. tWaypointID mDestination;

The destination of the edge uses the waypoint ID system to identify the destination we can reach. This ID is used to look up the waypoint in the network. float mCostModifier;

The cost modifier is used to scale the base cost of the edge. As mentioned before, the base cost of the edge is the Euclidean distance from the owner waypoint to the destination waypoint. bool mOpen;

This flag determines if this edge is currently traversable. Again, this is useful for such things as drawbridges or other passages which can become temporarily impassible.

195

Believe it or not, that is basically all there is to it. The class methods are basically just accessor methods or serialization methods, so we will not need to spend any time on those. Thus, on to the network!

5.1.3 The Waypoint Network

We have now seen the waypoints and we have examined the edges, so it is time to take a look at the network itself. As mentioned earlier, a waypoint network is simply a collection of waypoints and edges (a graph). In our implementation, the waypoints own the edges, so the network really just turns out to be a collection of waypoints and the class acts primarily as a waypoint manager.

The Waypoint Network Class

The waypoint network class is very straightforward. While it does contain the actual method for traversing the network, it really is not very complicated. So let us just dive right in, and see what we are getting ourselves into. // // cWaypointNetwork - a collection of waypoints and their associated edges // class cWaypointNetwork { public: cWaypointNetwork(); virtual ~cWaypointNetwork(); bool AddWaypoint(cWaypoint &waypoint); bool RemoveWaypoint(const tWaypointID &waypointID); void ClearWaypoints(); cWaypoint *FindWaypoint(const tWaypointID &waypointID) const; bool FindPathFromWaypointToWaypoint(const tWaypointID &fromWaypoint,

const tWaypointID &toWaypoint, tPath &path);

bool FindPathFromPositionToPosition(const D3DXVECTOR3 &origin, const D3DXVECTOR3 &destination, const cWaypointVisibilityFunctor

&visibilityFunc, tPath &path);

void GetExtents(D3DXVECTOR3 &minimum, D3DXVECTOR3 &maximum) const; const tWaypointMap &GetWaypoints() const { return mWaypoints; } int Serialize(ofstream &ar); int UnSerialize(ifstream &ar); private: bool FindClosestValidWaypointToPosition(const D3DXVECTOR3 &origin,

const cWaypointVisibilityFunctor

196

&visibilityFunc, tWaypointID &result);

float GoalEstimate(const tWaypointID &fromWaypoint, const tWaypointID &toWaypoint);

tWaypointMap mWaypoints; };

There it is; the waypoint network. As you can see, it really is just a collection of waypoints. Let us take a look at the specifics. tWaypointMap mWaypoints;

The sole data member in the waypoint network is a map of waypoint IDs to waypoints. This lets us quickly find the waypoints we want using our IDs. bool cWaypointNetwork::AddWaypoint(cWaypoint &waypoint) { // don't add the waypoint if its GUID already exists tWaypointID id = waypoint.GetID(); if (!FindWaypoint(id)) { mWaypoints[id] = &waypoint; return true; } return false; }

The AddWaypoint method does a quick check to see if the waypoint is already in the map, and if it is not, adds it to the map. bool cWaypointNetwork::RemoveWaypoint(const tWaypointID &waypointID) { // if the waypoint doesn't exist don't remove it tWaypointMap::iterator it = mWaypoints.find(waypointID); if (it == mWaypoints.end()) return false; // otherwise free the waypoint, and remove it cWaypoint *wp = it->second; if (wp) delete wp; wp = NULL; mWaypoints.erase(it); return true; }

The RemoveWaypoint method is slightly more complex, but not by much. It looks up the waypoint in the map, and if successful, deletes the waypoint, freeing its memory. It also removes the waypoint from the map.

197

void cWaypointNetwork::ClearWaypoints() { // delete all waypoints, and clear the map for (tWaypointMap::iterator it = mWaypoints.begin();

it != mWaypoints.end(); ++it) { cWaypoint *wp = it->second; if (wp) delete wp; wp = NULL; } mWaypoints.clear(); }

The ClearWaypoints method simply iterates through all of the waypoints in the map, and deletes them and frees their memory. It then clears the map of its entries. cWaypoint *cWaypointNetwork::FindWaypoint(const tWaypointID &waypointID) const { // if we find the waypoint, return it tWaypointMap::const_iterator it = mWaypoints.find(waypointID); if (it == mWaypoints.end()) return NULL; return it->second; }

The FindWaypoint method looks the waypoint up in the map, and simply returns it if found. void cWaypointNetwork::GetExtents(D3DXVECTOR3 &minimum, D3DXVECTOR3 &maximum) const { minimum.x = minimum.y = minimum.z = FLT_MAX; maximum.x = maximum.y = maximum.z = -FLT_MAX; for (tWaypointMap::const_iterator it = mWaypoints.begin();

it != mWaypoints.end(); ++it) { cWaypoint *wp = it->second; D3DXVECTOR3 pos = wp->GetPosition() + D3DXVECTOR3(wp->GetRadius(),

wp->GetRadius(), wp->GetRadius());

if (pos.x < minimum.x) minimum.x = pos.x; if (pos.y < minimum.y) minimum.y = pos.y; if (pos.z < minimum.z) minimum.z = pos.y; if (pos.x > maximum.x) maximum.x = pos.x; if (pos.y > maximum.y) maximum.y = pos.y; if (pos.z > maximum.z) maximum.z = pos.z; } }

198

The GetExtents method iterates through all of the waypoints, and expands the minimum and maximum vectors to build a bounding box for the waypoint network. There is one last method to discuss in the waypoint network class before delving into the pathfinding algorithm employed in the demo. bool cWaypointNetwork::FindClosestValidWaypointToPosition (

const D3DXVECTOR3 &origin, const cWaypointVisibilityFunctor &visibilityFunc, tWaypointID &result

) { float closestDistanceSq = FLT_MAX; bool foundClosest = false; // iterate through all the waypoints for (tWaypointMap::iterator it = mWaypoints.begin();

it != mWaypoints.end(); ++it) { cWaypoint *wp = it->second; const D3DXVECTOR3 &pos = wp->GetPosition(); // if we have a valid waypoint, and we can see it from this position if (wp && visibilityFunc.IsVisible(origin, pos)) { // check our distance to the waypoint D3DXVECTOR3 vec = origin - pos; float distsq = D3DXVec3LengthSq(&vec); // if our distance to the waypoint is the closest we've found yet if (distsq < closestDistanceSq) { // keep track of it closestDistanceSq = distsq; result = wp->GetID(); foundClosest = true; } } } // return if we've found any close waypoints we can see return foundClosest; }

FindClosestValidWaypointToPosition is a useful method employed during the pathfinding traversal of the network. We will discuss those methods in detail shortly, but first let us work out what this method does.

199

First the parameters of the method:

const D3DXVECTOR3 &origin,

The method takes an origin point in space, which is the location from which we want to find the closest waypoint in the network.

const cWaypointVisibilityFunctor &visibilityFunc,

The method also takes a visibility functor. This class is used to interface to your game system to provide visibility information between waypoints. Normally this is hooked up directly to a physics simulation system, which does ray casting into the physics representation of the world to see if you can draw a line from one point to another without running into anything (often called a “line of sight” test). The result is then used to determine if you can see between the points. If you can draw the line you can see from one point to the other; if not, you cannot. Let us take a quick look at that class. // // cWaypointVisibilityFunctor - a base class for determining visibility // for network waypoints // class cWaypointVisibilityFunctor { public: cWaypointVisibilityFunctor() {} virtual ~cWaypointVisibilityFunctor() {} bool IsVisible(const D3DXVECTOR3 &origin,

const D3DXVECTOR3 &destination) const; };

This is the interface for the waypoint visibility functor. The idea is that you would derive your own type and overload the IsVisible() method. That method would then do the line of sight test from the origin point to the destination point and return success if the line could be drawn. Let us return now to the parameters of the FindClosestValidWaypointToPosition method…

tWaypointID &result

The last parameter of the method is an address to a waypoint ID. This result will be set to the closest waypoint to the origin point that can be seen from the origin point (assuming there is one). Now that we know a bit about the parameters, let us discuss the algorithm. float closestDistanceSq = FLT_MAX; bool foundClosest = false;

First we set the closest distance squared value to the maximum float value. Thus, any distance will be less than this distance. We also set a flag noting we have not found a closest node yet.

200

// iterate through all the waypoints for (tWaypointMap::iterator it = mWaypoints.begin();

it != mWaypoints.end(); ++it)

We then start iterating through all of the waypoints. To be fair, this is not the best design strategy. Ideally, we would have a BSP or oct-tree that would assist us in finding the waypoint closest to our point. Since it could do a binary search, it would turn this search from an O(n) search to an O(log2 n) search. The linear search is good enough for the demo however, since we do not have many waypoints to hunt through. 3D Graphics Programming Module II here at the Game Institute covers BSP trees, oct-trees, and a host of other hierarchical spatial data structures in great detail. Be certain to check out that course at some point in the not too distant future so that you can integrate a search tree into your application and realize the benefits. cWaypoint *wp = it->second; const D3DXVECTOR3 &pos = wp->GetPosition();

As we get each waypoint out of the map, we get its position. // if we have a valid waypoint, and we can see it from this position if (wp && visibilityFunc.IsVisible(origin, pos))

We then call our visibility functor to see if we can see the waypoint from the origin point. Something that the default implementation of the visibility functor does is just check from the origin to the center of the waypoint. Another possible solution would be to check the cone from the origin point to the destination waypoint, taking into account its radius. This is a more complex test however, so it was not used for the demo. // check our distance to the waypoint D3DXVECTOR3 vec = origin - pos; float distsq = D3DXVec3LengthSq(&vec);

If we can see the waypoint, we compute the distance squared to the waypoint from our origin position. We use the squared result since we are just comparing the results in terms of magnitude, and as such it saves us a square root operation. Again, we could take into account the waypoint’s radius if we wanted, but our demo does not require such an approach. // if our distance to the waypoint is the closest we've found yet if (distsq < closestDistanceSq)

We then test this computed distance to see if it is less than the closest waypoint distance we have found so far. // keep track of it closestDistanceSq = distsq; result = wp->GetID(); foundClosest = true;

201

If we find that the newly computed distance is less than the closest distance we have found so far, we mark it as the closest distance we have found. We also store off the ID of the waypoint, and set our flag to say that we have found a closest waypoint. // return if we've found any close waypoints we can see return foundClosest;

After we have iterated across all the waypoints, we return our flag. The caller will check this flag to see if the waypoint ID reference passed in will contain the closest waypoint or not.

5.2 Navigating the Waypoint Network

Now that we have a waypoint network, navigating it is simple. In the demo, we use the same A* algorithm as in our initial pathfinding demo, only it has been modified to traverse the waypoint network structure instead of a fixed grid. Believe it or not, this actually simplifies the code somewhat. Let us take a look at what needs to be done. There are three separate cases that can occur when computing a path for an entity:

a. The entity is traveling from a point on the network to a point on the network. b. The entity is traveling from a point off the network to a point on the network. c. The entity is traveling to or from a point on the network to or from a point off the network.

When we say “on the network” versus “off the network,” we mean that an entity that is “on the network” is actually within the bounds of a waypoint; whereas “off the network” means the entity is not within the bounds of an actual waypoint. If we are dealing with case (a), then the entity can use the FindPathFromWaypointToWaypoint method directly, using the waypoint it is starting at, and the waypoint it is going to as the parameters. If we are dealing with case (b), then the entity uses the FindPathFromPointToPoint method, which internally finds the closest waypoints to the start and end, and then uses FindPathFromWaypointToWaypoint to compute the path between those waypoints. If we are dealing with case (c), then the entity again uses the FindPathFromPointToPoint method passing the position of the waypoint the entity is within or the position of the waypoint that is the goal; whichever we have a waypoint for. It would be an optimization to provide a method that could fast path the finding of the known waypoints, but that was not done for this demo. Primarily in this demo, the FindPathFromPointToPoint was used. Let us take a look at the implementation of this method. bool cWaypointNetwork::FindPathFromPositionToPosition (

const D3DXVECTOR3 &origin,

202

const D3DXVECTOR3 &destination, const cWaypointVisibilityFunctor &visibilityFunc, tPath &path

) { tWaypointID closestToOrigin; tWaypointID closestToDestination; // find the waypoint closest to the starting position if (!FindClosestValidWaypointToPosition(origin, visibilityFunc,

closestToOrigin)) return false; // find the waypoint closest to the destination position if (!FindClosestValidWaypointToPosition(destination, visibilityFunc,

closestToDestination)) return false; // clear the path path.clear(); // if the starting waypoint is the same as the ending waypoint,

// we can just walk straight to the // destination point if (closestToOrigin == closestToDestination) return true; return FindPathFromWaypointToWaypoint(closestToOrigin, closestToDestination,

path); }

Here is the algorithm in its entirety. It is not overly complex, so let us just walk through it right here. The method takes a position for the origin, a position for the destination, a visibility functor, and a reference to a path. The tPath typedef is simply an STL list of waypoint ID objects. We first try to find the closest valid waypoint to the origin. If we do not find one, we fail to make a path. We then find the closest valid waypoint to the destination. Again, if we do not find one, we fail to make a path. Next we clear the path, and early abort in the event the closest waypoint to the origin is the same waypoint as the destination. In that case, we can just go directly from the origin to the destination without acquiring the network. If we have a closest waypoint to the origin, and a closest waypoint to the destination, and they are not the same, we call upon the FindPathFromWaypointToWaypoint method to compute a path for us. We will talk about that method in a moment. Before we get going on the FindPathFromWaypointToWaypoint, we should discuss a helper class, the cAStarWaypointNode class. // // cAStarWaypointNode - a special node for scoring the weights for determining // pathing through a waypoint network // using A* // class cAStarWaypointNode { public: cAStarWaypointNode(const tWaypointID &id);

203

cAStarWaypointNode(const tWaypointID &id, float f, float g, float h); float GetCost() const { return m_f; } void GetCosts(float &f, float &g, float &h) const

{ f = m_f; g = m_g; h = m_h; } void SetCost(float cost) { m_f = cost; } void SetCosts(float f, float g, float h) { m_f = f; m_g = g; m_h = h; } cAStarWaypointNode *GetParent() const { return mParent; } void SetParent(cAStarWaypointNode *parent) { mParent = parent; } bool GetVisited() const { return mVisited; } void SetVisited(bool visited) { mVisited = visited; } const tWaypointID &GetWaypoint() const { return mWaypoint; } bool operator<(const cAStarWaypointNode &rhs) const; private: tWaypointID mWaypoint; cAStarWaypointNode *mParent; bool mVisited; float m_f; float m_g; float m_h; };

This class is used for managing the traversal of the network. This is done so we do not have to keep track of heuristic estimate costs and visited status on the waypoints themselves. It is very similar to the A* node classes we studied earlier in the course, but it is worth mentioning. Just as before, the class maintains the f, g and h values for computing the actual cost of the node, and it keeps track of the parent node that got us here. Once we find the path, we walk from end to start via the parent node pointers. Enough of the easy stuff; onto the pathfinding! bool cWaypointNetwork::FindPathFromWaypointToWaypoint (

const tWaypointID &fromWaypoint, const tWaypointID &toWaypoint, tPath &path

) { // ye verily do traversal of network here tAStarWaypointNodePriorityQueue open; tAStarWaypointNodeList closed; cAStarWaypointNode *n; cWaypoint *nwp; tNodeMap nodeMap; // seed the search with the starting point float g = 0.0f; float h = GoalEstimate(fromWaypoint, toWaypoint); float f = g + h; cAStarWaypointNode *node = new cAStarWaypointNode(fromWaypoint, f, g, h); nodeMap[fromWaypoint] = node;

204

open.push_back(node); // now iterate the search while(!open.empty()) { n = open.front(); nwp = FindWaypoint(n->GetWaypoint()); open.pop_front(); if (n->GetWaypoint() == toWaypoint) { path.clear(); node = n; while (node) { path.push_front(node->GetWaypoint()); node = node->GetParent(); } return true; } for (tEdgeList::iterator it = nwp->GetEdges().begin();

it != nwp->GetEdges().end(); ++it) { cNetworkEdge &edge = *it; cWaypoint *dest = FindWaypoint(edge.GetDestination()); if (!dest || !edge.IsOpen()) continue; n->GetCosts(f, g, h); float newg = g + nwp->GetCostForEdge(edge, *this); // first check the node map, if we don't have an

// entry in the node map, we've never been here // before. tNodeMap::iterator nodeIt = nodeMap.find(edge.GetDestination()); bool wasInMap = true; if (nodeIt == nodeMap.end()) { // never been here wasInMap = false; // add it to the map node = new cAStarWaypointNode(edge.GetDestination()); nodeMap[edge.GetDestination()] = node; } else node = nodeIt->second; node->GetCosts(f, g, h); if

( wasInMap && (open.contains(node) || closed.contains(node)) && g <= newg

)

205

{ // do nothing... we are already in the queue

// and we have a cheaper way to get there... } else { node->SetParent(n); g = newg; h = GoalEstimate(node->GetWaypoint(), toWaypoint); f = g + h; node->SetCosts(f, g, h); if(closed.contains(node)) { // remove it closed.remove_item(node); } if(!open.contains(node)) { open.add_item(node); } else { // update this item's position in the queue

// as its cost has changed // and the queue needs to know about it open.sort(); } } } closed.add_item(n); } // unable to find a route return false; }

This is obviously the core of our pathfinding on the waypoint network. This algorithm should look familiar, as it is the A* algorithm we have already discussed in this course. Even so, there are some small changes. Most importantly, it does the traversal all in one fell swoop rather than one iteration at a time like our last demo did. Let us take a look at what is going on. bool cWaypointNetwork::FindPathFromWaypointToWaypoint (

const tWaypointID &fromWaypoint, const tWaypointID &toWaypoint, tPath &path

)

206

The method takes a waypoint ID of the waypoint to start at, a waypoint ID of the waypoint to end at, and a reference to a path. The method returns true if a path was found, and false it no path from the starting waypoint to the ending waypoint can be found. // ye verily do traversal of network here tAStarWaypointNodePriorityQueue open;

The algorithm begins with a priority queue of A* waypoint nodes which is the “open” queue of nodes to examine. tAStarWaypointNodeList closed;

Next we have a list of A* waypoint nodes which is the “closed” list of already examined waypoints. cAStarWaypointNode *n;

We have an A* waypoint node object which is the node we are currently traversing. cWaypoint *nwp;

We also have a waypoint pointer which is the waypoint associated with the A* waypoint node object we are currently traversing. tNodeMap nodeMap;

Last, we have a node map. This map associates waypoint IDs to A* waypoint nodes. This also helps to let us know if we have visited a node or not. // seed the search with the starting point float g = 0.0f; float h = GoalEstimate(fromWaypoint, toWaypoint); float f = g + h; cAStarWaypointNode *node = new cAStarWaypointNode(fromWaypoint, f, g, h); nodeMap[fromWaypoint] = node; open.push_back(node);

The first thing we do is seed the search with the starting point. We estimate the distance from the start to the goal using our heuristic, create an A* waypoint node for this waypoint, associate the waypoint ID to the A* node in our map, and push the A* node onto our open queue. This gets us ready for the main loop. // now iterate the search while(!open.empty())

So long as we have nodes to investigate in our open queue, we will perform the search. n = open.front();

207

nwp = FindWaypoint(n->GetWaypoint()); open.pop_front();

During every iteration, we grab the top node off the priority queue, find the waypoint for the ID the node is associated with, and pop the node off the queue. We use a special derived version of the STL list container for our priority queue. We will not go into detail on how that works, as this is basic algorithms theory. The implementation lives in WaypointNetwork.h if you feel inclined to peruse it. if (n->GetWaypoint() == toWaypoint) { path.clear(); node = n; while (node) { path.push_front(node->GetWaypoint()); node = node->GetParent(); } return true; }

Next we see if the current node we popped off the queue happens to be the goal node. If the current node is the goal node, we clear out our path, and walk the parent pointers of the A* nodes, pushing the nodes onto the path in reverse order. This way the path has the nodes in the order they should be visited, not the other way around. for (tEdgeList::iterator it = nwp->GetEdges().begin();

it != nwp->GetEdges().end(); ++it)

Assuming we have not reached the goal node, we iterate across all of the edges of the current node. cNetworkEdge &edge = *it; cWaypoint *dest = FindWaypoint(edge.GetDestination()); if (!dest || !edge.IsOpen()) continue;

For each edge, we find the destination waypoint in the network. If we cannot find the destination, or the edge is closed, we skip this edge. n->GetCosts(f, g, h); float newg = g + nwp->GetCostForEdge(edge, *this);

Assuming we have a traversable edge with a valid destination waypoint, we get the costs for the current A* node, and compute a new g using the current g, and the cost for this edge. // first check the node map, if we don't have an

// entry in the node map, we've never been here // before. tNodeMap::iterator nodeIt = nodeMap.find(edge.GetDestination()); bool wasInMap = true; if (nodeIt == nodeMap.end())

208

{ // never been here wasInMap = false; // add it to the map node = new cAStarWaypointNode(edge.GetDestination()); nodeMap[edge.GetDestination()] = node; } else node = nodeIt->second;

Next we check to see if the destination waypoint has a representing A* node in the map yet. If it does not, we create a new A* node, and associate it with the destination waypoint ID in the map. If it does exist, we simply get the A* node from the map for destination waypoint ID. node->GetCosts(f, g, h);

We then get the current costs for the destination waypoint’s A* node. if

( wasInMap && (open.contains(node) || closed.contains(node)) && g <= newg

) { // do nothing... we are already in the queue

// and we have a cheaper way to get there... }

Here we check to see if we should update the destination’s A* node. If the node was previously in the map, and either the open or closed lists contain the node, and the existing g cost is less or equal the new g cost computed, we do nothing. node->SetParent(n); g = newg; h = GoalEstimate(node->GetWaypoint(), toWaypoint); f = g + h; node->SetCosts(f, g, h);

If we have determined that we need to update the destination A* node, we set its parent to the current node. We also set its g to the new g computed, compute the heuristic estimate for this node, and set the costs. if(closed.contains(node)) { // remove it closed.remove_item(node); }

209

Then, if we find the node in the closed list, we remove it from the closed list, as it needs re-examination. if(!open.contains(node)) { open.add_item(node); } else { // update this item's position in the queue

// as its cost has changed // and the queue needs to know about it open.sort(); }

If the open queue does not contain the node, we add it to the open queue. This action automatically keeps the queue sorted. If the open queue does contain the node however, we simply resort the open queue. This will ensure the node is properly placed in the queue based on its priority. closed.add_item(n);

After each of the edges for the current node have been visited, the current node is added to the closed list. // unable to find a route return false;

Lastly, if after exhausting the open queue, we do not find a path and return above, we return false, reporting failure to build the path. That is all there is to finding a path using a waypoint network. Given your experience with A* earlier in the course, this should be second nature to you. From a theoretical perspective, you now have all of the information you need to take your waypoint networks and use them to find paths from place to place in the game world. In the next few sections we are going to talk about how we decided to use this new system in our chapter demo. Our discussions will take us back to other topics covered in Chapters 3 and 4 so that we can see how these pieces can fit together to produce interesting results. This should provide you with a foundation to work from so that you can begin integrating your own ideas that suit your particular game.

210

5.3 Flocking and Waypoint Networks

In Chapter 3 when we were talking about flocking, we discussed the concept of behavior based movement. The idea is that you have a set of separate behaviors and that each could contribute to the final desired movement of the entity. Following up on that concept, our demo makes use of the behavioral movement system developed in the flocking demo, and creates a movement behavior that follows a waypoint network path.

5.3.1 The Pathfind Behavior

class cPathfindBehavior : public cBehavior { public: cPathfindBehavior(float turnRate, float goalRadius,

float avoidDist, float maxTimeBeforeAgitation, const D3DXVECTOR3 &upVector, cWaypointNetwork &waypointNetwork);

virtual ~cPathfindBehavior(void); virtual void Iterate(float timeDelta, cEntity &entity); void ApplyAvoidance(cEntity &entity); virtual string Name(void) { return("Pathfind Behavior"); } protected: float mTurnRate; float mGoalRadius; float mAvoidDist; float mMaxTimeBeforeAgitation; D3DXVECTOR3 mUpVector; cWaypointNetwork &mWaypointNetwork; };

Here we see our pathfind behavior. It has a decent number of data members, so we will examine those first. float mTurnRate;

Just as with many of our movement behaviors from the flocking demo, this behavior has a turning rate parameter which limits how quickly the entity can make turns. float mGoalRadius;

The goal radius parameter is used to determine if the entity has satisfactorily reached its goal. Bear in mind the goal is different than the current waypoint. The goal is the final destination, while the current waypoint is the next one.

211

float mAvoidDist;

The avoid distance is the distance at which the entities strive to remain apart from one another. More on this value later. float mMaxTimeBeforeAgitation;

The max time before agitation is another data member we will discuss a bit later. Basically it has to do with keeping the entity from getting stuck trying to reach the next waypoint. D3DXVECTOR3 mUpVector;

This is the vector which is “up” in the game world. We will discuss it in more detail later, but for now just know that it is used in conjunction with the max time before agitation variable. cWaypointNetwork &mWaypointNetwork;

This is a reference to the network we will be navigating. We need this to look up waypoints from IDs. Now that we have a good idea what kinds of data this behavior needs, let us look at how it does its job. void cPathfindBehavior::Iterate(float timeDelta, cEntity &entity) { // pathfinding only works on squad mate type entities! cSquadEntity &squadmate = dynamic_cast<cSquadEntity&>(entity); tPath &path = squadmate.GetPath(); tWaypointID wpID = squadmate.GetNextWaypoint(); D3DXVECTOR3 entityPos = entity.Position(); D3DXVECTOR3 desiredMoveAdj(0.0f, 0.0f, 0.0f); if (wpID == GUID_NULL && path.size() > 0) { // set the next waypoint! wpID = path.front(); path.pop_front(); squadmate.SetNextWaypoint(wpID); squadmate.ResetTimeSinceWaypointReached(); } else if (wpID == GUID_NULL || path.empty()) { desiredMoveAdj = squadmate.GetGoal() - entityPos; if (D3DXVec3Length(&desiredMoveAdj) < mGoalRadius) { // we made it... stand around entity.SetDesiredMove(-entity.Velocity()); ApplyAvoidance(entity); squadmate.ResetTimeSinceWaypointReached(); return; } }

212

cWaypoint *wp = mWaypointNetwork.FindWaypoint(wpID); if (wp) { D3DXVECTOR3 wppos = wp->GetPosition(); desiredMoveAdj = wppos - entityPos; if (D3DXVec3Length(&desiredMoveAdj) < wp->GetRadius()) { // close enough! next waypoint! if (path.size() > 0) { // set the next waypoint! wpID = path.front(); path.pop_front(); squadmate.SetNextWaypoint(wpID); wp = mWaypointNetwork.FindWaypoint(wpID); ASSERT(wp != NULL); wppos = wp->GetPosition(); desiredMoveAdj = wppos - entityPos; squadmate.ResetTimeSinceWaypointReached(); } else { // uh, no more waypoints,

// start walking toward our target position desiredMoveAdj = squadmate.GetGoal() - entityPos; squadmate.SetNextWaypoint(GUID_NULL); if (D3DXVec3Length(&desiredMoveAdj) < mGoalRadius) { // we made it... stand around entity.SetDesiredMove(-entity.Velocity()); ApplyAvoidance(entity); squadmate.ResetTimeSinceWaypointReached(); return; } } } } // move in the direction of your next pathnode or your goal position squadmate.IncrementTimeSinceWaypointReached(timeDelta); D3DXVECTOR3 currentDesiredMove = entity.DesiredMove(); D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= mTurnRate; currentDesiredMove += desiredMoveAdj * Gain(); if (squadmate.GetTimeSinceWaypointReached() > mMaxTimeBeforeAgitation) { // agitate the movement to get the guy moving properly D3DXVECTOR3 agitationVector; // nudge the desired move vector by using a vector perpendicular to the // current desired move's direction. D3DXVec3Cross(&agitationVector, &currentDesiredMove, &mUpVector); currentDesiredMove = agitationVector; squadmate.ResetTimeSinceWaypointReached(); } entity.SetDesiredMove(currentDesiredMove);

213

ApplyAvoidance(entity); }

That was a fairly big method, so we will tackle it a bit at a time. void cPathfindBehavior::Iterate(float timeDelta, cEntity &entity)

We begin with the prototype. The behavior requires the time passed since the last iteration as well as the entity upon which to iterate. // pathfinding only works on squad mate type entities! cSquadEntity &squadmate = dynamic_cast<cSquadEntity&>(entity);

This particular implementation only works on the special squad entities derived for this demo. So we make sure that is the case here. tPath &path = squadmate.GetPath(); tWaypointID wpID = squadmate.GetNextWaypoint(); D3DXVECTOR3 entityPos = entity.Position(); D3DXVECTOR3 desiredMoveAdj(0.0f, 0.0f, 0.0f);

First, we get the squad mate’s path, his next waypoint ID, his position, and initialize the desired movement adjustment to be nil. if (wpID == GUID_NULL && path.size() > 0) { // set the next waypoint! wpID = path.front(); path.pop_front(); squadmate.SetNextWaypoint(wpID); squadmate.ResetTimeSinceWaypointReached(); }

If the waypoint ID is NULL, and we have some nodes left in our path, we get the next waypoint from the path, and set it on the squad mate. We also reset a timer which keeps track of how long it has been since we reached a waypoint. else if (wpID == GUID_NULL || path.empty()) { desiredMoveAdj = squadmate.GetGoal() - entityPos; if (D3DXVec3Length(&desiredMoveAdj) < mGoalRadius) { // we made it... stand around entity.SetDesiredMove(-entity.Velocity()); ApplyAvoidance(entity); squadmate.ResetTimeSinceWaypointReached(); return; } }

214

If our waypoint ID is NULL, or our path is empty, we are moving directly towards our goal position. So we compute our desired move adjustment to be the vector to the goal from our position. If that vector’s length is less than the goal’s radius, we have reached the goal. In that case, we set our velocity to bring us to a stop by negating it, apply some avoidance measures (which we will cover later), and again reset the amount of time it has been since we last reach a waypoint. We then return out of the method. cWaypoint *wp = mWaypointNetwork.FindWaypoint(wpID);

Assuming we have not reached our goal, we find the waypoint for our current waypoint ID. if (wp)

Assuming we have a waypoint… D3DXVECTOR3 wppos = wp->GetPosition(); desiredMoveAdj = wppos - entityPos;

We get the position of the waypoint, and set our desired movement adjustment to be the vector from our current position to the waypoint’s position. if (D3DXVec3Length(&desiredMoveAdj) < wp->GetRadius())

If the magnitude of that vector is less than our waypoint’s radius, we have reached it. So… // close enough! next waypoint! if (path.size() > 0) { // set the next waypoint! wpID = path.front(); path.pop_front(); squadmate.SetNextWaypoint(wpID); wp = mWaypointNetwork.FindWaypoint(wpID); ASSERT(wp != NULL); wppos = wp->GetPosition(); desiredMoveAdj = wppos - entityPos; squadmate.ResetTimeSinceWaypointReached(); }

If we have path nodes left in our path, we grab the next waypoint from the path, and set the squad mate’s next waypoint to be that ID. We then find the waypoint for this new waypoint ID, and set our desired movement vector to be the vector from our current position to this new waypoint’s position. We then reset the amount of time it has been since this squad mate last reached a waypoint. else { // uh, no more waypoints,

// start walking toward our target position desiredMoveAdj = squadmate.GetGoal() - entityPos; squadmate.SetNextWaypoint(GUID_NULL);

215

if (D3DXVec3Length(&desiredMoveAdj) < mGoalRadius) { // we made it... stand around entity.SetDesiredMove(-entity.Velocity()); ApplyAvoidance(entity); squadmate.ResetTimeSinceWaypointReached(); return; } }

Otherwise, if we have no more nodes left in our path, we start walking towards our goal. We set the desired movement adjustment to be the vector from our current position to our goal. We also NULL out our squad mate’s next waypoint ID. If the magnitude of the vector from our current position to the goal is greater than the goal radius, we have reached the goal. At that point, we again negate our velocity and set it to be our desired movement adjustment to bring us to a halt. We apply some avoidance, and we reset the time since this squad mate last reached a waypoint. We then return from the method. // move in the direction of your next pathnode or your goal position squadmate.IncrementTimeSinceWaypointReached(timeDelta); D3DXVECTOR3 currentDesiredMove = entity.DesiredMove(); D3DXVec3Normalize(&desiredMoveAdj, &desiredMoveAdj); desiredMoveAdj *= mTurnRate; currentDesiredMove += desiredMoveAdj * Gain(); // // some mojo here… // entity.SetDesiredMove(currentDesiredMove); ApplyAvoidance(entity);

If we have not aborted from reaching a goal, we actually apply some clamps to the desired movement adjustments. But first, we increment the amount of time it has been since this squad mate reached a waypoint. We get our current desired move, normalize our desired movement adjustment, and apply the turn rate scalar. Then we add our desired move adjustment, scaled by the behavior gain, to the current desired move. At this point, we apply some mojo. We will go over exactly what that is shortly. Lastly, we set the desired move to the newly computed desired move and we apply some avoidance. We will talk about how that works very shortly.

216

Getting Stuck

There is a little complication that must be taken into account when applying turning radius limits. Imagine driving a car at 10 mph. The car corners pretty well and you can almost turn on a dime. Now start driving 50 mph. You cannot turn on a dime anymore because the car has a turning radius.

Figure 5.3

Take a look at Figure 5.3. If the car was trying to get into the blue circle, and its current velocity was in the direction of the green arrow, then the adjusted velocity would be the blue arrow. However, since the car has a turning radius, the red arrow is the adjusted velocity. This rate limit just keeps carrying it around the red circle! We never get to the blue circle because we keep trying to turn the same amount, while not slowing down. Sadly, our entities can suffer from this same problem. But there is a cure! By keeping track of how long it has been since we last reached the waypoint, we can detect if it has been an unacceptably long period of time since we made progress. If it has been too long, we can do something about it -- perturb their velocity. Enter the mojo… if (squadmate.GetTimeSinceWaypointReached() > mMaxTimeBeforeAgitation) { // agitate the movement to get the guy moving properly D3DXVECTOR3 agitationVector; // nudge the desired move vector by using a vector perpendicular to the // current desired move's direction. D3DXVec3Cross(&agitationVector, &currentDesiredMove, &mUpVector); currentDesiredMove = agitationVector; squadmate.ResetTimeSinceWaypointReached(); }

Right before we set our desired movement, we check to make sure that the amount of time it has been since this entity last reached a waypoint is not greater than the max time before we agitate his motion. If

217

it is, we cross the desired movement vector with the up vector, and get a vector perpendicular to our current motion. We then use that vector as our desired movement vector, for one and only one tick. This perturbs the motion of the entity just enough, and he can recover from the circular pattern we just witnessed.

5.3.2 A Word on Avoidance

There is one last bit remaining unexplained in the pathfinding behavior, and that is the ApplyAvoidance behavior. While we could have just used the avoidance behavior from the flocking demo, it did not provide exactly the same kind of avoidance that we wanted in this application. One thing to bear in mind is that avoidance is not supposed to keep the entities from ever running into each other. In order to do that, we would need to run a full scale simultaneous solve of the positions and velocities of the entities, and resolve interpenetrations. We did not want to worry about that in this demo since it introduces systems that are beyond the scope of this course. The idea here is that a proper collision system would keep you from being on top of one another, while the avoidance behavior would attempt minimize the number of times they keep running into each other. So the systems work in tandem. Discussion of a full blown collision system can be found in 3D Graphics Programming Module II. In a nutshell, the avoidance behavior checks to see if any entities are too close, and if so, computes an adjustment vector to move the entity on a travel path tangent to the other entities’ periphery.

Entity 1

Entity 1b

Entity 2

r

r

2r

θ

v

Figure 5.4

Looking at Figure 5.4, we compute the vector from Entity 1 to Entity 2. We then know if we want Entity 1 to pass by the side of Entity 2 without colliding, we should move in the direction from Entity 1

218

to Entity 1b. We can compute this by calculatingvr2tan 1−=θ . We then rotate v by θ to get the vector

pointing in the direction we need to go. Let us look at how the code does this. void cPathfindBehavior::ApplyAvoidance(cEntity &entity) { // pathfinding only works on squad mate type entities! cSquadEntity &squadmate = dynamic_cast<cSquadEntity&>(entity); D3DXVECTOR3 entityPos(entity.Position()); D3DXVECTOR3 currentDesiredMove(entity.DesiredMove()); // let's make sure we aren't bound to hit anyone else cWorld &world = squadmate.World(); for (tGroupList::iterator git = world.Groups().begin();

git != world.Groups().end(); ++git) { cGroup *grp = *git; for (tEntityList::iterator eit = grp->Entities().begin();

eit != grp->Entities().end(); ++eit) { cEntity *e = *eit; if (e == &entity) continue; D3DXVECTOR3 otherEntityPos(e->Position()); D3DXVECTOR3 toEntity = otherEntityPos - entityPos; if (D3DXVec3Length(&toEntity) < mAvoidDist) { // let's apply some avoidance. D3DXVec3Normalize(&toEntity, &toEntity); float zRotation = atan2f(toEntity.y, toEntity.x); D3DXMATRIX rotationMat; D3DXMatrixRotationZ(&rotationMat, zRotation); D3DXVECTOR4 rotatedDesiredMove; D3DXVec3Transform(&rotatedDesiredMove,

&currentDesiredMove, &rotationMat); D3DXVECTOR3 newDesiredMove(rotatedDesiredMove.x,

rotatedDesiredMove.y, rotatedDesiredMove.z);

D3DXVECTOR3 desiredMoveAdj = newDesiredMove – currentDesiredMove;

desiredMoveAdj *= mTurnRate; currentDesiredMove += desiredMoveAdj * Gain(); entity.SetDesiredMove(currentDesiredMove); } } } }

The method looks long, but the iteration takes up the bulk of the space. // pathfinding only works on squad mate type entities! cSquadEntity &squadmate = dynamic_cast<cSquadEntity&>(entity);

219

First off, this method only works on our special derived squad entity type of entity. So check that first. D3DXVECTOR3 entityPos(entity.Position()); D3DXVECTOR3 currentDesiredMove(entity.DesiredMove());

Next we get the current position and current desired move of the entity. cWorld &world = squadmate.World(); for (tGroupList::iterator git = world.Groups().begin();

git != world.Groups().end(); ++git) { cGroup *grp = *git; for (tEntityList::iterator eit = grp->Entities().begin();

eit != grp->Entities().end(); ++eit) {

We then iterate through all the groups in the world, and each of the entities in each group. cEntity *e = *eit; if (e == &entity) continue;

We then perform a sanity check, to ensure we are not trying to avoid ourselves. D3DXVECTOR3 otherEntityPos(e->Position()); D3DXVECTOR3 toEntity = otherEntityPos - entityPos;

Now we compute a vector from this entity to the other entity. if (D3DXVec3Length(&toEntity) < mAvoidDist)

If the magnitude of the vector is less than our avoid distance, we need to try to avoid this entity. // let's apply some avoidance. D3DXVec3Normalize(&toEntity, &toEntity); float zRotation = atan2f(toEntity.y, toEntity.x); D3DXMATRIX rotationMat; D3DXMatrixRotationZ(&rotationMat, zRotation);

First we normalize the vector to the entity, and compute the rotation based on the vector. We then build a rotation matrix using this vector. Note that this method is useful only in avoiding things in 2d, where Z is up. D3DXVECTOR4 rotatedDesiredMove; D3DXVec3Transform(&rotatedDesiredMove,

&currentDesiredMove, &rotationMat); D3DXVECTOR3 newDesiredMove(rotatedDesiredMove.x,

rotatedDesiredMove.y, rotatedDesiredMove.z);

220

Next we perform a little data hoop jumping. D3DXVec3Transform puts the results in a D3DXVECTOR4, because it wants to preserve the w value. That is fine, but there are no convenient operators to turn a vector 4 back to a vector 3 without constructing one by hand. So we rotate our current desired move vector by the rotation matrix we computed and then we put it into a usable vector. D3DXVECTOR3 desiredMoveAdj = newDesiredMove –

currentDesiredMove; desiredMoveAdj *= mTurnRate; currentDesiredMove += desiredMoveAdj * Gain(); entity.SetDesiredMove(currentDesiredMove);

Now we get a new desired move adjustment vector by getting the vector from our current desired move to new current desired move, apply our turn rate limit, and then add it into the our current desired move, making use of the behavior gain. Finally, we then set the desired move using computed vector. That is it for the behavioral movement component in the demo. Now we are ready to see how the squad members and squad leaders make their decisions about where to go in the world.

5.4 Squads and State Machines

So now we have waypoint networks, ways to pathfind through them, and a movement behavior to get our entities to follow a path. All that is left are the squad members themselves and how they decide what to do. In the last chapter, we discussed state machines as decision systems, and scripting as a means to extend what our games can do in a data driven way. We now take our next step, and integrate it into the demo with pathfinding and waypoint networks.

5.4.1 Methods of Squad Communication

In our demo, we will have a squad leader and three squad members. The squad leader tells the squad members what to do, and they do it. In truth, it does not get much simpler than this. So how do they manage this communication? There are a few typical ways to implement squad communication.

Direct Control

In this method, the squad leader directly calls methods to modify variables or cause transitions in the squad member’s state machine (or whatever decision system you are using). It is a simple approach, and the one we are using for our demo, but it is not the most flexible. The squad leader has to know all about the squad members, and make them do the right things to get them to behave as he wants them to. Basically the squad leader needs to know what to do, and how to do it.

221

Poll the Leader

Another approach is to have the squad members poll the leader to find out what he wants them to do. This pushes the responsibility of knowing how to do it onto the squad member. This is a slightly better design than the last one since it gives the squad members more autonomy and allows for more variety at the squad member level. The downside here is the squad members now need to know all about the squad leader. Although, in fairness, ‘many to one’ is often easier to manage than ‘one to many’ since the group members only need to know about one type of object – the leader.

Events

The last approach worth mentioning is an event system. The idea is that the squad leader decides what he wants to have happen, and sends an event to the squad members. They receive the event, process it, and decide how to do what the squad leader wants. When they are done, or if something comes up that requires them to seek redirection from the squad leader, they send an event back to the squad leader, who receives it, processes it, and sends an event back. It is a more complicated system, but fairly general and flexible.

5.4.2 The Squad Member

First, let us talk about the squad member itself, and the class that drives it. We will then discuss its state machine which gets it to do the things we want.

The Squad Entity Class

The squad entity class derives directly from our entity class from the flocking demo. It is specialized in a few ways, to give us the ability to recall the waypoints we were traversing, the network we are on, etc. Let us take a look at that class now. class cSquadEntity : public cEntity { public: cSquadEntity ( cWorld &world, unsigned type, float senseRange, float maxVelocityChange, float maxSpeed, float desiredSpeed, float moveXScalar, float moveYScalar,

222

float moveZScalar ); virtual ~cSquadEntity(void); virtual void Iterate(float timeDelta); void SetPath(const tPath &aPath) { mPath = aPath; } tPath &GetPath(void) { return mPath; } const tPath &GetPath(void) const { return mPath; } void SetGoal(const D3DXVECTOR3 &goal) { mGoalPosition = goal; } D3DXVECTOR3 &GetGoal(void) { return mGoalPosition; } const D3DXVECTOR3 &GetGoal(void) const { return mGoalPosition; } void SetNextWaypoint(const tWaypointID &wp) { mNextWaypoint = wp; } tWaypointID &GetNextWaypoint(void) { return mNextWaypoint; } const tWaypointID &GetNextWaypoint(void) const { return mNextWaypoint; } cWaypointNetwork *GetWaypointNetwork(void) { return mWaypointNetwork; } const cWaypointNetwork *GetWaypointNetwork(void) const

{ return mWaypointNetwork; } void SetWaypointNetwork(cWaypointNetwork *network)

{ mWaypointNetwork = network; } cStateMachine *GetStateMachine(void) { return mStateMachine; } const cStateMachine *GetStateMachine(void) const { return mStateMachine; } void SetStateMachine(cStateMachine *machine) { mStateMachine = machine; } COLORREF GetColor(void) { return mColor; } void SetColor(COLORREF color) { mColor = color; } bool WaypointReached(); void OnWaypointReached(); void ResetTimeSinceWaypointReached()

{ mTimeSinceNextWaypointReached = 0.0f; } void IncrementTimeSinceWaypointReached(float deltaTime)

{ mTimeSinceNextWaypointReached += deltaTime; } float GetTimeSinceWaypointReached() const

{ return mTimeSinceNextWaypointReached; } bool GoalReached(); void OnGoalReached(); void OnWaitingForCommand(); bool HasValidWaypoint() { return !mNextWaypoint.IsEqual(GUID_NULL); } bool HasValidPath() { return mPath.size() > 0; } cWorld &World() { return mWorld; } protected: cWaypointNetwork *mWaypointNetwork; tWaypointID mNextWaypoint; D3DXVECTOR3 mGoalPosition; tPath mPath; COLORREF mColor; float mTimeSinceNextWaypointReached; cStateMachine *mStateMachine; };

223

Most of this interface is accessors, so it is larger and more complicated looking than it really is. We will start our examination with the data members, and then some of the less obvious methods. cWaypointNetwork *mWaypointNetwork;

First off, we have the waypoint network that this entity is traveling on. tWaypointID mNextWaypoint; D3DXVECTOR3 mGoalPosition;

Next, we have the waypoint ID of the next waypoint we are traveling to, and our ultimate goal position. tPath mPath;

Here we have the path we will be using to find our way through the network to our ultimate goal. COLORREF mColor;

Here we have the color we are going to draw this entity in the waypoint network view. This gets changed when we walk over waypoints. float mTimeSinceNextWaypointReached;

Here we have the timer that keeps track of how long it has been since this entity has reached a waypoint. If this timer value gets to be too large, the entity has his velocity agitated in order to get him to his waypoint. cStateMachine *mStateMachine;

Last, we have the state machine for this entity. Now that we have seen the data this class uses, let us look at the non-trivial methods. void cSquadEntity::Iterate(float timeDelta) { // iterate our state machine if (mStateMachine) mStateMachine->Iterate(); cEntity::Iterate(timeDelta); // Update our orientation const float kEpsilon = 0.01f; if (D3DXVec3Length(&mVelocity) > kEpsilon) { D3DXVECTOR3 vec; D3DXVec3Normalize(&vec, &mVelocity); float zRotation = atan2f(vec.y, vec.x); D3DXQuaternionRotationYawPitchRoll(&mOrientation, 0.0f,

224

0.0f, zRotation); } }

Here we have the iterate method, which is called every frame. Its job is to execute the movement behaviors on the entity, and move the entity through the world. We do two things above and beyond the default implementation. We first update our state machine for this frame, then we call the default Iterate implementation, and finally we compute a new orientation vector which is compatible with this demo’s coordinate system. #define ENTITY_RADIUS 0.75f bool cSquadEntity::WaypointReached() { if (!mNextWaypoint.IsEqual(GUID_NULL) && mWaypointNetwork) { cWaypoint *wp = mWaypointNetwork->FindWaypoint(mNextWaypoint); if (wp) { D3DXVECTOR3 vec = wp->GetPosition() - Position(); float distToWP = D3DXVec3Length(&vec); if ((distToWP - ENTITY_RADIUS) < wp->GetRadius()) { return true; } return false; } } return false; }

The WaypointReached method is called to check to see if the next waypoint has been reached. If we have no next waypoint, or no waypoint network, we return false. Otherwise, we find the next waypoint in the network, and compute our distance to the waypoint. In this case, we take into account the radius of the entity to determine if we have reached the waypoint. #define GOAL_REACH_THRESHOLD 1.5f bool cSquadEntity::GoalReached() { D3DXVECTOR3 vec = mGoalPosition - Position(); if (D3DXVec3Length(&vec) < GOAL_REACH_THRESHOLD) { return true; } return false; }

225

The GoalReached method works similarly to the WaypointReached method, only it computes the distance to the goal position rather than to the next waypoint. void cSquadEntity::OnWaypointReached() { if (!mNextWaypoint.IsEqual(GUID_NULL) && mWaypointNetwork) { cWaypoint *wp = mWaypointNetwork->FindWaypoint(mNextWaypoint); if (wp) { COLORREF color; wp->GetBlindData(0, color); SetColor(color); return; } } SetColor(RGB(0, 0, 0)); }

The OnWaypointReached method gets called when the squad member has reached his next waypoint. It queries the blind data for the color value stored in it, and sets the color of the entity using that value. void cSquadEntity::OnGoalReached() { SetColor(RGB(255, 150, 150)); }

The OnGoalReached method gets called when the squad member has reached his goal. It simply changes the color of the entity. void cSquadEntity::OnWaitingForCommand() { SetColor(RGB(150, 255, 150)); }

The OnWaitingForCommand method gets called when the squad member is waiting for a command. It simply changes the color of the entity.

226

The Squad Member State Machine

Figure 5.5

As the state transition diagram shows (see Figure 5.5), the squad member has a simple state machine. He starts waiting for a command, and once he gets one, he moves to the goal using the network. Each time he reaches a waypoint (or the goal), he goes to the waypoint reached state. Once he has reached the goal point, and he has no more waypoints, he goes back to waiting for a command. Let us take a look at the Python scripted actions and transitions that get this job done for us. from GI_AISDK import * class WaitingForCommandAction(PythonScriptedAction): def execute(self): self.state().entity().on_waiting_for_command();

First we have the waiting for command action. It simply calls the entity’s on_waiting_for_command handler. This action is executed when the waiting for command state is entered. from GI_AISDK import * class CommandGiven(PythonScriptedTransition): def should_transition(self): return self.source().entity().has_valid_path() and \

not self.source().entity().has_valid_waypoint();

227

The CommandGiven transition checks to see if the entity has a valid path and not a valid waypoint. This means a new target has been set, but the entity has not yet begun walking that path. from GI_AISDK import * class HaveReachedWaypoint(PythonScriptedTransition): def should_transition(self): return self.source().entity().waypoint_reached() or \

self.source().entity().goal_reached();

The HaveReachedWaypoint transition checks to see if the entity has reached a waypoint, or the goal. from GI_AISDK import * class WaypointReachedAction(PythonScriptedAction): def execute(self): if self.state().entity().waypoint_reached(): self.state().entity().on_waypoint_reached(); else: self.state().entity().on_goal_reached();

The WaypointReachedAction checks to see if the waypoint or the goal has been reached, and calls the appropriate handler. This action is performed when the waypoint reached state is entered. from GI_AISDK import * class HasMoreWaypoints(PythonScriptedTransition): def should_transition(self): return self.source().entity().has_valid_path() or not \

self.source().entity().goal_reached();

The HasMoreWaypoints transition simply evaluates if the entity has a valid path, or has not yet reached the goal. from GI_AISDK import * class NoMoreWaypoints(PythonScriptedTransition): def should_transition(self): return self.source().entity().goal_reached() and not \

self.source().entity().has_valid_path()

The NoMoreWaypoints transition simply returns if the goal has been reached and the entity does not have a valid path.

228

That does it for the squad members. Now we just have to investigate how the squad leader works and we will be ready to set up this demo.

5.4.3 The Squad Leader

The squad leader derives from our squad entity class. This allows him to navigate the world if he so desires. He also has some specialized functionality, so let us take a look at that.

The Squad Leader Class

class cSquadLeaderEntity : public cSquadEntity { public: cSquadLeaderEntity ( cWorld &world, unsigned type, float senseRange, float maxVelocityChange, float maxSpeed, float desiredSpeed, float moveXScalar, float moveYScalar, float moveZScalar ); virtual ~cSquadLeaderEntity(void); void SendSquadToRandomPOI(); bool SquadArrivedAtGoal(); cPointOfInterest *GetSelectedPointOfInterest() const

{ return mSelectedPointOfInterest; } void SetSelectedPointOfInterest(cPointOfInterest *poi)

{ mSelectedPointOfInterest = poi; } tPointOfInterestMap *GetPointsOfInterest()

{ return mPointsOfInterest; } void SetPointsOfInterest(tPointOfInterestMap *pointsOfInterest)

{ mPointsOfInterest = pointsOfInterest; } void AddSquadMember(cSquadEntity *member); void RemoveSquadMember(cSquadEntity *member); void ClearSquadMembers(); protected: vector<cSquadEntity*> mSquadMembers; tPointOfInterestMap *mPointsOfInterest; cPointOfInterest *mSelectedPointOfInterest; };

229

One of the first things you will notice is the point of interest class. A point of interest is a lot like a waypoint, only it is not connected to the network. Typically, you create a point of interest for anything in the game world that is going to be important to the decision maker. Consider the example of a stealth action game, where the hero is trying to sneak into a compound filled with bad guys. If the hero makes a noise, the game could create a “noise” point of interest at the position where he made the noise. The bad guys would then see the point of interest, and go investigate. In the demo, the squad leader has a large set of points of interest, and he directs his squad to investigate them at random. Again, the points of interest are not on the network. This demonstrates that most of the time in a continuous world environment, the things you care about will not be on the pre-computed network (although certainly, you can include points of interest on the waypoint network if desired). Ultimately, we need to be able to get on and off the network without mishap. Getting back to the squad leader class, you will notice that there are a few methods and some data that need a little explaining. vector<cSquadEntity*> mSquadMembers;

First we have the vector of squad members in our squad. This allows the leader to keep track of them, and tell them what to do. tPointOfInterestMap *mPointsOfInterest;

Next we have a map of point of interest IDs to points of interest. This is basically just like the waypoint map the waypoint network has, only with points of interest instead. cPointOfInterest *mSelectedPointOfInterest;

Last, we have the point of interest we are currently directing our squad to investigate. There are two methods of interest in the squad leader class. Let us take a look at them. void cSquadLeaderEntity::SendSquadToRandomPOI() { if (!mPointsOfInterest && mSquadMembers.size() > 0) return; tPointOfInterestID closestPOI = FindPointOfInterestNearestPosition

( *mPointsOfInterest, mSquadMembers[0]->GetGoal()

); tPointOfInterestID poiID = SelectRandomPointOfInterest(*mPointsOfInterest,

closestPOI); cPointOfInterest *poi = FindPointOfInterest(*mPointsOfInterest, poiID); if (!poi) return;

230

SetSelectedPointOfInterest(poi); tPath pathToWP; cWaypointVisibilityFunctor functor; for (vector<cSquadEntity*>::iterator it = mSquadMembers.begin();

it != mSquadMembers.end(); ++it) { cSquadEntity *entity = *it; entity->SetNextWaypoint(GUID_NULL); pathToWP.clear(); GetWaypointNetwork()->FindPathFromPositionToPosition

( entity->Position(), poi->GetPosition(), functor, pathToWP

); entity->SetPath(pathToWP); entity->SetGoal(poi->GetPosition()); } }

First we have the SendSquadToRandomPOI method. This method selects a random point of interest, and sends every squad member to investigate. Let us take a closer look. if (!mPointsOfInterest && mSquadMembers.size() > 0) return;

For starters, if we have no points of interest, or no squad members, we do not have any work to do. tPointOfInterestID closestPOI = FindPointOfInterestNearestPosition

( *mPointsOfInterest, mSquadMembers[0]->GetGoal()

);

Next we find the closest POI to the squad’s current goal position. The idea here is that we do not want to select this point of interest again, since they are already going there, or are already there. tPointOfInterestID poiID = SelectRandomPointOfInterest(*mPointsOfInterest,

closestPOI);

Now we select a random point of interest, ignoring the one we just found as the closest. This method is pretty simple, so we will not go into it here (see PointOfInterest.cpp). cPointOfInterest *poi = FindPointOfInterest(*mPointsOfInterest, poiID); if (!poi) return;

231

Next, we find the point of interest in the map using the ID we got back from the random selector. If we do not find it, we bail out. SetSelectedPointOfInterest(poi);

We then select this as our current point of interest. tPath pathToWP; cWaypointVisibilityFunctor functor; for (vector<cSquadEntity*>::iterator it = mSquadMembers.begin();

it != mSquadMembers.end(); ++it) {

Next, we iterate through all of our squad members. We will be finding a path for each of them, so we will need a path, and a visibility functor. cSquadEntity *entity = *it; entity->SetNextWaypoint(GUID_NULL); pathToWP.clear();

For each entity, we will null out their current waypoint, and clear our path that we are going to find for them. GetWaypointNetwork()->FindPathFromPositionToPosition

( entity->Position(), poi->GetPosition(), functor, pathToWP

); entity->SetPath(pathToWP); entity->SetGoal(poi->GetPosition());

We then find the path from the entity’s current position to the position of the point of interest we selected. We set that path on the entity, and set his goal to the position of the point of interest we selected. It is worth noting here, that we could easily modify this method to select a different point of interest for each squad member by moving the random waypoint selection code inside the loop. bool cSquadLeaderEntity::SquadArrivedAtGoal() { for (vector<cSquadEntity*>::iterator it = mSquadMembers.begin();

it != mSquadMembers.end(); ++it) { cSquadEntity *entity = *it; if (!entity->GoalReached()) return false; } return true; }

232

The other method we should talk about is the SquadArrivedAtGoal method. This method returns success if all of the squad members have arrived at their goals. It is done by simply iterating through all the squad members, and if any one of them has not reached their goal, we return false. Otherwise, we return true. That is everything we need to discuss about the squad leader class, so let us take a look at the state machine the squad leader uses.

The Squad Leader State Machine

Figure 5.6

The squad leader has three states: the awaiting squad task completion state, the wait to give orders state, and command squad to POI state. He begins in the command squad to POI state, and directs the squad to a point of interest. He then waits for the squad to arrive at their destination, whereupon he waits a second, then commands them to another point of interest. Let us take a look at the Python scripted actions and transitions that make this work. from GI_AISDK import * class SquadArrivedAtPOI(PythonScriptedTransition): def should_transition(self):

233

return self.source().entity().squad_arrived_at_goal();

The SquadArrivedAtPOI transition simply returns if the squad has arrived at the goal. from GI_AISDK import * class CommandSquadMembersToPOI(PythonScriptedAction): def execute(self): self.state().entity().send_squad_to_random_poi();

The CommandSquadMembersToPOI action sends the squad to a random point of interest using the method provided by the squad leader. This action is performed when the state is entered. from GI_AISDK import * class SquadsProceedingToPOI(PythonScriptedTransition): def should_transition(self): return True

The SquadsProceedingToPOI transition simply returns true. Since the squad leader issues the orders for the squad to proceed to their goal upon entering the CommandSquadToPOI state, there is no need to remain in the state afterwards. That is really all there is to the squad leader. Obviously there is much more behavior you are probably thinking about adding, and with the tools provided you will be able to develop some very cool and interesting AI. To wrap things up in this chapter, let us see how we set up our demo to bring it all together.

234

5.5 Setting up the Demo

Figure 5.7

The waypoint network and squads demo is shown in Figure 5.7. On the left, we have a tree view that represents the state machines of the squad leader and his squad members. This tree view is just like the one in the state machine demo, but does not allow editing of the state machines. If you want, you can use the state machine demo to edit your state machines, since this demo uses those files to load the squad leader and squad member state machines. The top pane on the right is the waypoint network view. It displays the waypoints, their edges, and the points of interest, along with the squad members. The arrows inside the waypoints show the orientation of the waypoint. While this demo makes no use of that information, as mentioned before, orientation on waypoints can be very useful, so it deserves attention. The circles without the arrows are the points of interest. The purple circle with the arrow is the squad leader, and the blue circles with the (in the case of this diagram orange) arrows are the squad members. As the squad members cross the waypoints, their arrow will change to the color of the waypoint crossed. The currently selected entity will be filled in gray, and his path to the currently selected point of interest (also filled in gray) will be shown in black. You may select squad members by clicking on them in this view, or in the state machine tree on the left. The pane on the bottom right is the state machine view. It is similar to the state machine view in the state machine demo, as it shows the currently active squad member’s state machine. The state with the red border is the state the squad member is currently residing in. Let us quickly go over the initialization code for the demo, and then you should be ready to jump in and have some fun with it!

235

BOOL CWaypointNetworksAndSquadsDoc::OnNewDocument() { if (!CDocument::OnNewDocument()) return FALSE; // free the old network and world if (mWaypointNetwork) delete mWaypointNetwork; mWaypointNetwork = NULL; if (mWorld) delete mWorld; mWorld = NULL; if (mPFbeh) delete mPFbeh; mPFbeh = NULL; ClearPointsOfInterest(mPointsOfInterest); // allocate a new ones mWaypointNetwork = new cWaypointNetwork(); mWorld = new cWorld; cGroup *squadGroup = new cGroup(*mWorld); mWorld->Add(*squadGroup); const int kSquadType = 0x1; const float kSenseDistance = 20.0f; const float kMaxVelocityChange = 1.0f; const float kMaxSpeed = 5.0f; const float kMoveXScalar = 1.0f; const float kMoveYScalar = 1.0f; const float kMoveZScalar = 0.0f; // don't allow z moment const float kSeparationDist = 1.5f; const float kPathfindingMaxRateChange = 0.2f; const float kGoalReachedRadius = 1.0f; const float kMaxTimeBeforeAgitation = 5.0f; // seconds to reach a

// path node before the entity // gets agitation stimulus

D3DXVECTOR3 upVector(0.0f, 0.0f, 1.0f); // The entities in this demo // live in the XY plane

const int kNumSquadMembers = 3; // make pathfinding behavior mPFbeh = new cPathfindBehavior(kPathfindingMaxRateChange, kGoalReachedRadius,

kSeparationDist, kMaxTimeBeforeAgitation, upVector, *mWaypointNetwork);

// load a default waypoint network ifstream wpnfile("demo.wpn"); mWaypointNetwork->UnSerialize(wpnfile); ClearPointsOfInterest(mPointsOfInterest); UnSerializePointsOfInterest(mPointsOfInterest, wpnfile); tPointOfInterestID startPoiID = SelectRandomPointOfInterest(

236

mPointsOfInterest, GUID_NULL); cPointOfInterest *startPoi = FindPointOfInterest(mPointsOfInterest,

startPoiID); D3DXVECTOR3 origin = startPoi->GetPosition(); tPointOfInterestID endPoiID = SelectRandomPointOfInterest(mPointsOfInterest,

startPoiID); cPointOfInterest *endPoi = FindPointOfInterest(mPointsOfInterest, endPoiID); D3DXVECTOR3 destination = endPoi->GetPosition(); // setup the squad leader mSquadLeader = new cSquadLeaderEntity ( *mWorld, kSquadType, kSenseDistance, kMaxVelocityChange, kMaxSpeed, kMaxSpeed * 0.4f, kMoveXScalar, kMoveYScalar, kMoveZScalar ); mSquadLeader->SetPointsOfInterest(&mPointsOfInterest); D3DXQUATERNION squadLeaderOrientation(0.0f, 0.0f, 1.0f, 0.0f); mSquadLeader->SetOrientation(squadLeaderOrientation); mSquadLeader->SetSelectedPointOfInterest(endPoi); cStateMachine *statemachine = new cSquadStateMachine(mSquadLeader); ifstream squadleaderfsm("SquadLeader.stm"); try { if (statemachine->UnSerialize(squadleaderfsm) == FALSE) { // error handling mojo... delete statemachine; statemachine = NULL; } } catch(error_already_set) { // error handling mojo... delete statemachine; statemachine = NULL; } statemachine->Reset(); D3DXVECTOR3 squadleaderpos(34, 10, 0.0f); mSquadLeader->SetPosition(squadleaderpos); mSquadLeader->SetStateMachine(statemachine); mSquadLeader->SetWaypointNetwork(mWaypointNetwork); squadGroup->Add(*mSquadLeader); // setup some squad mates to roam the network for (int i = 0; i < kNumSquadMembers; ++i) { D3DXVECTOR3 pos(0.0f, (float)i, 0.0f); cSquadEntity *squadmate = new cSquadMemberEntity ( *mWorld,

237

kSquadType, kSenseDistance, kMaxVelocityChange, kMaxSpeed, kMaxSpeed * 0.4f, kMoveXScalar, kMoveYScalar, kMoveZScalar ); statemachine = new cSquadStateMachine(squadmate); ifstream ar("SquadMember.stm"); try { if (statemachine->UnSerialize(ar) == FALSE) { // error handling mojo... delete statemachine; statemachine = NULL; } } catch(error_already_set) { // error handling mojo... delete statemachine; statemachine = NULL; } squadmate->AddBehavior(*mPFbeh); squadmate->SetPosition(startPoi->GetPosition() + pos); squadmate->SetWaypointNetwork(mWaypointNetwork); statemachine->Reset(); squadmate->SetStateMachine(statemachine); squadGroup->Add(*squadmate); mSquadLeader->AddSquadMember(squadmate); if (!mSelectedEntity) mSelectedEntity = squadmate; cWaypointVisibilityFunctor visibilityFunctor; tPath thePath; if (mWaypointNetwork->FindPathFromPositionToPosition(origin,

destination, visibilityFunctor, thePath))

{ squadmate->SetPath(thePath); squadmate->SetGoal(endPoi->GetPosition()); } } mPause = false; return TRUE; }

238

As with most of our prior demos, this demo is implemented in MFC, so all of the data lives in the document class. When a new document is created, this method is called. Let us go over it to see how it is put together. // free the old network and world if (mWaypointNetwork) delete mWaypointNetwork; mWaypointNetwork = NULL; if (mWorld) delete mWorld; mWorld = NULL; if (mPFbeh) delete mPFbeh; mPFbeh = NULL; ClearPointsOfInterest(mPointsOfInterest);

First, if we have a waypoint or world or pathfinding behavior or any points of interest around, we free them so we can start with a clean slate. // allocate a new ones mWaypointNetwork = new cWaypointNetwork(); mWorld = new cWorld; cGroup *squadGroup = new cGroup(*mWorld); mWorld->Add(*squadGroup);

We then allocate a new waypoint network, a new world, create a group for our squad, and add it to the world. const int kSquadType = 0x1; const float kSenseDistance = 20.0f; const float kMaxVelocityChange = 1.0f; const float kMaxSpeed = 5.0f; const float kMoveXScalar = 1.0f; const float kMoveYScalar = 1.0f; const float kMoveZScalar = 0.0f; // don't allow z moment const float kSeparationDist = 1.5f; const float kPathfindingMaxRateChange = 0.2f; const float kGoalReachedRadius = 1.0f; const float kMaxTimeBeforeAgitation = 5.0f; // seconds to reach a

// path node before the entity // gets agitation stimulus

D3DXVECTOR3 upVector(0.0f, 0.0f, 1.0f); // The entities in this demo // live in the XY plane

const int kNumSquadMembers = 3;

We set up some constants for use in creating the behaviors and entities in the world. // make pathfinding behavior mPFbeh = new cPathfindBehavior(kPathfindingMaxRateChange, kGoalReachedRadius,

239

kSeparationDist, kMaxTimeBeforeAgitation, upVector, *mWaypointNetwork);

We then create a pathfinding behavior. This behavior is shared by all of the entities. // load a default waypoint network ifstream wpnfile("demo.wpn"); mWaypointNetwork->UnSerialize(wpnfile); ClearPointsOfInterest(mPointsOfInterest); UnSerializePointsOfInterest(mPointsOfInterest, wpnfile);

Next we load in the default waypoint network file (feel free to open that file up in a text editor and try modifying it). We also load in the default points of interest from that file. tPointOfInterestID startPoiID = SelectRandomPointOfInterest(

mPointsOfInterest, GUID_NULL); cPointOfInterest *startPoi = FindPointOfInterest(mPointsOfInterest,

startPoiID); D3DXVECTOR3 origin = startPoi->GetPosition(); tPointOfInterestID endPoiID = SelectRandomPointOfInterest(mPointsOfInterest,

startPoiID); cPointOfInterest *endPoi = FindPointOfInterest(mPointsOfInterest, endPoiID); D3DXVECTOR3 destination = endPoi->GetPosition();

We then select a random starting point of interest and a random goal point of interest to start off with. // setup the squad leader mSquadLeader = new cSquadLeaderEntity ( *mWorld, kSquadType, kSenseDistance, kMaxVelocityChange, kMaxSpeed, kMaxSpeed * 0.4f, kMoveXScalar, kMoveYScalar, kMoveZScalar ); mSquadLeader->SetPointsOfInterest(&mPointsOfInterest); D3DXQUATERNION squadLeaderOrientation(0.0f, 0.0f, 1.0f, 0.0f); mSquadLeader->SetOrientation(squadLeaderOrientation); mSquadLeader->SetSelectedPointOfInterest(endPoi); cStateMachine *statemachine = new cSquadStateMachine(mSquadLeader); ifstream squadleaderfsm("SquadLeader.stm"); try { if (statemachine->UnSerialize(squadleaderfsm) == FALSE) { // error handling mojo... delete statemachine;

240

statemachine = NULL; } } catch(error_already_set) { // error handling mojo... delete statemachine; statemachine = NULL; } statemachine->Reset(); D3DXVECTOR3 squadleaderpos(34, 10, 0.0f); mSquadLeader->SetPosition(squadleaderpos); mSquadLeader->SetStateMachine(statemachine); mSquadLeader->SetWaypointNetwork(mWaypointNetwork); squadGroup->Add(*mSquadLeader);

Now we set up the squad leader. We set his points of interest, give him an initial position and orientation, and set his initial selected point of interest. We also load his state machine from a file (again, feel free to experiment with this machine using the State Machine demo from the last chapter) and set it. Finally, we give him the waypoint network and finally add him to the squad. // setup some squad mates to roam the network for (int i = 0; i < kNumSquadMembers; ++i)

Here we create a set of squad members. D3DXVECTOR3 pos(0.0f, (float)i, 0.0f); cSquadEntity *squadmate = new cSquadMemberEntity ( *mWorld, kSquadType, kSenseDistance, kMaxVelocityChange, kMaxSpeed, kMaxSpeed * 0.4f, kMoveXScalar, kMoveYScalar, kMoveZScalar ); statemachine = new cSquadStateMachine(squadmate); ifstream ar("SquadMember.stm"); try { if (statemachine->UnSerialize(ar) == FALSE) { // error handling mojo... delete statemachine; statemachine = NULL; } } catch(error_already_set) { // error handling mojo... delete statemachine;

241

statemachine = NULL; } squadmate->AddBehavior(*mPFbeh); squadmate->SetPosition(startPoi->GetPosition() + pos); squadmate->SetWaypointNetwork(mWaypointNetwork); statemachine->Reset(); squadmate->SetStateMachine(statemachine); squadGroup->Add(*squadmate); mSquadLeader->AddSquadMember(squadmate); if (!mSelectedEntity) mSelectedEntity = squadmate; cWaypointVisibilityFunctor visibilityFunctor; tPath thePath; if (mWaypointNetwork->FindPathFromPositionToPosition(origin,

destination, visibilityFunctor, thePath))

{ squadmate->SetPath(thePath); squadmate->SetGoal(endPoi->GetPosition()); }

For each squad member, we load his state machine from a file (which can be edited in the State Machine Editor from the previous chapter). We add the pathfinding behavior, initialize his position, set his waypoint network, set his state machine, and add him to the group. We also inform the squad leader about the new squad mate and find a path for him from his current position to the goal position of the initial goal point of interest. That is it! The demo is now initialized. There is one other place that needs noting -- the UpdateWorld method in the document. This method calls the Iterate method on the world object, which ensures the entities get iterated (making sure that their behaviors and state machines get iterated as well). This method gets called by a timer set in the waypoint network view, once every 33 milliseconds.

Conclusion

In this chapter, we discussed how to bring together the various AI components we learned about in this course to get them to cooperate in a single project. We talked about how we can build a waypoint network for traversing a continuous world, and a movement behavior for traversing the network. We looked at squad entities, and saw how we can use scripted state machines to drive their behavior. We even talked about squad leaders, and some different ways that we can get them to command their squads. Ultimately we built a practical example that demonstrated a squad examining points of interest using a waypoint network, while using scripting to extend our state machines. With the material discussed in this course, it should be possible to build a robust AI system for your games using the framework provided as a starting point. Hopefully you have found our discussions and demonstrations enjoyable. There is obviously a lot to learn as a game developer, but for many of us, AI

242

programming is certainly one of the most fun and exciting. This is because you really get a chance to be as creative as you want to be (within reason!) and see the results of your work acted out by the little virtual characters in your world. It is a very satisfying feeling to watch your team of AI entities travel from place to place in the world in a realistic manner, seeming to communicate and cooperate with one another, doing things that make you feel that these guys are really thinking for themselves. You will get the chance soon enough to experience this for yourself. And thus we have reached the end of our course, but certainly not to the end of the road. In this course we have covered a lot of the core AI topics that you will need to understand if you want to develop AI for games. But as mentioned right at the beginning, there is a lot more to study in the field of AI then what we were able to talk about in our short time together. At this point you have a very solid set of working code that can serve as the basis for further exploration and enhancement. If you enjoyed the materials we encountered and remain interested in taking your game AI even further, there are plenty of books and internet tutorials available for you to examine. And of course, if you do create some interesting AI for your game using the systems we discussed, please come by and let us know. We would love to hear about it and see your AI in action! We wish you the very best of luck in your future game programming adventures!