CS-378: Game Technology

Lecture #17: AI

Prof. Okan ArikanUniversity of Texas, Austin

Thanks to James O’Brien, Steve Chenney, Zoran Popovic, Jessica HodginsV2005-08-1.1

Fuzzy LogicPhilosophical approach

Decisions based on “degree of truth”

Is not a method for reasoning under uncertainty – that’s probability

Crisp Facts – distinct boundaries

Fuzzy Facts – imprecise boundaries

Probability - incomplete facts

Example – Scout reporting an enemy

“Two tanks at grid NV 54“ (Crisp)

“A few tanks at grid NV 54” (Fuzzy)

“There might be 2 tanks at grid NV 54 (Probabilistic)

Apply to Computer Games

Can have different characteristics of players

Strength: strong, medium, weak

Aggressiveness: meek, medium, nasty

If meek and attacked, run away fast

If medium and attacked, run away slowly

If nasty and strong and attacked, attack back

Control of a vehicle

Should slow down when close to car in front

Should speed up when far behind car in front

Provides smoother transitions – not a sharp boundary

Fuzzy SetsProvides a way to write symbolic rules with terms like “medium” but evaluate them in a quantified way

Classical set theory: An object is either in or not in the set

Can’t talk about non-sharp distinctions

Fuzzy sets have a smooth boundary

Not completely in or out – somebody 6” is 80% in the tall set tall

Fuzzy set theory

An object is in a set by matter of degree

1.0 => in the set

0.0 => not in the set

0.0 < object < 1.0 => partially in the set

Example Fuzzy Variable

Each function tells us how much we consider a character in the set if it has a particular aggressiveness value

Or, how much truth to attribute to the statement: “The character is nasty (or meek, or neither)?”

Aggressiveness

Membership (Degree of Truth)

1.0

0.0 1 0.5

MediumNastyMeek

-1 -0.5

Fuzzy Set Operations: Complement

The degree to which you believe something is not in the set is 1.0 minus the degree to which you believe it is in the set

Membership

Units

1.0

0.0

FS

¬FS

Fuzzy Set Ops: Intersection (AND)

If you have x degree of faith in statement A, and y degree of faith in statement B, how much faith do you have in the statement A and B?

Eg: How much faith in “that person is about 6’ high and tall”

Does it make sense to attribute more truth than you have in one of A or B?

Membership

Height

1.0

0.0

About 6’ Tall

Assumption: Membership in one set does not affect membership in another

Take the min of your beliefs in each individual statement

Also works if statements are about different variables

Dangerous and injured - belief is the min of the degree to which you believe they are dangerous, and the degree to which you think they are injured

Membership

Height

1.0

0.0

About 6’ Tall

About 6’ high and tall

Fuzzy Set Ops: Intersection (AND)

Fuzzy Set Ops: Union (OR)

If you have x degree of faith in statement A, and y degree of faith in statement B, how much faith do you have in the statement A or B?

Eg: How much faith in “that person is about 6’ high or tall”

Does it make sense to attribute less truth than you have in one of A or B?

Membership

Height

1.0

0.0

About 6’ Tall

Take the max of your beliefs in each individual statement

Also works if statements are about different variables

Membership

Height

1.0

0.0

About 6’ Tall

About 6’ high or tall

Fuzzy Set Ops: Union (OR)

Fuzzy Rules

“If our distance to the car in front is small, and the distance is decreasing slowly, then decelerate quite hard”

Fuzzy variables in blue

Fuzzy sets in red

Conditions are on membership in fuzzy sets

Actions place an output variable (decelerate) in a fuzzy set (the quite hard deceleration set)

We have a certain belief in the truth of the condition, and hence a certain strength of desire for the outcome

Multiple rules may match to some degree, so we require a means to arbitrate and choose a particular goal - defuzzification

Fuzzy Rules Example(from Game Programming Gems)

Rules for controlling a car:

Variables are distance to car in front and how fast it is changing, delta, and acceleration to apply

Sets are:

Very small, small, perfect, big, very big - for distance

Shrinking fast, shrinking, stable, growing, growing fast for delta

Brake hard, slow down, none, speed up, floor it for acceleration

Rules for every combination of distance and delta sets, defining an acceleration set

Assume we have a particular numerical value for distance and delta, and we need to set a numerical value for acceleration

Extension: Allow fuzzy values for input variables (degree to which we believe the value is correct)

Set Definitions for Example

distance

v. small small perfect big v. big

delta

< = > >>

acceleration

slow present fast fastest

<<

brake

Instance for Example

Distance could be considered small or perfect

Delta could be stable or growing

What acceleration?

distance

v. small small perfect big v. big

delta

< = > >>

acceleration

slow present fast fastest

<<

brake

????

Matching for Example

Relevant rules are:

If distance is small and delta is growing, maintain speed

If distance is small and delta is stable, slow down

If distance is perfect and delta is growing, speed up

If distance is perfect and delta is stable, maintain speed

For first rule, distance is small has 0.75 truth, and delta is growing has 0.3 truth

So the truth of the and is 0.3

Other rule strengths are 0.6, 0.1 and 0.1

Fuzzy Inference for Example

Convert our belief into action

For each rule, clip action fuzzy set by belief in rule

acceleration

present

acceleration

presentacceleration

slow

acceleration

fast

Defuzzification Example

Three actions (sets) we have reason to believe we should take, and each action covers a range of values (accelerations)

Two options in going from current state to a single value:

Mean of Max: Take the rule we believe most strongly, and take the (weighted) average of its possible values

Center of Mass: Take all the rules we partially believe, and take their weighted average

In this example, we slow down either way, but we slow down more with Mean of Max

Mean of max is cheaper, but center of mass exploits more information

Evaluation of Fuzzy LogicDoes not necessarily lead to non-determinism

Advantages

Allows use of continuous valued actions while still writing “crisp” rules – can accelerate to different degrees

Allows use of “fuzzy” concepts such as medium

Biggest impact is for control problems

Help avoid discontinuities in behavior

In example problem strict rules would give discontinuous acceleration

Disadvantages

Sometimes results are unexpected and hard to debug

Additional computational overhead

There are other ways to get continuous acceleration

References

Nguyen, H. T. and Walker, E. A. A First Course in Fuzzy Logic, CRC Press, 1999.

Rao, V. B. and Rao, H. Y. C++ Neural Networks and Fuzzy Logic, IGD Books Worldwide, 1995.

McCuskey, M. Fuzzy Logic for Video Games, in Game Programming Gems, Ed. Deloura, Charles River Media, 2000, Section 3, pp. 319-329.

Neural NetworksInspired by natural decision making structures (real nervous systems and brains)

If you connect lots of simple decision making pieces together, they can make more complex decisions

Compose simple functions to produce complex functions

Neural networks:

Take multiple numeric input variables

Produce multiple numeric output values

Normally threshold outputs to turn them into discrete values

Map discrete values onto classes, and you have a classifier!

But, the only time I’ve used them is as approximation functions

Simulated Neuron - Perceptron

Simulated Neuron - Perceptron

Network Structure

Single perceptron can represent AND or OR, but not XOR

Combinations of perceptron are more powerful

Perceptron are usually organized on layers

Input layer: takes external input

Hidden layer(s)

Output layer: external output

Feed-forward vs. recurrent

Feed-forward: outputs only connect to later layers

Learning is easier

Recurrent: outputs can connect to earlier layers or same layer

Internal state

Neural network for QuakeFour input perceptron

One input for each condition

Four perceptron hidden layer

Fully connected

Five output perceptron

One output for each action

Choose action with highest output

Or, probabilistic action selection

Choose at random weighted by output

Enemy Sound

Dead

Low Health

Attack

Retreat

Wander

Chase

Spawn

Neural Networks Evaluation

Advantages

Handle errors well

Graceful degradation

Can learn novel solutions

Disadvantages

“Neural networks are the second best way to do anything”

Can’t understand how or why the learned network works

Examples must match real problems

Need as many examples as possible

Learning takes lots of processing

Incremental so learning during play might be possible

References

Mitchell: Machine Learning, McGraw Hill, 1997.

Russell and Norvig: Artificial Intelligence: A Modern Approach, Prentice Hall, 1995.

Hertz, Krogh & Palmer: Introduction to the theory of neural computation, Addison-Wesley, 1991.

Cowan & Sharp: Neural nets and artificial intelligence, Daedalus 117:85-121, 1988.

Rule Based Systems

And decision trees coming next lecture …

Now, some lower level issues …

Path Finding

Very common problem in games:

In FPS: How does the AI get from room to room?

In RTS: User clicks on units, tells them to go somewhere. How do they get there? How do they avoid each other?

Chase games, sports games, …

Very expensive part of games

Lots of techniques that offer quality, robustness, speed trade-offs

Path Finding ProblemProblem Statement (Academic): Given a start point, A, and a goal point, B, find a path from A to B that is clear

Generally want to minimize a cost: distance, travel time, …

Travel time depends on terrain, for instance

May be complicated by dynamic changes: paths being blocked or removed

May be complicated by unknowns – don’t have complete information

Problem Statement (Games): Find a reasonable path that gets the object from A to B

Reasonable may not be optimal – not shortest, for instance

It may be OK to pass through things sometimes

It may be OK to make mistakes and have to backtrack

Search or Optimization?

Path planning (also called route-finding) can be phrased as a search problem:

Find a path to the goal B that minimizes Cost(path)

There are a wealth of ways to solve search problems, and we will look at some of them

Path planning is also an optimization problem:

Minimize Cost(path) subject to the constraint that path joins A and B

State space is paths joining A and B, kind of messy

There are a wealth of ways to solve optimization problems

The difference is mainly one of terminology: different communities (AI vs. Optimization)

But, search is normally on a discrete state space

Brief Overview of Techniques

Discrete algorithms: BFS, Greedy search, A*, …

Potential fields:

Put a “force field” around obstacles, and follow the “potential valleys”

Pre-computed plans with dynamic re-planning

Plan as search, but pre-compute answer and modify as required

Special algorithms for special cases:

E.g. Given a fixed start point, fast ways to find paths around polygonal obstacles

Graph-Based AlgorithmsIdeally, path planning is point to point (any point in the world to any other, through any unoccupied point)

But, the search space is complex (space of arbitrary curves)

The solution is to discretize the search space

Restrict the start and goal points to a finite set

Restrict the paths to be on lines (or other simple curves) that join points

Form a graph: Nodes are points, edges join nodes that can be reached along a single curve segment

Search for paths on the graph

Waypoints (and Questions)

The discrete set of points you choose are called waypoints

Where do you put the waypoints?

There are many possibilities

How do you find out if there is a simple path between them?

Depends on what paths you are willing to accept - almost always assume straight lines

The answers to these questions depend very much on the type of game you are developing

The environment: open fields, enclosed rooms, etc…

The style of game: covert hunting, open warfare, friendly romp, …

Where Would You Put Waypoints?

Waypoints By HandPlace waypoints by hand as part of level design

Best control, most time consuming

Many heuristics for good places:

In doorways, because characters have to go through doors and straight lines joining rooms always go through doors

Along walls, for characters seeking cover

At other discontinuities in the environments (edges of rivers, for example)

At corners, because shortest paths go through corners

The choice of waypoints can make the AI seem smarter

Waypoints By GridPlace a grid over the world, and put a waypoint at every gridpoint that is open

Automated method, and maybe even implicit in the environment

Do an edge/world intersection test to decide which waypoints should be joined

Normally only allow moves to immediate (and maybe corner) neighbors

What sorts of environments is this likely to be OK for?

What are its advantages?

What are its problems?

Grid Example

Note that grid points pay no attention to the geometry

Method can be improved:

Perturb grid to move closer to obstacles

Adjust grid resolution

Use different methods for inside and outside building

Join with waypoints in doorways

Waypoints From Polygons

Choose waypoints based on the floor polygons in your world

Or, explicitly design polygons to be used for generating waypoints

How do we go from polygons to waypoints?

Hint: there are two obvious options

Waypoints From Polygons

!

Could also add points on walls

Waypoints From CornersPlace waypoints at every convex corner of the obstacles

Actually, place the point away from the corner according to how wide the moving objects are

Or, compute corners of offset polygons

Connects all the corners that can see each other

Paths through these waypoints will be the shortest

However, some unnatural paths may result

Particularly along corridors - characters will stick to walls

Waypoints From CornersNOTE: Not every edge is drawn

Produces very dense graphs

Getting On and OffTypically, you do not wish to restrict the character to the waypoints or the graph edges

When the character starts, find the closest waypoint and move to that first

Or, find the waypoint most in the direction you think you need to go

Or, try all of the potential starting waypoints and see which gives the shortest path

When the character reaches the closest waypoint to its goal, jump off and go straight to the goal point

Best option: Add a new, temporary waypoint at the precise start and goal point, and join it to nearby waypoints

Getting On and Off

Best-First-SearchStart at the start node and search outwards

Maintain two sets of nodes:

Open nodes are those we have reached but don’t know best path

Closed nodes that we know the best path to

Keep the open nodes sorted by cost

Repeat: Expand the “best” open node

If it’s the goal, we’re done

Move the “best” open node to the closed set

Add any nodes reachable from the “best” node to the open set

Unless already there or closed

Update the cost for any nodes reachable from the “best” node

New cost is min(old-cost, cost-through-best)

Best-First-Search Properties

Precise properties depend on how “best” is defined

But in general:

Will always find the goal if it can be reached

Maintains a frontier of nodes on the open list, surrounding nodes on the closed list

Expands the best node on the frontier, hence expanding the frontier

Eventually, frontier will expand to contain the goal node

To store the best path:

Keep a pointer in each node n to the previous node along the best path to n

Update these as nodes are added to the open set and as nodes are expanded (whenever the cost changes)

To find path to goal, trace pointers back from goal nodes

Expanding Frontier

Definitionsg(n): The current known best cost for getting to a node from the start point

Can be computed based on the cost of traversing each edge along the current shortest path to n

h(n): The current estimate for how much more it will cost to get from a node to the goal

A heuristic: The exact value is unknown but this is your best guess

Some algorithms place conditions on this estimate

f(n): The current best estimate for the best path through a node: f(n)=g(n)+h(n)

Using g(n) OnlyDefine “best” according to f(n)=g(n), the shortest known path from the start to the node

Equivalent to breadth first search

Is it optimal?

When the goal node is expanded, is it along the shortest path?

Is it efficient?

How many nodes does it explore? Many, few, …?

Behavior is the same as defining a constant heuristic function: h(n)=const

Why?

Breadth First Search

Breadth First Search

Using h(n) Only (Greedy Search)

Define “best” according to f(n)=h(n), the best guess from the node to the goal state

Behavior depends on choice of heuristic

Straight line distance is a good one

Have to set the cost for a node with no exit to be infinite

If we expand such a node, our guess of the cost was wrong

Do it when you try to expand such a node

Is it optimal?

When the goal node is expanded, is it along the shortest path?

Is it efficient?

How many nodes does it explore? Many, few, …?

Greedy Search (Straight-Line-Distance Heuristic)

A* SearchSet f(n)=g(n)+h(n)

Now we are expanding nodes according to best estimated total path cost

Is it optimal?

It depends on h(n)

Is it efficient?

It is the most efficient of any optimal algorithm that uses the same h(n)

A* is the ubiquitous algorithm for path planning in games

Much effort goes into making it fast, and making it produce pretty looking paths

More articles on it than you can ever hope to read

A* Search (Straight-Line-Distance Heuristic)

A* Search (Straight-Line-Distance Heuristic)

Note that A* expands fewer nodes than breadth-first, but more than greedy

It’s the price you pay for optimality

See Game Programming Gems for implementation details. Keys are:

Data structure for a node

Priority queue for sorting open nodes

Underlying graph structure for finding neighbors

Heuristics

For A* to be optimal, the heuristic must underestimate the true cost

Such a heuristic is admissible

Also, not mentioned in Gems, the f(n) function must monotonically increase along any path out of the start node

True for almost any admissible heuristic, related to triangle inequality

If not true, can fix by making cost through a node max(f(parent) + edge, f(n))

Combining heuristics:If you have more than one heuristic, all of which underestimate, but which give different estimates, can combine with: h(n)=max(h1(n),h2(n),h3(n),…)

Inventing HeuristicsBigger estimates are always better than smaller ones

They are closer to the “true” value

So straight line distance is better than a small constant

Important case: Motion on a grid

If diagonal steps are not allowed, use Manhattan distance

General strategy: Relax the constraints on the problem

For example: Normal path planning says avoid obstacles

Relax by assuming you can go through obstacles

Result is straight line distance

is a bigger estimate than

Non-Optimal A*Can use heuristics that are not admissible - A* will still give an answer

But it won’t be optimal: May not explore a node on the optimal path because its estimated cost is too high

Optimal A* will eventually explore any such node before it reaches the goal

Non-admissible heuristics may be much faster

Trade-off computational efficiency for path-efficiency

One way to make non-admissible: Multiply underestimate by a constant factor

See Gems for an example of this