Artificial Intelligence in Game Design

Heuristics and Other Ideas in Board Games

Good and Bad Heuristics

• Heuristic for evaluating board must be accurate– Directly related to “likelihood” of win– Inversely related to “distance” from win

• Example: TicTacToe heuristicH(board) = 2 × # of possible rows/columns/diagonals where X could win in one move + + 1 × # of possible rows/columns/diagonals where X could win in two moves + - 2 × # of possible rows/columns/diagonals where O could win in one move + - 1 × # of possible rows/columns/diagonals where O could win in two moves

• Gives these two boards same measure

-2-2

X

O X

O X O

O X

X

O X O

Guaranteed loss for AI

Good and Bad Heuristics

• Better heuristic must take this into account!• H(board) =

– if my move next and I have 2 in row MAXINT

– if opponent move next and they have 2 in row -MAXINT

– if my move and opponent has > 1 instance of 2 in row -MAXINT

– if opponent move and I have > 1 instance of 2 in row MAXINT

– else (above function)

• Will work better, but more complex to compute!

Speed to computeheuristic value

Accuracy ofheuristic value tradeoff

Linear Heuristic Functions

• Heuristic is some function of individual pieces on board– Usually weighted in some way

– Overall board value = Σ H(piecei)

i

• Very fast to compute

• Example: Chess– Pieces have “standard values”

– Heuristic = sum of AI pieces – sum of player pieces

9 5 3 3 1

Linear Heuristic Functions

• Often based on position of pieces on board• Examples:

– Games where purpose is to move all pieces to some goal

• Backgammon

• Sorry

• Can often send other player back to “start”

– Heuristic value = total distance of AI pieces from goal – total distance of player pieces from goal

Linear Heuristic Functions

• Reversi (Othello)– Board value of based entirely on piece positions

• Corners very valuable (can’t be flipped)• Sides somewhat valuable (very difficult to flip)• Middle little value (will easily be flipped)

– H(board) = C1 * number of pieces in corner +

C2 * number of pieces on side +

C3 * number of pieces in middle +

C1 >> C2 >> C3

• Reversi easiest type of game for AI– Low branching factor (5 to 15 possible moves)

– Good heuristics

– Single move can greatly change board• Hard for human player to see• Easy for MinMax lookahead to see

Nonlinear Heuristics

• Based on relationships between pieces on board• Simple example:

– Prefer chess pieces to protect one another– Piece value = piece value * 1.5 if protected by another– Piece value = piece value * 0.5 if not protected and threatened

by opponent piece

Nonlinear Heuristics

• Drawback: Usually much more expensive to compute– n pieces O (n2) relationships between them

• May be able to explore more moves with simpler (linear) heuristic

• Example:– Simple linear heuristic takes k ms to evaluate per board– Complex nonlinear heuristic takes 400k ms to evaluate per

board– Average of 20 possible next moves per board

• 400 possible next two moves

– On average, could explore two additional moves if use linear heuristic

Linear vs. Nonlinear Heuristics

• Nonlinear heuristics often detect future changes in board– Pieces threatened by others might be captured– Pieces protected by others less likely to be captured

• Often see effect of nonlinear heuristic with additional levels of linear heuristic

• Example: “knight fork” in chess– Can look for this using nonlinear

heuristic

Linear vs. Nonlinear Heuristics

• Searching 2 more moves in game tree will show result of knight fork– Shows state where rook now gone

– Higher value for linear heuristic based on piece values

– MinMax then gives “knight fork” state a high value

Board with knight fork

Board with knight threatening rook

Black moves king

Other moves give checkmate (value = MAXINT)

Board with blackrook gone

Knight takes rook

High value h

High value h

High value h

Max level

Min level

Horizon Effect

• Major weakness of purely linear heuristics based on piece values– “Throwing good money after bad”

• May need to recognize when move does not improve board position• Chess example:

Queen pinned by bishops Could move rook in way But rook captured and will still lose queen

Horizon Effect

Board with queen pinned

Queen takes bishop

Board with queen lost

Bishop takes queen

White down 6 points

Move rook in front

Board with rook lost

Bishop takes rook

White down 5 points

Queen takes bishop

Board with queen and

rook lost

Bishop takes queen

White now down 11 points!

This looks worse at cutoff level, but is actually best in long run!

Trappy Minmax

• Idea: Use minmax to set traps for player– Branch appears to have short-term benefit for player– In long run, branch has benefit for AI

AI move

player move

Ok result for AI Great result for AI

This state looks good to player

This state looks ok to player

Good chance player might choose this branch

So worth considering making this move

Trappy Minmax

Queen cancapture rook

Board with rook ignored

Some other move

White unchanged Board with rook captured

Queen takes rook

White up 5 points

Board with queen lost

Knight captures queen

White now down 4 points!

Player did not lookahead far enough to see this

Takes advantage of player horizon effect

Factors in Trappy Minmax

• Trappiness: Estimate of how likely player will choose branch corresponding to trap– Usually computed based on median or maximum of player

heuristics down best branch

5

9

8

-7

Possible player move

Path player thinks game will follow if they make this move

6

Median = 7

2

Factors in Trappy Minmax

• Profitability: Score if player follows “trap” branch – score if player follows their best branch

• Trap Quality: Trappiness * Profitability

• Use trap quality to adjust heuristic measure of a state:

Trappy_heuristic(state) = normal_heuristic(state) + f(trap_quality(state))

Data-Driven Approaches

Basing actions on known strategies rather than tree search

• Opening books of initial moves (Chess)– Often 20-30 moves at start of Grandmaster match

• Each book consists of:– List of moves

– Evaluation of final outcome• Should we follow this strategy

• Allows faster processing– No need to search game tree until

end of sequence

– Can just use evaluation as heuristic

Currentboard

Boards in sequence

End of sequence

Now start branching

Opening Books

• Choose as own opening strategy• Must have good final evaluation!

– Make moves according to script– If opponent follows script, keep following– If opponent leaves script, start MinMax

• Will probably be in same way that benefits us

• Must also recognize when opponent uses an opening book– Keep database of moves in opening books– Match current board to those in database to find whether it is part of

a sequence– If so, make decision about whether following script is good idea

Other Set Plays

• End Games– Many games have different strategies when few pieces left

• Forcing checkmate in chess• Kings vs. kings in checkers• Getting last pieces home in backgammon

– Recognize based on pieces left

– Follow set strategies

• Set Evaluation Values– No heuristic evaluation of board – instead,

match board to database to get evaluation

– Works best if can match subboards• Example: Edge configurations in Othello• This edge has a known value

Alternative Approaches

• Go – 19 19 board branching factor of 361

– Impossible for MinMax

• Only known approaches based on template matching– Look in local area for configurations

that match known strategies

– Still very open problem• Best AI for Go only plays at

amateur level