GAMES WITH A HISTORY - Essential...

Essential Microeconomics -1-

© John Riley January 8, 2014

GAMES WITH A HISTORY

Finitely repeated games 2

Commitment? 5

Equilibrium threats 13

Sequential move games 16

Sub-game perfect equilibrium 27

One-stage deviation principle 30



FINITELY REPEATED GAMES

Stage Game: Simultaneous move game played in each of T rounds or “stages”.

Strategy space in the t-th stage game: 1

...t

IS S S

Strategy profile 1( ,..., )Ts s s 1 ... TS S S

**






...t

IS S S


Payoffs: The stage t payoff to player i J is ( )t

iu s .

Future payoffs are discounted at the rate . Therefore if the strategy profile is sS , the payoff to

player i is

1

1

( ) ( )T

t t

i it

U s u s

where t t

iis S S

I

*






...t

IS S S


Payoffs: The stage t payoff to player i J is ( )t

iu s .

Future payoffs are discounted at the rate . Therefore if the strategy profile is sS , the payoff to

player i is

1

1

( ) ( )T

t t

i it

U s u s

where t t

iis S S

I

History

Player i can base her strategy in stage t on the prior actions of players, that is, on the history t

ih of the

game. Initially we shall assume that players observe the prior actions of all their opponents * .

Then 1 1( ,..., )t t

ih a a .

*A player who is randomizing could also make his mixed strategy public by allowing a third party to monitor his randomization device. For the cases we shall

consider, the equilibrium strategies are pure strategies so observing past actions is the same as observing past strategies.



Commitment?

Consider this simple stage game.

Regardless of player 2’s strategy, player 1’s

best response is U. Thus her NE best response is U.

And given that player 1 chooses U,

player 2’s best response is L. Thus

the unique NE is of the stage game is * ( , )s U L .

Player 2

L R

Player 1

U 4,4 2,2

D 3,5 1,1



Two stage game

Next suppose that the stage game is played twice.

For simplicity assume no discounting.

Stage 1 strategies:

1 1 1

1 2( , )s s s

Stage 2 strategies

In stage 2 the history of the game is 2 1h s so the strategy of player i is

2 2 2 1( ) ( )i is h s s .

It is readily confirmed that mutual best responses are to ignore history and repeat the unique NE

strategy of the stage game.

That is, the following strategy profile is an NE.

1 * 2 2 *, ( )s s s h s where * ( , )s U L

Player 2

L R

Player 1

U 4,4 2,2

D 3,5 1,1



We now show that the following strategy profile is also a NE.

Player 1: 1 1 1

1 2, ( )s D s h L .

Player 2: 1

2s L , 1

2 1

2 1

, if ( , )( )

, if ( , )

L h D Ls h

R h D L

Player 2 would like player 1 to choose D in the first stage so that his payoff is 5. He achieves this with

the threat to play R in the second stage if player 1 does not play D in the first stage.

Player 2

L R

Player 1

U 4,4 2,2

D 3,5 1,1



Player 1: 1 1 2

1 2, ( )s D s h U .

Player 2 1

2s L , 2

2 2

2 2

, if ( , )( )

, if ( , )

L h D Ls h

R h D L

Best response by player 2:

In the first stage player 2’s best response is L. Then 2 1 ( , )h s D L and so 2 2

2 ( )s h L . Player 1’s best

response in the second stage is therefore U

Best response by player 1:

If player 1 chooses U in the first stage then 1 ( , )s U L so player 2 chooses R in the second stage.

Hence player 1’s best response in the second stage is U.

Player 1’s total payoff is therefore 4+2 = 6.

If player 1 chooses D in the first stage then 1 ( , )s D L so player 2 chooses L in the second stage.

Hence player 1’s best response in the second stage is U. Her total payoff is therefore 3+4 =7.

Thus 1 1 2

1 2, ( )s D s h U is indeed a best response.

Player 2

L R

Player 1

U 4,4 2,2

D 3,5 1,1



In most economic games this kind of outcome seems distinctly unlikely. Crucially it requires that

player 2 can commit to her strategy. For if not player 1 can reason as follows.

“When stage 2 is reached player 2’s strictly dominant strategy is L so she will never follow through on

her threat. Thus player 2’s threat to play R is not credible.”

Equilibrium in the final stage without commitment

Let 1 2 2, ( )s s h be a NE of the two stage game.

In the absence of commitment, the equilibrium strategy in the second stage 2 2( )s h must also be a NE

of the stage game.

In the example the stage game has a unique NE * ( , )s U L therefore 2 2( ) ( , )s h U L

Thus the NE strategy of the two stage game is 1 2 2 1 *{ , ( )} { , }s s h s s .

But 1 2 2 1 *( ) ( ) ( ( )) ( ) ( )i i i i iU s u s u s h u s u s

The last term is a constant so the payoff in the two stage game is the payoff in the one stage game plus

a constant.



Adding a constant to the stage game payoffs does not change the payoff to deviating.

Since *s is the unique equilibrium of the stage game it follows that 1 *s s .

Proposition 9.2-1: Nash equilibria of a finitely repeated game

Suppose that , 1,...,ts t T is a NE of the stage game. Then (i) the strategy profile 1( ,..., )Ts s s is a

NE of the repeated game and (ii) in the absence of commitment, if *s is the unique NE of the stage

game then * *( ,..., )s s is the unique NE of the repeated game.

Proof of (i):

1 1 1( , ) ( ) ... ( ( ), ) ... ( ( ), )t t t T T T

i i i i i i i i i iU s s u s u s h s u s h s

Since ( , )t t

i is s is an equilibrium of the stage game ( ( ), ) ( , )t t t t

i i i i i iu s h s u s s .

Since this holds for all t,

1 1 1( , ) ( ) ... ( ) ... ( )t t T T

i i i i i iU s s u s u s u s



Proof of (ii):

Let 1( ,..., ( ))T Ts s s h be an NE of the finitely repeated game.

1 1 1( ) ( ) ... ( ( ), ) ... ( ( ), ( ))t t t T T T T

i i i i i i i iU s u s u s h s u s h s h

We apply a simple backwards induction argument.

In the absence of commitment ( )T Ts h must be an equilibrium of the stage game played in the T-th

stage. By hypothesis the unique NE of the stage game is *s so *( )T Ts h s .

Therefore

1 1 1 * *( ) ( ) ... ( ( ), ( )) ... ( , )t t t t T

i i i i i i i iU s u s u s h s h u s s

The last term is a constant. Thus the T stage game reduces to a T-1 stage game. In the absence of

commitment 1 1( )T Ts h must be an equilibrium of the stage game played in stage T-1. By hypothesis

the unique NE of the stage game is *s so 1 1 *( )T Ts h s .

Applying this argument inductively, it follows that *( ) , 1,...,t ts h s t T



Repeated games with multiple equilibria of the stage game

Example: Two-stage repeated partnership game

player i’s strategy : 1 2 2( , ( ))i i i i i i

s s s h S S .

player i’s payoff: 1 1 2 2( , ) ( , ) ( , )i i i i i i i i i

U s s u s s u s s where 2 2

1( , ( ))t

i i i i is s s h S S

Class Exercise:

(a) What are the NE strategies of the stage game?

(b) What are some NE strategies of the

repeated game?

Player 2

2 1a

2 2a 2 3a

1 1a 5,5 11,4 17,-9

Player 1 1 2a 4,11 16,16 28,9

1 3a -9,17 9,28 27,27

2 1a 2 2a 2 3a

1 1a 5,5 11,4 17,-9

Player 1 1 2a 4,11 16,16 28,9

1 3a -9,17 9,28 27,27



Equilibrium threats

We now argue that, for any sufficiently high discount factor, if the stage game has multiple

equilibria with different equilibrium payoffs, a player can use the threat to play the bad NE in the later

periods period to induce a more favorable behavior in the first period.

Consider the two-stage partnership game.

**

Player 2

2 1a

2 2a 2 3a

1 1a 5,5 11,4 17,-9

Player 1 1 2a 4,11 16,16 28,9

1 3a -9,17 9,28 27,27

2 1a 2 2a 2 3a

1 1a 5,5 11,4 17,-9

Player 1 1 2a 4,11 16,16 28,9

1 3a -9,17 9,28 27,27



Equilibrium threats





Let ( )t t

is h be the strategy of player i in period t

Consider the following strategy:

1 3i

s , 2 2 2( ) 2 if (3,3)i

s h h , 2 2 2( ) 1 if (3,3)i

s h h .

Class discussion:

Is ( , )s s a NE strategy profile.

*

Player 2

2 1a

2 2a 2 3a

1 1a 5,5 11,4 17,-9

Player 1 1 2a 4,11 16,16 28,9

1 3a -9,17 9,28 27,27

2 1a 2 2a 2 3a

1 1a 5,5 11,4 17,-9

Player 1 1 2a 4,11 16,16 28,9

1 3a -9,17 9,28 27,27



Equilibrium threats





Let ( )t t

is h be the strategy of player i in period t

Consider the following strategy:

1 3i

s , 2 2 2( ) 2 if (3,3)i

s h h , 2 2 2( ) 1 if (3,3)i

s h h .

Argue that ( , )s s is a NE strategy profile.

Note that with more than 2 periods, the threat to switch thereafter to action 1 is an even stronger

deterrent.

Player 2

2 1a

2 2a 2 3a

1 1a 5,5 11,4 17,-9

Player 1 1 2a 4,11 16,16 28,9

1 3a -9,17 9,28 27,27

2 1a 2 2a 2 3a

1 1a 5,5 11,4 17,-9

Player 1 1 2a 4,11 16,16 28,9

1 3a -9,17 9,28 27,27



Sequential move games

We now consider games in

which players move sequentially.

As an example, consider the

three-player penny matching

game. But now players move

one at a time.

*

Fig. 9.2-1: Game tree

(0,2,1)

(2,2,2)

(0,0,0)

(1,0,4)

(-1,3,3)

(0,0,0)

(0,0,0)

(0,0,0) T

H

H

H

T

T H

T

T

H

H

H T

T 3

3

2

3

2

3

1



Sequential move games

We now consider games in

which players move sequentially.

As an example, consider the

three-player penny matching

game. But now players move

one at a time.

Backwards induction

The key to solving sequential move

games is to begin by examining the

final stage of the game and then to

work backwards.

Arrows in the top figure indicate branches that are best responses for player 3.

Replace the third node by the best response payoffs….continue…

Fig. 9.2-1: Game tree

(0,2,1)

(2,2,2)

(0,0,0)

(1,0,4)

(-1,3,3)

(0,0,0)

(0,0,0)

(0,0,0) T

H

H

H

T

T H

T

T

H

H

H T

T 3

3

2

3

2

3

1

Fig. 9.2-2: Backwards induction

(2,2,2)

(4,0,1)

(0,0,0)

(0,2,1 )

H T 1 2

H

T

H

T

2



Language of a sequential move game

Nodes: At each node of the tree one player takes an action

******

2

T

B

(2,2)

(24,1)

(8,1)

(0,3 )

T

1

T

B

B

2 2

2

1




Nodes: At each node of the tree one player takes an action.

Branches: Each action is depicted as a branch in the tree

*****

2

T

B

(2,2)

(24,1)

(8,1)

(0,3 )

T

1

T

B

B

2 2

2

1






Initial node: Starting point of the game is called the initial node.

Convention: The player making the choice at the initial node is called player 1.

Other players are labeled according to the order of their first moves

****

2

T

B

(2,2)

(24,1)

(8,1)

(0,3 )

T

1

T

B

B

2 2

2

1









Terminal node: The end of each branch from the last decision node. This is a possible outcome of

the game. Associated with each outcome is a vector of payoffs ( )u a .

***

2

T

B

(2,2)

(24,1)

(8,1)

(0,3 )

T

1

T

B

B

2 2

2

1











Action profile: List of actions leading from the initial node to any terminal node.

2

T

B

(2,2)

(24,1)

(8,1)

(0,3 )

T

1

T

B

B

2 2

2

1



**












Extensive form: The depiction of the game as a tree is called the extensive form representation of

the game or more simply the “extensive form” of the game.

*

2

T

B

(2,2)

(24,1)

(8,1)

(0,3 )

T

1

T

B

B

2 2

2

1












Extensive form: The depiction of the game as a tree is called the extensive form representation of

the game or more simply the “extensive form” of the game.

NOTE: All nodes except the initial node are connected to a single prior node and at least

two successor nodes. This is what gives the graph its tree structure.

2

T

B

(2,2)

(24,1)

(8,1)

(0,3 )

T

1

T

B

B

2 2

2

1



Sub-games

As we now show, there can be NE of sequential move games that are highly implausible. We present

an example and then show that by introducing a modest “refinement” of the NE, the silly equilibrium

is eliminated. In the example, an Incumbent firm faces a potential Entrant. The game tree is depicted

in Fig. 9.2-3. Player 1 is the Entrant. Player 2 is the Incumbent.

Claims: (i) There are two NE. (ii) Only one is plausible.

*

Out

Enter

Fight

Share

(0,6)

(-2,1)

(3,3)

1

2

Fig. 9.2-3: Entry Game with sub-game



Sub-games

As we now show, there can be NE of sequential move games that are highly implausible. We present

an example and then show that by introducing a modest “refinement” of the NE, the silly equilibrium

is eliminated. In the example, an Incumbent firm faces a potential Entrant. The game tree is depicted

in Fig. 9.2-3. Player 1 is the Entrant. Player 2 is the Incumbent.

Claims: (i) There are two NE. (ii) Only one is plausible.

Note that the part of the tree inside the dotted rectangle

looks just like a game. This has all the requirements of a game;

an initial node and terminal nodes.

Out

Enter

Fight

Share

(0,6)

(-2,1)

(3,3)

1

2




Definition: Sub-game

Any branch of a game tree that begins with a single node is a sub-game.

Sub-game Perfect Equilibrium (SPE)

Consider the sub-game inside the dashed rectangle. We can

strengthen our definition of strategic equilibrium by requiring

that the NE strategies must also be equilibrium strategies of

each sub-game.

*

Out

Enter

Fight

Share

(0,6)

(-2,1)

(3,3)

1

2




Definition: Sub-game

Any branch of a game tree that begins with a single node is a sub-game.

Sub-game Perfect Equilibrium (SPE)

Consider the sub-game inside the dashed rectangle. We can

strengthen our definition of strategic equilibrium by requiring

that the NE strategies must also be equilibrium strategies of

each sub-game.

Definition: Sub-game perfect equilibrium

A NE strategy profile of a sequential move game is sub-game perfect if the strategy profile is also a

NE strategy of each of the sub-games.

Out

Enter

Fight

Share

(0,6)

(-2,1)

(3,3)

1

2




For our example, the unique equilibrium of the single sub-game

is for player 2 to choose Share. We can then lop off this part of

the tree and replace it by the NE payoffs of the sub-game.

Player 1’s unique SPE strategy is therefore to choose Enter.

1

Out

Enter

(0,6)

(3,3)

Fig. 9.2-4: Entry Game with payoffs from sub-game

1 1



The one-stage deviation principle

There is an easy way to check whether a strategy profile is sub-game perfect. It is enough to consider

one-stage deviations by players at each decision node.

Proposition 9.2-2: One-stage deviation principle

In a T stage sequential move game, suppose that for the strategy profile 1 2 2( , ( ),.., ( ))T Ts s s h s h there

is no one stage deviation by a player that raises that player’s payoff. Then the strategy profile is sub-

game perfect.



Proof: The proof follows by backwards induction.

1 2 2( , ( ),.., ( ))T Ts s s h s h NE strategy profile.

1 2 2( , ( ),.., ( ))T Ts s s h s h some other strategy profile.

1 1 1( ) ( ,...., ( ), ( ),..., ( ))T Ts s s h s h s h strategy profile that agrees with s for t and with

fors t

By hypothesis there is no one-stage deviation from s that benefits any player. So there is no one stage

deviation from ( )s for any stage t sub-game where t .

We consider deviations by player i. Suppose that stage a is the last stage at which, for some history,

player i deviates. Then the strategy profile s agrees with s for 1t a so that ( 1)s s a . We now

argue that player i must be at least as well off under strategy profile ( )s a as ( 1)s a .



Consider the stage a sub-game. By hypothesis, there is no one stage deviation from ( )s a that benefits

player i. But the only difference between ( )s a and ( 1)s a is the strategy at stage a . Therefore

( ( )) ( ( 1)) ( )i i i

U s a U s a U s . (0.0-1)

*



Consider the stage a sub-game. By hypothesis, there is no one stage deviation from ( )s a that benefits

player i. But the only difference between ( )s a and ( 1)s a is the strategy at stage a . Therefore

( ( )) ( ( 1)) ( )i i i

U s a U s a U s . (*)

Let stage b be the second last stage at which player i deviates. Consider the sub-game at stage b . Note

that ( 1) ( )s b s a since there are no deviations between the two stages. Arguing as before ( 1)s b is

a one stage deviation from ( )s b . Therefore ( ( )) ( ( 1))i i

U s b U s b . Combining this result with (*) it

follows that ( ( )) ( )i i

U s b U s . Repeating this argument for every stage in which player i deviates it

follows that ( ) ( )i i

U s U s .

Q.E.D.



While we have considered a sequential move game, the argument is almost identical for a finitely

repeated game. We therefore have the following corollary.

Corollary 9.2-3: One-stage deviation principle in finitely repeated games

In a finitely repeated game suppose that for the strategy profile 1 2 2( , ( ),.., ( ))T Ts s s h s h there is no one

stage deviation by a player that raises that player’s payoff. Then the strategy profile is sub-game

perfect.

Date post:	07-May-2018
Category:	Documents
Upload:	vohanh
View:	215 times
Download:	2 times

GAMES WITH A HISTORY - Essential...

Documents