MixedStrategies

CMPT882ComputationalGameTheory

SimonFraserUniversitySpring2010

Instructor:OliverSchulte

Motivation

•  Some games like Rock, Paper, Scissor don’t have a Nash equilibrium as defined so far.

•  Intuitively, the reason is that there is no steady state where players have perfect knowledge of each other’s actions: knowing exactly what the other player will do allows me to achieve an optimal payoff at their expense.

•  Von Neumann and Morgenstern observed that this changes if we allow players to be unpredictable by choosing randomized strategies.

Definition

•  Consider a 2-player game (A1,A2,u1,u2). •  The members of Ai are the pure strategies for

player i. •  A mixed strategy is a probability distribution

over pure strategies. •  Or, if we have k pure strategies, a mixed strategy

is a k-dimensional vector whose nonnegative entries sum to 1.

•  Note: this is the notation from the text. More usual are Greek letters for mixed strategies, e.g. !.

Example:MatchingPennies

Heads: 1/2 Tails: 1/2

Heads: 1/3 1,-1 -1,1

Tails: 2/3 -1,1 1,-1

MixedStrategiesctd.

•  Supposethateachplayerchoosesamixedstrategysi.•  Theexpectedutilityofpurestrategya1isgivenbyEU1(a1,s2)=Σku1(a1,ak)xs2(ak).whereajrangesoverthestrategiesofplayer2.

•  Theexpectedutilityofmixedstrategys1istheexpectationoverthepurestrategies:EU1(s1,s2)=ΣjEU1(aj,s2)xs1(aj)=ΣjΣku1(aj,ak)xs1(aj)xs2(ak).

•  Wealsowriteu1(s1,s2)forEU1(s1,s2).Thenwehaveineffectanewgamewhosestrategysetsarethesetsofmixedstrategies,andwhoseutilityfunctionsaretheexpectedpayoffs.

Nashequilibriuminmixedstrategies

•  Thedefinitionofbestresponseisasbefore:amixedstrategys1isabestresponsetos2ifandonlyifthereisnoothermixedstrategys’1withu1(s’1,s2)>u1(s1,s2).

•  Similarly,twomixedstrategies(s1,s2)areaNashequilibriumifeachisabestresponsetotheother.

ExamplesandExercises

•  FindamixedNashequilibriumforthefollowinggames.– MatchingPennies.– CoordinationGame.– Prisoner’sDilemma.

– BattleoftheSexesorChicken.•  CanyoufindalltheNashequilibria?

VisualizationofEquilibria

•  Ina2x2game,wecangraphplayer2’sutilityasafunctionofp,theprobabilitythat1choosesstrategy1.Similarlyforplayer1’sutility.

StrategiesandTopology

•  Considermixedstrategyprofilesask‐dimensionalvectors,wherekisthetotalnumberofpurestrategiesforeachplayer.Thissetisconvexandcompact(why?).

VisualizationofMatchingPennies

VisualizationofM.P.equilibrium

ComputationofEqulibria

•  DefinitionThesupportofaprobabilitymeasurepisthesetofallpointsxs.t.p(x)>0.

•  Proposition.Amixedstrategypair(s1,s2)isanN.E.ifandonlyifforallpurestrategiesaiinthesupportofsi,thestrategysiisabestreplysos‐i.

•  Corollary.If(s1,s2)isanN.E.anda1,b1areinthesupportofs1,thenu1(a1,s2)=u1(b1,s2).Thatis,player1isindifferentbetweena1andb1.Dittofora2,b2inthesupportofs2.

AgeneralNPprocedureforfindingaNashEquilibrium

1.  Choosesupportforplayer1,supportforplayer2.

2.  CheckifthereisaNashequilibriumwiththosesupports.

ExistenceofNashEquilibrium

•  Theorem(Nash1950).Inanyfinitegame(anynumberofplayers)thereexistsaNashequilibrium.

•  ShortproofbyNash.

ExistenceProof(1)

•  Transformagivenpairofmixedstrategies(s1,s2)asfollows.Foreachpurestrategya1ofplayer1,setc(a1):=max(0,[u1(a1,s2)–u1(s1,s2)]).s’1(a1):=[s1(a1)+c(a1)]/[1+Σbc(b)].Dittoforplayer2.

•  Thisdefinesacontinuousoperatorf(s1,s2)=(s’1,s’2)onthespaceofmixedstrategypairs.

•  Astrategyaisabestreplyifandonlyifc(a)=0.•  Soif(s1,s2)isanN.E.,thenc(a)=0foralla,sof

(s1,s2)=(s1,s2).

ExistenceProof(2)

•  ThegeneralizedBrouwerfixedpointtheoremstatesthatifKisconvexandcompact,andf:KKiscontinuous,thenfhasafixedpointf(x)=x.

•  If(s1,s2)isafixedpointofthemappingonthepreviousslide,then(s1,s2)isanN.E.

•  Proofoutline:Someactiona1withs1(a)>0mustbeabestreplyagainstanys2.Thereforec(a1)=0.Sincewehaveafixedpoint,1+Σbc(b)=1.Thisimpliesthatc(b)=0foreachb.

IllustrationinExcel

•  IllustrateconstructioninExcel.•  Notethattheupdateoperationcanbeviewedasalocalcomputationmethod,andevenasalearningmethod!

MaxminandMinmax

Considera2‐playergame.•  Amaxminstrategyforplayer1solvesmaxs1mins2u1(s1,s2).Dittoforplayer2.

•  Interpretation:Conservativelychoosestrategyagainstworst‐caseadversary.

•  Thevaluemaxs1mins2u1(s1,s2)iscalledthesecuritylevelofplayer1.

•  Aminmaxstrategyforplayer1solvesmins1maxs2u2(s1,s2).Dittoforplayer2.

•  Interpreation:Punishtheotherplayerbyminimizingthebestpayoffshecanget.

N.E.inZero‐SumGames:TheMinimaxTheorem(vonNeumann1928).

.Considera2‐playerzero‐sumgame.1.  siisamaxminstrategyifandonlyifsiisa

minmaxstrategyfori=1,2.2.  Forbothplayers,themaxminvalue=minmax

value.3.  Ifs1,s2areeachmaxmin(minmax)strategies,

then(s1,s2)isaNashequilibrium.4.  If(s1,s2)isanN.E.,thens1ands2aremaxmin

(minmax)strategies.

InterpretationofmixedN.E.

N.E.inPopulationGames