Polymatrix Games:Algorithms and Applications
Rahul Savani
Department of Computer ScienceUniversity of Liverpool
Tutorial at theConference on Web and Internet Economics
WINE 2015
Some of talk relates to joint work with Argyrios Deligkas, John Fearnley,Paul Goldberg, Paul Spirakis, and Bernhard von Stengel
What is a polymatrix game?
Polymatrix games are many-player games
For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)
They model pairwise interactions
Nodes correspond to players
Edges correspond to bimatrix games
Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games
What is a polymatrix game?
Polymatrix games are many-player games
For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)
They model pairwise interactions
Nodes correspond to players
Edges correspond to bimatrix games
Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games
What is a polymatrix game?
Polymatrix games are many-player games
For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)
They model pairwise interactions
Nodes correspond to players
Edges correspond to bimatrix games
Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games
What is a polymatrix game?
Polymatrix games are many-player games
For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)
They model pairwise interactions
Nodes correspond to players
Edges correspond to bimatrix games
Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games
What is a polymatrix game?
Polymatrix games are many-player games
For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)
They model pairwise interactions
Nodes correspond to players
Edges correspond to bimatrix games
Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games
What is a polymatrix game?
Polymatrix games are many-player games
For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)
They model pairwise interactions
Nodes correspond to players
Edges correspond to bimatrix games
Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games
History of polymatrix games
Introduced in:
Janovskaya (1968)Equilibrium points in polymatrix games (in Russian)Latvian Mathematical Collection
We will touch on the following papers here:
Both classical:
Eaves 1973 [9]
Howson 1972 [15]
Howson & Rosenthal 1974 [16]
Miller & Zucker 1991 [19]
And more recent:
Cai et al 2015 [4]
Fearnley et al 2015 [8]
Mehta 2012 [18]
Govindan & Wilson 2004 [14]
Rubinstein 2015 [21]
Polymatrix game
n players i = 1, . . . , n
finite pure strategy sets Si
payoff matrices for every player i and j , i
A ij∈ R
|Si |×|Sj |
For mixed profile (x1, . . . , xn), the payoff to player i is
ui(x1, . . . , xn) =∑i,j
(xi)>A ijx j
Polymatrix game
n players i = 1, . . . , n
finite pure strategy sets Si
payoff matrices for every player i and j , i
A ij∈ R
|Si |×|Sj |
For mixed profile (x1, . . . , xn), the payoff to player i is
ui(x1, . . . , xn) =∑i,j
(xi)>A ijx j
Example polymatrix game
1 2
3
a ba 0,0 2,2b 2,2 0,0
a ba 0,0 2,2b 2,2 0,0
a ba 0,0 1,1b 1,1 0,0
Equilibria:
1 2 3
a b b
b b a
(0.5, 0.5) (0.5, 0.5) (0.5, 0.5)
Example polymatrix game
1 2
3
a ba 0,0 2,2b 2,2 0,0
a ba 0,0 2,2b 2,2 0,0
a ba 0,0 1,1b 1,1 0,0
Equilibria:
1 2 3
a b b
b b a
(0.5, 0.5) (0.5, 0.5) (0.5, 0.5)
Advantage: succinctness
In terms of the number of players, the size of a
strategic-form game is exponential
polymatrix game is polynomial (quadratic)
# players # actions(per player)
# payoffentries
strategic-formn k
n × k n
polymatrix 2k 2 × (n2)
Applications
Polymatrix games are general modelling tool for multi-playergames via pairwise interactions
We will also discuss some other applications from theliterature:
1 Relaxation Labelling Problems for Artificial Neural Networks [19]
2 Graph Transduction in Machine Learning [10]
3 To model 2-player Bayesian Games [16]
4 As a sub-routine for solving general multi-player games [14]
Take-home message
Many things carry over from bimatrix to polymatrix games:
Rational equilibria
Formulation as a Linear Complementarity Problem
Applicability of complementary pivoting algorithms (e.g.Lemke-Howson, Lemke)
Descent methods using Linear Programming for findingApproximate Equilibria
There are also important differences. For polymatrix games:
PPAD-hard to find ε-Nash equilibrium for constant ε
Finding a pure equilibrium is PLS-hard
Take-home message
Many things carry over from bimatrix to polymatrix games:
Rational equilibria
Formulation as a Linear Complementarity Problem
Applicability of complementary pivoting algorithms (e.g.Lemke-Howson, Lemke)
Descent methods using Linear Programming for findingApproximate Equilibria
There are also important differences. For polymatrix games:
PPAD-hard to find ε-Nash equilibrium for constant ε
Finding a pure equilibrium is PLS-hard
Outline
1 Nash equilibria of bimatrix games
2 Linear Complementarity Problems (LCPs)
3 The Lemke–Howson Algorithm and the class PPAD
4 Lemke’s algorithm
5 PLS-hardness of pure equilibria, Graph Transduction
6 Reduction from Polymatrix Game to LCP
7 Descent method for ε-Nash equilibria of polymatrix games
8 Other recent work on polymatrix games
Nash equilibria of bimatrix games
@@I
II
T
M
B
l r
3 31 0
2 50 2
0 64 3
Nash equilibria of bimatrix games
@@I
II
T
M
B
l r
3 31 0
2 50 2
0 64 3
Nash equilibrium =
pair of strategies x, y with
x best response to y andy best response to x
Mixed equilibria
@@I
II
T
M
B
l r
3 31 0
2 50 2
0 64 3
Ay =
3 32 50 6
( 1/3 2/3)T
=
344
xT B =
01/32/3
T 1 0
0 24 3
=(
8/3 8/3)
only only pure best responses canhave
probability > 0
Outline
1 Nash equilibria of bimatrix games
2 Linear Complementarity Problems (LCPs)
3 The Lemke–Howson Algorithm and the class PPAD
4 Lemke’s algorithm
5 PLS-hardness of pure equilibria, Graph Transduction
6 Reduction from Polymatrix Game to LCP
7 Descent method for ε-Nash equilibria of polymatrix games
8 Other recent work on polymatrix games
Linear Complementarity Problem
Given: q ∈ Rn, M ∈ Rn×n Find: z, w ∈ Rn so that
z ≥ 0 ⊥ w = q + Mz ≥ 0
⊥ means orthogonal:
zT w = 0⇔ ziwi = 0 all i = 1, . . . , n
If q ≥ 0, the LCP has trivial solution w = q , z = 0.
Linear Complementarity Problem
Given: q ∈ Rn, M ∈ Rn×n Find: z, w ∈ Rn so that
z ≥ 0 ⊥ w = q + Mz ≥ 0
⊥ means orthogonal:
zT w = 0⇔ ziwi = 0 all i = 1, . . . , n
If q ≥ 0, the LCP has trivial solution w = q , z = 0.
LP in inequality form
primal : max cT xsubject to Ax ≤ b
x ≥ 0
dual : min yT b
subject to yT A ≥ cT
y ≥ 0
LP in inequality form
primal : max cT xsubject to Ax ≤ b
x ≥ 0
dual : min yT b
subject to yT A ≥ cT
y ≥ 0
Weak duality: x, y feasible (fulfilling constraints)
⇒ cT x ≤ yT Ax ≤ yT b
LP in inequality form
primal : max cT xsubject to Ax ≤ b
x ≥ 0
dual : min yT b
subject to yT A ≥ cT
y ≥ 0
Strong duality: primal and dual feasible
⇒ ∃ feasible x, y : cT x = yT b (x, y optimal)
LCP generalizes LP
LCP encodes complementary slackness of strong duality:
cT x = yT Ax = yT b
⇔ (yT A − cT )x = 0, yT (b − Ax) = 0.
≥ 0 ≥ 0 ≥ 0 ≥ 0
LP⇔ LCP
(xy
)︸︷︷︸
z
≥ 0 ⊥(−c
b
)︸ ︷︷ ︸
q
+
(0 AT
−A 0
)︸ ︷︷ ︸
M
(xy
)︸︷︷︸
z
≥ 0
LCP generalizes LP
LCP encodes complementary slackness of strong duality:
cT x = yT Ax = yT b
⇔ (yT A − cT )x = 0, yT (b − Ax) = 0.
≥ 0 ≥ 0 ≥ 0 ≥ 0
LP⇔ LCP
(xy
)︸︷︷︸
z
≥ 0 ⊥(−c
b
)︸ ︷︷ ︸
q
+
(0 AT
−A 0
)︸ ︷︷ ︸
M
(xy
)︸︷︷︸
z
≥ 0
Outline
1 Nash equilibria of bimatrix games
2 Linear Complementarity Problems (LCPs)
3 The Lemke–Howson Algorithm and the class PPAD
4 Lemke’s algorithm
5 PLS-hardness of pure equilibria, Graph Transduction
6 Reduction from Polymatrix Game to LCP
7 Descent method for ε-Nash equilibria of polymatrix games
8 Other recent work on polymatrix games
Symmetric equilibria of symmetric games
Given: n n payoff matrix A for row player AT for column player
mixed strategy x = probability distribution on {1,...,n} x 0 , 1Tx = 1
equilibrium (x, x) x best response to x
Remark: As general as m n games (A, B).
Best responses
Given: n n payoff matrix A, mixed strategy y of column player
Ay = vector of expected payoffs against y, components (Ay)i
x best response to y
x maximizes expected payoff xTAy
best response condition:
∀i : xi > 0 (Ay)i = u = maxk (Ay)k
Symmetric equilibria as LCP solutions
equilibrium (x, x) of game with payoff matrix A x best response to x
1Tx = 1,
x 0 Ax ≤ 1u
w.l.o.g. A > 0 u > 0,
equilibrium (x, x)
z = (1/u) x ( 1/u = 1Tz ),
z 0 Az ≤ 1 "equilibrium z"
Best response polyhedron
0
2
1
1
1 2
2 0A =
1x
2x
u<>x 0,{ ( , ) |x u }1Tx= 1, x uA 1
1
Best response polyhedron
1
1
2
2
211
2
0
2
1
1
1 2
2 0A =
1x
2x
u<>x 0,{ ( , ) |x u }1Tx= 1, x uA 1
1
Best response polyhedron
1
1
2
2
21
1
2
0
2
1
1
1 2
2 0A =
1x
2x
u<>x 0,{ ( , ) |x u }1Tx= 1, x uA 1
(2/3, 1/3)
(completely labeled)equilibrium
1
Projective transformation
1 2
2 0A =
1x
2x
u<>x 0,{ ( , ) |x u }1Tx= 1, x uA 1
>x 0, <xA 1{ ( , ) |1x }1
>z 0, <zA 1
Best response polytope
{ |z }
2
1
2
1
1 2
2 0A =
2z
1z
Symmetric Lemke−Howson algorithm
1z
2z
z3
(bottom)
(back)
2
1
1
2
33
Symmetric Lemke−Howson algorithm
1missing label
1z
2z
z3
(bottom)
(back)
2
1
1
2
33
Symmetric Lemke−Howson algorithm
1missing label
1z
2z
z3
(bottom)
(back)
2
1
1
2
33
Symmetric Lemke−Howson algorithm
1missing label
1z
2z
z3
(bottom)
(back)
2
1
1
2
33
1missing label
Symmetric Lemke−Howson algorithm
1z
2z
z3
(bottom)
(back)
2
1
1
2
33
1missing label
Symmetric Lemke−Howson algorithm
1z
2z
z3
(bottom)
(back)
2
1
1
2
33
found label 1
Symmetric Lemke−Howson algorithm
1z
2z
z3
(bottom)
(back)
2
1
1
2
33
Why Lemke-Howson works
LH finds at least one Nash equilibrium because
• finitely many "vertices"
for nondegenerate (generic) games:
• unique starting edge given missing label
• unique continuation
precludes "coming back" like here:
END OF LINE (Papadimitriou 1991)
start
end
Given a graph G ofindegree/outdegree at most 1,and a start vertex of indegree 0and outdegree 1,find another vertex of degree1
END OF LINE (Papadimitriou 1991)
start0000
0101
end
Catch:graph is exponentially largedefined by two boolean circuitsS , P that take a vertex in {0, 1}n
and output its successor andpredecessor
S(0000) = 0101
P(0101) = 0000
END OF LINE (Papadimitriou 1991)
start
end
A problem belongs to PPAD if itis reducible in poly-time to ENDOF LINE; and PPAD-completeif END OF LINE is reducible toit.
END OF LINE (Papadimitriou 1991)
start
end
A problem belongs to PPAD if itis reducible in poly-time to ENDOF LINE; and PPAD-completeif END OF LINE is reducible toit.
Not to be confused with
OTHER END OF THIS LINE
output unique vertex endfound by “following the line”from the start – this isPSPACE-hard
PPAD-hardness for bimatrix games
Theorem (DGP06, CDT06 [5, 6])
It is PPAD-complete to compute an exact Nash equilibrium of abimatrix game.
Later we will see PPAD-hardness for approximate equilibriaof bimatrix and polymatrix games
Outline
1 Nash equilibria of bimatrix games
2 Linear Complementarity Problems (LCPs)
3 The Lemke–Howson Algorithm and the class PPAD
4 Lemke’s algorithm
5 PLS-hardness of pure equilibria, Graph Transduction
6 Reduction from Polymatrix Game to LCP
7 Descent method for ε-Nash equilibria of polymatrix games
8 Other recent work on polymatrix games
Costs instead of payoffs
1 2 2 1
2 0 1 3
aik 3 − aik
payoff cost
with new cost matrix A > 0 :
equilibrium z z 0 Az 1
Polyhedral view
1 + 3
2 + 1
1z ≥ 0
2z1z ≥ 1
2z1z ≥ 1
2z ≥ 0
1z
2z
1
2
1
2
Lemke's algorithm
given LCP
z 0 w = q + Mz 0
Lemke's algorithm
augmented LCP
z 0 w = q + Mz + dz0 0 z0 0
Lemke's algorithm
augmented LCP
z 0 w = q + Mz + dz0 0 z0 0
where
d > 0 covering vectorz0 extra variable
z0 = 0 z w solves original LCP
Lemke's algorithm
augmented LCP
z 0 w = q + Mz + dz0 0 z0 0
Initialization:
z 0 w = q + dz0 0
z0 0 minimal wi = 0 for some i
pivot z0 in, wi out,
can increase zi while maintaining z w .
Lemke's algorithm for
M = 2 1 , d = 2 1 3 1
w1 −1 2 1 2= + z1 + z2 + z0
w2 −1 1 3 1
w1 1 0 −5 −2= + z1 + z2 + w2
z0 1 −1 −3 −1
w1 −1 2 1 2= + z1 + z2 + z0
w2 −1 1 3 1
w1 1 0 −5 −2= + z1 + z2 + w2
z0 1 −1 −3 −1
z2 0.2 0 −0.2 −0.4= + z1 + w1 + w2
z0 0.4 −1 0.6 0.2
w1 1 0 −5 −2= + z1 + z2 + w2
z0 1 −1 −3 −1
z2 0.2 0 −0.2 −0.4= + z1 + w1 + w2
z0 0.4 −1 0.6 0.2
z2 0.2 0 −0.2 −0.4= + z0 + w1 + w2
z1 0.4 −1 0.6 0.2
Polyhedral view of Lemke
Polyhedral view of Lemke
1z
2z
1
2
1
2
Polyhedral view of Lemke
0z
1z
2z
1
2
1
2
Polyhedral view of Lemke
1z
2z
0z
1
2
1
2
Polyhedral view of Lemke
1z
2z
0z
1
2
1
2
Polyhedral view of Lemke
1z
2z
0z
1
2
1
2
Polyhedral view of Lemke
1z
2z
0z
0z = 0
1
2
1
2
Outline
1 Nash equilibria of bimatrix games
2 Linear Complementarity Problems (LCPs)
3 The Lemke–Howson Algorithm and the class PPAD
4 Lemke’s algorithm
5 PLS-hardness of pure equilibria, Graph Transduction
6 Reduction from Polymatrix Game to LCP
7 Descent method for ε-Nash equilibria of polymatrix games
8 Other recent work on polymatrix games
The class PLS (Polynomial Local Search)
s Given a starting solutions ∈ S = Σn
a P-time algorithm thatcomputes the cost c(s)
a P-time function that computesa neighbouring solutions′ ∈ N(s) with lower cost, i.e.s.t. c(s′) < c(s), or reportsthat no such neighbour exists:
find a local optimum of thecost function c
“every DAG has a sink”
Local Max Cut
Find local optimum ofMax Cut with the FLIP-neighbourhood (exactly onenode can change sides)
Schaffer and Yannakakis [22] showed that Local Max Cutis PLS-complete (via an extremely involved reduction)
Local Max Cut is to PLS what 3-SAT is to NP
1 2
3 4
1
1
−4
31
−2
Local Max Cut
Find local optimum ofMax Cut with the FLIP-neighbourhood (exactly onenode can change sides)
Schaffer and Yannakakis [22] showed that Local Max Cutis PLS-complete (via an extremely involved reduction)
Local Max Cut is to PLS what 3-SAT is to NP
1 2
3 4
1
1
−4
31
−2
Local Max Cut
Find local optimum ofMax Cut with the FLIP-neighbourhood (exactly onenode can change sides)
Schaffer and Yannakakis [22] showed that Local Max Cutis PLS-complete (via an extremely involved reduction)
Local Max Cut is to PLS what 3-SAT is to NP
1 2
3 4
1
1
−4
31
−2
Solutions:
{{1, 3, 4}, {2}} (actual Max Cut)
Local Max Cut
Find local optimum ofMax Cut with the FLIP-neighbourhood (exactly onenode can change sides)
Schaffer and Yannakakis [22] showed that Local Max Cutis PLS-complete (via an extremely involved reduction)
Local Max Cut is to PLS what 3-SAT is to NP
1 2
3 4
1
1
−4
31
−2
Solutions:
{{1, 3, 4}, {2}} (actual Max Cut){{3}, {1, 2, 4}}
Pure Equilibrium in Polymatrix Game
1 2
3
2
−1 2
Pure Equilibrium in Polymatrix Game
1 2
3
a ba 0,0 2,2b 2,2 0,0
a ba 0,0 2,2b 2,2 0,0
a ba 0,0 -1,-1b -1,-1 0,0
Pure Equilibrium in Polymatrix Game
1 2
3
a ba 0,0 2,2b 2,2 0,0
a ba 0,0 2,2b 2,2 0,0
a ba 0,0 -1,-1b -1,-1 0,0
The bimatrix games (A ,B) we used are examples of teamgames because A = B; also called coordination games
Proof that the reduction is correct
Define potential function for “team” polymatrix games
Φ(S) =12
∑i
ui(S)
This is an exact potential function:when i changes strategy then the potential functionchanges by exactly i’s change in utilityFact: in exact potential games,pure equilibria↔ local optima of exact potentialfunctionOur exact potential function value equals value of the cutfor all strategy profiles
�
Summary on PLS and polymatrix games
In contrast to bimatrix games, computing a pureequilibrium in polymatrix games is PLS-hard
Next, an application of team polymatrix games
Application: Graph Transduction
semi-supervised learning: estimate a classificationfunction defined over graph of labeled and unlabeled nodes
ie. propagate labels to unlabelled nodes in consistent way
INPUT: Weighted graph, where some nodes are labelled;
edge weights represent similarities
one approach is to use global optimization
an alternative approach is to use a polymatrix game
Note: without the labelled examples, this is a clusteringproblem; also see e.g., “Hedonic Clustering Games” [12, 2]
Application: Graph Transduction
semi-supervised learning: estimate a classificationfunction defined over graph of labeled and unlabeled nodes
ie. propagate labels to unlabelled nodes in consistent way
INPUT: Weighted graph, where some nodes are labelled;
edge weights represent similarities
one approach is to use global optimization
an alternative approach is to use a polymatrix game
Note: without the labelled examples, this is a clusteringproblem; also see e.g., “Hedonic Clustering Games” [12, 2]
Application: Graph Transduction
semi-supervised learning: estimate a classificationfunction defined over graph of labeled and unlabeled nodes
ie. propagate labels to unlabelled nodes in consistent way
INPUT: Weighted graph, where some nodes are labelled;
edge weights represent similarities
one approach is to use global optimization
an alternative approach is to use a polymatrix game
Note: without the labelled examples, this is a clusteringproblem; also see e.g., “Hedonic Clustering Games” [12, 2]
Application: Graph Transduction
1 2
3
a ba 2,2 0,0b 0,0 2,2
a ba 2,2 0,0b 0,0 2,2
a ba -1,-1 0,0b 0, 0 -1,-1
Note: asymmetric similarity measures have also beenconsidered. Then we may no longer have pure equilibria, butmixed equilibria are still considered meaningful
Application: Graph Transduction
1 2
3
a ba 2,2 0,0b 0,0 2,2
a ba 2,2 0,0b 0,0 2,2
a ba -1,-1 0,0b 0, 0 -1,-1
Note: asymmetric similarity measures have also beenconsidered. Then we may no longer have pure equilibria, butmixed equilibria are still considered meaningful
Open question for team polymatrix games
Can we compute a mixed Nash equilibrium of a teampolymatrix game in polynomial-time? [7]
Note that this problem lies in PPAD ∩ PLS so is unlikely to behard for either of them
Question:
Can anyone think of an easy mixed equilibrium for thelocal max cut game?
Suggested reading:
Daskalakis & PapadimitriouContinuous local search SODA 2011
Open question for team polymatrix games
Can we compute a mixed Nash equilibrium of a teampolymatrix game in polynomial-time? [7]
Note that this problem lies in PPAD ∩ PLS so is unlikely to behard for either of them
Question:
Can anyone think of an easy mixed equilibrium for thelocal max cut game?
Suggested reading:
Daskalakis & PapadimitriouContinuous local search SODA 2011
Open question for team polymatrix games
Can we compute a mixed Nash equilibrium of a teampolymatrix game in polynomial-time? [7]
Note that this problem lies in PPAD ∩ PLS so is unlikely to behard for either of them
Question:
Can anyone think of an easy mixed equilibrium for thelocal max cut game?
Suggested reading:
Daskalakis & PapadimitriouContinuous local search SODA 2011
Outline
1 Nash equilibria of bimatrix games
2 Linear Complementarity Problems (LCPs)
3 The Lemke–Howson Algorithm and the class PPAD
4 Lemke’s algorithm
5 PLS-hardness of pure equilibria, Graph Transduction
6 Reduction from Polymatrix Game to LCP
7 Descent method for ε-Nash equilibria of polymatrix games
8 Other recent work on polymatrix games
Polymatrix games→ LCPs
At least three different reductions to LCP; each gives analmost-complementarity algorithm
1 Howson 1972 [15]2 Eaves 1973 [9] (more general)3 Miller and Zucker 1991 [19]
Instead we are going to present bilinear games whichappeared in Ruta Mehta’s thesis [18, 13], and which are aspecialization of Eave’s games
Bilinear Games
Inspired by sequence form of Koller, Megiddo, von Stengel(1996) [17]
They turn out to be are a special case of Eaves’ polymatrixgames with joint constraints [9], where we restrict to:
two players
polytopal strategy constraint sets
Bilinear Games
A bilinear game is given by:
two m × n dimensional payoff matrices A and B
polytopal strategy constraint sets:
X = {x ∈ Rm| Ex = e, x ≥ 0}
Y = {y ∈ Rn| Fy = f , y ≥ 0}
With payoffs xT Ay and xT By
for the strategy profile (x, y) ∈ X × Y
Bilinear Games
A bilinear game is given by:
two m × n dimensional payoff matrices A and B
polytopal strategy constraint sets:
X = {x ∈ Rm| Ex = e, x ≥ 0}
Y = {y ∈ Rn| Fy = f , y ≥ 0}
(x, y) ∈ X × Y is a Nash equilibrium iff
xT Ay ≥ xT A for all x ∈ X and
xT By ≥ xT By for all y ∈ Y
An LCP for Bilinear Games
Encode best response condition via an LP:
maxx
x>(Ay)
s.t. x>E> = e>, x ≥ 0
An LCP for Bilinear Games
Encode best response condition via an LP:
maxx
x>(Ay)
s.t. x>E> = e>, x ≥ 0
The dual LP has an unconstrained vector p:
miny
e>p
s.t. E>p ≥ Ay
We will again use complementary slackness:
An LCP for Bilinear Games
Encode best response condition via an LP:
maxx
x>(Ay)
s.t. x>E> = e>, x ≥ 0
The dual LP has an unconstrained vector p:
miny
e>p
s.t. E>p ≥ Ay
We will again use complementary slackness:
Feasible x, p are optimal iff x>(Ay) = e>p = x>E>p, i.e.,
An LCP for Bilinear Games
Encode best response condition via an LP:
maxx
x>(Ay)
s.t. x>E> = e>, x ≥ 0
The dual LP has an unconstrained vector p:
miny
e>p
s.t. E>p ≥ Ay
We will again use complementary slackness:
x>(−Ay + E>p) = 0
An LCP for Bilinear Games
Given: q ∈ Rn, M ∈ Rn×n Find: z, w ∈ Rn so that
M =
−A E> −E>
−B> F> −F>
−EE−F
F
q =
00e−e
f−f
z = (x, y, p′, p′′, q′, q′′)>
wherep = p′ − p′′, q = q′ − q′′
Lemke’s algorithm for Bilinear Games
Theorem 4.1 in [17] says:
If we have
1 z>Mz ≥ 0 for all z ≥ 0, and
2 z ≥ 0, Mz ≥ 0 and z>Mz = 0 imply that z>q ≥ 0
then
Lemke’s algorithm computes an solution to the LCP M, q
Polymatrix games as Bilinear Games
Polymatrix game (with complete interaction graph):
players i = 1, . . . , n, with pure strategy sets Si
and payoff matrices for player i,
A ij∈ R
|Si |×|Sj |
for pairs of players (i, j)
let (x1, . . . , xn) in ∆(Si) × · · · ×∆(Sn) be a mixed strategyprofile, then the payoff to player i is
ui(x1, . . . , xn) =∑i,j
(xi)>A ijx j
Polymatrix games as Bilinear Games
(Symmetric) bilinear game: (A ,A>,E,E, e, e)
payoff matrices (A ,A>)
strategy constraints Ex = e
where e = 1n, and
A =
0 A12 · · · A1n
A21 0 A2n
.... . .
An1 An2 · · · 0
E =
1>
|S1|0 · · · 0
0 1>
|S2|· · · 0
.... . .
0 0 · · · 1>
|Sn |
Reductions for sparse polymatrix games
Existing reductions apply to polymatrix games oncomplete interaction graphs
For other interactions graphs, missing edges are replacedwith games with all 0 payoffs
Can we come up with more space efficient reductionsfor non-complete interaction graphs?
Outline
1 Nash equilibria of bimatrix games
2 Linear Complementarity Problems (LCPs)
3 The Lemke–Howson Algorithm and the class PPAD
4 Lemke’s algorithm
5 PLS-hardness of pure equilibria, Graph Transduction
6 Reduction from Polymatrix Game to LCP
7 Descent method for ε-Nash equilibria of polymatrix games
8 Other recent work on polymatrix games
Approximation - Background
Definition (ε-Nash equilibrium)
A strategy profile is an ε-Nash equilibrium if:
no player can gain more than ε by a unilateral deviation
(additive notion of approximation)
Approximation - Background
Definition (ε-Nash equilibrium)
A strategy profile is an ε-Nash equilibrium if:
no player can gain more than ε by a unilateral deviation
(additive notion of approximation)
Theorem (Rubinstein 2014)
There exists a constant ε such that it is PPAD-hard to find anε-Nash equilibrium of a n-player polymatrix game.
Approximation - Background
Definition (ε-Nash equilibrium)
A strategy profile is an ε-Nash equilibrium if:
no player can gain more than ε by a unilateral deviation
(additive notion of approximation)
Theorem (CDT 2006)
If there is an FPTAS for computing an ε-Nash of a bimatrixgame, then PPAD = P.
Background: bimatrix games
What is the smallest ε such that an ε-Nash equilibrium can becomputed in polynomial time (payoffs in [0, 1])?
HISTORY:
0.5 Daskalakis Mehta Papadimitriou (WINE 06)
0.382 DMP (EC 2007)
0.364 Bosse Byrka Markakis (WINE 07)
0.339 Tsaknakis Spirakis (WINE 07)
Tsaknakis & Spirakis use gradient descent
Background: bimatrix games
What is the smallest ε such that an ε-Nash equilibrium can becomputed in polynomial time (payoffs in [0, 1])?
HISTORY:
0.5 Daskalakis Mehta Papadimitriou (WINE 06)
0.382 DMP (EC 2007)
0.364 Bosse Byrka Markakis (WINE 07)
0.339 Tsaknakis Spirakis (WINE 07)
Tsaknakis & Spirakis use gradient descent
Background: many-player games
Two players: 0.3393 [Tsaknakis and Spirakis]
n players: 1 − 1/n [obvious extension of DMP]
DMP idea extends solution for n − 1 players to n players:
Three players: 0.6022
Four players: 0.7153
Guarantee goes to 1 as n goes to infinity
Next we show for the class of n-player polymatrix games:(0.5 + δ) in time polynomial in the input size and 1/δ
Background: many-player games
Two players: 0.3393 [Tsaknakis and Spirakis]
n players: 1 − 1/n [obvious extension of DMP]
DMP idea extends solution for n − 1 players to n players:
Three players: 0.6022
Four players: 0.7153
Guarantee goes to 1 as n goes to infinity
Next we show for the class of n-player polymatrix games:(0.5 + δ) in time polynomial in the input size and 1/δ
Background: many-player games
Two players: 0.3393 [Tsaknakis and Spirakis]
n players: 1 − 1/n [obvious extension of DMP]
DMP idea extends solution for n − 1 players to n players:
Three players: 0.6022
Four players: 0.7153
Guarantee goes to 1 as n goes to infinity
Next we show for the class of n-player polymatrix games:(0.5 + δ) in time polynomial in the input size and 1/δ
Gradient descent on max regret
Extend method of Tsaknakis and Spirakis
Definition
For a strategy profile x we define f(x) as the regret:
f(x) := maxi∈players
u∗i(x) − ui(x)
define δ-stationary point of f via combinatorial “gradient”
LP to find a corresponding steepest descent direction
Gradient descent on max regret
Extend method of Tsaknakis and Spirakis
Definition
For a strategy profile x we define f(x) as the regret:
f(x) := maxi∈players
u∗i(x) − ui(x)
define δ-stationary point of f via combinatorial “gradient”
LP to find a corresponding steepest descent direction
The algorithm
1 Choose an arbitrary strategy profile x ∈ ∆
2 Solve steepest descent LP with input x to obtain x′
3 Set x := x + α(x′ − x), where α = δδ+2
4 If f(x) ≤ 0.5 + δ then stop, otherwise go to step 2
The result
Theorem
A (0.5 + δ)-Nash equilibrium of a polymatrix game can befound in time polynomial in the size of the game and in 1/δ.
Proof sketch:
We do not get stuck at a bad point: Every δ-stationarypoint x∗ of f is a (0.5 + δ)-NE, i.e., f(x∗) ≤ 0.5 + δ
Each descent step makes enough progress in reducing f ,so that after polynomially many iterations f(x) ≤ 0.5 + δ
Open questions on approximate equilibria
Better upper bounds:
Constant number of players or strategies
Extend methods for bimatrix games that solve a single LP
ε-well-supported approximate equilibria
Lower bounds:
It is PPAD-hard to find an ε-Nash equilibrium of a polymatrixgame for a constant but very small ε [Rubinstein]
Improve the value of ε in such a lower bound
Application: 2-player Bayesian games
Howson and Rosenthal (1974) observed that these gamescan be written as a complete bipartite polymatrix games
Types of P1 Types of P2
1
2
3
1
2
The descent algorithm gives a 1/2-Nash but this is easilyachievable by the DMP method
Open question: do other methods for bimatrix games alsoextend to Bayesian two-player games?
Application: 2-player Bayesian games
Howson and Rosenthal (1974) observed that these gamescan be written as a complete bipartite polymatrix games
Types of P1 Types of P2
1
2
3
1
2
The descent algorithm gives a 1/2-Nash but this is easilyachievable by the DMP method
Open question: do other methods for bimatrix games alsoextend to Bayesian two-player games?
Enumerating equilibria
All methods we discussed are to find one sampleequilibrium
Often a proper analysis of a game requires anenumeration of all equilibria
Well-developed enumeration methods for bimatrixgames [1]
It is an interesting direction to develop similar methodsfor polymatrix games
Outline
1 Nash equilibria of bimatrix games
2 Linear Complementarity Problems (LCPs)
3 The Lemke–Howson Algorithm and the class PPAD
4 Lemke’s algorithm
5 PLS-hardness of pure equilibria, Graph Transduction
6 Reduction from Polymatrix Game to LCP
7 Descent method for ε-Nash equilibria of polymatrix games
8 Other recent work on polymatrix games
Other recent work on polymatrix games
Solving general multi-player games [14] (also see [11])
Zero-sum polymatrix games [4]
Efficiency of equilibria in polymatrix coordinationgames [20]
QPTAS for tree polymatrix games [3]
References I
[1] David Avis, Gabriel D. Rosenberg, Rahul Savani, and Bernhardvon Stengel.Enumeration of Nash equilibria for two-player games.Economic Theory, 42(1):9–37, 2009.
[2] Haris Aziz and Rahul Savani.Hedonic Games, chapter 15.Cambridge University Press, 2015.In press.
[3] Siddharth Barman, Katrina Ligett, and Georgios Piliouras.Approximating nash equilibria in tree polymatrix games.In Algorithmic Game Theory - 8th International Symposium,(SAGT), pages 285–296, 2015.
References II
[4] Yang Cai, Ozan Candogan, Constantinos Daskalakis, andChristos Papadimitriou.Zero-sum polymatrix games: A generalization of minmax.Mathematics of Operations Research, To appear.
[5] Xi Chen, Xiaotie Deng, and Shang-Hua Teng.Settling the complexity of computing two-player Nash equilibria.Journal of the ACM, 56(3):14:1–14:57, 2009.
[6] Constantinos Daskalakis, Paul W. Goldberg, and Christos H.Papadimitriou.The complexity of computing a Nash equilibrium.SIAM Journal on Computing, 39(1):195–259, 2009.
References III
[7] Constantinos Daskalakis and Christos Papadimitriou.Continuous local search.In Proceedings of the twenty-second annual ACM-SIAMsymposium on Discrete Algorithms, pages 790–804. SIAM,2011.
[8] Argyrios Deligkas, John Fearnley, Rahul Savani, and PaulSpirakis.Computing approximate Nash equilibria in polymatrix games.Algorithmica, 2015.Online first; Preliminary conference version appeared at WINE2014.
[9] B Curtis Eaves.Polymatrix games with joint constraints.SIAM Journal on Applied Mathematics, 24(3):418–423, 1973.
References IV
[10] Aykut Erdem and Marcello Pelillo.Graph transduction as a noncooperative game.Neural Computation, 24(3):700–723, 2012.
[11] Uriel Feige and Inbal Talgam-Cohen.A direct reduction from k-player to 2-player approximate Nashequilibrium.In Algorithmic Game Theory - Third International Symposium(SAGT), pages 138–149, 2010.
[12] Moran Feldman, Liane Lewin-Eytan, and Joseph Seffi Naor.Hedonic clustering games.In Proceedings of the 24th Annual ACM symposium onParallelism in Algorithms and Architectures SPAA, pages267–276. ACM, 2012.
References V
[13] Jugal Garg, Albert Xin Jiang, and Ruta Mehta.Bilinear games: Polynomial time algorithms for rank basedsubclasses.In Internet and Network Economics - 7th InternationalWorkshop, WINE, pages 399–407, 2011.
[14] Srihari Govindan and Robert Wilson.Computing Nash equilibria by iterated polymatrix approximation.Journal of Economic Dynamics and Control, 28(7):1229–1241,April 2004.
[15] Joseph T. Howson.Equilibria of polymatrix games.Management Science, 18(5):pp. 312–318, 1972.
References VI
[16] Jr. Howson, Joseph T. and Robert W. Rosenthal.Bayesian equilibria of finite two-person games with incompleteinformation.Management Science, 21(3):pp. 313–315, 1974.
[17] Daphne Koller, Nimrod Megiddo, and Bernhard von Stengel.Efficient computation of equilibria for extensive two-persongames.Games and Economic Behavior, 14(2):247–259, 1996.
[18] Ruta Mehta.Nash Equilibrium Computation in Various Games.PhD thesis, Dept. of CSE, IIT-Bombay, 8 2012.
References VII
[19] Douglas A. Miller and Steven W. Zucker.Copositive-plus lemke algorithm solves polymatrix games.Operations Research Letters, 10(5):285 – 290, 1991.
[20] Mona Rahn and Guido Schafer.Efficient equilibria in polymatrix coordination games.CoRR, abs/1504.07518, 2015.
[21] Aviad Rubinstein.Inapproximability of Nash equilibrium.In Proceedings of the Forty-Seventh Annual ACM onSymposium on Theory of Computing, STOC, pages 409–418,2015.
[22] Alejandro A Schaffer and Mihalis Yannakakis.Simple local search problems that are hard to solve.SIAM journal on Computing, 20(1):56–87, 1991.