Post on 15-Jan-2016
transcript
1
Smooth Games and Intrinsic Robustness
Christodoulou and Koutsoupias,Roughgarden
Slides stolen/modified from Tim Roughgarden
2
Congestion Games
• Agent i has a set of strategies, Si, each strategy s in Si is a set of resources
• The cost to an agent is the sum of the costs of the resources r in s used by the agent when choosing s
• The cost of a resource is a function of the number of agents using the resource
fr(# agents)3
4
Price of Anarchy
Price of anarchy: [Koutsoupias/Papadimitriou 99] quantify inefficiency w.r.t some objective function.– e.g., Nash equilibrium: an outcome such that
no player better off by switching strategies
Definition: price of anarchy (POA) of a game (w.r.t. some objective function):
optimal obj fn value
equilibrium objective fn value
the closer to 1 the better
5
Network w/2 players:
s t
2x 12
5x50
Atomic identical flowis a Congestion game
6
Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate)
Formally: if cP(f) = sum of costs of edges of P (w.r.t. the flow f), then:
C(f) = P fP • cP(f)
s ts t
x
1½½
Cost = ½•½ +½•1 = ¾
Costs: atomic and non-atomic flow
7
Def: linear cost fn is of form ce(x)=aex+be
Linear costs
8
Nash Equilibrium: To Minimize Cost:
Price of anarchy = 28/24 = 7/6.• if multiple equilibria exist, look at the worst
one
s t
2x 12
5x5
cost = 14+10 = 24
cost = 14+14 = 28
s t
2x 12
5x5
00
Atomic identical flowLinear costs
9
Theorem: [Roughgarden/Tardos 00] for every non-atomic flow network with linear cost fns:
≤ 4/3 ×
i.e., price of anarchy non atomic flow ≤ 4/3 in the linear latency case.
cost of non-atomic Nash flow
cost of opt flow
POA non-atomic flow
10
Abstract Setup
• n players, each picks a strategy si
• player i incurs a cost Ci(s)
Important Assumption: objective function is cost(s) := i Ci(s)
Key Definition: A game is (λ,μ)-smooth if, for every pair s,s* outcomes (λ > 0; μ < 1):
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)
11
Smooth => POA Bound
Next: “canonical” way to upper bound POA (via a smoothness argument).
• notation: s = a Nash eq; s* = optimal
Assuming (λ,μ)-smooth:
cost(s) = i Ci(s) [defn of cost]
≤ i Ci(s*i,s-i) [s a Nash
eq] ≤ λ●cost(s*) + μ●cost(s)
[(*)]
Then: POA (of pure Nash eq) ≤ λ/(1-μ).
12
“Robust” POA
Best (λ,μ)-smoothness parameters:
cost(s) = i Ci(s)
≤ i Ci(s*i,s-i)
≤ λ●cost(s*) + μ●cost(s)Minimizing: λ/(1-μ).
Congestion games with affine cost functions are (5/3,1/3)-
smooth• Claim: For all non-negative integers y,
z :
13
y(z + 1) ·53y2 +
13z2:
Thus,
14
y(z + 1) ·53y2 +
13z2
) ay(z + 1) + by ·53(ay2 + by) +
13(az2 + bz)
Let s, s¤ be any two vectors of strategies in a congestion game,with loads x and x¤,in (s¤
i ;s¡ i ) the number of users of e is · xe + 1, we have
kX
i=1
Ci (s¤i ;s¡ i ) ·
X
e2E
(ae(xe + 1) + be)x¤e
·X
e2E
53(aex¤
e + be)x¤e +
X
e2E
13(aexe + be)xe
=53C(s¤) +
13C(s):
y = x¤e; z = xe
POA 5/ 2 Any congestion game(includes atomic unit
flow)
a,b ≥0
15
Why Is Smoothness Stronger?
Key point: to derive POA bound, only needed
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)
to hold in special case where s = a Nash eq and s* = optimal.
Smoothness: requires (*) for every pair s,s* outcomes.– even if s is not a pure Nash equilibrium
16
The Need for Robustness
Meaning of a POA bound: if the game is at an equilibrium, then outcome is near-optimal.
17
The Need for Robustness
Meaning of a POA bound: if the game is at an equilibrium, then outcome is near-optimal.
Problem: what if can’t reach equilibrium?• (pure) equilibrium might not exist• might be hard to compute, even
centrally– [Fabrikant/Papadimitriou/Talwar], [Daskalakis/
Goldberg/Papadimitriou], [Chen/Deng/Teng], etc.
• might be hard to learn in a distributed way
Worry: are POA bounds “meaningless”?
18
Robust POA Bounds
High-Level Goal: worst-case bounds that apply even to non-equilibrium outcomes!
• best-response dynamics, pre-convergence– [Mirrokni/Vetta 04], [Goemans/Mirrokni/Vetta 05],
[Awerbuch/Azar/Epstein/Mirrokni/Skopalik 08]
• correlated equilibria– [Christodoulou/Koutsoupias 05]
• coarse correlated equilibria aka “price of total anarchy” aka “no-regret players”– [Blum/Even-Dar/Ligett 06],
[Blum/Hajiaghayi/Ligett/Roth 08]
19
Lots of previous work uses smoothness Bounds
• atomic (unweighted) selfish routing [Awerbuch/Azar/Epstein 05], [Christodoulou/Koutsoupias 05], [Aland/Dumrauf/Gairing/Monien/Schoppmann 06], [Roughgarden 09]
• nonatomic selfish routing [Roughgarden/Tardos 00],[Perakis 04] [Correa/Schulz/Stier Moses 05]
• weighted congestion games [Aland/Dumrauf/Gairing/Monien/Schoppmann 06],
[Bhawalkar/Gairing/Roughgarden 10]
• submodular maximization games [Vetta 02], [Marden/Roughgarden 10]
• coordination mechanisms [Cole/Gkatzelis/Mirrokni 10]
Beyond Pure Nash Equilibria (Static)
20
pureNash
mixed Nash
correlated eq
CCE
For all s;s0i : Es» ¾[Ci (s)] · E s¡ i » ¾¡ i [Ci (s0
i ;s¡ i )]¾= ¾1 £ ¾2 £ ¢¢¢£ ¾k
For all s;s0i : Es» ¾[Ci (s)] · E s» ¾[Ci (s0
i ;s¡ i )]
For all s;s0i : E s» ¾[Ci (s)jsi ] · E s» ¾[Ci (s0
i ;s¡ i )jsi ]
Mixed:
Correlated:
Coarse Correlated:
¾6= ¾1 £ ¾2 £ ¢¢¢£ ¾k
Beyond Nash Equilibria (non-Static)
Definition: a sequence s1,s2,...,sT of outcomes is no-regret if:
• for each player i, each fixed action qi:– average cost player i incurs
over sequence no worse than playing action qi every time
– if every player uses e.g. “multiplicative weights” then get o(1) regret in poly-time
– empirical distribution = "coarse correlated eq" 21
pureNash
mixed Nash
correlated eq
no-regret
An Out-of-Equilibrium Bound
Theorem: [Roughgarden STOC 09] in a (λ,μ)-smooth game, average cost of every no-regret sequence at most
[λ/(1-μ)] x cost of optimal outcome.
(the same bound we proved for pure Nash equilibria)
22
23
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st) [defn of cost]
24
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st) [defn of cost]
= t i [Ci(s*i,st
-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st
-
i)]
25
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st) [defn of cost]
= t i [Ci(s*i,st
-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st
-
i)]
≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]
26
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st) [defn of cost]
= t i [Ci(s*i,st
-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st
-
i)]
≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]
No regret: t ∆i,t ≤ 0 for each i.
To finish proof: divide through by T.
Intrinsic Robustness
Theorem: [Roughgarden STOC 09] for every set C, unweighted congestion games with cost functions restricted to C are tight:
maximum [pure POA] = minimum [λ/(1-μ)]congestion games
w/cost functions in C(λ ,μ): all such gamesare (λ ,μ)-smooth
27
Intrinsic Robustness
Theorem: [Roughgarden STOC 09] for every set C, unweighted congestion games with cost functions restricted to C are tight:
maximum [pure POA] = minimum [λ/(1-μ)]
• weighted congestion games [Bhawalkar/ Gairing/Roughgarden ESA 10] and submodular maximization games [Marden/Roughgarden CDC 10] are also tight in this sense
congestion gamesw/cost functions in C
(λ ,μ): all such gamesare (λ ,μ)-smooth
28