Date post: | 22-Dec-2015 |
Category: |
Documents |
View: | 213 times |
Download: | 0 times |
Inefficiency of Equilibria: POA, POS, SPOA
Based on Slides by Amir Epstein and by Svetlana Olonetsky
Modified/Corrupted by Michal Feldman and Amos Fiat
Inefficiency of equilibria
• Outcome of rational behavior might be inefficient• How to measure inefficiency?
– E.g., prisoner’s dilemma
• Define an objective function– Social welfare (= sum of players’ payoffs): utilitarian– Maximize mini ui (egalitarian)– …
3,3 0,5
5,0 1,1
Inefficiency of equilibria
• To measure inefficiency we need to specify: – Objective function– Definition of approximately optimal– Definition of an equilibrium– If multiple equilibria exist, which one do we
consider?
4
More Equilibrium Concepts
pureNash
mixed Nash
correlated eq ???
no regret ???
best-responsedynamics
Strong Nash
Common measures
• Price of anarchy (POA)=cost of worst NE / cost of OPT
• Price of stability (POS)=cost of best NE / cost of OPT
• Approximation ratio: Measures price of limited computational resources
• Competitive ratio: Measures price of not knowing future
• Price of anarchy: Measures price of lack of coordination
Pure
Pure
Pure
Pure
/Mixed
/Mixed
/Mixed
/Mixed
/Strong/Strong
/Strong /Strong
Corr POA ≤ Pure POA ≤ Mixed POA ≤ Strong POA
Corr POS ≥ Pure POS ≥ Mixed POA ≥ Strong POA
Price of anarchy
• Example: in prisoner’s dilemma, POA = POA = 3– But can be as large as desired
• Wish to find games in which POS or POA are bounded– NE “approximates” OPT– Might explains Internet efficiency.
• Suppose we define POA and POS w.r.t. NE in pure strategies– we first need to prove existence of pure NE
3,3 0,5
5,0 1,1
Prisoner’s dilemma
Max-cut game
• Given undirected graph G = (V,E)
• players are nodes v in V
• An edge (u,v) means u “hates” v (and vice versa)
• Strategy of node i: si {Black,White}
• Utility of node i: # neighbors of different color
• Lemma: for every graph G, corresponding game has a pure NE
Proof 1• Claim: OPT of max-cut defines a NE• Proof:
– Define strategies of players by cut (i.e., one side is Black, other side is White)
– Suppose a player i wishes to switch strategies: i’s benefit from switching = improvement in value of the cut
– Contradicting optimality of cut
ui=1 ui=2
Proof 2
• Algorithm greedy-find-cut (GFC):– Start with arbitrary partition of nodes into two sets– If exists node with more neighbors in other side, move
it to other side (repeat until no such node exists)• Claim 1: GFC provides 2-approx. to max-cut, and
runs in polynomial time• Proof:
– Poly time: GFC terminates within at most |E| steps (since every step improves the value of the solution in at least 1, and |E| is a trivial upper bound to solution)
– 2-approx.: Each node ends up with more neighbors in other side than in own side, so at least |E|/2 edges are in cut (since #edges in cut > #edges not in cut)
Proof 2 (cont’d)
• Claim 2: cut obtained by GFC defines a NE
• Proof: obvious, as each player stops only if his strategy is the best response to the other players’ strategies
• Conclusion: max-cut game admits a NE in pure strategies
11
Wardrop Equilibria: Traffic Flow: the Mathematical Model
• a directed graph G = (V,E)
• k source-destination pairs (s1 ,t1), …, (sk ,tk)
• a rate (amount) ri of traffic from si to ti
• for each edge e, a cost function ce(•)
s1 t1
c(x)=x Flow = ½
Flow = ½c(x)=1
Example: (k,r=1)
12
Routings of Traffic
Traffic and Flows:
• fP = amount of traffic routed on si-ti path P
• flow vector f routing of traffic
Selfish routing: what are the equilibria?
s t
13
Wardrop Flows
Special case, assumptions:• agents small relative to network (nonatomic game)• want to minimize cost of their path
Def: A flow is at [Pure] Nash equilibrium (or is a Nash flow) if all flow is routed on min-cost paths [given current edge congestion]
xs t
1Flow = .5
Flow = .5
s t1
Flow = 0
Flow = 1
x
Example:
14
History + Generalizations
• model, defn of Nash flows by [Wardrop 52]
• Nash flows exist, are (essentially) unique– due to [Beckmann et al. 56]– general nonatomic games: [Schmeidler 73]
• congestion game (payoffs fn of # of players)– defined for atomic games by [Rosenthal 73]– previous focus: Nash eq in pure strategies exist
• potential game (equilibria as optima)– defined by [Monderer/Shapley 96]
15
The Cost of a Flow
Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate)
s t
x
1
½
½
Cost = ½•½ +½•1 = ¾
16
The Cost of a Flow
Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate)
Formally: if cP(f) = sum of costs of edges of P (w.r.t. the flow f), then:
C(f) = P fP • cP(f)
s ts t
x
1
½
½
Cost = ½•½ +½•1 = ¾
17
Inefficiency of Nash FlowsNote: Nash flows do not minimize the cost • observed informally by [Pigou 1920]
• Cost of Nash flow = 1•1 + 0•1 = 1• Cost of optimal (min-cost) flow = ½•½ +½•1 = ¾• Price of anarchy := Nash/OPT ratio = 4/3
s t
x
1
0
1 ½
½
18
Braess’s Paradox
Initial Network:
s tx 1
½
x1½
½
½
cost = 1.5
19
Braess’s Paradox
Initial Network: Augmented Network:
s tx 1
½
x1½
½
½
cost = 1.5
s tx 1
½
x1½
½
½0
Now what?
20
Braess’s Paradox
Initial Network: Augmented Network:
s tx 1
½
x1½
½
½
cost = 1.5 cost = 2
s t
x 1
x1
0
21
Braess’s Paradox
Initial Network: Augmented Network:
All traffic incurs more cost! [Braess 68]
• see also [Cohen/Horowitz 91], [Roughgarden 01]
s tx 1
½
x1½
½
½
cost = 1.5 cost = 2
s t
x 1
x1
0
Special Case of routing: Equal Machine Load Balancing = Parallel
Links• Two nodes
• m parallel (related) links
• n jobs (communication requests)
• User cost (delay) is proportional to link load
• Global cost (maximum delay) is the maximum link load
Price of Anarchy
• Price of Anarchy:
The worst possible ratio between: - Objective function in Nash Equilibrium and- Optimal Objective function
• Objective function: total user cost, total user utility, maximal/minimal cost, utility, etc., etc.
Identical machines
• Main results (objective function – maximum load)- For m identical links, identical jobs (pure) R=1- For m identical links (pure) R=2-1/(m+1)- For m identical links (mixed)
m
mR
loglog
log
Lower bound – easy : uniformly choose machine with prob. 1/mUpper bound – assume opt = 1, opt = max expected ≤ 2 in NE (otherwise not NE,
NE = expected max ≤ log m / loglog m due to Hoeffding concentration inequality
Related Work (Cont’)
• Main results
- For 2 related links R=1.618
- For m related links (pure)
- For m related links (mixed)
- For m links restricted assignment (pure)
- For m links restricted assignment (mixed)
m
mR
loglog
log
m
mR
logloglog
log
m
mR
loglog
log
m
mR
logloglog
log
2 2
1
Identical machines
Highest load machine (#1), lowest weight job on #1 (1)
Lowest weight job on highest load machine ≤ ½ HL (3)
Every other machine has load ≥ ½ HL
2 1.5
• m (=3) machines• n (=4) jobs
• vi – speed of machine i
• wj – weight of job j
v1 = 4 v2 = 2 v3 = 1
1 (4) 2 (4) 2 (2)
1 (2)
L1 = 1 L2 = 3 L3 = 2
Related machines
• Li – load on machine i
Price of Anarchy: Lower Bound
k! / (k-i)!
Gi
k-i
k !1
k
Gk
k
k-1
k(k-1)
k-2
G0 G1 G2
v=2k-i v=1v=2k
w=2k-iw=2k
v=2w=2
v=2k-1
w=2k-1
Price of Anarchy: Lower Bound
Gi
k-i
k !1
k
Gk
k
k-1
k(k-1)
k-2
G0 G1 G2k! ~ m
k ~ log m / log log m
k! / (k-i)!
v=2k-i v=1v=2k
w=2k-iw=2k
v=2w=2
v=2k-1
w=2k-1
11
Its a Nash Equilibrium
Gi
k-i
k !1
k
Gk
k
k-1
k(k-1)
k-2
G0 G1 G2
k! / (k-i)!
2
v=2k-i v=1v=2k
w=2k-iw=2k
v=2w=2
v=2k-1
w=2k-1
1
Its a Nash Equilibrium
Gi
k-i
k !1
k
Gk
k
k-1
k(k-1)
k-2
G0 G1 G2
k! / (k-i)!
2 4
v=2k-i v=1v=2k
w=2k-iw=2k
v=2w=2
v=2k-1
w=2k-1
1
The social optimum
k! / (k-i)!
Gi
k-i
k !1
k
Gk
k
k-1
k(k-1)
k-2
G0 G1 G2
21
v=2k-i v=1v=2k
w=2k-iw=2k
v=2w=2
v=2k-1
w=2k-1
The social optimum
k! / (k-i)!
Gi
k-i
k !1
k
k
k-1
k(k-1)
k-2
G0 G1 G2
2
v=2k-i v=1v=2k
w=2k-iw=2k
v=2w=2
v=2k-1
w=2k-1
1
Gk
12
The social optimum
k! / (k-i)!
Gi
k-i
k !1
k
k
k-1
k(k-1)
k-2
G0 G1 G2
2
2 22 2
v=2k-i v=1v=2k
w=2k-iw=2k
v=2w=2
v=2k-1
w=2k-1
Gk
Related Machines: Price of Anarchy upper bound
• Normalize so that Opt = 1
• Sort machines by speed
• The fastest machine (#1) has load Z, no machine has load greater than Z+1 (otherwise some job would jump to machine #1)
• We want to give an upper bound on Z
Related Machines: Price of Anarchy upper bound
• Normalize so that Opt = 1• The fastest machine (#1) has load Z, but
Opt is 1, consider all the machines that Opt uses to run these jobs.
• These machines must have load ≥ Z-1 (otherwise job would jump from #1 to this machine)
• There must be at least Z such machines, as they need to do work ≥ Z
Related Machines: Price of Anarchy upper bound
• Take the set of all machines up to the last machine that opt uses to service the jobs on machine #1.
• The jobs on this set of machines have to use Z(Z-1) other machines under opt.
• Continue, the bottom line is that n ≥ Z!, or that Z ≤ log m / log log m
Restricted Assignment to Machines
m0 m0 m0 m0 m0 m1m0 m1 m1 m1 m1 m1 m2 m2 m2 m3
NASH
Group 1
m0 m0 m0 m0 m0 m1m0 m1 m1 m1 m1 m1 m2 m2 m2 m3
Group 2 Group 3
Group 1
Group 2
Group 3
OPT
l=3
Restricted Assignment to Machines
m0 m0 m0 m0 m0 m1m0 m1 m1 m1 m1 m1 m2 m2 m2 m3
NASH
Group 1
m0 m0 m0 m0 m0 m1m0 m1 m1 m1 m1 m1 m2 m2 m2 m3
Group 2 Group 3
Group 1
Group 2
Group 3
OPT
l=3
Price of anarchy for unrelated machines
• POA for unrelated machines is unbounded
1
1
Job 1
Job 2
Machine 1 Machine 2
1 1
Machine 2Machine 1Machine 2
Machine 1
Social optimum Nash equilibrium
makespan= makespan=
PoA=1/
Allowing Coordination in Equilibrium
• Strong Equilibrium [Aumann’59]– No coalition can deviate and strictly improve
the utility of all of its members• very robust concept• may be a better prediction of rational
behavior• most games do not admit Strong Eq.
– usually applied to pure Eq with pure deviations
Example 1: Prisoner’s Dilemma
0,5
5,0
cooperate
cooperate
defect
defect
Unique Nash Eq.
Strong Eq. ?
Prisoner’s dilemma does not admit any Strong Eq.
Strong Price of Anarchy
• Determining SPoA requires two parts:– Proving existence of Strong Eq– Bounding the worst ratio
• SE NE SPoA ≤ PoA
Price of Anarchy (PoA) [KP00]:
optimum social
mequilibriuNash worst PoA
Strong Price of Anarchy (SPoA):
optimum social
mequilibriu Strongworst SPoA
k-Strong Equilibrium
• A joint action sS is not resilient to a pure deviation of a coalition if there is a pure action profile of such that ci(s-,)<ci(s) for any i – e.g., (defect,defect) in Prisoner’s dilemma
• A pure Nash Eq sS is resilient to pure deviation of coalitions of size k if there is no coalition of size at most k such that s is not resilient to a pure deviation by
• A k-Strong Equilibrium is a pure Nash Eq that is resilient to pure deviation of coalitions of size at most k
S=S1x…xSn
Strong Equilibrium Hierarchy
1-SE
2-SE
n-SE
= NE
=SE [Aumann]
Related Work
• Existence of Strong Equilibrium– monotone decreasing congestion games [Holzman+Lev-
tov 1997, 2003]
– monotone increasing congestion games + correlated SE [Rosenfeld+Tennenholtz 2006]
• Related solution concepts– Coalition-proof Eq. [Bernheim 1987]
– Group-strategyproof mechanisms [Moulin+Shenker 2001]
– Coalitions with transferable utilities [Hayrapetyan et al 2006]
SE
CPE
NE
Existence of Strong Equilibrium in load balancing games
• Is every Nash Eq. on identical machines also a Strong Eq ?– NO ! (for m ≥ 3)
5
5
4 4
3 3
10 7 7
s
55
4 43
3
6 99
s’Coalition: 5,5,3,3
Strong Eq. Existence
• Theorem: in any load balancing game, the lex. minimal joint action s is a k-SE for any k
Recall Lexicographic Order
• Definition: a vector (l1,…lm) is smaller than (l1’,…,lm’) lexicographically if for some i, li < li’ and lk = lk’ for all k<I
• Definition: A joint action s is smaller than s’ lex. (ss’) if the vector of machine loads L(s), sorted in non-decreasing order, is smaller lex. than L(s’)
s
s’ss’
Proof of SE Existence• Suppose in contradiction that s (lex. minimal) is not a
SE, and let be the smallest coalition (deviating to s’).• Claim: the same set of machines are chosen by in s
and in s’ (denote it M())– If a job migrates TO some
machine, another jobmigrates FROM it
• else contradicting s is NE – If a job migrates FROM some
machine, another jobmigrates TO it
• else contradicting minimality of • Since all jobs in must benefit, all loads of M() in s’
must be smaller than max load of M() in s – Contradicting minimality of s
Price of Anarchy (PoA)• Recall: for unrelated machines, PoA may be
unbounded
1
1
Job 1
Job 2
Machine 1 Machine 2
Objective: min makespan
Social optimumNash equilibrium Nash equilibrium
PoA=1/
1 1
M1
makespan=
M2
makespan=
M1 M2
Strong equilibrium Strong equilibrium
SPoA=1
Strong Price of Anarchy
• Theorem: for any job scheduling game with m unrelated machines and n jobs, SPoA ≤ m
Proof for SpoA ≤ m
• Claim 1: L1(s) ≤ OPT– else: coalition of all jobs to OPT
M1Mm Mi Mi-1 M1Mm Mi Mi-1
OPT
L1(s)
OPT
L1(s)
Proof for SpoA ≤ m
• Claim 1: L1(s) ≤ OPT– else: coalition of all jobs to OPT
• Claim 2: i Li(s)-Li-1(s) ≤ OPT – else: consider s’, where all jobs on machines i..m go to OPT.
For all J • cJ(s) > Li-1(s) + OPT• cJ(s’) ≤ Li-1(s) + OPT (since all J together add at most OPT)
M1Mm Mi Mi-1 M1Mm Mi Mi-1
> OPT OPT
Lm(s) ≤ m OPT
Li-1(s) L1(s)
Li(s)
Lower Bound (m machines)• Theorem: there exists a job scheduling game with m
unrelated machines for which SPoA ≥ m
• Proof:
M1 M2 M3 M4 Mm
J1 1 1
J2 1 2
J3 1 3
J4 1 4
Jm 1 m
OPT = 1
makespan=mSE
Identical Machines
• Theorem: there exists a job scheduling game with m identical machines and n jobs, such that
m
SPoA1
1
2
1 2 m-1 m
J1
Jm
Jm+1
J2m
1
1/m1 m-2 m-1 m
OPT
SE
1+1/m
2
Results - machines
• Objective function – maximum load- For m identical links, identical jobs (pure) R=1- For m identical links (pure) POA= SPOA = 2-1/(m+1),
- For m related links (pure)
- For m links restricted assignment (pure)
- For m unrelated machines,
m
mPOA
loglog
log
m
mPOA
loglog
log
Homework assignment
• Fill in tables:
– (pure POA, pure SPOA, pure POS, pure SPOS) * (identical, related, unrelated)
– (mixed POA, mixed SPOA, mixed POS, mixed SPOS) * (identical, related, unrelated)
• Why?
• Send me e-mail with pptx presentation