Convergence Time to Nash Equilibria
in Load Balancing
Eyal Even-Dar, Tel-Aviv UniversityAlex Kesselman, Tel-Aviv UniversityYishay Mansour, Tel-Aviv University
Game Theory and CS
• AI: – Machine Learning– Reinforcement learning– Multiple agents
• Communication Networks:– Huge networks with little control– Futuristic control mechanisms
• CS Theory
Motivation: Internet
• Diverse set of users• Very large scale system
– Extremely hard to optimize
• Selfish goals and behavior– Relative anonymity
• Consider game theory– Non-cooperative players
Routing: Motivating example
• Each source can select a route• Source Goal: Minimize latency
– Solution concept: Nash Eq.
• Global Goal: Maximize utilization• Coordination Ratio
– How bad can Nash Eq. be?
Job Scheduling
• Classic setting:– Centralize control– Optimize an objective function
• minimize MAX load
– Full cooperation
• Game theory setting:– Congestion and Potential games– Each job optimizes its objective
• Load of the machine it selects.
What are we after?
• Convergence TIME to reach a Nash Eq.
• Major issue– For implementation– Understanding the constraints– Theoretical interest.
• Non Issue (here)– The quality of the resulting Nash.
Model: Jobs and Machines
• Job Scheduling:– m machines– n jobs
• Machine Model: – Machine Mi has speed Si
– m machines ands speeds in [Smin , Smax]
• Jobs Model:– Unrelated: job J a weight wk(J) on Mk
– Otherwise: job J a weight w (J)
– Restricted: job J can be assign to M in R(J)
Model: Weights and Load
• Weights: – Total sum of weights W
– Maximum weight wmax
– integer versus arbitrary weights.– Discrete weights: K different weights
• Load Model:– Machine Mi at time t:
– Bi(t) = jobs running on Mi
– Li(t) = Σj in Bi(t) wi (j) ; Lmax = MAX Li
– Ti(t) = Li(t) / Si ; Tmax = MAX Ti
Machines
M4M2 M3M1
L4
B2
Model: Nash Equilibrium
• No job can move and lower its load.
• For a job J at Mi
– For any Mj
– If J moves to Mj
– Then Ti Tj + wj(J)/Sj
• The load after the move is not lower than before!
Model: Migration
• Elementary step system (ESS):– Only one job moves at a time.– Job’s aim: minimize its observed load– A(t) = jobs wanting to move at time t– Job’s move
• improvement• best reply
– Scheduler:• arbitrary; • Specific: random; FIFO; Max Weight; Max Load
Our motivation
• Study the time it takes the system to converge.
• Arbitrary schedule: – Universal guarantee.
• Specific Schedule:– Optimize convergence
Results overview
• Unrelated Machines– Always converges
• Identical Machines:– Arbitrary [Min Weight]:
• Lower bound (Exponential in m)
– Max Weight: n– FIFO: (n2) – Random: O(n2)
Results Overview
• Related Machines [ignore speeds]:– Arbitrary: O(W2)
– Max Load: O(W√m + n (wmax)2)
– Restricted and unit weight Jobs:• strategy with O( m n )[ideas similar to Milchtaich ’96]
• Discrete weights:– wmax = O( K n4K )
Max Job First
M4M1 M2 M3
Min Job First
M4M1 M2 M3
Upper bound: Unrelated
• Claim: No global system state occurs twice.– Lexicographic sorted order of states
• 90, 100, 20, 1, 3, 100, 5, 2, 90, 90, 90• 100, 100, 90, 90, 90,90, 20, 5, 3, 2, 1
– improvement step -> lower order• move from load 90 to 1 • 90 lowers to 85• 1 increases to 89• 100, 100, 85, 90, 90,90, 20, 5, 3, 2, 89• 100, 100, 90, 90,90, 89, 85, 20, 5, 3, 2
Upper bound: Unrelated
• Convergence bounds:– Number of state
• General: mn
• K Weights:
Kmk
i
i cKm
nc
m
nm
1
Upper Bound: Unrelated
• Integer Weights:– Potential Function:
– Each move the Potential reduces:• Consider a move from Mi to Mk
m
i
tLitP1
)(4)(
)1()1()()( 4444)1()( tLtLtLtL kikitPtP
2/4424)1()( )(1)()( tLtLtL iiitPtP
Upper Bound: Unrelated
• Convergence bound: • Arbitrary:
– Let W= J maxj wj(J)– Initially 4W max{P(t)}, – Each move drops by at least 2– Bound O(4W)
• Max Load Machine:– Initially, each move drops by P(t)/2m– O(mW + m 4W/m+ wmax)
m
i
tLitP1
)(4)(
Lower Bound: Two identical Machines
m1 2 m1 2m1 2
Theorem: Min Job First requires at least (n2) steps to converge.
Lower bound: Identical Machines
• K distinct weights– Each weight has n/K=r jobs– weight wi+1 >> wi
• m=K+1 Machines numbered 0 to K• Initial configuration
– Machine Mi has all jobs of weight wi
• Scheduler: Min job First– Each move of job wi creates an avalanche
Lower Bound: Identical Machines
Lower Bound: Identical Machines
Lower Bound: Identical Machines
Lower Bound: Identical Machines
Lower Bound: Identical Machines
Lower Bound: Identical Machines
Lower Bound: Identical Machines
Identical Machines
• Lower Bound– duration of phase i is ri/(2 i!)– lower bound (n/k)k/(2 k!)
• Upper bound– arbitrary schedule– upper bound (n/k +1)k
Upper Bound : Identical machinesMax Weight + Best response
Theorem: Max Weight + Best response: stabilizes in at most n moves
Claim: Using Best Response in identical machine, after job J stabilizes it will moveonly after a larger job reached its machine.
Upper Bound : Identical machinesMax Weight + Best response
• Consider job J which moved to Mi
– At time of move its stable
• Job J’ moves to Mk
– J’ improved– J did not want to move to Mk
• Job J’ moves to Mi and w(J’) < w(J) – This was the best response of J’
Upper Bound: Related Machines• Potential function:
– Smin =1 and wmin =1
• Lemma– When a job of size w move from Mu to Mv
Pr(t+1)-Pr(t) = 2 w (Tv(t+1)-Tu(t)) < 0
n
i tiM
im
jjj
n
i tiM
im
j j
jr S
wtTS
S
w
S
tLtP
1 ),(
2
1
2
1 ),(
2
1
2
))(())((
)(
Upper Bound: Related
• Theorem: – Related & Restricted assignments -Nash: O(W2/)
• Initial potential =O(W2)• Each move improves by
– Nash [integer weights]: O(W2 (Smax)2)
• Smallest = O(1/ (Smax)2)
Upper Bound: Related
• Theorem:– Max Load [Related & unrestricted] -Nash
2max
max
nwmSWO
Discrete Integer Weights
• Bound the maximum weight • A priori, unbounded
• Two weights: wmaxn
• Beyond two: much more tricky!• Define equivalence of weights
– Same “relative size” for two assignments
• View it as an integer program
• Bound solution size: K (c Smax n)4K
What’s next
• Consider paths in graphs– General Load and Additive cost:
• No DET Nash [LO]
– Max Cost: Always converges.
• General Congestion games– Personal Preferences and weights [M]
• Beyond DET Nash.