Distributed Computingwith Adaptive Heuristics

Michael SchapiraPrinceton

Innovations in Computer Science09 January 2011

Partially supported by NSF

Aaron D. JaggardRutgers/Colgate

Rebecca N. WrightRutgers

This Talk

• Identify new aspects of the boundary between game theory and distributed computing

• Look at some initial results

• Identify various avenues for future work

nodes interacting in an asynchronous environment

simple, myopic rules of behavior;convergence in synchronized

environment

Background Areas

Motivation

Many real-world settings involve both simple, myopic behavior and asynchronicity

– Network protocols• Routing• Congestion control• ...

– Asynchronous circuits

Goal: Explore dynamics of adaptive heuristics when

asynchrony is allowed

Goal: Explore dynamics of adaptive heuristics when

asynchrony is allowed

Research Agenda

novel applications in

distributed computing

(congestion control, asynchronous circuits,…)

new questionsin game theory and economics

(fictitious play, regret minimization,…)

Understanding when Dynamics Converge: A Simple Example

• Two stable states (pure Nash equilibria)

(5,4)(5,4) (1,1)(1,1)

(0,0)(0,0) (4,5)(4,5)

Bach

Stravinsky

Bach Stravinsky

Understanding when Dynamics Converge: A Simple Example

• If either player is activated alone, the system converge

(5,4)(5,4) (1,1)(1,1)

(0,0)(0,0) (4,5)(4,5)

Bach

Stravinsky

Bach Stravinsky

Understanding when Dynamics Converge: A Simple Example

• Without control over who is activated, the system might not converge

(5,4)(5,4) (1,1)(1,1)

(0,0)(0,0) (4,5)(4,5)

Bach

Stravinsky

Bach Stravinsky

Basic Model

• n nodes (the players)

• Node i has action space Ai

• Each node i has a reaction functionfi: A1 x A2 x ...x An→Ai that determines i’s nextaction based on other current actions– No dependence on own action

Dynamics

• Infinite sequence of discrete timesteps t = 1, …

• Schedule s:{1,…} → 2[n] determines which set of players is activated at time t.– Fair schedules

• Start at an initial state; at each time step t, let the nodes in s(t) react using their reaction functions

Convergence

• The players’ action profile a=(a1,…, an) is a stable state if fi(a) = ai for every i.

• The system is convergent if the dynamics always converge (for all initial states and all fair schedules)

Two High-Level Questions

• What classes of systems are guaranteed (or cannot be guaranteed) to always converge to a stable state?

• How hard is it to determine whether a system always converges to a stable state?

Basic Result

Theorem: If the system has multiple stable states, then the system is not convergent. (I.e., there is some initial state and some schedule that diverge.)

• Actually, can strengthen this:– Allow some history-dependence– Allow randomness in reaction functions

Revisiting Our Example

(5,4)(5,4) (1,1)(1,1)

(0,0)(0,0) (4,5)(4,5)

Bach

Stravinsky

Bach Stravinsky

A Few Words About the Proof

• Inspired by approach to FLP result on impossibility of resilient consensus

Applications

• Interdomain routing

• Congestion control

• Best-reply dynamics in general games

• Diffusion of technologies in social networks

• Asynchronous circuits

• …

Communication Complexity

Theorem: Determining whether a system of n nodes, each with two actions, is convergent may require Ω(2n) bits.– Even if all reaction functions are deterministic,

and do not depend on history or own action

• Uses a reduction from SET DISJOINTNESS.– Constructed system has a unique stable state

Computational Complexity

Theorem: Determining whether a system of n nodes, each with deterministic and historyless reaction function, is convergent is PSPACE-complete.

• So, difficult even if the reaction functions are succinctly represented (so that they could be transmitted quickly)

• Under complexity assumptions, no short witnesses (in general) of being convergent

Scheduling

Question: Does randomness help?

(5,4)(5,4) (1,1)(1,1)

(0,0)(0,0) (4,5)(4,5)

Bach

Stravinsky

Bach Stravinsky

Scheduling

Question: Does randomness help?

Answer: No. Divergence may not only be possible, but overwhelmingly likely.

• Issues of r-fairness

Open Questions

• What are the convergence guarantees and impossibility results– For other heuristics– For other notions of convergence– For other notions of equilibrium

• We’ve taken first steps in the context of regret-minimizing dynamics

Other Open Questions

• Variations in information– Outdated information– Knowledge only of own utility function (uncoupled

dynamics)

• Lots of others

Summary

• Simple behaviors show up in lots of settings– Important to understand dynamic behavior when

asynchrony is allowed

• Initial results on the impossibility of guaranteeing convergence

• Lots of open questions– What can we say about the dynamic behavior in

other natural asynchronous settings?