CS483/683 Multi-Agent Systems

Lectures 5-6:From Satisfaction to Optimization

ADOPT: Asynchronous Distributed Optimization

2-4 February 2010Instructor: Kostas Bekris

Computer Science & Engineering, University of Nevada, Reno

CS483

Distributed Constrained Optimization

DCOP:

•A set of n variables V={X1, ..., Xn}

•Each variable has a discrete domain: D1, ..., Dn

•Each variable is assigned to an agent

- Only the agent who is assigned a variable has control over its value

- And knowledge of its domain

•Assign values to variable so as to optimize a global objective function

- The optimization of the function satisfies a set of constraints

Requirements:

•Distributed computation using only local communication

•Fast, asynchronous computation - agents should work in parallel

•Quality guarantees are needed

- Provably optimal solutions whenever possible

- Trade-off between computation and solution quality

✓Bounded-error approximation: Guarantee solution within a distance from optimal, less time than the optimal

CS483 ADOPT: Asynchronous Distributed Optimization

The objective function is a summation over a set of cost functions

F(A)=∑Xi,Xj∈V fij(di,dj)

Xi←di, Xj←dj in A

We want to find A* that minimizes F(A)

e.g. F( { (X1,0), (X2,0), (X3,0), (X4,0) } ) = 4

F( { (X1,1), (X2,1), (X3,1), (X4,1) } ) = 0

XX11

XX22

XX33 XX44

di djfij(di,dj

)

0 0 1

0 1 2

1 0 2

1 1 0Constraint Graph

CS483

Assumptions

1. Summation operation over cost function

•Associative

•Commutative

•Monotonic

- Cost of a solution can only increase as more costs are aggregated (i.e., we cannot have negative cost)

2. Constraints are at most binary

•There are ways to extend to constraints that involve a larger number of variables

3. Each agent is assigned a single variable

•There is a way to extend to the case that an agent must handle multiple variables

CS483

Key Ideas in ADOPT

1. Opportunistic best-first search

•Agents are prioritized in a tree structure

- an agent has a single parent and multiple children

•Each agent keeps on choosing the best value based on the current available information

- i.e., chooses the variable which implies the smallest lower bound

- lower bounds do not need global information to be estimated

•Each agent maintains a lower and an upper bound for the cost of its subtrees

- and informs its parent about its own bounds

•Strategy allows agents to abandon partial solutions which have not been proven to be suboptimal

- they may have to reconsider the same assignments into the future

XX11

XX22

XX33 XX44

Communication Graph

CS483

Key Ideas in ADOPT

2. Backtrack Threshold

•When an agent knows from previous search experiences that lb is a lower bound for its subtree

- inform the subtree agents not to bother searching for a solution whose cost is lower than lb

•In the general case, remembering these lower bounds for past assignments requires exponential space

- Approach remembers only one value and then cost is subdivided to children arbitrarily and adapted on the fly as new computations are executed

3. Built-in Termination Detection

•Keeping track of bounds (lower and upper bound for the cost function) on each agent

- allows to keep track of the progress towards the optimum solution

- and automatically terminates when necessary

CS483

Messages and Data Structures

VALUE message (like ok?)

Send selected value to children along the

constraint graph

COST message (like NoGood)

Send to parents along the communication graph

THRESHOLD message

Send to children along the communication graph

Each agent maintains the “context”

(like the “agent_view”)

•A recorf of higher priority neighbors’ current variable assignment

Two contexts are compatible if they do not disagree on any variable assignment

XX11

XX22

XX33 XX44

Constraint Graph

XX11

XX22

XX33 XX44

Communication Graph

CS483

COST message

Xk transmits COST message to Xi

Message contains

•context of Xk

•lb of Xk

•up of Xk

When Xi receives the message it stores

•lb(d,Xk)

•ub(d,Xk)

where d is the assignment of Xi in Xk’s context

If context of Xi is incompatible with the context of Xk:

•lb(d,Xk) = 0

•ub(d,Xk) = ∞

XXii

XXkk

CS483

Costs and Bounds

Local cost: δ(di) = ∑(Xj,dj) fij(di,dj)

di : assignment of agent Xi

Xj : higher priority neighbors than Xi

Lower bound for value d:

∀ d ∈ Di:

LB(d) = δ(d) + ∑ Xk ∈ children lb(d,Xk)

similarly for the upper bound for value d

Lower bound:

LB = min d ∈ Di LB(d)

similarly for the upper bound

For leaves: LB(d) = UB(d) = δ(d)

If not a leaf but has not get received a COST message: LB = δ(d) and UB = ∞

XX11

XX22

XX33 XX44

Constraint Graph

XX11

XX22

XX33 XX44

Communication Graph

CS483

When does Xi change value?

Whenever LB(di) exceeds the backtrack threshold value, Xi changes its variable value to one with smaller lower bound

The threshold is updated with the following three ways:

•Its value can increase whenever Xi determines that LB is greater than the current threshold

- guarantees that there is always a variable with a lower bound than the threshold

•Its value can decrease whenever Xi determines that UB is lower than the current threshold

Invariant: LB ≤ threshold ≤ UB

•Its value is also updated whenever a THRESHOLD message is received from a parent

- a parent subdivides its own threshold value among its children

- t(d,Xk): the threshold on cost allocated by parent Xi to child Xk

- then the value of t(d,Xk) respects the following invariants:

✓threshold = δ(di) + ∑ Xk ∈ children t(di,Xk)

✓∀ d ∈ Di, ∀ Xk ∈ children: lb(d,Xk) ≤ t(d,Xk) ≤ ub(d,Xk)

CS483

Example

All agents begin concurrently choosing 0.

Each agents send a VALUE message to lower priority agents along the constraint graph

We will follow one specific execution path - there are many possible

XX11

XX22

XX33 XX44

Communication Graph

di djfij(di,dj

)

0 0 1

0 1 2

1 0 2

1 1 0

XX11

XX22

XX33 XX44

Constraint Graph

CS483

Example

X2 receives X1’s VALUE message

and records this value to its context

X2’s context: {X1=0}

Then it computes bounds:

•LB(0) = δ(0) + lb(0,X3) + lb(0,X4) = 1

•LB(1) = δ(1) + lb(1,X3) + lb(1,X4) = 2

•LB(0) < LB(1) ⇒ LB = LB(0) = 1

Similarly: UB = ∞

threshold was set to LB(0), equal to 1 so the invariant holds

Transmits a COST message to X1:

COST( {X1=0}, 1, ∞ )

XX11=0=0

XX22=0=0

XX44=0=0XX33=0=0

di djfij(di,dj

)

0 0 1

0 1 2

1 0 2

1 1 0

CS483

Example

X3 receives X1’s and X2’s VALUE messages

and records these values to its context

X3’s context: {X1=0, X2=0}

Then it computes bounds:

•LB(0) = δ(0) = 1 + 1 = 2

•LB(1) = δ(1) = 2 + 2 = 4

•LB(0) < LB(1) ⇒ LB = LB(0) = 2

Similarly: UB = 2

threshold was LB(0) = 2 so the invariant holds

Transmits a COST message to X2:

COST( {X1=0, X2=0}, 2, 2 )

Similarly with X4... but no reference to X1

XX11=0=0

XX22=0=0

XX44=0=0XX33=0=0

di djfij(di,dj

)

0 0 1

0 1 2

1 0 2

1 1 0

CS483

Example

X1 receives X2’s COST message

COST( {X1=0}, 1, ∞ )

•test if compatible with its current context

•store: lb(0,X2) = 1 and ub(0,X2) = ∞

Then it computes bounds:

•LB(0) = δ(0) + lb(0,X2) = 0 + 1 = 1

•LB(1) = δ(1) + lb(1,X2) = 0 + 0 = 0

•LB(1) < LB(0) ⇒ LB = LB(1) = 0

Similarly: UB = ∞

threshold was 0, but LB(0) = 1:

•violation of the invariant, change assignment

Send VALUE messages to children

XX11=1=1

XX22=0=0

XX44=0=0XX33=0=0

di djfij(di,dj

)

0 0 1

0 1 2

1 0 2

1 1 0

CS483

Example

Assume COST messages from X3 and X4 are delayed... instead VALUE message from X1 arrives first at X2

•Current context at X2: {X1=1}

When X2 receives the COST messages from X3 its context will be incompatible with X2’s

•the bounds in the message will not be stored

The message from X4 is not incompatible:

•store lb(0,x4) = 1 and up(0,x4) = 1

Then it computes bounds:

•LB(0) = δ(0) + lb(0,X3) + lb(0,X4) = 2+ 0 +1 = 3

•LB(1) = δ(1) + lb(1,X3) + lb(1,X4) = 0 +0 +0 = 0

•LB(1) < LB(0) ⇒ LB = LB(1) = 1

Similarly: UB = ∞

XX11=1=1

XX22=1=1

XX44=0=0XX33=0=0

di djfij(di,dj

)

0 0 1

0 1 2

1 0 2

1 1 0

CS483

Example

X2 will inform X3 and X4 about the changes.

Similar changes will take place on X3 and X4:

•1 will be selected as the value

COST messages will be transmitted:

•from X4 to X2: ( {X2=1}, 0, 0 )

•from X3 to X1 and X2: ( {X1=1,X2=1}, 0, 0 )

•from X2 to X1: ( {X1=1}, 0, 0)

- this is after receiving the COST messages from X3 and X4

Upon receipt of the COST message of X2 at X1:

•LB = UB = threshold = 0

X1 sends TERMINATE messages to other agents.

XX11=1=1

XX22=1=1

XX44=1=1XX33=1=1

di djfij(di,dj

)

0 0 1

0 1 2

1 0 2

1 1 0