Decentralised Coordination of Mobile Sensors using the Max-Sum Algorithm 

Ruben Stranders, Alex Rogers, Nick JenningsSchool of Electronics and Computer ScienceUniversity of Southampton{rs06r, acr, nrj}@ecs.soton.ac.uk

2

This presentation focuses on the use of Max-Sum to coordinate mobile sensors

Sensor Architecture & Max-Sum

Empirical Evaluation

Speeding up Max-Sum

Model

Value

Coordinate

This work can be applied to improve situational awareness in dynamic scenarios

Disaster Response

Military Surveillance

Climate Research

Our contribution is a coordination mechanism for a team of autonomous mobile sensors

These mobile sensors continuously monitor spatial phenomena

The main challenge is to coordinate the sensors in order to the state of these spatial phenomena

The main challenge is to coordinate the sensors in order to the state of these spatial phenomena

?

The main challenge is to coordinate the sensors in order to the state of these spatial phenomena

Limited Communication

The main challenge is to coordinate the sensors in order to the state of these spatial phenomena

No centralised control

The main challenge is to coordinate the sensors in order to the state of these spatial phenomena

No centralised control

To solve this coordination problem, we had to address three challenges

1. How to model the phenomena?2. How to value potential samples?3. How to coordinate to gather

samples of highest value?

The three central challenges are clearly reflected in the architecture of our sensing agents

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Model

Value

Coordinate

These three challenges are clearly reflected in the architecture of our sensing agents

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Model

The sensors model the spatial phenomenon using the Gaussian Process

Weak Strong

Spatial Correlations

The sensors model the spatial phenomenon using the Gaussian Process

Areas of Rapid Change

The sensors model the spatial phenomenon using the Gaussian Process

Weak Strong

Temporal Correlations

0 0.5 1 1.5 2 2.5 3 3.5 4

-2

Time

Tem

pera

ture

0 0.5 1 1.5 2 2.5 3 3.5 4

-2

Time

Tem

pera

ture

The value of a sample is determined how much it reduces uncertainty

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Value

The value of a sample is based on how much it reduces uncertainty

But how to determine uncertainty reduction before collecting a sample?

The value of a sample is based on how much it reduces uncertainty

But how to determine uncertainty reduction before collecting a sample?

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

PredictionConfidence IntervalCollected Sample

Gaussian Process not only gives predictions, but also confidence intervals

The value of a sample is based on how much it reduces uncertainty

But how to determine uncertainty reduction before collecting a sample?

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

PredictionConfidence IntervalCollected Sample

Gaussian Process not only gives predictions, but also confidence intervals

Potential Sample Location

The value of a sample is based on how much it reduces uncertainty

But how to determine uncertainty reduction before collecting a sample?

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

PredictionConfidence IntervalCollected Sample

Gaussian Process not only gives predictions, but also confidence intervals

Measure of uncertainty

The value of a sample is based on how much it reduces uncertainty

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

PredictionConfidence IntervalCollected Sample

Specifically, we can use information metrics such as Entropy ,or Mutual Information

)(XH

);( YXMI

The sensor agents coordinate using the Max-Sum algorithm

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Coordinate

24

Max-Sum is a powerful algorithm for solving DCOPs

Complete Algorithms

DPOPOptAPOADOPT

Communication Cost

Iterative Algorithms

Best Response (BR)Distributed Stochastic

Algorithm (DSA) Fictitious Play (FP)

Max-SumAlgorithm

Optimality

Max-Sum solves the social welfare maximisation problem in a decentralised way

Mobile Sensors

Max-Sum solves the social welfare maximisation problem in a decentralised way

1x

2x

3x

4x

5x

6x

7x8x

Movement Parameters

Max-Sum solves the social welfare maximisation problem in a decentralised way

1U

2U

3U

4U

5U

6U

7U8U

Utility Functions

Max-Sum solves the social welfare maximisation problem in a decentralised way

)( 33 xU

Localised Interaction

},,,{ 54313 xxxxx

Max-Sum solves the social welfare maximisation problem in a decentralised way

M

iiiU

1

)(maxarg xx

Social welfare:

Mobile Sensors

The input for the Max-Sum algorithm is a graphical representation of the problem: a Factor Graph

Variable nodes Function nodes

1x

2x

3x

1U

2U

3U

Agent 1Agent 2

Agent 3

Max-Sum solves the social welfare maximisation problem by message passing

1x

2x

3x

1U

2U

3U

Variable nodes Function nodes

Agent 1Agent 2

Agent 3

Max-Sum solves the social welfare maximisation problem by message passing

jiadjk

iikiji xrxq\)(

)()(

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx

From variable i to function j

From function j to variable i

To use Max-Sum, we encode the mobile sensor coordination problem as a factor graph

1x

2x

3x

1U

2U

3U

Sensor 1Sensor 2

Sensor 3

Sensor 1

Sensor 2

Sensor 3

Variables represent the sensors’ movements

1x

2x

3x

1U

2U

3U

Sensor 1Sensor 2

Sensor 3

Functions represent the uncertainty reduction that results from collecting a sample

1x

2x

3x

1U

2U

3U

Sensor 1Sensor 2

Sensor 3

Functions represent the uncertainty reduction that results from collecting a sample

1x

2x

3x

1U

2U

3U

Sensor 1Sensor 2

Sensor 3

Functions represent the uncertainty reduction that results from collecting a sample

1x

2x

3x

1U

2U

3U

Sensor 1Sensor 2

Sensor 3

Functions represent the uncertainty reduction that results from collecting a sample

1x

2x

3x

1U

2U

3U

Sensor 1Sensor 2

Sensor 3

Functions represent the uncertainty reduction that results from collecting a sample

1x

2x

3x

1U

2U

3U

Sensor 1Sensor 2

Sensor 3

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx

Unfortunately, the straightforward application of Max-Sum is too computationally expensive

jiadjk

iikiji xrxq\)(

)()(

From variable i to function j

From function j to variable i

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx

Unfortunately, the straightforward application of Max-Sum is too computationally expensive

jiadjk

iikiji xrxq\)(

)()(

From variable i to function j

From function j to variable i

Bottleneck!

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx

Therefore, we developed two general pruning techniques that speed up Max-Sum

Goal: Make as small as possible

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx

Therefore, we developed two general pruning techniques that speed up Max-Sum

Goal: Make as small as possible

1. Try to prune the action spaces of individual sensors

2. Try to prune joint actions

ix

ij \x

The first pruning technique prunes individual actions by identifying dominated actions

The first pruning technique prunes individual actions by identifying dominated actions

1. Neighbours send bounds

↑ [2, 2]↓ [1, 1]

↑ [5, 6]↓ [0, 1]

↑ [1, 2]↓ [3, 4]

The first pruning technique prunes individual actions by identifying dominated actions

2. Bounds are summed[8, 10]

[4, 7]

The first pruning technique prunes individual actions by identifying dominated actions

3. Dominated actions are pruned [8, 10]

[4, 7]X

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx

We developed two general pruning techniques that speed up Max-Sum

Goal: Make as small as possible

1. Try to prune the action spaces of individual sensors

2. Try to prune joint actions

ix

ij \x

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

Action

Sensor 1 Sensor 2 Sensor 3

Action

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

Sensor 1 Sensor 2 Sensor 3

Action

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

Sensor 1 Sensor 2 Sensor 3

Action

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

Sensor 1 Sensor 2 Sensor 3

Action

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

Sensor 1 Sensor 2 Sensor 3

Action

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

Sensor 1 Sensor 2 Sensor 3

Action

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

Sensor 1 Sensor 2 Sensor 3

Action

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

Sensor 1 Sensor 2 Sensor 3

Etcetera…

The second pruning technique prunes the joint action space using branch and bound

Sensor 1

Sensor 2

Sensor 3

The second pruning technique prunes the joint action space using branch and bound

[7, 13][0, 4] [2, 6]

Sensor 1

Sensor 2

Sensor 3

The second pruning technique prunes the joint action space using branch and bound

[7, 13][0, 4] [2, 6]XX

Sensor 1

Sensor 2

Sensor 3

The second pruning technique prunes the joint action space using branch and bound

9 10 7 8

[7, 13][0, 4] [2, 6]XX

Sensor 1

Sensor 2

Sensor 3

The second pruning technique prunes the joint action space using branch and bound

9 10 7 8

[7, 13][0, 4] [2, 6]XX

X X XO

Sensor 1

Sensor 2

Sensor 3

This demonstration shows four sensors monitoring a spatial phenomenon

This demonstration shows four sensors monitoring a spatial phenomenon

Sensors

This demonstration shows four sensors monitoring a spatial phenomenon

UncertaintyContours

This demonstration shows four sensors monitoring a spatial phenomenon

To empirically evaluate our algorithm, we measured speed up and prediction error

UncertaintyContours

9 10 7 8

[7, 13][0, 4] [2, 6]XX

X X X

The two pruning techniques combined prune 95% of the action space with 6 neighbouring sensors

2 2.5 3 3.5 4 4.5 5 5.5 60

25

50

75

100

Number of neighbouring sensors

% o

f joi

nt a

ction

s pr

uned

Aver

age

RMSE

Our Algorithm reduces Root Mean Squared Error of predictions up to 50% compared to Greedy

Our Al-gorithm

Greedy Random Fixed0.0

0.2

0.4

0.6

0.8

1.0

In conclusion, the use of Max-Sum leads to an effective coordination algorithm for mobile sensors

1. Decentralised

In conclusion, the use of Max-Sum leads to an effective coordination algorithm for mobile sensors

1. Decentralised

2. Fast

% P

rune

d

In conclusion, the use of Max-Sum leads to an effective coordination algorithm for mobile sensors

1. Decentralised

2. Fast

3. Accurate predictions

% P

rune

d

Pred

ictio

n Er

ror

Our

Greedy

Random

Fixed

For future work, we wish to extend the algorithm to do non-myopic planning

References

• R. Stranders, A. Farinelli, A. Rogers and N.R. Jennings (2009): Decentralised Coordination of Mobile Sensors Using the Max-Sum Algorithm. In: Proc 21st Int. Joint Conf on AI (IJCAI), Pasadena, USA. (In Press)

• R. Stranders, A. Farinelli, A. Rogers and N.R. Jennings (2009): Decentralised Coordination of Continuously Valued Control Parameters using the Max-Sum Algorithm. 8th Proc. Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS), Budapest. (In Press)

In conclusion, the use of Max-Sum leads to an effective coordination algorithm for mobile sensors

1. Decentralised

2. Fast

3. Accurate predictions

% P

rune

d

Pred

ictio

n Er

ror

Our

Greedy

Random

Fixed

Questions?