Decentralised Coordination of Mobile Sensors using the Max-Sum Algorithm
School of Electronics and Computer ScienceUniversity of Southampton{rs06r2, af2, acr, nrj}@ecs.soton.ac.uk
Ruben Stranders, Alessandro Farinelli, Alex Rogers, Nick Jennings
2
This presentation focuses on the use of Max-Sum to coordinate mobile sensors
Sensor Architecture
Decentralised Control using Max-Sum
Model
Value
Coordinate
Problem Formulation
The key challenge is to monitor a spatial phenomenon with a team of autonomous sensors
Sensors
The key challenge is to monitor a spatial phenomenon with a team of autonomous sensors
LimitedCommunication
The key challenge is to monitor a spatial phenomenon with a team of autonomous sensors
No centralised control
Spatial phenomena are modelled as a spatial field over two spatial and one temporal dimensions
The aim of the sensors is to collectively minimise predictive uncertainty of the spatial phenomenon
PredictiveUncertaintyContours
The main challenge is to coordinate the sensors in order to the state of these spatial phenomena
How to move to minimiseuncertainty?
To solve this coordination problem, we had to address three challenges
1. How to model the phenomena?2. How to value potential samples?3. How to coordinate to gather
samples of highest value?
The three central challenges are clearly reflected in the architecture of our sensing agents
Samples sent toneighbouring agents
Samples received fromneighbouring agents
Information processing
Model of Environment
Outgoing negotiation messages
Incomingnegotiation messages
Value of potential samples Action
Selection
Move
Samples from own sensor
SensingAgent
Rawsamples
Model
Value
Coordinate
These three challenges are clearly reflected in the architecture of our sensing agents
Samples sent toneighbouring agents
Samples received fromneighbouring agents
Information processing
Model of Environment
Outgoing negotiation messages
Incomingnegotiation messages
Value of potential samples Action
Selection
Move
Samples from own sensor
SensingAgent
Rawsamples
Model
The sensors model the spatial phenomenon using the Gaussian Process
Weak Strong
Spatial Correlations
The sensors model the spatial phenomenon using the Gaussian Process
Weak Strong
Temporal Correlations
The value of a sample is determined how much it reduces uncertainty
Samples sent toneighbouring agents
Samples received fromneighbouring agents
Information processing
Model of Environment
Outgoing negotiation messages
Incomingnegotiation messages
Value of potential samples Action
Selection
Move
Samples from own sensor
SensingAgent
Rawsamples
Value
The value of a sample is based on how much it reduces uncertainty
But how to determine uncertainty reduction before collecting a sample?
The value of a sample is based on how much it reduces uncertainty
But how to determine uncertainty reduction before collecting a sample?
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
8
PredictionConfidence IntervalCollected Sample
Gaussian Process not only gives predictions, but also confidence intervals
The value of a sample is based on how much it reduces uncertainty
But how to determine uncertainty reduction before collecting a sample?
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
8
PredictionConfidence IntervalCollected Sample
Gaussian Process not only gives predictions, but also confidence intervals
Potential Sample Location
The value of a sample is based on how much it reduces uncertainty
But how to determine uncertainty reduction before collecting a sample?
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
8
PredictionConfidence IntervalCollected Sample
Gaussian Process not only gives predictions, but also confidence intervals
Measure of uncertainty
The value of a sample is based on how much it reduces uncertainty
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
8
PredictionConfidence IntervalCollected Sample
Specifically, we use Entropy , as information metric
)(XH
The sensor agents coordinate using the Max-Sum algorithm
Samples sent toneighbouring agents
Samples received fromneighbouring agents
Information processing
Model of Environment
Outgoing negotiation messages
Incomingnegotiation messages
Value of potential samples Action
Selection
Move
Samples from own sensor
SensingAgent
Rawsamples
Coordinate
Using the Entropy criterion, the sum of the conditional values equals the team utility
)( 1XH )|( 12 XXH ),|( 321 XXXH
1U 2U 3U
),|()|()(),,( 213121321 XXXHXXHXHXXXH
iU
The key problem is to maximise the social welfare of the team of sensors in a decentralised way
M
iiiU
1
)(maxarg xx
Social welfare:
Mobile Sensors
1x
2x
3x
4x
5x
6x
7x8x
Variables Encode Movement
The key problem is to maximise the social welfare of the team of sensors in a decentralised way
1U
2U
3U
4U
5U
6U
7U8U
Utility Functions
The key problem is to maximise the social welfare of the team of sensors in a decentralised way
(These encode information value)
)( 33 xU
Localised Interaction
},,,{ 54313 xxxxx
1x
3x
4x
5x
The key problem is to maximise the social welfare of the team of sensors in a decentralised way
26
We can now use Max-Sum to solve the social welfare maximisation problem
Complete Algorithms
DPOPOptAPOADOPT
Communication Cost
Iterative AlgorithmsBest Response (BR)
Distributed Stochastic Algorithm (DSA)
Fictitious Play (FP)
Max-SumAlgorithm
Optimality
The input for the Max-Sum algorithm is a graphical representation of the problem: a Factor Graph
Variable nodes Function nodes
1x
2x
3x
1U
2U
3U
Agent 1Agent 2
Agent 3
Max-Sum solves the social welfare maximisation problem by local computation and message passing
1x
2x
3x
1U
2U
3U
Variable nodes Function nodes
Agent 1Agent 2
Agent 3
Max-Sum solves the social welfare maximisation problem by local computation and message passing
jiadjk
iikiji xrxq\)(
)()(
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
From variable i to function j
From function j to variable i
To use Max-Sum, we encode the mobile sensor coordination problem as a factor graph
1x
2x
3x
1U
2U
3U
Sensor 1Sensor 2
Sensor 3
Sensor 1
Sensor 2
Sensor 3
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
Unfortunately, the straightforward application of Max-Sum is too computationally expensive
jiadjk
iikiji xrxq\)(
)()(From variable i to function j
From function j to variable i
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
Unfortunately, the straightforward application of Max-Sum is too computationally expensive
jiadjk
iikiji xrxq\)(
)()(From variable i to function j
From function j to variable i
Bottleneck!
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
Therefore, we developed two general pruning techniques that speed up Max-Sum
Goal: Make as small as possible
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
Therefore, we developed two general pruning techniques that speed up Max-Sum
Goal: Make as small as possible
1. Try to prune the action spaces of individual sensors
2. Try to prune joint actions
ix
ij \x
The first pruning technique prunes individual actions by identifying dominated actions
The first pruning technique prunes individual actions by identifying dominated actions
1. Neighbours send bounds
↑ [2, 2]↓ [1, 1]
↑ [5, 6]↓ [0, 1]
↑ [1, 2]↓ [3, 4]
The first pruning technique prunes individual actions by identifying dominated actions
↑ [2, 2]↓ [1, 1]
↑ [5, 6]↓ [0, 1]
↑ [1, 2]↓ [3, 4]
2. Bounds are summed
↑ [8, 10]↓ [4, 7]
The first pruning technique prunes individual actions by identifying dominated actions
2. Bounds are summed
↑ [8, 10]↓ [4, 7]
↓ [4, 7]↑ [8, 10]
The first pruning technique prunes individual actions by identifying dominated actions
3. Dominated actions are pruned
[8, 10][4, 7]
X
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
We developed two general pruning techniques that speed up Max-Sum
Goal: Make as small as possible
1. Try to prune the action spaces of individual sensors
2. Try to prune joint actions
ix
ij \x✔
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
Sensor 1 Sensor 2 Sensor 3
The second pruning technique reduces the joint action space because exhaustive enumeration is too costly
Sensor 1 Sensor 2 Sensor 3
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
The second pruning technique reduces the joint action space because exhaustive enumeration is too costly
132 \)(},{
11 )()(max)(xjadjk
kjkjjxx
j xqUxr x
132 \)(},{
11 )()(max)(xjadjk
kjkjjxx
j xqUxr x
Sensor 1 Sensor 2 Sensor 3
The second pruning technique reduces the joint action space because exhaustive enumeration is too costly
),,(max)( 32},{
132
xxUr jxx
j
The second pruning technique reduces the joint action space because exhaustive enumeration is too costly
),,(max)( 32},{
132
xxUr jxx
j
),,,(),,,(max jj UU
),,,(),,,( jj UU...),,,(),,,( jj UU
The second pruning technique prunes the joint action space using Branch and Bound
Sensor 1
Sensor 2
Sensor 3
[7, 13][0, 4] [2, 6]
Sensor 1
Sensor 2
Sensor 3
The second pruning technique prunes the joint action space using Branch and Bound
[7, 13][0, 4] [2, 6]XXSensor 1
Sensor 2
Sensor 3
The second pruning technique prunes the joint action space using Branch and Bound
The second pruning technique prunes the joint action space using Branch and Bound
9 10 7 8
[7, 13][0, 4] [2, 6]XXSensor 1
Sensor 2
Sensor 3
The second pruning technique prunes the joint action space using Branch and Bound
9 10 7 8
[7, 13][0, 4] [2, 6]XX
X X XO
Sensor 1
Sensor 2
Sensor 3
This demonstration shows four sensors monitoring a spatial phenomenon
This demonstration shows four sensors monitoring a spatial phenomenon
Sensors
This demonstration shows four sensors monitoring a spatial phenomenon
UncertaintyContours
This demonstration shows four sensors monitoring a spatial phenomenon
The two pruning techniques combined prune 95% of the action space with 6 neighbouring sensors
2 2.5 3 3.5 4 4.5 5 5.5 60
25
50
75
100
Number of neighbouring sensors
% o
f joi
nt a
ction
s pru
ned
Avg.
Roo
t Mea
n Sq
uare
d Er
ror
Our Algorithm reduces Root Mean Squared Error of predictions up to 50% compared to Greedy
Our Al-gorithm
Greedy Random Fixed0.0
0.2
0.4
0.6
0.8
1.0
In conclusion, the use Max-Sum leads to an effective coordination algorithm for mobile sensors
1. Decentralised
In conclusion, the use Max-Sum leads to an effective coordination algorithm for mobile sensors
1. Decentralised
2. Fast
% P
rune
d
In conclusion, the use Max-Sum leads to an effective coordination algorithm for mobile sensors
1. Decentralised
2. Fast
3. Accurate predictions
% P
rune
d
Pred
ictio
n Er
ror
Our
Greedy
Random
Fixed
For future work, we wish to extend the algorithm to do non-myopic planning
In conclusion, the use Max-Sum leads to an effective coordination algorithm for mobile sensors
1. Decentralised
2. Fast
3. Accurate predictions
% P
rune
d
Pred
ictio
n Er
ror
Our
Greedy
Random
Fixed
Questions?