Foraging Swarms:
From Biology to Engineering
Applications
Kevin M. Passino
Dept. Electrical EngineeringThe Ohio State University
OHIOSTATE
T . H . E
UNIVERSITY
Acknowledgement: Thanks to IEEE CSS Distinguished Lecturer Program.
2
OHIOSTATE
T . H . E
UNIVERSITY
Swarms
Biological swarms... foraging, seeking protection, etc.
Science: “Emergent behaviors/intelligence,” etc.
3
OHIOSTATE
T . H . E
UNIVERSITY
Vehicular swarms... formation/pattern/group
(satellites, aircraft, ground/undersea vehicles).
Manufacturing facility Goal
4
OHIOSTATE
T . H . E
UNIVERSITY
Mat
hem
atic
s
Phys
ics
Che
mis
try
Bio
logy
Engineering, Computer Science
Modeling/analysis
Intelligent vehicle swarms
Social foraging
Biomimicry for solvingtechnological problems
5
OHIOSTATE
T . H . E
UNIVERSITY
Philosophy...
Biomimicry: Organisms designed (evolved) to solve
technological problems?
Mathematics/Physics: Models not perfect, analysis
limited, need ideas?
Exploit best of both!
Contributions? Technology? Science?
6
OHIOSTATE
T . H . E
UNIVERSITY
Foraging Theory
• Animals search for and obtain nutrients to maximize
E
T
where E is energy obtained per time T
• Foraging constraints: Physiology, predators/prey,
environment
Evolution optimizes foraging
7
OHIOSTATE
T . H . E
UNIVERSITY
Group of predators
Forager
Nutient patch
Search/foraging strategies, use dynamic
programming to find “optimal policies.”
Social foraging: Costs, but get “collective
intelligence”
8
OHIOSTATE
T . H . E
UNIVERSITY
Chemotactic Behavior of E. coli
• E. coli: Diameter: 1µm, Length: 2µm
Figure 1: E. coli bacterium.
• Sensors/actuators/controller, an autonomous
underwater vehicle – “nanotechnologist’s dream”!
9
OHIOSTATE
T . H . E
UNIVERSITY
Clockwise rotation of flagella, tumble
Counterclockwise rotation of flagella, swim
10
OHIOSTATE
T . H . E
UNIVERSITY
Swarms
E. coli and S. typhimurium can form intricate stable
spatio-temporal patterns in certain semi-solid
nutrient media
• Eat radially, cell-to-cell attractant signals.
11
OHIOSTATE
T . H . E
UNIVERSITY
Bacterial Swarm Foraging as
Optimization
• Find the minimum of
J(θ), θ ∈ p
when we do not have ∇J(θ).
Suppose θ is the position of a bacterium, and J(θ)
represents an attractant-repellant profile so:
1. J > 0 ⇒ noxious
2. J = 0 ⇒ neutral
3. J < 0 ⇒ food
12
OHIOSTATE
T . H . E
UNIVERSITY
Set of bacteria (positions):
P (j, k, ) =θi(j, k, )|i = 1, 2, . . . , S
at the jth chemotactic step, kth reproduction step,
and th elimination-dispersal event.
• Let J(i, j, k, ) denote the cost at the location of the
ith bacterium θi(j, k, ) ∈ p.
• Let φ(j) be a random vector of unit length and C(i)
be a step size, then
θi(j + 1, k, ) = θi(j, k, ) + C(i)φ(j)
If go down then continue for a few steps, if not then
generate random vector
13
OHIOSTATE
T . H . E
UNIVERSITY
Swarming: Add on inter-bacterial nutrient profiles
for each bacterium
Optimization model:
– Chemotaxis for stochastic gradient climbing
– Attraction/repulsion for social aspect, inter-agent
effects → parallel optimization characteristics
– Elimination/dispersion, evolution
Biologically valid model?
A good engineering optimization method?
• See: “Biomimicry of Bacterial Foraging for
Distributed Optimization and Control” [5]
14
OHIOSTATE
T . H . E
UNIVERSITY
0 10 20 300
5
10
15
20
25
30Bacteria trajectories, Generation=1
θ1
θ 2
0 10 20 300
5
10
15
20
25
30Bacteria trajectories, Generation=2
θ1
θ 2
0 10 20 300
5
10
15
20
25
30Bacteria trajectories
θθ 2
1
Figure 2: Function optimization, swarm behavior.
15
OHIOSTATE
T . H . E
UNIVERSITY
Other Social Foraging Models...
-30 -20 -10 0 10 20 30-30
-20
-10
0
10
20
30
x1=θ
1
x 2=θ 2
Nectar concentration (contour plot) and forage sites
M. xanthus: Optimization on noisy surfaces, cellular
automaton approach [3]
Ant colony optimization methods (e.g. shortest path)
Social foraging of honey bees: Optimal resource
allocation model
16
OHIOSTATE
T . H . E
UNIVERSITY
Intelligent Social Foraging
Learning/attentional/planning/“social” approach:
– Construct representation as “cognitive maps”
(learn)
– Focus on parts of the map (attention)
– Predict using these (plan)
– Share the maps (communications → “social”)
17
OHIOSTATE
T . H . E
UNIVERSITY
Stable “Dumb” Foraging Swarms:
Concepts & Challenges
18
OHIOSTATE
T . H . E
UNIVERSITY
Literature: Biology, physics, autonomous vehicles
(Beni, Leonard, Murray, Morse, ...),
Here: Lyapunov stability anlaysis of cohesion
• N “agents:”
xi = vi
vi =1
Miui
• Agent to agent interactions – “attract-repel” to seek
“comfortable” inter-agent distances.
19
OHIOSTATE
T . H . E
UNIVERSITY
Attract: Term in ui like −ka (xi − xj), ka > 0
Repel: Term in ui like
kr exp
(−12‖xi − xj‖2
r2s
) (xi − xj
)
kr > 0, rs > 0
An “equilibrium” inter-agent distance?
20
OHIOSTATE
T . H . E
UNIVERSITY
Environment Model
Move along negative gradient of a “resource profile”
(e.g., nutrient profile) J(x), x ∈ n.
• Plane: J(x) = Jp(x) = Rx + ro
• Quadratic: J(x) = Jq(x) = rm
2‖x − Rc‖2 + ro
Sensor noise ↔ noise on profile
21
OHIOSTATE
T . H . E
UNIVERSITY
Stability Analysis of
Swarm Cohesion Properties
Swarm center, velocity:
x =1
N
N∑i=1
xi v =1
N
N∑i=1
vi
Agent objective: Move to x and v (time-varying)
Error system: eip = xi − x, ei
v = vi − v
eip = ei
v
eiv =
1
Miui − 1
N
N∑j=1
1
Mjuj
22
OHIOSTATE
T . H . E
UNIVERSITY
Cohesive Social Foraging in Noise:
Constant Noise Bounds
Noise: ‖dip‖ ≤ Dp, ‖di
v‖ ≤ Dv, ‖dif‖ ≤ Df
Agents can sense: vi and...
eip = ei
p − dip
eiv = ei
v − div
∇Jp
(xi
)− di
f
23
OHIOSTATE
T . H . E
UNIVERSITY
Agents steer themselves (use Jp):
ui = −Mikaeip − Mikae
iv − Mikvv
i
+ Mikr
N∑j=1,j =i
exp
(−12‖ei
p − ejp‖2
r2s
) (ei
p − ejp
)
− Mikf
(∇Jp
(xi
)− di
f
)
Effects on error: eip − ej
p = (xi − xj) −(di
p − djp
) What are the effects of noise?
Stability/cohesion possible?
24
OHIOSTATE
T . H . E
UNIVERSITY
Consider terms of: eiv = vi − ˙v
• Symmetry gives repel term in ˙v as zero, and:
˙v = −kvv + kadp + kadv + kf df − kfR︸ ︷︷ ︸z(t)
‖z(t)‖ ≤∥∥∥kadp
∥∥∥ +∥∥∥kadv
∥∥∥ +∥∥∥kf df
∥∥∥ + ‖kfR‖ ≤ δ
δ = kaDp + kaDv + kfDf + kf‖R‖
25
OHIOSTATE
T . H . E
UNIVERSITY
Exponentially stable system with a time-varying but
bounded input z(t) → v(t) is bounded:
1. For some positive constant 0 < θ < 1 and some
finite T we have
‖v(t)‖ ≤ exp [−(1 − θ)kvt] ‖v(0)‖ , ∀ 0 ≤ t < T
2. Also, we have the bound
‖v(t)‖ ≤ δ
kvθ, ∀ t ≥ T
26
OHIOSTATE
T . H . E
UNIVERSITY
Remarks:
• Fix δ, θ: kv ↑ ⇒ (faster, smaller bound)
• Dp + Dv + Df ↑ ⇒ δ ↑ ⇒ bound ↑ (e.g., the average
velocity could oscillate).
• Average sensing errors change direction of the
group’s movement relative to nutrients (can get lost).
N → ∞ ⇒ could have dp ≈ dv ≈ df ≈ 0 →“Grunbaum’s principle” of social foraging (compare
to N = 1 case). Groups can climb noisy gradients
better.
Sensor noise leads to “group inertia” (e.g., bee
swarms)
27
OHIOSTATE
T . H . E
UNIVERSITY
• Let Ei = [eip, ei
v]
and E = [E1, E2, . . . , EN]
Theorem 1: Swarm trajectories will converge (in finite
time) to the compact set
Ωb =
E :
∥∥∥Ei∥∥∥ ≤ 2
λmax(P )
λmin(Q)β, i = 1, 2, . . . , N
β = 2ka (Dp + Dv) + 2kfDf + krrs(N − 1) exp(−1
2
)• Proof outline:
1. Lyapunov function V (E) =∑N
i=1 Vi (Ei),
Vi (Ei) = EiPEi
28
OHIOSTATE
T . H . E
UNIVERSITY
2. We have λmax(P ), the maximum eigenvalue of P ,
Vi ≤ −λmin(Q)
(∥∥∥Ei∥∥∥ − 2λmax(P )
λmin(Q)‖gi(E)‖
) ∥∥∥Ei∥∥∥
3. ‖gi(E)‖ < β? Finite repel!
−10
−5
0
5
10
−10
−5
0
5
−15
−10
−5
0
5
x
Swarm agent position trajectories
y
z
Remarks: Effect of parameters on |Ωb|?
29
OHIOSTATE
T . H . E
UNIVERSITY
No sensing errors (Dp = Dv = Df = 0), chooseQ = kaI:
Ωb =
E :
∥∥Ei∥∥ ≤ 2krrs(N − 1)
λmax(P )
λmin(Q)exp
(−1
2
), i = 1, 2, . . . , N
– N , kr, rs fixed: ka ↑ ⇒ |Ωb| ↓, up to a point
(collisions).
– Fixed N , ka, and kr: rs ↑ ⇒ |Ωb| ↑.– Fixed kr, ka, and rs: N → ∞ ⇒ |Ωb| → ∞ (line),
but average errors could be small.
30
OHIOSTATE
T . H . E
UNIVERSITY
Sensing errors:
– Dp ↑ Dv ↑ Df ↑ ⇒ |Ωb| ↑ (no R effect)
– Fix noise at some level, effect of ka?
– Choose Q = kaI, let Ds = Dp + Dv.
Let J = 12|Ωb| and suppose that parameters are
chosen (by evolution) to minimize this.
31
OHIOSTATE
T . H . E
UNIVERSITY
1 2 3 4 5 6 7 8 9 100
2
4
6
8
10
12
0
100
200
300
400
500
600
700
800
900
1000
Ds
J, ka values that minimize J for each D
s
ka
J, s
ize
of b
ound
32
OHIOSTATE
T . H . E
UNIVERSITY
Cohesive Social Foraging in Noise: Extensions
More general noise (work with Yanfei Liu):
‖df‖ ≤ Df
‖dip‖ ≤ Dp1
∥∥∥Ei∥∥∥ + Dp2
‖div‖ ≤ Dv1
∥∥∥Ei∥∥∥ + Dv2
Geometric meaning?
Conditions for swarm cohesion?
Non-identical agents
Trajectory tracking
33
OHIOSTATE
T . H . E
UNIVERSITY
Cohesive Social Foraging, No Noise
Goal: Show connections to optimization perspective
Modify above theory to get:
Ω′b =
E :
∥∥∥Ei∥∥∥ ≤ 2krrs(N − 1)
kaexp
(−1
2
), i = 1, 2, . . . , N
Choose V o(E) =∑N
i=1 V oi (Ei)
V oi
(Ei
)=
1
2kaei
p
eip +
1
2kaei
v
eiv + krr2
s
N∑j=1,j =i
exp
(− 1
2‖ei
p − ejp‖2
r2s
)
• Not a standard Lyapunov function
View ui as being chosen to minimize V o(E)
34
OHIOSTATE
T . H . E
UNIVERSITY
LaSalle’s Invariance Principle: If E(0) ∈ Ω (invariant
set) then E(t) will converge to the largest invariant
subset of
Ωe = E : eiv = 0, i = 1, 2, . . . , N ⊂ Ω
Hence eiv(t) → 0 as t → ∞.
Follow profile? v(t) → −kf
kvR and vi(t) → −kf
kvR for
all i as t → ∞ (group follows the profile)
35
OHIOSTATE
T . H . E
UNIVERSITY
Additional work...
• “Stability Analysis of Swarms,” [1]
• “Stability Analysis of M -Dimensional Asynchronous
Swarms with a Fixed Communication Topology,” [4]
• Model/analyze bee swarms, [2]
Current work with Yanfei Liu (CDC/TAC):
– General noise conditions
– Network effects (delays, topology, reconfiguration)
– Why should we be able to get a result?
36
OHIOSTATE
T . H . E
UNIVERSITY
Biology: Cooperative Foraging?
Groups can climb noisy gradients better than
individuals (some organisms can forage more
successfully in groups than by
themselves–Grunbaum)
In getting your next meal it is best to cooperate!
Why cooperate?
1. Gain since individuals exploit group information
about best direction to go
2. Lose since group moves slower to better sources
3. Overall is there a gain? Apparently so...
37
OHIOSTATE
T . H . E
UNIVERSITY
−250−200
−150−100
−500
−20
0
20
40
60
80−400
−350
−300
−250
−200
−150
−100
−50
0
x
Swarm agent position trajectories
y
z
0 10 20 30 40 50 60 70 80−20
−10
0
10
20Swarm velocities, x dimension
0 10 20 30 40 50 60 70 80−20
−10
0
10
20Swarm velocities, y dimension
0 10 20 30 40 50 60 70 80−20
−10
0
10
20Swarm velocities, z dimension
Time, sec.
(a) Agent positions. (b) Agent velocities.
Figure 3: Linear noise bounds case, plane profile (N = 1).
38
OHIOSTATE
T . H . E
UNIVERSITY
−200−150
−100−50
050
−200
−150
−100
−50
0
50−250
−200
−150
−100
−50
0
50
x
Swarm agent position trajectories
y
z
0 10 20 30 40 50 60 70 80−20
−10
0
10
20Swarm velocities, x dimension
0 10 20 30 40 50 60 70 80−20
−10
0
10
20Swarm velocities, y dimension
0 10 20 30 40 50 60 70 80−20
−10
0
10
20Swarm velocities, z dimension
Time, sec.
(a) Agent positions. (b) Agent velocities.
Figure 4: Linear noise bounds case, plane profile (N = 50).
39
OHIOSTATE
T . H . E
UNIVERSITY
What about group climbing of more
interesting surfaces? Mountains?
40
OHIOSTATE
T . H . E
UNIVERSITY
41
OHIOSTATE
T . H . E
UNIVERSITY
Social Coffee Foraging
Arabica coffee bean grows best at elevations of about
1000 to 1800 meters
Topographical data for Colombia:
– National Geophysical Data Base, 5 minute data
– Use linear interpolation for points in between
available data
42
OHIOSTATE
T . H . E
UNIVERSITY
-84 -82 -80 -78 -76 -74 -72 -70 -68 -66 -64
-4
-2
0
2
4
6
8
10
12
Degrees Longitudinal (- = west of Meridian of Greenwich)
Deg
rees
Nor
th o
f Equ
ator
(-
= s
outh
)
Topographical map of Colombia
Figure 5: Topographical map of Colombia.
43
OHIOSTATE
T . H . E
UNIVERSITY
Given an altimeter can agents socially climb
mountains to find all coffee growing regions in
Colombia?
1. Avoid each other
2. But try to stay together (helps each other)
3. Use modified terrain map...
Cost function: Gaussian function of elevation,
centered at 1400 meters
Movie: Due to Yanfei Liu...
44
OHIOSTATE
T . H . E
UNIVERSITY
Movie...
45
OHIOSTATE
T . H . E
UNIVERSITY
Application: Robotic Swarms
Manufacturing facility Goal
46
OHIOSTATE
T . H . E
UNIVERSITY
“Potential fields approach” to autonomous vechicle
guidance, no noise...
47
OHIOSTATE
T . H . E
UNIVERSITY
With noise...
48
OHIOSTATE
T . H . E
UNIVERSITY
Intelligent Vehicle Swarms
Use ideas from intelligent social foraging?
Planning, attention, learning, etc. How?
What are network effects (delays, topology)?
Mathematical analysis possible? Important? Yes!
(verification and validation)
What can we achieve via cooperative robotic
systems?
Many challenges!
49
OHIOSTATE
T . H . E
UNIVERSITY
Concluding Remarks
Foraging swarms:
1. Bio-inspiration, optimization models
2. Mathematical stability analysis of swarm cohesion
3. Application: Robotic swarms in manufacturing
Book: “Biomimicry for Optimization, Control, and
Automation,” to appear
http://eewww.eng.ohio-state.edu/˜passino/ciiee03.html
50
OHIOSTATE
T . H . E
UNIVERSITY
References
[1] V. Gazi and K. M. Passino. Stability analysis of swarms. To appear, IEEE
Trans. on Automatic Control, 2003.
[2] V. Gazi and K.M. Passino. Modeling and analysis of the aggregation and
cohesiveness of honey bee clusters and in-transit swarms. Submitted to J.
of Theoretical Biology, 2002.
[3] Y. Liu and K. Passino. Biomimicry of social foraging behavior for
distributed optimization: Models, principles, and emergent behaviors. J. of
Optimization Theory and Applications, 115(3), December 2002.
[4] Y. Liu, K. M. Passino, and M. M. Polycarpou. Stability analysis of
m-dimensional asynchronous swarms with a fixed communication topology.
IEEE Transactions on Automatic Control, 48(1):76–95, 2003.
[5] K.M. Passino. Biomimicry of bacterial foraging for distributed optimization
and control. IEEE Control Systems Magazine, 22(3):52–67, June 2002.
