Biomimetic searching strategies

Massimo Vergassola

CNRS, URA 2171

Institut Pasteur, Unit “In Silico Genetics”

2m

wind

MaleMoth released

Zigzag

Casting: Extended crosswind

Source OdorsDirection and velocity of the wind are determined by air currents and visualclues.

Zigzagging and casting (J.S. Kennedy, e.g. in Physiological Entomology,1983)

Sniffers

Olfactory robots with applications to the detection of chemical leaks, drugs, bombs, land and/or sea mines.

D. Martinez “On the right scent” Nature, 445, 371-372, 2007 (N&V).

Micro vs macro-organisms: the role of size and transport

Chemotaxis of living organisms

Temporal or spatial gradients are sensed and either climbed or descended.

Crucial that the chemoattractant field be smooth and the concentration high enough to be measurable.

Gradients ought to provide a reliable local cue.

Physical constraints on concentration measurements

(Berg & Purcell, Biophys. J., 1977)

Smoluchowski’s diffusion-limited rate of encounters

Reliable measurement of concentration requires:

Measured hits in the time Tint >> fluctuations:

€

J(r) = 4πDac(r)

€

Dac(r)Tint >>1

Bottomline: Chemotaxis requires exponential integration times for exponentially small

concentrations

Searches by macroorganisms

Responses times are O(ms)

Away from the source, gradients are not effectively traceable and do not always point to the source.

Odor encounters are sparse and sporadic.

Yet, birds respond Km’s away and moths locate females hundreds of meters away.

Existing sniffers rely on micro-organism mimetic strategies

• Chemotactic methods, e.g. Ishida et al. (1996); Kuwana et al. (1999); the robolobster by Grasso & Atema et al. (2000); Russell et al. (2003).

• Plume-tracking, e.g. Belanger & Arbas (1998); Li, Farrell, Cardé (2001); Farrell, Pang, Li (2003)&(2005); Ishida et al. (2005); Pang, Farrell (2006).

Effective in dense conditions (relatively close to the source)

Sniffer front view

Sniffer in action

Strategies for searches starting far away from the source, in dilute

conditions?M.V., E. Villermaux, B.I. Shraiman Infotaxis as a strategy for searching without gradients. Nature, 445: 406-9, 2007.

In a nutshell

Concentration is not a good local clue in dilute conditions.

What else could we track in the “desert”, when nothing is detected?

1. Build a map of probability for the source position on the basis of the history of

receptions.

2. Move locally to make the map sharp as fast as possible, i.e. maximize the rate of entropy

reduction.

The message of odor encounters

The source emits particles that are transported in the (random)

environment.

Consider them as a message sent to the searcher.

Message in a random medium.

Use the trace of odor encounters experienced by the searcher to infer the position of the source.

r1, t1

r2, t2

r3, t3

Decoding the message

€

R r | r0( ) Hit rate at position r if source located at r0 .

€

Pt r0( ) =

e− R(r(s)|r0 )ds

0

t

∫R(r(th ) | r0

h=1

H

∏ )

dze− R(r(s)|z )ds

0

t

∫R(r(th ) | z

h=1

H

∏ )∫

As in message decoding, construct the posterior distribution Pt(r0) for the position of the source r0 from the trace ((r1,t1),(r2,t2),…,(rH,tH)) of the hits.

A simple model of random medium

“Particles” are patches of odors where mixing has not dissipated them below the detectibility threshold.

Particles emitted at rate R, advected by a mean wind V, having a finite lifetime and diffused with diffusivity D.

€

0 = V ⋅∇c(r | r0) + DΔc(r | r0) −1

τc(r | r0) + Rδ r − r0( )

€

c(r | r0) =R

4πD(r − r0)e−V (y−y0 ) 2De−r λ

€

λ =Dτ

1+ V 2τ 4D

€

R(r | r0) = 4πDac(r | r0)

After some algebra

General problem: How should we exploit the posterior and deal with its uncertainties?

The “unusual” feature is that the field cannot be quite trusted and is continuously

updated. ML is not suitable. €

Pt r0( ) =

e− R(r(s)|r0 )ds

0

t

∫R(r(th ) | r0

h=1

H

∏ )

dze− R(r(s)|z )ds

0

t

∫R(r(th ) | z

h=1

H

∏ )∫

Search time-entropy relationship

N points to visit. Probability at the j-th visited point is pj and neighborhood constraints dismissed.

€

Τ = jp j

j

∑ + α p j

j

∑ −1 ⎛

⎝ ⎜ ⎜

⎞

⎠ ⎟ ⎟+ β −1 p j ln p j + S

j

∑ ⎛

⎝ ⎜ ⎜

⎞

⎠ ⎟ ⎟

Gibbs distribution

€

p j ∝ e−βj

€

S = T lnT − (T −1)ln(T −1)

reducing to (T>>1)

€

T ≥ eS−1 ≡ N /e

Search times vs entropy

€

T ≥ eS−1 ≡ N e

Note the exponential dependence on S, contrary to the “standard” optimal code length inequality:

The reason is that the “search alphabet” is degenerate, i.e. made of a single letter. Words are

discriminated by their length only

(no coalescence as in Huffman coding)

€

l j

j

∑ p j ≥ S

Infotaxis

Choose the local direction of motion maximizing the rate of information acquired: Maximum expected reduction <S> of the entropy of the field Pt(r0) .

€

S r → rj( ) = Pt rj( ) 0 − S[ ] + 1− Pt rj( )( ) ρ 0ΔS0 + ρ1ΔS1 + ...[ ]

€

ρk =h ke−h

k!

€

h rj( ) = Δt Pt∫ r( )R rj | r( )dr

With the expected hit rate

Exploitation vs exploration

Gradients of concentration in chemotaxis

Rate of acquisition of information, i.e. reduction of entropy of the posterior field Pt(r0).

€

S r → rj( ) = Pt rj( ) 0 − S[ ] + 1− Pt rj( )( ) ρ 0ΔS0 + ρ1ΔS1 + ...[ ]

Exploration: passive gathering of information.

Exploitation: maximum likelihood.

RS Sutton, AG Barto Reinforcement Learning MIT Press, 1998.

Infotactic trajectories

pM sperm responding (sea urchin)

Kaupp et al., Nature Cell Biology, 2003

Search time statistics

Infotaxis is the most robust and rapid among a set of alternative

strategies

Robustness to inaccuracies in the model of the environment

Independent detection model in a real jet flow

Spatial maps in animal brains

Microstructure of a spatial map in the entorhinal cortex Nature, 2005 and following papers by E.I. Moser and colls. Spatial cues are transmitted to the hippocampus

J. O’Keefe, J. Dostrovsky Brain Research 1971 discovery of place cells in hippocampus (see also The Hippocampus as a Cognitive Map, 1978)

In collaboration with

Boris Shraiman (Kavli Inst. Theor. Phys., UCSB)

Emmanuel Villermaux (IRPHE, Marseille)

A simple possible way to account for time correlations

A model where consecutive detections have a space-independent rate give:

€

Pt r0( ) =

e− R(r(s)|r0 )ds

Vi

∫i

∑R(r(th ) | r0

h=1

H

∏ )

dze− R(r(s)|z )ds

Vi

∫i

∑R(r(th ) | z

h=1

H

∏ )∫

Consecutive detections are counted just once

Learning about the source and the medium

Start the searcher with rough estimates of the parameters which make the rate function R(r|

r0) flatter than in realitynot stuck.

The searcher will get to the source slowly but steadily. Once there, infer from its odor

encounter trace the parameters of the medium and the source.

Learning about the source and the medium

Log-likelihood of the experienced series of odor encounters.