4. Larkin Sensory

8/7/2019 4. Larkin Sensory

1/29

SENSORY INFORMATIONSYSTEMS14 March 2011

WILLARD LARKINProgram Manager

AFOSR/RSL

Air Force Office of Scientific Research

AFOSR

Distribution A: Approved for public release; distribution is unlimited. 88ABW-2011-0780


2/29

2

2307/C PORTFOLIO OVERVIEW

Program Manager: Willard Larkin

BRIEF DESCRIPTION OF PORTFOLIO:

Auditory modeling for acoustic analysis

Biological polarization optics & vision Sensori-motor control of bio- flight & navigation

SUB-AREAS IN PORTFOLIO:Sensory Information Systems (2307/C)


3/29

3

Program Trends and Strategy:TOPIC AREA OVERVIEW

Polarization Vision & Optics:

Sensorimotor Control of Flight & Navigation:

Scientific Question: How do natural photoreceptors detect and how doanimal brains interpret polarization information? How is it used for nocturnalnavigation or recognition of obscured targets? Can these unique bio-opticalstructures be emulated?

Scientific Question: How does neural control make natural, low-ReynoldsNo. flight autonomous, efficient, and robust? Discover principles ofmultisensory fusion, distributed sensors and actuators. Develop control lawsfor emulation in MAVs.

Advanced Auditory Modeling:Scientific Question: How does the auditory brain parse acousticlandscapes, bind sensory inputs, adapt its filters, hear through noise anddistortion? Could autonomous listening devices emulate neurology to match orexceed human auditory analysis, e.g., to detect and identify speech targets innoise and reverberation?

42%

11%

47%

Primary Strategy: Forge useful connections between math and biology

3 +6


4/29

4

Other OrganizationsThat Fund Related Work

Funds 5 current AFOSR P.I.s in a new MURI, Animal Inspired Robust

Flight with Outer and Inner Loop Strategies. (M. Steinberg)ONR

ONRProgram, Bio-Inspired Autonomous Systems, (T. McKenna) has focuson aquatic environment, held joint review with AFOSR in May 2010

WRAMCCoordinates psychoacoustics with AFRL/RH, and extends our

6.1 research to hearing-impaired populations. (D. Brungart)

ARL MASTMicro Autonomous Systems and Technology program forurban and complex terrain funds 6.2 work on MAVs.

NSF Partners with AFOSR to support the annual workshop onNeuromorphic Modeling, involving several AFOSR P.I.s

NSF Broad programs in biomathematics (M. Horn), perception, action &cognition (B. Tuller), NSF-NIH computational neuroscience

5

NRL Applies AFOSR 6.1 to MAV development; funds related 6.2, 6.3

. . . plus international & 6.2 coordination


5/29

5

International and 6.2Coordination

U.K.

T. Daniel

S. Humbert

M. Willis

U.S.AFRL-Dstl Working Group

Biologically-Motivated Micro-Air-Vehicles

STATE OF THE ART REVIEW

Georgia Tech15-18 June, 2010

Organizers: M. Wehling, AFRL. P. Biggins, Dstl

Presentations:https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=24091294

&objAction=browse&viewType=1

30 Participants from UK, US, Industry, Academia, & Gov.

H. Krapp

J. Niven

G. Taylor
https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=24091294&objAction=browse&viewType=1https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=24091294&objAction=browse&viewType=1https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=24091294&objAction=browse&viewType=1https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=24091294&objAction=browse&viewType=1http://en.wikipedia.org/wiki/File:Georgia-Tech-Insignia.svg


6/29

6

Recent Highlights

Best Paper, 2010Journal of

ExperimentalBiology

Chiu, et al., Paper oncompetitive target capture

by echolocating bats

AFRL Fellow2010

Richard McKinleyHuman Effectivenes Directorate

Acoustic Research

Special IssueHearing Research

Mechanics ofHearing Workshop


7/29

7

2 Photoreceptor

Rows havemembrane for wave phase delay

Crossed parallelmicrovilli in lowersection conferlinear pol

sensitivity

Photoreceptorrows 1-6 in O.Scyllaruscompound eye

Visual Photoreceptor

Discriminates L vs RCircular Polarization

Receptor Membrane

Imparts a Uniform 1/4Wave Delay, 400 700 nm

Comparison with Best Available Materials

QUARTZOPTIMAL GRATING

Two Recent Discoveries

T. Cronin, (UMBC), J. Marshal (Queensland), N. Roberts (Manchester)
http://en.wikipedia.org/wiki/File:Circular.Polarization.Circularly.Polarized.Light_Right.Handed.Animation.305x190.255Colors.gif


8/29

8

AFOSR Young Investigator DevelopsTransformative Science

Math predicts a previously unknown &unexpected mode of neural behavior:prolonged electrical silencing, thought tobe physiologically unsustainable.

Math captures details of intracellular

gene expression / transcription andmembrane electrochemistry.

Confirmed during 2010 in several labs

Finding revises basic assumptions forneural computation in cells & networks.

MEMBRANEDEPOLARIZATION

Science 326, 9 Oct. 2009D. Forger, AFOSR YIP.

NORMALACTIONPOTENTIALS

ELECTRICALLYSILENT

~ 5 Hrs.
http://www.umich.edu/http://www.umich.edu/


9/29

9

Recent Transitions

To Army: VLSI implementation of insect visual sensory fusion for a motion-

sensitive guidance system. Contract W56HZV-100192. (Dr. P. Shoemaker)

To Oticon A/S: Method to boost directional hearing via monaural tandemanalysis of pitch and voice segmentation. Dr.Ulrik Kjems, J. Woodruff. (Dr. Wang)

To NRL: Methods for cooperative steering of autonomous surface & air vehicles,from control law mathematics based on bats & dragonflies. Dr. Justh, NRL/TEMD

To AFRL 6.2: Spatial audio sorting, annotation, and retrieval system enhancesvoice communications for multiple sources. Victor Finomore, RHCB. (Dr. N. Iyer)

To NSMRL: Simulators and technical method to calibrate underwater acousticstimulation of the human head. Dr. Michael Qin. (Creare, Inc. STTR, Dr. A. Dietz.)

To AFRL 6.2: Speaker recognition method, based upon binary mask technique,boosts performance in additive noise. Brett Smolenski, RADC/Rome Lab. (Dr. Wang)

To EmergentViews.com: New technique for polarization imaging, basedupon 6.1 AFOSR work accomplished under the BioInspired Theme. (Dr. N. Engheta)

To NSA & CIA: Method to sort speech from non-speech in noisy electronicsignals, based upon cortical model from experiments with ferrets. (Dr. S. Shamma)

7


10/29

10

150 dBA SPL

Most AF Flight-Line Crews

suffer Hearing Loss

A 50 dB Technical Transition

ResearchGoal:

AFOSR LaunchedMultiple Efforts:

Discover how high-level soundtransmits through air, bone, andtissue to the human cochlea.

Enable 50 dB acoustic isolation,with no sacrifice of voicecommunications.(Legacy level was 30 dB)

6.1 Research in Support of DTO HS-33

Measure nonlinear loudness compression in bone conduction Model acoustic wave propagation through skull Develop a physiologically realistic, instrumented human head simulator Measure and model dynamics of middle ear transduction

Develop new techniques for noise cancellation, active and passive

6.16.1STTRSTTR

STTR


11/29

11

Technical TransitionAchieved 50 dB Attenuation Goal of DTO HS-33

Program Coordination with: NATO, ONR, ARO, AFRL 711th HPW

Wright-Patt Bioacoustics Lab tests New

Helmet for Navy Carrier Crew

Instrumented Head Simulatorenabled acoustic tests withoutrisk to human listenersHanover, N.H.

Coordinated 6.1, 6.2efforts enabled a majorbreakthrough in hearing

protection
http://en.wikipedia.org/wiki/File:CardSeal-1.gifhttp://en.wikipedia.org/wiki/File:CardSeal-1.gifhttp://en.wikipedia.org/wiki/File:UIUC_seal.svghttp://en.wikipedia.org/wiki/File:Dartmouth_College_shield.svg


12/29

12

Auditory Modeling for Acoustic Analysisand Audio Displays

0 0.2 0.4 0.6 0.8 1 1.2-0.1

-0.05

0

0.05

0.1

Time (s)

Amplitude

Speech signals encode information in low-frequencyenvelopes modulating high-frequency carriers

Cortical theory for speech detection and recognition

Innovative signal processing ideas

based upon neural mechanisms

ResearchTopics

Modulation analysis of acoustic signals

3D spatial audio displays to optimize human performance

Biophysical basis of3D spatial hearing.

(Wm. Hartmann, MSU)

Computational auditory scene analysis

3 +6

REPORTEDIN RECENT

AFOSRREVIEWS

NEW IN 2011:

NEXT SLIDES:


13/29

13

INNERROW

OUTERROWS

COCHLEAR HAIR CELLS

Cochlea behaves as a system ofadaptive, tunable oscillators.

Triplet Tuning

Adjacent groups of neural fibersrespond strongly to the same key

frequencies in a vowel sound

R. Kumaresan, V. Peddinti (U. Rhode Island) & P. Cariani (Harvard), ICASSP 2011

Synchrony Capture in Auditory Nerve:inspires method for multi-frequency tracking


14/29

14

Auditory Model emulates SynchronyCapture to achieve multi-pitch tracking

R. Kumaresan, V. Peddinti (U. Rhode Island) & P. Cariani (Harvard), ICASSP 2011

New architecture for

auditory signal processingemploys synchrony capture

via adaptive band-passfilters that emulate cochlear

mechanics.

NEXT GOAL: Use model to explain other phenomena, e.g., distortion

products, two-tone suppression, gain control.

Spectrogram of a speech fragment overlaid

with multiple frequency tracks obtained viasynchrony capture.
http://en.wikipedia.org/wiki/File:Uriseal.pnghttp://en.wikipedia.org/wiki/File:Uriseal.png


15/29

15

Neural Model generates Auditory Nonlinear

Resonances to match Neural Brain Signals

Florida Atlantic Univ., E. W. Large, F. Almonte, unpublished (used with permission.) Data from Lee, et al. 2009

INPUT ENERGY

F1 F2Background:

Conventional auditory modelingdoes not account for highlynonlinear neural responsepatterns, e.g. in inferior colliculusor brainstem EEG.

Hypothesis:These nonlinear patterns arekey to auditory cognition.

Progress:Arithmetic combinations of F1 and F2,arise here in brainstem data and in anonlinear coupled oscillator model,fitted with one free parameter (gain).

Neural Responses to a two-tone complex

F2 / F1 = 1.7

STTR:CircularLogic
http://en.wikipedia.org/wiki/File:Florida_Atlantic_University_seal.svg


16/29

16

1 1( ) ( ) ( ( ))k

K

k

k k

k

K

m tx t x t c t

Complex Modulators Carriers

AFOSR / Rome Lab Symposium onCoherent Modulation Analysis

Acoustic Signal

. . .A key problem in decomposition of speech & other acoustic waveforms

Dr. Les Atlas, University of Washington

Coherent Demodulationis a new

approach for feature extraction andimproved intelligibility. Out-performs conventional incoherent,AM/FM, and phase vocoding methods.

Participants8 July 2010 J.Grieco, D. Harris, B. Pokines, J. Parker, S. Wenndt, K. Godin, A. Noga,J. Cupples, S. Johns, B. King, P. Clark, L. Atlas, W. Larkin

Time

Frequency

Eight harmonics

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.10

500

1000

1500

2000

2500

3000

8-Component Modulation Spectrum for aSpeech Sample
http://en.wikipedia.org/wiki/File:University_of_Washington_Seal.pnghttp://en.wikipedia.org/wiki/File:University_of_Washington_Seal.png


17/29

17

ResearchTopics

NEXT: BIOSENSORY

Flight and Navigation

Neuromorphic emulation of inner- and outer-loop control

Airfoil mechanosensors in bats

Nocturnal navigation by echolocation or optical polarization

Airfoil, antennae mechanosensors inhawkmoths, insects Sensorimotor reflex basis forcooperative formation control

Bio-clock compensated opticalcompass navigation

Insect flight stabilization & self-motion tuning

Target tracking and pursuit in the dragonfly

REPORTEDIN RECENT

AFOSRREVIEWS

NEW IN 2010 & 2011:

NEXT SLIDES -- RECENT DISCOVERIES:


18/29


19/29

19

Virtual Reality wind tunnelallows simultaneous tracking offree flight & computer-projected

imagery on the walls and floor

Discovery: Insects Sensorimotor

Altimeter Ignores Ventral Angular Velocity

Scientific Question:How do insects controlflying altitude?

Background:

Flies were assumed to matchflying height to a preferred rate ofoptic flow on the ground.

Caltechs Velocity Clamp lab

enables experimental control ofdynamic optic flow, independent offlight velocity in a wind tunnel.

Discovery: Flies adjust height to match nearby horizontal features. They do not regulate optic flow rate for this purpose. They rely on optomotor and collision-avoidance reflexes.

AFOSR Young Investigator. A. Straw, Caltech. Current Biology 19 Aug. 2010.

Di R ti Fi ld S lf M ti
http://en.wikipedia.org/wiki/File:Caltech_logo.svghttp://en.wikipedia.org/wiki/File:Caltech_logo.svghttp://en.wikipedia.org/wiki/File:Caltech_logo.svg


20/29

20

Maps of motion sensitivity in right visual field show a strong responseto yaw motion to the right and inhibition to forward translation

Discovery: Receptive Field Self-MotionTuning is Conserved across Species

Self-motion tuning is key to understanding gaze stabilization

Tuning organization is conserved across these 3 species.

Gaze stabilization is essential for flight control keeps visual

sensors aligned with the inertial reference frameH. Krapp, et al., AFOSR Report Nov. 2010. Imperial College, London
http://en.wikipedia.org/wiki/File:Imperial_College_London_crest.svghttp://en.wikipedia.org/wiki/File:Imperial_College_London_crest.svg


21/29

21

3 Ocelli

(polarizationreceptors)

Lobula PlateVS Neuron

RotationTuning Curve

Primary RotationAxes of Ocellar

Neurons

Flight Control relies on Sensor Fusion inInsect Optical Processing

M. Parsons, et al., Current Biology 2010. Cambridge University.

Yaw

Rotation Fusion Site Discovered inLobula Plate Neurons

Slow Inputs from compoundeye combine with fast inputsfrom polarization receptors

(reflex latencies are 20 to 30ms versus 6 ms for ocelli.)

Both are tuned to specific,but different axes of rotation

Ocellar system has lowerangular precision than thecompound eye system.

Sensory information alignswith axes of natural modes of

insects flight instability


22/29

22

Insect Gaze Stabilization viaFeedback/Feedforward Sensor Fusion

HeadDeflection

HeadRoll

HALTERES

COMPOUND EYESNECK MOTOR

SYSTEM

THORAX

ROLL

Theory: Short latency mechanosensors firstdetect body rotation, then feed forwardto induce compensatory head roll via

neck motor system

Long latency visual system detectsresidual optic flow feedback fromincomplete compensation. Data fit preliminary linear model.

Compound eyes plus halteres and ocelli

Compound eyes plus halteresCompound eyes only

H. Krapp, et al., AFOSR Report Nov. 2010. Imperial College, London


23/29

23

Target trajectory

Tracking?

Interception?

OR

R. Olberg, Union College, A. Leonardo, Howard Hughes Medical Institute

Dragonfly Attacks Moving Target

21 sec. VIDEO


24/29

24

t = -28ms

Evidence for Interception

R. Olberg, Union College, Schenectady, NY

Dragonfly movesgaze prior to launch

Launch angleleads target

Launch angle depends ontarget velocity prior to

takeoff

Dragonfly plans launch direction while still on its perch

Dragonfly Predicts Prey Flight Path
http://en.wikipedia.org/wiki/File:UnionCollegeSeal.svghttp://en.wikipedia.org/wiki/File:UnionCollegeSeal.svg


25/29

25Dragonfly perch

Dragonfly Predicts Prey Flight Path,Updates Interception Path in Real Time

Neural System Analysis:Relates pursuit time &

accuracy to small-targetselective descending (TSD)

neurons in ventral nerve cord.

~ 300 ms

Dragonfly Eyes


Dragonfly visual neurons code for
http://en.wikipedia.org/wiki/File:UnionCollegeSeal.svghttp://en.wikipedia.org/wiki/File:UnionCollegeSeal.svg


26/29

26

Dragonfly visual neurons code for

both past and future target positions

Neurons tuned to targetlocations in the visual fieldalso convey informationabout past, current, orfuture target positions

Predictive CodingCoding Past Position

Receptive fieldInformation peaks 30 ms

after cell spikes

Receptive fieldInformation peaks 30 ms

before cell spikes


Neurons have nearlyorthogonal sensitivity

to direction of motion

Next Research Stage:
http://en.wikipedia.org/wiki/File:UnionCollegeSeal.svghttp://en.wikipedia.org/wiki/File:UnionCollegeSeal.svghttp://www.google.com/imgres?imgurl=http://www.globalcraftsb2b.com/catalog/images/dragonfly.jpg&imgrefurl=http://www.globalcraftsb2b.com/catalog/index.php%3FcPath%3D121_69&h=400&w=400&sz=28&tbnid=jDbdE_V39SNQcM:&tbnh=124&tbnw=124&prev=/images%3Fq%3Ddragonfly%2Bpictures&zoom=1&q=dragonfly+pictures&hl=en&usg=__IAK85ePRP-6nUwcU9VQXDGqHrck=&sa=X&ei=v5ULTaz2HYL6lweS-qGlDA&ved=0CCYQ9QEwAw


27/29

27

Next Research Stage:Neural Telemetry in Free Flight

Flight Arena

Perch Platform

Cameras

Multi-contactprobe in

mesothoracicganglion

Telemetry chipmounts behind legs,recharges on perch

platform.

R. Olberg, Union College,R. Harrison, Univ. of Utah A. Leonardo, Howard Hughes Medical Institute

SUMMARY:
http://en.wikipedia.org/wiki/File:Hhmi_logo_small.pnghttp://en.wikipedia.org/wiki/File:Hhmi_logo_small.pnghttp://en.wikipedia.org/wiki/File:UofU_official_seal.pnghttp://en.wikipedia.org/wiki/File:UnionCollegeSeal.svg


28/29

28

SUMMARY:Transformational Impacts & Opportunities

4

Advanced auditory modeling:

Hearing protection:

Optical processing:

Autonomous flight control:

Adaptive airfoils based upon bio-sensory mechanisms Guidance from neural systems, not from networks Discover sensorimotor basis of formation flight

Polarization vision and signaling adapted from biology Achromatic 1/4 wave optical retarders Emulating compound eye in new optical devices

Mathematics for coherent modulation analysis Neural-Inspired analyses to parse acoustic scenes

Massive improvements in high-noise attenuation.
http://images.google.com/imgres?imgurl=http://evolution.berkeley.edu/evosite/evo101/images/bat_echo.gif&imgrefurl=http://evolution.berkeley.edu/evosite/evo101/IIIE5cExaptations.shtml&usg=__xm5JuB-ltO06AUA5RIKytq8Co1g=&h=208&w=233&sz=7&hl=en&start=4&um=1&itbs=1&tbnid=J6qsPgdIc7X-aM:&tbnh=97&tbnw=109&prev=/images%3Fq%3Dbat%2Becholocation%26hl%3Den%26sa%3DX%26um%3D1


29/29

29

Questions?

Thank you for your attention

Willard Larkin, Program Manager, AFOSR/NL

703-696-7793

[email protected]
mailto:[email protected]:[email protected]

Date post:	08-Apr-2018
Category:	Documents
Upload:	afosr
View:	231 times
Download:	0 times

4. Larkin Sensory

Documents