Area Access Surveillance Technologies

S. Fioravanti, A. Tesei November 21, 2012

Outline •  CMRE history and mission •  Underwater Monitoring Technologies •  Hardware design and development •  Detection and Localization •  Classification •  Experimental results •  Conclusions

•  world-class NATO scientific research and experimentation facility, La Spezia, Italy –  ocean science, modeling

and simulation, acoustics and other disciplines

•  over 50 years of service

• The Centre disposes of an unique research structure in the European panorama –  employs scientists from all NATO countries –  two research platforms for experiments at sea –  development systems and laboratories for acoustic

and oceanographic studies –  facilities for various instrument calibration –  electronic and mechanical design laboratories –  autonomous underwater and surface vehicles

1989 End of cold war: many examples of dual-use military technologies →  Design and implementation of a calibration facility for oceanographic

instrumentation which provides assistance to nearly all the Italian marine research institutions and to other countries in southern Europe

→  Design and implementation of advanced environmental monitoring systems →  CMRE starts studies on the effect of anthropogenic noise on marine animals

– main purpose is to draw a mitigation protocol on the influence on animals made by artificial acoustic emissions used for military or geological applications

–  joined several national and international research institution expert on this topic –  from 1999, carried out many big experimental campaigns at sea with the

participation of research institutions from all over the world . . . . .

→ ARGOMARINE: area access monitoring technologies

UW ModemPassive acoustic

Passive acoustic

Autonomous sensing

Acoustic Monitoring

Underwater Monitoring Technologies

3D VIEW

Shore Lab

Acous&c:  Triangula&on  among  distributed  sensors  

Cell-phone GPS

Major differences: •  Complex environment

•  Noise & Sound propagation •  Variety of unknown sound sources (blind monitoring)

EM waves

Acoustic waves

Examples of noise from vessels

Time (sec)

Freq

uenc

y (kH

z)

Slow, mid-size, leisure boat. Spectrogram (dB re. 1µPa)

0 1 2 3 4 5 6 7 8

5

10

15

20

25

30

40

50

60

70

80

90

100

110

Time (sec)

Freq

uenc

y (k

Hz)

NURC rubber boat. Spectrogram (dB re. 1µPa)

5 6 7 8 9 10 11 12 13

5

10

15

20

25

30

40

50

60

70

80

90

100

110

Time (sec)

Freq

uenc

y (k

Hz)

Mid-speed, small ship. Spectrogram (dB re. 1µPa)

0 1 2 3 4 5 6 7 8

5

10

15

20

25

30

40

50

60

70

80

90

100

110

High level @ LF Limited prop. cavitation Few spectral lines

High level in Wide Band Strong prop. cavitation Several spectral lines

High level @ LF & MF No cavitation Many spectral lines

Exploitation of time coherence of signal received by each sensor pair of each array

Requirements: •  Sparse hydrophones (d>>λ) •  High sampling frequency

τ: Time Delay α : Bearing angle

α  = acos ( τ cw / d )

Slant Plane

Hydrophone Pair

α d

Hyd 1 Hyd 2

ΔT

Hyd 1

Hyd 2

τ X-Correlogram EF2 Tripod 1 - Pair 23

Time (sec)

Bear

ing

(deg

)

0 10 20 30 40 50

0

50

100

150

System design •  Each Station

–  Sparse Tetrahedral Array of Four low-noise, preamp hydrophones (100 kHz bandwidth)

–  SCU & Digitalizer (192 kHz SF) –  Pan-Tilt-Compass-Depth sensor (serial data

integrated into digital data flow to shore)

•  Fiber-optic-cable connection to shore

•  Simultaneous acquisition of continuous data flow from both stations on shore –  Real-time Reception, Acquisition, Display

& Processing of data from both stations –  Integration in the same data files of

acoustic, orientation, and other possible serial data

Deployments

•  2011-2012: La Spezia harbor –  Test of performances and

assessment of performances degradation

•  May 2012: 3 weeks in Elba Island at sea recording data with and without ground truth tracks –  collected several Terabyte of data

to be used for algorithm assessment and validation and for classifier training set

Localization from one station and from two stations

y

x

z

Azimuth  Eleva.on  

k

Water D

epth

P

Two Tripods: 2D Triangulation

One Tripod

Tripod1

Tripod2

Top View

θ1

θ2

ARGOMARINE Sea Trials 2012 (NURC & Elba Island)

•  Acoustic characterization of the test sites (ambient noise)

•  Oceanographic data on the field (SVP)

•  Acoustic data collection under controlled conditions: –  Simultaneous data acquisition from BOTH

uw stations during the run of an inflatable boat equipped with GPS antenna

–  Integration of GPS ground-truth position data into acoustic data files

•  Blind acoustic monitoring

GPS  antenna  

1510 1515 1520 1525

5 10 15 20 25 30 35

Sound Speed (m/s)

Dep

th (m

)

CTD 28/05/'12

Tripods deployment: •  40m depth •  About 450-550m off shore •  Sandy-posidonia seabed •  Relatively quiet environment

         Tripod  distance:  120  m  

No evidence of thermoclyne

Enfola (Elba Island) test site

10.269 10.27 10.271 10.272 10.273 10.274 10.275 10.276 42.83

42.831

42.832

42.833

42.834

42.835

Lon E (deg)

Lat N

(deg

)

Fused Track in Geographic Coordinates

GPS ground-truth Estimate

5.446 5.448 5.45 5.452 5.454 x 10 4

200

300

400

500

600

700

Time (sec)

(m)

Fused Horizontal Range seen from Tripod 1

Experimental Results

Station 2 Station 1

GPS ground-truth Acoustic Estimate

Experimental Results Performances: 40 m max error over 700m range (6 %)

ClassificationRVM (Relevance Vector Machine) classifier •  Supervised statistical method (19 features

selected) •  Binary •  Fully Bayesian model ⇒ provides probabilistic

predictions •  No need of a-priori statistics •  Provides selection of most significant features •  Nice balance between simplicity and power

Feature extraction from data

0 20 40 60

40

50

60

70

80

90

100

Frequency (kHz)

dB re

. 1µ P

a/√

Hz

PSD function

0 20 40 60

-20

-10

0

10

20

30

40

Frequency (kHz)

dB

X-PSD function

50 100 150 2000

0.02

0.04

0.06

0.08

Frequency (Hz)

Nor

mal

ized

Am

plitu

de

DEMON SpectrumFrequency (kHz)

Tim

e (s

ec)

Spectrogram.

10 20 30 40

22

24

26

28

Classification results

Slow small boats

Fast small/mid-sized boats

Ships

Slow boats 92 7.0 1.0

Fast mid-sized boats 8.0 89.0 3.0

Ships 0.0 0.0 100

Multi-class confusion rate matrix (%) (Threshold = 0.5)

Pred

True

Three  classes  

-200

2040

05

10150

2

4

6

8

10

Feature # 2Feature # 16

Feat

ure

# 17

Slow, small boats

Fast small/mid-sized boats

Ships (down to ferry size)

Autonomous Sensing Vehicles

•  eFolaga with e-nose

Complete MIS Integration

MOOS Database

HTTP

XML files To & From CNR

XML Transfor

mer

XML Style sheet

MOOS variables

Tracks &

Aco features

AUV positions &

E-nose data

XML status update

Integration with ARGOMARINE MIS

Acoustic Detection, Localization and Classification

•  Concept successfully validated

•  Advanced prototype system

•  Possible exploitations: –  Marine mammal survey –  Monitoring of noise sources with important

environmental impacts (wind-farm piling, regasification ships, etc.)

–  Port protection

Test at sea

•  May 2012: eNose mission integrated into ARGOMARINE MIS

•  Sep. 2012: acquisition of sampling oil signals

•  Nov. 2012: eNose missions with optimal sampling trajectory and real time MIS integration

•  Cooperation with CNR-IFC, Graaltech and CNR-ISTI

OBJECTIVE: Find the optimum sampling designs for an AUV-mooring ocean observing network •  METHODOLOGY: The problem is decoupled into

a) finding the most adequate •  sampling locations for the AUV and b)

to visit these locations in the fastest way.

•  Definition of a space filling design. Try to spread sampling locations throughout the region, leaving as few holes as possible. Sampling points are located to minimize a criterion

•  Solution of the Travel-salesman Problem. Once the sampling locations have been defined, a trajectory of the AUV is computed to visit all the locations selected in the fastest way.

Floats Network

-Unevenly distributed-Same cycling period-Synoptic measures

Glider Network

AUV mission planner Definition of Operational Constraints

• Area • Time constraints • Vehicle speed • Number of vehicles • Obstacles

Planning Module

Space-filling Design

AUV Mission

• Waypoints • Travelled Distance

Genetic Algorithm

Feedback between the space-filling design generator and the genetic algorithm until operational conditions are satisfied.

Find an optimum mission for Folaga AUV to sample the selected marine area, considering the existence of a monitoring buoy and denied areas(red). Mission should take around 1 hr ( 1m/s)

Experimental Design for ARGOMARINE

Optimum trajectory for the Folaga-AUV (dash-dot black line) compatible with operational constraints. The traveled distance is 2962 m.

AUV mission -Result for ARGOMARINE

WP4.5 Integration •  Current vehicle capabilities •  Macro Tasks

–  Surface navigation

–  Gliding mission –  Underwater navigation

–  Idle

–  Vertical Profiler

•  User control mode –  Controlled by external software –  Simple command interface

–  Complete control on devices

•  Emergency –  Release drop weight

Macro tasks and state machine

balloon Valve

pump

MOOS-IvP •  MOOS: Mission Oriented Operating Suite •  IvP: Interval Programming a mathematical programming model

for multi-objective optimization

•  MOOS-IvP is a set of open source C++ modules for providing autonomy on robotic platforms, in particular autonomous marine vehicles –  It provides a framework for data exchange/communication –  separation of overall capability into separate and distinct modules

–  Front-seat/Back-seat concepts

An Overview of MOOS-IvP and a Users Guide to the IvP HelmMichael R. Benjamin, Henrik Schmidt, Paul Newman, and John J. Leonard

Moos-IVP integration:

Behavior examples –  Wait on position

–  Search pattern (lawnmower)

–  Goto location –  E-nose mission on location

–  Go home

E-Folaga Main controller

(front-seat driver)

MOOS-ivp User Control Mode (back-seat driver)

Set of behaviors

Navigation data,

vehicle status

Navigation commands Heading, speed, depth

Acoustic modem

GPRS modem

Radio modem

Argomarine MIS

MOOS database

MOOS DB: Log all variables n  Position n  Comms n  Payloads data n  Mission status n  Etc.

Shoreside station MOOS-iVp

Definition of communication

protocol

TCP/IP GPRS/3G

Many thanks to CMRE team

•  Alberto A. •  Alberto G. •  Alessandra T. •  Federico C. •  Lavinio G. •  Piero G. •  Vittorio G. Alessandro, Marco, Salvatore, Piero

Conclusions

•  Mission accomplished

Hyd. Preamp.

H.P. Filter* VGA* 2-ch

24 bit ADC

to FPGA

* pre-selected in hardware