Joint Research Centre the European Commission's
in-house science service
The Blue Hub: https://bluehub.jrc.ec.europa.eu
Exploiting maritime Big Data,
the Blue Hub
Marlene Alvarez Alvarez,
Michele Vespe, Harm Greidanus
2
Available (big) data on ship traffic:
Ship position reports
(AIS, LRIT)
Earth observation
satellites (e.g., Copernicus)
From data to knowledge
Contents
3
Irregular migration
Maritime Security
Maritime Traffic
Piracy Illegal Fishing
Oil Pollution
Maritime Situational Awareness (MSA)
Which ships are on the sea?
What are they doing?
4
Ship reporting Data
DC
INMARSAT GPS
Long Range Identification and Tracking (LRIT) Automatic Identification System (AIS)
REGULAR (space and time)
IRREGULAR (spatial/temporal gaps)
FREQUENCY Fix (to 6 hours)
FREQUENCY High (few sec to min)
ACCESS Restricted
ACCESS Free
5
Collection of AIS ship reports
Received by satellites,
when they pass over
Collected by coastal receiver
networks, continuously
Picture: FleetMon.com
Picture: exactEarth
GLOBAL
COVERAGE LIMITED
COVERAGE
CONSTELLATIONS improved frequency revisit
6
Radarsat-1 Fine © CSA/MDA
Earth observation satellites
Optical - high resolution
- Small area
- Daytime, clear skies
Radar - low resolution
- Wide area
- Through clouds and night
DigitalGlobe
Used for recognition Used for detection
7
Vessel Movement Data
Big Data (historical)
Real-Time Maritime
Surveillance
Data Fusion & Target Tracking
Routes Extraction
Anomaly Detection
(deviation from normality)
Routes prediction
Data Mining & Knowledge Discovery
Fishing grounds and fishing
efforts
Fisheries socio-economics
Mapping Off-shore and exploration
Maritime Spatial Planning
Gridding Shipping Emissions
Mapping activities at sea
Vessel profiling/ intelligence
Customs & Agencies
Ex-post policy evaluation
Geopolitical issues impact on
transport
Fisheries management
Trade Indicators
The Blue Hub – exploiting maritime Big Data
8
…
Ship Name
…
…
…
Data Fusion & Target Tracking at global scale
Track built by integrating multiple AIS data providers (ship-type color coded).
Vessel Movement Data
Real-Time Maritime
Surveillance
Data Fusion & Target Tracking
“BIG DATA”
- ~150,000 ships
carry transponders
- each may send
10,000 messages /
day
9
1,557 S-1 images (3 Oct 2014 - 11 Sep 2015)
41,928 ships detected
correlated with AIS
34,263 ships
uncorrelated with AIS
AIS Coverage
Data: Copernicus
Sentinel-1: Long term ship monitoring
10
One-month of EU LRIT CDC data, revealing the main global traffic routes and
enabling the implementation of innovative decision support tools
→ Video Link
From EU to worldwide tracking and traffic routes
Big Data (historical)
Vessel Movement Data
11
FishingDredging
Tug
Passenger
Cargo
Cargo,
Haz. Cat. X
Cargo,
Haz. Cat. Y
Cargo,
Haz. Cat. Z
Cargo,
Haz. Cat. OS
Tanker
Tanker,
Haz. Cat. X
Tanker,
Haz. Cat. YTanker,
Haz. Cat. Z
Other Type,
Haz. Cat. X
Travel Time Histogram
2-week raw AIS data
Pattern discovery
Entry area
Exit areas
Port area
1.2 1.3 1.4 1.5 1.6 1.7 1.8
50.6
50.7
50.8
50.9
51
51.1
51.2
Longitude
Latit
ude
Network representation
1.2 1.3 1.4 1.5 1.6 1.7 1.8
50.6
50.7
50.8
50.9
51
51.1
51.2
Longitude
Latit
ude
Single route analysis
Vessel pattern knowledge discovery: from raw data to support to decision
Fernandez Arguedas et al.: „Automatic Generation of Geographical Networks for Maritime Traffic Surveillance‟, Int. Conf. on Information Fusion (2014)
Big Data (historical)
Routes Extraction
Data Mining & Knowledge Discovery
Ship type distribution
Maritime Spatial Planning
12 Zampieri A., Vespe M., Westra M., Alvarez M., Greidanus H., „A future for historical LRIT data‟, International Maritime Organization (IMO) – Maritime Safety Committee (MSC) 95th session, London, 2015
Historical LRIT data can be used to predict where a vessel will be up to a few days in advance
Route prediction
Big Data (historical)
Routes Extraction
Data Mining & Knowledge Discovery
Routes prediction
13 Greidanus H, Vespe M., Alvarez M., „Detection of Anomalous behaviour in ship reporting data for improved maritime security‟, Future Security Conference, Berlin, 2016
Anomaly Detection (deviation of normality)
- Density map based on 2-y LRIT data - Overlaid real-time positions of all cargo ships - Multi-day past track from one cargo ship
14
Event-based Knowledge Discovery
Raw Data
Event map generation
Vessel based event detection
Pre-Processing
Trajectory Extraction Georeferenced grid
Vessel Objects
Event Objects
• Vessel Id • Dynamic • Static/Voyage
• Event type • Dynamic • Static/Voyage
Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., “AIS Event-Based Knowledge Discovery for Maritime Situational Awareness”, International Conference on Information Fusion 2016.
15
Vessel-based Event Detection
Longitude
Latitu
de
Event Type Relevance
AOI Enter/Exit Access to protected areas or areas regulated by traffic routing schemes
Cell Enter/Exit Traffic density maps
Birth/Death Coverage maps
Start/Stop Detection of stop areas (ports, offshore platforms)
Proximity Discovery of illegal activities
Fishing/Steaming Mapping activities at sea
AIS on/off switching Discovery of illegal activities
Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., “AIS Event-Based Knowledge Discovery for Maritime Situational Awareness”, International Conference on Information Fusion 2016.
16
Event Map Generation
Event Object
MMSI
Longitude
Latitude
SOG
COG
ROW
COL
Event timestamp
Vessel type
Event type
DWH
Presentation Layer
Analysis Cube
Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., “AIS Event-Based Knowledge Discovery for Maritime Situational Awareness”, International Conference on Information Fusion 2016.
17
Event Map Generation: Results
AOI: [15ᵒ W 17ᵒ E; 43ᵒ N 46ᵒ N]
Time interval: October 1st - December 1st, 2015
Terrestrial AIS data from Italian Coast Guard
Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., “AIS Event-Based Knowledge Discovery for Maritime Situational Awareness”, International Conference on Information Fusion 2016.
Distribution of generated events
0.004 degree
18
Entry into cells
Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., “AIS Event-Based Knowledge Discovery for Maritime Situational Awareness”, International Conference on Information Fusion 2016.
Event Map Generation: Results
19
Entry into cells – Fishing Vessels
Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., “AIS Event-Based Knowledge Discovery for Maritime Situational Awareness”, International Conference on Information Fusion 2016.
Event Map Generation: Results
20
Entry into cells – Fishing Vessels (Fishing behaviour only)
Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., “AIS Event-Based Knowledge Discovery for Maritime Situational Awareness”, International Conference on Information Fusion 2016.
Event Map Generation: Results
21
Entry into cells – Cargo Vessels
Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., “AIS Event-Based Knowledge Discovery for Maritime Situational Awareness”, International Conference on Information Fusion 2016.
Event Map Generation: Results
22
Entry into cells – Tanker Vessels
Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., “AIS Event-Based Knowledge Discovery for Maritime Situational Awareness”, International Conference on Information Fusion 2016.
Event Map Generation: Results
23
Entry into cells – Passenger Vessels
Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., “AIS Event-Based Knowledge Discovery for Maritime Situational Awareness”, International Conference on Information Fusion 2016.
Event Map Generation: Results
24
Stop events
Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., “AIS Event-Based Knowledge Discovery for Maritime Situational Awareness”, International Conference on Information Fusion 2016.
Event Map Generation: Results
25
Left: North- (green) and South-bound (red) traffic crossing the Indian Ocean
Right: Traffic extraction during the second semester 2009 using historical LRIT data
Analysis of geopolitical developments:
the declining impact of piracy in the Indian Ocean
Vespe M., Greidanus H., Alvarez M. : „The Declining Impact of Piracy on Maritime Transport in the Indian Ocean: Statistical Analysis of 5-year Vessel Tracking Data‟, Marine Policy, 2015.
Big Data (historical)
Data Mining & Knowledge Discovery
Mapping activities at sea
Ex-post policy evaluation
Geopolitical issues impact on
transport
26
Time series of traffic crossing the Indian Ocean in 5 years (each figure shows 6 months),
showing the effect of piracy and its progressive decline in deviating maritime traffic
Vespe M., Greidanus H., Alvarez M. : „The Declining Impact of Piracy on Maritime Transport in the Indian Ocean: Statistical Analysis of 5-year Vessel Tracking Data‟, Marine Policy, 2015.
Analysis of geopolitical developments:
the declining impact of piracy in the Indian Ocean
27
Piracy Incidents Source: EU Naval Force – Somalia
Analysis of geopolitical developments:
the declining impact of piracy in the Indian Ocean
Time series of traffic crossing the Indian Ocean in 5 years (each figure shows 6 months),
showing the effect of piracy and its progressive decline in deviating maritime traffic
Vespe M., Greidanus H., Alvarez M. : „The Declining Impact of Piracy on Maritime Transport in the Indian Ocean: Statistical Analysis of 5-year Vessel Tracking Data‟, Marine Policy, 2015.
28
Analysis of geopolitical developments:
the declining impact of piracy in the Indian Ocean
Vespe M., Greidanus H., Alvarez M. : „The Declining Impact of Piracy on Maritime Transport in the Indian Ocean: Statistical Analysis of 5-year Vessel Tracking Data‟, Marine Policy, 2015.
LRIT derived Speed Over Ground [kn] distributions change over the 5 years:
a progressive reduction of speed + the second peak (around 18 knots) disappears mid-2012
Higher speed: - reduces risk of piracy attacks - raises fuel costs
29
Track and speed profile of a trawler showing three clusters of velocities corresponding to in port, fishing (position highlighted in yellow) and steaming behaviours. Vespe M., Gibin M., Alessandrini A., Natale F. Mazzarella F., Osio G., „Mapping EU
fishing activities using ship tracking data‟, Journal of Maps, 2016.
Track and speed profile of vessels
30
1) Raw data 2) Fishing Activities
4) Real time fishing matching 3) Density of Fishing Activities
Mazzarella F., Vespe M., Damalas D., Osio G.: „Discovering Vessel Activities at Sea using AIS Data: Mapping of Fishing Footprints‟, Proc. 17th Int. Conf. on Information Fusion, 2014.
Knowledge Discovery: fishing activities
31 Vespe M., Gibin M., Alessandrini A., Natale F. Mazzarella F., Osio G., „Mapping EU fishing activities using ship tracking data‟, Journal of Maps, 2016.
EU trawlers Fishing intensity
Coverage
32 10 October 2016 32
Trajectories cleaning and splitting
Port activity and trade indicators
33 10 October 2016 33
CB-SMOT
Port activity and trade indicators
34 10 October 2016 34
DB-SCAN
Port activity and trade indicators
35 10 October 2016 35
Concave-hull and buffer
Port activity and trade indicators
36 10 October 2016 36
Semantic annotation
Port activity and trade indicators
37
Weekly Distribution
Port of Gioia Tauro Port of Genoa
Port activity and trade indicators
38
Time Distribution In/Out
Port of Gioia Tauro Port of Genoa
Port activity and trade indicators
39
Port of Genoa: link to Mediterranean Ports
Port activity and trade indicators
40
Port of Genoa: link to Mediterranean Ports
Port activity and trade indicators
41
Port of Genoa: link to Mediterranean Ports
Port activity and trade indicators
42
Conclusions
• Vessel tracking data can be used to understand,
map and quantify activities at sea, including how
they change over time
• It is possible to turn large amount of data into
elements that can be useful to policy makers
• The knowledge it is extracted directly from the
data, using data mining and knowledge discovery
techniques
43
Stay in touch
JRC Science Hub:
ec.europa.eu/jrc
Twitter and Facebook:
@EU_ScienceHub
LinkedIn:
european-commission-joint-research-centre
YouTube:
JRC Audiovisuals
Vimeo:
Science@EC
Thank you
