28-11-17 1www.bdva.eu

Florin Șerban, Anca Liana Costea, Codrina IlieTERRASIGNA

on behalf of TF7 / SG5

Big Data Earth Observation

28-11-17 2www.bdva.eu

Main objective: linking the Earth Observation community and ICT community, in an effective dialog on tackling the big data from space related issues.

Earth Observation white paper

BDVA Earth Observation working group

28-11-17 3www.bdva.eu

BDVA Earth Observation groupGroupleader:

28-11-17 4www.bdva.eu

1. CONTEXT & OPPORTUNITIES2. ECONOMICS & BUSINESS OF EO3. EO DATA APPLICATION AREAS4. EUROPEAN JOINT EFFORTS ON BIG DATA5. BIG DATA IN SPACE – TECHNOLOGIES6. BIG DATA – INFRASTRUCTURE7. EO BIG DATA – SOCIAL DIMENSION8. BIG DATA – EDUCATION9. CONCLUSIONSANNEX I – EU DATA POLICYANNEX II - EARTH OBSERVATION DATA APPLICATION DOMAINS

a. AGRICULTUREb. FORESTRYc. SAFETY / SECURITY (INCLUDING DISASTER MONITORING / RISK ASSESSMENT/ INSURANCE

RISK ASSESSMENT)d. ENERGY - OIL AND GASe. MARINE AND MARITIME (INCLUDING COASTAL ZONE MANAGEMENT/ FISHERIES AND

AQUACULTURE)f. PUBLIC HEALTHg. URBAN DEVELOPMENT (INCLUDING SMART CITIES AND CULTURAL HERITAGE)

ANNEX III - COPERNICUS CORE SERVICESANNEX IV - ESA THEMATIC EXPLOITATION PLATFORMSANNEX V – EC AND ESA EO BIG DATA PROJECTS

BDVA Earth Observation White Paper

28-11-17 5www.bdva.eu

1. Context and Opportunities

4TB per day of Earth Observation open data!

Unprecedented access to a wealth of data that allows analysis and understanding of Earth as never before in human history

Copernicus Market Report 2016evaluated downstream market EUR 2.8 billion for 2015 and predicted EUR 5.3 billion for 2020

28-11-17 6www.bdva.eu

2. Economics and business of EOSector Revenues

for EO in 2015

Revenues for EO in

2020

Comments

Agriculture 71M€ 221 M€ Focus on precision farming onlyForestry 36 M€ 41M€Renewable Energy 22 M€ 44 M€ Solar, wind, hydro, biomassUrban development 45 M€ 99 M€ Urban monitoring

Ocean monitoring 103 M€ 116 M€Oil & Gas 73 M€ -- Consistent estimates for 2020 are not

provided

1. The EO community must implement the appropriate business models to transformthe EO activities into a sustainable, self-supporting business addressing thedemand of the users in the various economic sectors in the most efficient andcost-effective way.

2. The EO and ICT technologies together must manage to make available suitablesolutions to address the Big Data challenges of exploiting EO data.

28-11-17 7www.bdva.eu

3. EO DATA APPLICATION AREASAgricultureForestrySafety / Security / Disaster Monitoring / Risk Assessment/ Insurance Energy / Oil and Gas/ Resource ManagementOcean Monitoring: Marine and Maritime / coastal zone management/ Fisheries and AquaculturePublic HealthUrban Development/ Smart cities/ Cultural HeritageTransport - Navigation

Services / products using EO data (+ standards)Typical clients for EO services and applicationsEO companies: business models Collaboration between domain experts and data analysts Interdisciplinary collaboration Awareness / stimulating the use of EO dataConclusions: EO Big data related needs, barriers, challenges

28-11-17 8www.bdva.eu

4. European Joint Efforts on Big Data

28-11-17 9www.bdva.eu

4. European Joint Efforts on Big Data

H2020 2014, 2014-2015 and 2016-2017 –LEIT SPACEAdditional topics for EO - Societal Challenges (1+2)

Societal Challenge for climate action, environment, resource efficiency and raw materialsBlue Growth (Ocean observation technologies) Sustainable Food Security (smart farming),Competitiveness of the European Space Sector (COMPET)

28-11-17 10www.bdva.eu

4. European Joint Efforts on Big Data

Collaborative Ground SegmentThematic Exploitation PlatformsRegional Exploitation PlatformsData Access and Information Services

28-11-17 11www.bdva.eu

5. EO Big Data in Space - TechnologiesData Vizualization

Visual Data Mining3D Visualisation

Data AnalyticsMachine and Deep Learning

Data ProcessingComplex event processing

Data ManagementSatellite Image Time Series Content Based Image Retrieval

The3DvisualizationofaLandsat7ETM+imagebasedonspectraldescriptors,notingfivesemantic clusters

SampleofimmersivevisualizationofaRS

imagedataset

28-11-17 12www.bdva.eu

6. EO Big Data - InfrastructurePublic Infrastructure

NationalTHEIA Land Data Center (France)Climate, Environment and Monitoring from Space (UK)Etc.

InternationalGeohazard Exploitation Platform (GEP) integrated within EGI infrastructure

Private InfrastructureGoogle Earth Engine Amazon Web ServiceEO Cloud - Earth Observation Innovative Platform Testbed Poland

Hybrid Infrastructure Projected: The European Open Science Cloud

28-11-17 13www.bdva.eu

Weather and climate forecasts Monitoring our water and food resources Ensuring healthy, productive oceans Vulnerable ecosystems monitoring Contributing to human health

7. EO Big Data – Social Dimension

EO technologies should support policymakers in reaching the Sustainable Development Goals agenda of the United Nation.

28-11-17 14www.bdva.eu

8. EO Big Data Education

Key Challenge for Europe: to ensure the availability of highly skilled people

European Data Science Academy Eit Digital Master School

What is a Data Scientist? – the answer given within the H2020 Edison projectBig Data Value Ecosystem: activities with the objective of developing skills, education, and Centers of Excellence around Big Data (H2020)

28-11-17 15www.bdva.eu

Outside Europe

FIELDS program – by the University of California, Riverside (UCR) and NASA’s Jet Propulsion Laboratory - to train the next generation of scientists and engineers in large-scale data analysis and visualization.Initiative Building a Big Data Analytics Workforce in iSchools –University Penn State develops:

Geospatial Big Data Analytics and Geographic EducationGeospatial Big Data Visualization with Technology

8. EO Big Data Education

28-11-17 16www.bdva.eu

9. Conclusions – Challenges

Challenge-1: Generic framework for multi-level Big Data processing

standardization of further specialized data fusion, analytics and adaptive modelling with uncertainty control servicesestimating the propagation of uncertainty across the multiple Big Data sources at pre-processing and post-processing levels

28-11-17 17www.bdva.eu

9. Conclusions – ChallengesChallenge-2: Technical priorities

1. Data Management 2. Data Analytics 3. Data Advanced Visualisation 4. Data Standardization 5. Data ProcessingScalability Quality of data Multiple axis/parameters Ontologies Robustness

Adequate and affordable infrastructure tostore and process data.

Generating reliable high-quality andaffordable information for fieldmanagement (especially in complexurban systems).

Implementation of 3dimensionalvisualization services to improve theunderstanding of spatial relationshipsbetween image texture and topography,allowing land use features to be observednot only from the normal vertical view, butalso at different scales, and differentorientations and perspectives. As example,using this method, the range of sprawl andgrowth of cities can be identified anddetermined.

The lack of some standards, particularly tofacilitate data analysis at multiscale levels, (e.g.TileMatrixSet).

Data Fusion of multisource data.

Enterprise-class data integration that isdynamic - meeting current and futureperformance requirements - andextendable - partitioning for fast and easyscalability.

Evidence-based knowledge isrequired for the management of keypublic health issues.Detailed travel survey data to captureindividual activity-based behavior isrequired for Human Mobility PatternsInferred from Mobile Phone Data.

Access to a business intelligence (BI) andanalytics data in eye-catching and easy-to-understand formats.

Advanced instruments for territorialplanning/management are required. Data set hasto be accurate, objective, reliable, comprehensiveand always up-to-date, following an integration ofmultispectral (and hyperspectral) data fromvarious satellites.

Develop automatic or semi-automatedprocessing techniques of very highresolution imagery to successfully provideproducts that cover extended areas inshort time.

Cloud computing platforms to providevirtual clusters and elastic auto-scaling ofresources.

The lack of trained people onInnovative Data Analysis (SpatialAnalysis and Statistics)

Definition of advanced and integrated datavisualization solutions allowing new ways toview EO and non-EO data, throughinteractive visuals such as bubble charts,word clouds and dynamic geospatial maps.

Implementation of new standards for Big Data(EO, non-EO and fused products) metadata andmanagement.

For multi-temporal monitoring of changeson the land surface implement integratedservices with: Image differencing,vegetation index differencing, principalcomponent analysis, direct multi-dateunsupervised classification, post-classification change differencing and acombination of image enhancement andpost-classification comparison. Developspectral mixture analysis, artificial neuralnetworks for change detectionapplications. Developed a curvelet-basedchange detection algorithm for radarimages.

Designing isolation approach to use adata processing design in any hardwareconfiguration without needing to redesignand retune.

Use of multi-temporal imageryapproach - combined spring andsummer images.

Implementation of customized, innovative,networked and highly affordablevisualization solutions (e.g. for Smart CityPlatforms - indoor and outdoor areas).

Developing interoperable data infrastructurethrough extensible governance and metadatalifecycle framework.

Developing systems for a stable and fastreal-time data handling that can interpretand provide results in real-time.

28-11-17 18www.bdva.eu

9. Conclusions – Challenges

Challenge-3: Data Analysts and Knowledge Experts

Challenge-4: Data driven servicesThe main challenges of this kind of services are heterogeneity, lack of structure, error-handling, privacy, timeliness, interoperability and effective dissemination at all stages of the analysis workflow. These challenges are common across the different business models and it is not possible to find a single solution applicable to all of them.

28-11-17 19www.bdva.eu

8. ConclusionsSocial: The development of a fruitful and efficient ecosystem of Big Datain EO would have a great social impact in critical areas such as Weatherand climate forecasts, management of water resources, ensuring healthyand productive oceans, keeping vulnerable ecosystems safe, humanhealth and much more.

Economic: Business opportunities must be promoted to capitalizecurrent EO data. Public End-Users adoption needs to be encouraged bymarket-oriented policies (i.e. governance that promote privatedownstream services). In the near future, the innovation should focus onEO data feeds in other domains than the ‘traditional’ ones and oningesting other available data (than the ones already ‘traditionally’integrated) into EO services / applications.

Technical: Big EO Data collection, dissemination, processing and addedvalue service delivery for End-Users represent existing technologicalchallenges.

28-11-17 20www.bdva.eu

Thank you!Florin Șerban: florin.serban@terrasigna.comAnca Costea: anca.costea@terrasigna.comCodrina Ilie: codrina.ilie@terrasigna.comTERRASIGNA & BDVA TF7 / SG5