33
AGENDA
Time Content Presenter
12:00 Introduction Phil Delaney
12:05 DEA Overview Phil Delaney
12:15 Data and Metadata Alex Leith
12:25 What is the Open Data Cube Alex Leith
12:35 Using the Open Data Cube Felix Lipkin
12:50 Cube in a Box Alex Leith
1:00 Questions All
1:20 Next Steps and Close Eva Rodriguez
WEBINAR OUTCOMES
• Purpose: Understand how you can use data and services from DEA
• Collaborate: Discuss the opportunities for the private sector to utilise DEA data
• Learn
• Data and Metadata:
• Find information about data
• Access and download data
• The Open Data Cube:
• What is the Open Data Cube
• How can you use the Open Data Cube to analyse and process
• Support: Know where to go for help, support and questions
BEING AGILE:
REQUIREMENTS AND SOLUTIONS EVOLVE COLLABORATIVELY
RELEASING THINGS BEFORE THEY
ARE PERFECT
5
DEA Overview
6
• Australia used to be a ‘downlink’ for Landsat
• Data was delivered back to the US
• Raw data was stored on tapes
• Data eventually was digitized in Unlocking the Landsat Archive project (with CRCSI)
• The Australian Geoscience Data Cube (ARDC) was developed
• Rewrite was undertaken making it more generic (ARDCv2)
• Launched as Digital Earth Australia in 2017
• Code released as Open Data Cube
A brief history of the DEA
DIGITAL EARTH AUSTRALIA – WHY?
DEA will support innovation and growth in the digital economy and drive increased productivity across a wide range of sectors.
DEA will improve the efficiency and effectiveness of the Australian Government’s investments and improve how we manage our natural resources.
Tracking agricultural change
DEA ROADMAP
The DEA Roadmap:
• outlines our strategic framework and provides an overview of the tactical direction of the DEA program.
• is constantly reviewed and regularly updated with the goal of increasing the value of the program to its users.
• is not an immutable set of projects and products that will be delivered to a specific timeline.
www.ga.gov.au/dea/news
15
Data and Metadata
17
What’s a COG
A Cloud Optimized GeoTIFF (COG) is a regular GeoTIFF file, aimed at being hosted on a HTTP file server, with an internal organization that
enables more efficient workflows on the cloud. It does this by leveraging the ability of clients issuing HTTP GET range requests to ask for
just the parts of a file they need.
https://www.cogeo.org/
18
https://blog.mapbox.com/combining-the-power-of-aws-lambda-and-rasterio-8ffd3648c348
19
COGs are supported natively in the ODC
https://blog.mapbox.com/combining-the-power-of-aws-lambda-and-rasterio-8ffd3648c348
20
What is the Open Data Cube
And why am I here talking to you
21
A definition of the project
According to the OpenDataCube.org website:
“The objective of the ODC is to increase the impact of satellite data by providing an open and freely accessible exploitation tool, and to foster a community to develop, sustain, and grow the breadth and depth of applications.”
A Python library that facilitates working with raster data
22
The technical components
In a nutshell:
• Data
• An index
• Software
23
Using the Open Data Cube
Felix Lipkin and Phil Delaney
24
DEMONSTRATION
25
Cube in a Box
26
Cube in a Box
• Works locally (Docker Compose)
• Works on AWS (CloudFormation and Docker Compose)
• Enables several things:
• Test environment
• Development environment
• Disposable workspace
• Rapid evaluation
27
Cube in a BoxEnabling technology
• Docker
• Jupyter
• Cloud Optimised GeoTIFFs (and relevant cloud data stores)
• GDAL and RasterIO
• PostgreSQL
• The Open Data Cube
28
Running Cube in a Box Locally
• Requirements: Docker and Docker Compose
• Very fast to start• Easy to scrap workspace and
start again• Configured with Landsat 8 and
DEA data on AWS for auto-indexing
• Get running with Open Data Cube in minutes
29
Running Cube in a Box on the cloud
• Template based using AWS’ CloudFormation
• Deploys in 3 minutes
• Takes 5 minutes to index data
• Ready to run with Landsat 8 data from AWS auto-indexed
• Also able to index DEA data
ACCESSING DEA NOTEBOOKS
• Register for a GitHub account here:
https://github.com/join?source=header-home
• Use the link below to access various notebooks:
• FrontierSI DEA Sandbox – smaller study areas, amazon deployment
https://jupyterhub.test.frontiersi.io
For users who have access to the National Computational Infrastructure:
• Geoscience Australia DEA User Guide
http://geoscienceaustralia.github.io/digitalearthau/
RESOURCES TO ANSWER YOUR QUESTIONS
• DEA technical questions
▪ Open Data Cube Slack Channel http://slack.opendatacube.org/
• Webinars, Workshops and DEA Industry Consultation questions
▪ Contact FrontierSI Direct [email protected]
• General DEA and data questions
▪ Contact Geoscience Australia Direct [email protected]
• DEA Product Descriptions
▪ http://www.ga.gov.au/dea/products
• Digital Earth Australia Github Account
▪ https://github.com/GeoscienceAustralia/digitalearthau
• CSIRO Open Data Cube
▪ Email Dr Alex Held on [email protected] or Rob Woodcock [email protected]
32
Q&A
