nci.org.au
GSKY
Geospatialrequest
FEATURES• Distributed• Scalable• Concurrent
INPU
T
OUTPUT
Request Response
User’sclient
GSKY:AScalable,DistributedGeospatialDataServer
© NCI Australia 2019
nci.org.au
GSKY: HighLevelArchitecture
Worker 1
Worker 2
Worker n
OWS
/g/dataGeospatial
Index
GeoCrawl
...
MAS
GSKY
OGCRequest
OGCResponse
© NCI Australia 2019
nci.org.au
GSKYrespondstoOpenGeospatialConsortium(OGC)APIoverhttpprotocol:• WebMapService(fordisplayingtheimagesonthemapserver)• WebCoverageService(fordeliveringtheactualdataas“coverages”- independentofthe
underlyingstorageformatorfiles)• WebProcessingservice
GSKYallows:• Performantaggregations,subsetting,subsampling,polygon/pencil/pixeldrills• Executionofon-the-flydatatransformations,re-projectionsandotheralgorithms
GSKYisimplementedusing• Richmetadataserverfordataquerye.g.,spatial,temporal,otherphysicalvariables• Clusteredbackendworkers– highperformanceI/Oandscale-outserver-sidecompute
© NCI Australia 2019
GSKYinanutshell:GeospatialBigDataService
nci.org.au
Ongoingimprovementstoseamlesslytransformingmoredataonthefly
GSKYfilehandlingofcommonlargeEarthObservationdatasetsAmountofdataisscalingupwithmoremodernsatelliteinstruments
MODIS(CSIRO&TERN)MODISdatafrom2001– present,with4timestampspermonthand1kfilespertimestamp.GSKYishandlingapprox.900kMODISfiles
Landsat-8dataset(DEA)Landsat8datasethasdatafrom2013– present,approximately3kfilespertimestamp.GSKYishandlingapprox.7millionfiles.
Sentinel2ARD(DEA)Approx 1400availabledays(2015-present)andabout12kfilesperdayforthisdataset.GSKYishandlingapprox.17millionfiles.
© NCI Australia 2019
nci.org.au
GSKY:• CurrentlyWMS,WPS,WCSforearthobs,
climateandgeophysicsdatasets• OpenSource:http://github.com/nci/gsky
Formoreinformation,see:http://gsky.nci.org.au/https://gsky.readthedocs.io/
Quicklook:http://nationalmap.gov.au/Adddata->MyData->AddWebDataURLs:http://gsky.nci.org.au/ows/dea
http://gsky.nci.org.au/ows/geoglam
NCI’sGSKY– AScalableGeospatialDataServer
© NCI Australia 2019
