Ocean Colour Climate Change Initiative
AI in Ocean ColourCarsten Brockmann (BC), Thomas Jackson (PML)
Material by M. Paperin, J. Wevers, K. Stelzer, D. Müller & Roland Doerffer (BC, HZG)
Slide 2
Ocean Colour Problem
• Radiative transfer – highly non linear process
▪ Not uniquely reversible
• Additional problems
▪ (S)IOPs highly variable
– space, time
▪ parametrisation of
radiative transfer equation
– inherent optical properties
of atmosphere and water
▪ Clouds
Slide 3
Cloud Screening using Machine Learning
• Idea:
▪ our eye and brain is the best cloud detector
▪ → train a machine to mimic a human‘s eye/brain for cloud detection
▪ Eumetsat IAVISA Study, 2008
• Implementation
▪ Collection of manually labelled pixels = training dataset
– No algorithm or any other machine involved in the process of identification
and labelling of a pixel
▪ Training of a neural network
– Classical fully connected multi-layer perceptron
– Feedforward – backpropagation training
– (SNNS toolkit, German award for educational software 1991)
Slide 4
Training Dataset
Slide 5
Training Dataset
• A priori definition of classes and frequency
distribution
• Hierarchy of classes
MERIS: 110 000 pixels
VIIRS: 60 000 pixels
OLCI: 44 100 pixels
53000
30654
12395
17509
750
22306
11522
5422
4042
1320
4987
2751
1265
971
0 10000 20000 30000 40000 50000 60000
Total number of pixels
Cloudy
Totally Cloudy
Semi-transparent clouds
Other turbid atmosphere
Clear
Clear sky land
Clear sky water
Clear sky snow/ice
Other clear cases
Other
Floating ice
Glint
Cloud shadow
distribution of surface types (PB-V)
PB-V: 53 000 pixels
Slide 6
NN Performance
opaque cloud
clear Land
semi-transparent cloud
spatially mixed cloudclear water
clear snow/ice
Slide 7
Validation
1 = Opaque
Slide 8
Validation
1 = Opaque
2 = Semi-transparent cloud
3 = Thick semi-transparent cloud
4 = Average density
semi-transparent cloud
5 = Thin semi-transparent cloud
Slide 9
Validation
1 = Opaque
2 = Semi-transparent cloud
3 = Thick semi-transparent cloud
4 = Average density
semi-transparent cloud
Slide 10
Example OLCI, 2016428
Slide 11
Inversion of the radiative transferCoupled ocean – atmosphere system
• Idea:
▪ Radiative transfer physics are well understood
▪ Formulation of „forward“ problem possible
▪ Numerical RT models well advanced and validated
→ Calculate a comprehensive database of spectra for representative waterand atmosphere conditions
→ Inversion by machine learning
• Implementation:
▪ Decomposition of problem into 2 parts (otherwise the manifold of thesolution space would be too large): ocean and atmosphere
▪ Set of neural nets for the inversions
▪ Starting with SNNS in mid-1990‘s for MERIS
– MLP with ffbp training
▪ Switching to Tensorflow/KERAS in 2018
– Experimenting with different architectures
– Same quality can be achieved with much less training samples
– Speed of the training significantly improved
NEURAL NETWORK BASED PROCESSING
water bio-opticalmodel
atmosp. parametrisation
aerosol
SIOPs
RT atm.
RT ocean
RT simulations: MERIS, OLCI,
MODIS, VIIRS, SeaWiFS,
S2 MSI, L8 OLI, RE
NNs training
FeedforwardBackpropagation MLP
aaNN
IOP
fwd
kd
rw
unc
SNAP C2RCC S2 Processor
SNAP C2RCC S3 Processor
ProcessorOLCI GS
SNAP C2RCC S2 Processor
TRAINING DATASETS:ATMOSPHERE MODEL
Solar zenith angle: 0-75 deg
Surface pressure: 800 – 1040 hPa
Max. rho_toa at 865 nm limited to 0.8
AOD Angstrom coeff.
AOD
frequency
frequency
Angstr.
TRAINING DATASET: BIO-OPTICAL MODEL
ranges derived from in-situ measurements
frequency
frequency
frequency
frequency
frequency
ad agapig
bp bw
TRAINING DATASET: BIO-OPTICAL MODEL
btot ad
apig
ag
Co-variances derived from in-situ measurements
NEURAL NETWORK BASED PROCESSING
water bio-opticalmodel
atmosp. parametrisation
aerosol
SIOPs
RT atm.
RT ocean
RT simulations: MERIS, OLCI,
MODIS, VIIRS, SeaWiFS,
S2 MSI, L8 OLI, RE
NNs training
FeedforwardBackpropagation MLP
aaNN
IOP
fwd
kd
rw
unc
SNAP C2RCC S2 Processor
SNAP C2RCC S3 Processor
ProcessorOLCI GS
SNAP C2RCC S2 Processor
VERIFICATON (SIMULATED DATA)ATMOSPHERE
water leaving reflectance,400 nm water leaving reflectance, 560 nm
„truth“
Re
trie
va
l (N
N)
Re
trie
va
l (N
N)
„truth“
VERIFICATON (SIMULATED DATA, WATER)
apig
Only water part(NN validation) Atmospere + Water
Adding extreme water cases(masking effect)
„truth“
retr
ieved
by
NN
apig
apig
„truth“
retr
ieved
by
NN
„truth“
retr
ieved
by
NN
VALIDATION COMPARISON AGAINST IN-SITU
Comparison OLCI S3A rho_w_nn with
AAOT rhon_w_is
NEURAL NETS FOR CONSISTENCY CHECKS
water → forward net fed
with retrived IOPs
atmosphere →
autoassociatove neural net
TO
A r
efle
cta
nce
wavelength wavelength
wa
ter
lea
vin
gre
fle
cta
nce
UNCERTAINTIES
RT Database
IOPs
NNIOP
rho_w
IOPs, estimated
∆(IOPs)
traininguncer-tainty
net
NNuncer-tainty
IOPs, estimated
∆(IOPs)
UNCERTAINTIESa
pig
longitude
apig Uncert. of apig
CH
L c
onc.
longitude
S3B OLCI 20190104
rho_toarho_w
Chlorophyll and TSM
Adg and z90max
TSM wit al3ex model
Slide 27
Conclusion
• The construction of the population (training sample, validation sample) ismost critical for the quality of the retrieval quality
▪ Cloud screening: representing all different types of clear sky and cloudyconditions
▪ Covering the range of optical properties of the water body and the atmosphere
▪ Reflecting the inner structure (dependencies, co-variances) of the IOP space
▪ Containing sufficient samples of everything which shall be retrieved
– Constructing the training data set such that it represents the frequency distribution of conditions as they appear in reality is a wrong approach; It would cause rare cases being poorly retrieved.
• The choice among different AI methods (deep learning, RF, conv.NNs, …) has a minor effect.
▪ All tested methods so far deliver excellent performance of inverting the validationdataset.
▪ However, a 99% accuracy on the validation dataset (which is from the same population as the training dataset) is irrelevant if the population is not properlyrepresenting nature.
Slide 28
Future use – Water Type Classification
Objective: Increased automation of processing up to end of water class set
generation allows more time for scientific interpretation and rapid
updates/application to new data sources.
Slide 29
Last Slide
• RT inversion in a coupled ocean-atmosphere system is a highly
non-linear, underdetermined problem
▪ „Ocean Colour retrieval seems impossible“ (Roland Doerffer)
• Articifial Intelligence is a method to address this problem
▪ „Let the data tell us the solution“ (Helmut Schiller)