Date post: | 16-Dec-2015 |
Category: |
Documents |
Upload: | nikki-lakeman |
View: | 215 times |
Download: | 0 times |
EOLDAS_ngThe new generation
Prof. P. Lewis, Dr. Jose Gomez-Dans, UCL/NCEO
Task 3: EO-LDAS baselineWP3.1: EO-LDAS v1 consolidation
o Autumn 2013/workshopo Publicly available software & documentation
WP3.2: Integration of enhanced componentso Dec/2013o Integration of other modelling efforts in variational
DA
What is EO-LDAS?
• generic variational DA system, weak or strong constraint
• different observation operators (RT models)
• different process models, including regularisation
• tutorial documentation -- how to do DA with the system
• Demonstrated so far with optical data
Lewis et al (2012), Rem Sens Env
Lewis et al (2012), ESA SP
The concept
Observations(RT codes)
Dynamic Model
Prior
Define a cost function J as a sum of however many components, but typically:
• Each component calculates the cost and the partial derivatives of the cost function• Use gradient descent minimisation
o Need partial derivatives for Observation Operator & Dynamic model• Uncertainty is calculated using the Hessian (matrix of 2nd order derivatives) @ minimum
J, J'
What have we learned with EO-LDAS?
• v1 design controlled by configuration fileso minimise programming requirementso generic structure probably soundo teach users new metalanguage to describe problem
• Too much flexibility leads to inefficiencieso Difficult to add extra functionalityo Difficult to maintain
EO-LDAS v21. Assume user knows Python (e.g. I/O, plotting)2. Streamline system by reducing flexibility3. Simple components - readily extended4. Decouple solver from operators
a. Essentially, list of J, J'
EO-LDAS should be a Python library of state definitions and generic operators, requiring the user to interface to other:• ObsOps,• Dynamic Models,• Data,• Plotting, etc
Interactive documentation
IPython notebooks to provide interactive documentation on library brary
http://nbviewer.ipython.org/urls/github.com/jgomezdans/eoldas_ng/raw/master/notebooks/ASimpleSmoother.ipynb
Faster models through emulation
• First steps in using Gaussian Processes (GP) Provides a fast version of the code
• ... but also of its partial derivatives!1. User generates training set with RT model
2. GP is trained to the above
3. This is used in EO-LDAS
• Much more flexible: o if the model can be emulated, any model can go ino no need to waste time on adjointso very fasto simplifies library structure very much
Emulator testing
Use IPython notebooks to provide the user with interactive documentation on how to do emulationhttp://nbviewer.ipython.org/urls/raw.github.com/profLewis/emulate_test/master/emulate_test.ipynb
Spares
Multi-resolution processing
GotchasObservation Operator gotchas• Typical RT codes are science grade software:
o Limited support from the authorso Inefficiento Difficult to follow & modify
• Few RT models have adjoints readily available• If the RT model is modified, the adjoint needs to be re-created
Dynamic model gotchas• All of the above• Ended up implemented a linear model: if you can express your model as Ax, then you can slot
it in• Most dynamic models are highly non-linear, and linearisation requires automatic differentiation• So ended up using Tikhonov-type regularisation