Date post: | 30-Mar-2015 |
Category: |
Documents |
Upload: | trenton-toll |
View: | 212 times |
Download: | 0 times |
Tuning GENIE Earth System Model Components using a Grid Enabled
Data Management SystemAndrew Price, Gang Xue, Andrew Yool, Dan Lunt, Tim Lenton,
Jasmin Wason, Graeme Pound, Simon Cox and the GENIE team.
http://www.genie.ac.uk/
UK e-Science – All Hands Meeting
3rd September 2004
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 2/22
Outline
• Introduction• Scientific aims of GENIE• e-Science tools
– Data Management System– Geodise Toolboxes– OPTIONS Design Search and Optimisation
• Results• Future work• Conclusions
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 3/22
Introduction
The GENIE project is developing a Grid-based system to:
• Flexibly couple together state-of-the-art components to form a unified Earth system model
• Execute the resulting model on the Grid
• Share the distributed data produced in simulations
• Provide high-level open access to the system, creating and supporting virtual organisations of Earth system modellers
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 4/22
Scientific Aims
ice-age ice-ageice-ageice-age
• Orbital parameters affect incident radiation and climate• Biological and geological processes interact with, and
feedback upon, the climate (via, for instance, CO2)
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 5/22
The target GENIE Model
3D atmosphere
3D ocean
2D sea ice
AtmosphericCO2
2D land surface
Land vegetation
Ocean biogeochemistry
Ocean sediments
3D ice sheets
Atmosphere – Bristol’s IGCM3Ocean – SOC’s GOLDSTEIN
Land – Met. Office’s TRIFFIDLand ice – Bristol’s GLIMMER
Ocean biogeochemistry andsediments – UEA’s BioGEM
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 6/22
Initial GENIE experiments
• Initial studies in GENIE performed parameter sweeps to investigate the properties of the model
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 7/22
e-Science Tools
• Data Management System (augmented version of the Geodise Database System)
• Matlab scripting environment• Geodise Toolboxes• XML Toolbox• OPTIONS Design Search and Optimisation
package• Template and Example scripts
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 8/22
Data Management System
JavaClient Code
Apache Axis
CoG
JythonFunctions
Globus Server
Geodise Database Toolbox
Metadata Database
Client Grid
SOAP
MatlabFunctions
DatabaseWeb Services
AuthorisationService
LocationService
Metadata Archive & Query
Services
Jython
XML Schema
GridFTP
Portal
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 9/22
Grid Computation
National Grid Service (GT2)Oxford Leeds
RAL Manchester
Jython
Local Resources (GT2)
Java CoG
Imperial Condor PoolSouthampton Condor Pool
Flocked Condor Pools
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 10/22
Geodise Toolboxes
Geodise Compute Toolbox
gd_createproxy.m Creates a Globus proxy certificate for the user's credentials
gd_destroyproxy.m Destroys the local copy of the user's Globus proxy certificate
gd_jobsubmit.m Submits a compute job to a Globus GRAM job manager
gd_jobstatus.m Gets the status of a Globus GRAM job
gd_putfile.m Puts a remote file using GridFTP
gd_getfile.m Retrieves a remote file using GridFTP
gd_rmfile.m Deletes a remote file using GridFTP
gd_makedir.m Creates a remote directory using GridFTP
gd_rmdir.m Deletes a remote directory using GridFTP
Geodise Database Toolbox
gd_archive.m Archives a file or data structure to the database
gd_query.m Query the database for data matching specified criteria.
gd_retrieve.m Retrieves a file or data structure from the database
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 11/22
Scripting a Tuning Study
GENIE Database
Grid Resource
MATLAB
function RMS_Error = cgoldstein(params)
config file
results file
CG binary
return RMS_Error
optimum = fminsearch( … @cgoldstein, params, … )
gd_query(results)
gd_putfile(CG binary)
gd_putfile(config file)
gd_jobsubmit(RSL)
gd_getfile(results file)
gd_archive(results)
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 12/22
Matlab Optimisation Toolbox% ************************% Specify a starting point% ************************parameters = [ 0.5 ];
% ************************% Perform the minimisation% ************************optimum = fminsearch( @cgoldstein_1D, parameters, optimisation_parameters )
% ************************% Specify a starting point% ************************parameters = [ 420 5000000 ];
% ************************% Perform the minimisation% ************************optimum = fminsearch( @cgoldstein_2D, parameters, optimisation_parameters )
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 13/22
OPTIONS
• Matlab interface to the Options design exploration system– http://www.soton.ac.uk/~ajk/options/welcome.html
• State of the art design search and optimisation algorithms– Design of Experiment methods– Response Surface Modelling– Over 30 search methods including:
• Adaptive Random Search (ADRANS), Powell's Direct Search (PDS),• Simplex Method (SIMP), Genetic Algorithm (GA),• Simulated Annealing (SA), Evolutionary Programming (EP)
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 14/22
Grid Computation
OptionsMatlab
National Grid Service (GT2)
Oxford Leeds RAL Manchester
Local Resource (GT2)
GENIE Database
objfun.m objfun_parse.m
optjobparallel.m
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 15/22
OptionsMatlab
>> OptionsInput = createOptionsStructure(4.0)
DNULL: -777
OLEVEL: 2
MAXJOBS: 100
NVRS: 12
VNAM: {'SCLTAU' 'INVDRAG' 'OCNHORZDF' ... }
LVARS: [1.3000 2.0000 2500 ... ]
UVARS: [2.1000 4.8000 5700 ... ]
VARS: [1.7000 3.4000 4100 ... ]
ONAM: 'RMSERROR'
OMETHD: 4.0000
DIRCTN: -1
NITERS: 1000
OPTFUN: 'cgoldstein_12D'
OPTJOB: 'optjobparallel'
GA_NPOP: 100
>> OptionsOutput = OptionsMatlab(OptionsInput);
Available Optimisation Methods:
1.1 for OPTIVAR routine ADRANS 1.2 for OPTIVAR routine DAVID 1.3 for OPTIVAR routine FLETCH 1.4 for OPTIVAR routine JO 1.5 for OPTIVAR routine PDS 1.6 for OPTIVAR routine SEEK 1.7 for OPTIVAR routine SIMPLX 1.8 for OPTIVAR routine APPROX 1.9 for OPTIVAR routine RANDOM 2.1 for user specified routine OPTUM1 2.2 for user specified routine OPTUM2 2.3 for NAG routine E04UCF 2.4 for bit climbing 2.5 for dynamic hill climbing 2.6 for population based incremental learning 2.7 for numerical recipes routines 2.8 for design of experiment based routines 3.11 for Schwefel library Fibonacci search 3.12 for Schwefel library Golden section search 3.13 for Schwefel library Lagrange interval search 3.2 for Schwefel library Hooke and Jeeves search 3.3 for Schwefel library Rosenbrock search 3.41 for Schwefel library DSCG search 3.42 for Schwefel library DSCP search 3.5 for Schwefel library Powell search 3.6 for Schwefel library DFPS search 3.7 for Schwefel library Simplexsearch 3.8 for Schwefel library Complexsearch 3.91 for Schwefel library two membered evolution strategy 3.92 for Schwefel library multi membered evolution strategy 4 for genetic algorithm search 5 for simulated annealing 6 for evolutionary programming 7 for evolution strategy
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 16/22
Twin-Test Experiment
Attempt to recover a known state of the model using a Genetic Algorithm.
Performed 10 generations of a 100 member population. Then applied a local Simplex search of the best candidate.
Population too small to find optimal solution – suitable for finding local minima
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 17/22
Tuning using Observational Data
Model Sea Surface Temperatures
NCEP Sea Surface Temperatures
Model Air Temperatures
NCEP Air Temperatures
• Apply the same method but calculate the RMS error statistic by comparing the model state with NCEP observational data.
• The lack of a land surface in the model means tuning cannot match the observational data.
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 18/22
IGCM3 Atmosphere Model
iii
cellscells
ncepi
cellscells
ncepi
obsi
i
xfwxobjfun
N
Areax
N
Areaxx
xf2
2
• The objective function is a weighted sum of the RMS differences between the model state and NCEP data.• Winter and Summer averages for a number model fields.
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 19/22
IGCM Results
• 25% reduction in error statistic compared to default parameters
• Similar result to a parallel study performed using the Ensemble Kalman Filter
• Model physics insufficient to perfectly match observational data.
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 20/22
e-Science Summary
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 21/22
Conclusions
• Provided the environmental scientist with a toolset for tuning GENIE models:– Scripting environment– Database repository– Computational Grid interface– Suite of generic optimisation algorithms
• A Global minimum can reliably be found in low dimensional problem space.
• For higher dimensional problems, the tools are appropriate for locating local minima in the state space.
03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 22/22
The GENIE Team
Coordinator: Tim Lenton – CEH Edinburgh
Principal investigator:Paul Valdes – Bristol
Research Team and Collaborators:James Annan – FRSGC, Japan Chris Brockwell – UEA Norwich
David Cameron – CEH Edinburgh
Peter Cox – Hadley Centre (UKMO)
Neil Edwards – Bern, Switzerland
Murtaza Gulamali – London e-Science Centre Julia Hargreaves – FRSGC, Japan Phil Harris – CEH Wallingford
Dan Lunt – Bristol
Bob Marsh – SOC
Andrew Price – Southampton e-Science Centre
Andy Ridgwell – UBC, Canada
Ian Rutt – Bristol
Gang Xue – Southampton e-Science Centre
Andrew Yool – SOC
Management Team:Melvin Cannell – CEH Edinburgh
Trevor Cooper-Chadwick – Southampton e-Sci. Centre
Simon Cox – Southampton e-Sci. Centre
John Darlington – London e-Science Centre
Richard Harding – CEH Wallingford
Tony Payne – Bristol
John Shepherd – SOC
Andrew Watson – UEA Norwich
Thanks toSteven Newhouse