Post on 13-Dec-2015
transcript
Development of Neural Network Emulations of Model Radiation for
Improving the Computational Performance of the NCEP Climate
Simulations and Seasonal ForecastsVladimir Krasnopolsky
NOAA/NCEP/SAIC University of Maryland/ESSIC
In collaboration with: M. Fox-Rabinovitz, Y-T. Hou,
S. Lord, and A. Belochitski
Acknowledgements: H. Pan, S. Saha, S. Moorthi, M. Iredell
CTB Seminar Series
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 2
Outline• Background
– CFS; Motivation for Development of NN Radiation
– Neural Networks
• NN Radiation:– Accurate and Fast NN Emulations of LWR
and SWR Parameterizations– Validation
• Approximation Accuracy• Parallel Runs
– 17 year climate simulation– Seasonal predictions
• Conclusions
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 3
CFS Background and
Motivations for Development of NN
Radiation
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 4
NCEP Climate Forecast System (CFS) (1)The set of conservation laws (mass, energy,
momentum, water vapor, ozone, etc.)• Deterministic First Principles Models, 3-D Partial Differential Equations on the Sphere:
- a 3-D prognostic/dependent variable, e.g., temperature – x - a 3-D independent variable: x, y, z & t– D - dynamics (spectral)– P - physics or parameterization of physical processes (1-D vertical r.h.s. forcing)
• Continuity Equation• Thermodynamic Equation• Momentum Equations
( , ) ( , )D x P xt
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 5
NCEP CFS (2)Physics – P, currently represented by 1-D (vertical) parameterizations
• Major components of P = {R, W, C, T, S, CH}:– R - radiation (long & short wave processes): AER Inc.
rrtm, ncep0, and ncep1– W – convection, and large scale precipitation processes– C - clouds– T – turbulence – S – land (noah), ocean (MOM3/4), ice – air interaction– CH – chemistry (aerosols)
• Components of P are 1-D parameterization of complicated set of multi-scale theoretical and empirical physical process models simplified for computational reasons
• P is the most time consuming part of climate/weather models!
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 6
Distribution of NCEP CFS Calculation TimeNCEP CFS T126L64
Radiation Dynamics Other
~60%
~20%
~20%
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 7
Motivations• Calculation of model radiation takes usually a
very significant part (> 50%) of the total model computations.
• Calculation of model radiation is always a trade-off between the accuracy and computational efficiency:– NCEP and UKMO reduce the frequency of
calculations– ECMWF:
• reduces horizontal resolution of radiation calculations in climate and NWP models
• uses neural network long wave radiation in DAS
– Canadian Meteorological Service reduces vertical resolution of radiation calculations
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 8
Developed Accurate and Fast NN Radiation:
• Allows sufficiently frequent calculations of radiation
• Allows radiation calculations at each grid point of high resolution 3D grid
• NN developed for both long and short wave radiations
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 9
NN Background
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 10
Mapping and NNs• MAPPING (continuous or almost
continuous) is a relationship between two vectors: a vector of input parameters, X, and a vector of output parameters, Z,
• NN is a generic approximation for any continuous or almost continuous mapping given by a set of its input/output records:
SET = {Xi, Zi}i = 1, …,N
mn ZandXXFZ );(
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 11
Linear part Nonlinear part
x1
xn
xi
x2
xn-1
NN - Continuous Input to Output MappingMultilayer Perceptron: Feed Forward, Fully Connected
1x
2x
3x
4x
nx
1y
2y
3y
my
1t
2t
kt
NonlinearNeurons
LinearNeurons
X Y
Input Layer
Output Layer
Hidden Layer
Y = FNN(X)
Jacobian !
Neuron
tj
0 01 1 1
01 1
( )
tanh( ); 1,2, ,
k k n
q q qj j q qj j ji ij j i
k n
q qj j ji ij i
y a a t a a b x
a a b x q m
1
1
( )
tanh( )
n
j j ji ii
n
j ji ii
t b x
b x
jjTj sbX jj ts )(
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 12
NN as a Universal Tool for Approximation of Continuous & Almost Continuous Mappings
Some Basic Theorems:
Any function or mapping Z = F (X), continuous on a compact subset, can be approximately represented by a p (p 3) layer NN in the sense of uniform convergence (e.g., Chen & Chen, 1995; Blum and Li, 1991, Hornik, 1991; Funahashi, 1989, etc.) The error bounds for the uniform approximation on compact sets (Attali & Pagès, 1997):
||Z -Y|| = ||F (X) - FNN (X)|| ~ O(1/k) k -number of neurons in the hidden layer
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 13
NN{W}
X Training Set Z
ErrorE = ||Z-Y||X
Input
Y
Output
Z DesiredOutput
Weight AdjustmentsW
E No
Yes EndTraining
E
BP
NN TrainingOne Training Iteration
W
E ≤
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 14
Major Advantages of NNs:NNs are generic, very accurate and convenient mathematical (statistical) models which are able to emulate complicated nonlinear input/output relationships (continuous or almost continuous mappings ).
NNs are robust with respect to random noise and fault- tolerant.
NNs are analytically differentiable (training, error and sensitivity analyses): almost free Jacobian!
NNs emulations are accurate and fast but NO FREE LUNCH!
Training is complicated and time consuming nonlinear optimization task; however, training should be done only once for a particular application!
NNs are well-suited for parallel and vector processing
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 15
Basis for Accurate and Fast NN Emulations of
Model Physics
• Any parameterization of model physics is a continuous or almost continuous mapping
• NN is a generic tool for emulating such mappings
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 16
NN Emulations of Model Physics Parameterizations Learning from Data
GCM
X Y
Original Parameterization
F
X Y
NN Emulation
FNN
TrainingSet …, {Xi, Yi}, … Xi Dphys
NN Emulation
FNN
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 17
NN for RadiationLong Wave Radiation
• Long Wave Radiative Transfer:
• Absorptivity & Emissivity (optical properties):
4
( ) ( ) ( , ) ( , ) ( )
( ) ( ) ( , ) ( )
( ) ( )
t
s
p
t t t
p
p
s
p
F p B p p p p p dB p
F p B p p p dB p
B p T p the Stefan Boltzman relation
0
0
{ ( ) / ( )} (1 ( , ))
( , )( ) / ( )
( ) (1 ( , ))
( , )( )
( )
t t
tt
dB p dT p p p d
p pdB p dT p
B p p p d
p pB p
B p the Plank function
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 18
NN Emulation of Input/Output Dependency:Input/Output Dependency:
The Magic of NN Performance
Xi
OriginalParameterization Yi
Y = F(X)
Xi
NN EmulationYi
YNN = FNN(X)
Mathematical Representation of Physical Processes
4
( ) ( ) ( , ) ( , ) ( )
( ) ( ) ( , ) ( )
( ) ( )
t
s
p
t t t
p
p
s
p
F p B p p p p p dB p
F p B p p p dB p
B p T p the Stefan Boltzman relation
0
0
{ ( ) / ( )} (1 ( , ))
( , )( ) / ( )
( ) (1 ( , ))
( , )( )
( )
t t
tt
dB p dT p p p d
p pdB p dT p
B p p p d
p pB p
B p the Plank function
Numerical Scheme for Solving Equations Input/Output Dependency: {Xi,Yi}I = 1,..N
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 19
NCEP LW Radiation and NN Characteristics
• 612 Inputs:– 10 Profiles: temperature, humidity, ozone, pressure, cloudiness, CO2, etc– Relevant surface and scalar characteristics
• 69 Outputs:– Profile of heating rates (64)
– 5 LW radiation fluxes • Hidden Layer: One layer with 50 to 300 neurons • Training: nonlinear optimization in the space with
dimensionality of 15,000 to 100,000– Training Data Set: Subset of about 200,000 instantaneous profiles
simulated by CFS for 17 years– Training time: about 1 to several days– Training iterations: 1,500 to 8,000
• Validation on Independent Data:– Validation Data Set (independent data): about 200,000 instantaneous
profiles simulated by CFS
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 20
NCEP SW Radiation and NN Characteristics
• 650 Inputs:– 10 Profiles: pressure, temperature, water vapor, ozone
concentration, cloudiness, CO2, etc– Relevant surface and scalar characteristics
• 73 Outputs:– Profile of heating rates (64)– 9 LW radiation fluxes
• Hidden Layer: One layer with 50 to 200 neurons • Training: nonlinear optimization in the space with
dimensionality of 25,000 to 130,000– Training Data Set: Subset of about 200,000 instantaneous profiles
simulated by CFS for 17 year– Training time: about 1 to several days – Training iterations: 1,500 to 8,000
• Validation on Independent Data:– Validation Data Set (independent data): about 200,000
instantaneous profiles simulated by CFS
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 21
NN Approximation Accuracy and Performance vs. Original Parameterization
(on independent data set)Parameter Model Bias RMSE RMSEt RMSEb Performance
LWR(K/day)
NCEP CFSAER rrtm
2. 10-3 0.40 0.09 0.64 12
times faster
NCAR CAMW.D. Collins
3. 10-4 0.28 0.06 0.86 150
times faster
SWR(K/day)
NCEP CFSAER rrtm 5. 10-3 0.20 0.21 0.22
~45
times faster
NCAR CAM W.D. Collins
-4. 10-3 0.19 0.17 0.43 20
times faster
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 22
Error Vertical Variability Profiles
LWR – solid line; SWR – dashed line
RMSE profiles in K/day
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 23
Individual Profiles (NCEP CFS)
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 24
Validation of Full NN Radiation in CFS• The Control CFS run with the original LWR
and SWR parameterizations is run for 17 years.
• The NN Full Radiation run: CFS with LWR and SWR NN emulations is run for 17 years.
• Another Control CFS Run after updates of FORTRAN compiler and libraries
• Validation of the NN Full Radiation run is done against the Control run. The differences/biases are less than/within observation errors and uncertainties of reanalysis
• The differences between two controls (“butterfly”/”round off” differences) have been also calculated and shown for comparison.
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 25
Climate Simulation17 years:
1990 – 2006
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 26
Zonal and time mean Top of Atmosphere Upward Fluxes (Winter)
The solid line – the difference (the full radiation NN run – the control (CTL)),
the dash line – the background differences (the differences between two
control runs). All in W/m2.
LWR
SWR
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 27
Zonal and time annual mean Downward and Upward Surface Long Wave Fluxes
The solid line – the difference (the full radiation NN run – the control (CTL)),
the dash line – the background differences (the differences between two
control runs). All in W/m2.
Downward Upward
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 28
The time mean (1990-2006) SST statistics for summer & winter
Control RunNN Full
Radiation
Run
NN - ControlControl1 –
Control2
The contour intervals for the SST fields are 5º K
and for the SST differences are 0.5º K.
Fields
Differences
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 29
CTL NN FR
NN - CTL
CTL_O – CTL_N
SST
CTL1 – CTL2
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 30
CTL NN FR
NN - CTL CTL1 – CTL2
SST
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 31
The time mean (1990-2006) total precipitation rate (PRATE) statistics
for summer & winter
Control RunNN Full
Radiation
Run
NN - ControlControl1 –
Control2
The contour intervals for the PRATE fields are 1 mm/day for the
0 – 6 mm/day range and 2 mm/day for the 6 mm/day and higher;
for the PRATE differences the contour intervals are 1 mm/day
Fields
Differences
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 32
CTL NN FR
NN - CTL
PRATE
CTL1 – CTL2
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 33
CTL NN FR
NN - CTL CTL1 – CTL2
PRATE
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 34
The time mean (1990-2006) total) total clouds statistics for summer & winter
Control RunNN Full
Radiation
Run
NN - ControlControl1 –
Control2
The contour intervals for the total clouds fields the cloud fields
are 10% and for the differences – 5%.
Fields
Differences
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 35
JJACTL NN FR
NN - CTL CTL1 – CTL2
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 36
DJFCTL NN FR
NN - CTL CTL1 – CTL2
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 37
The time mean (1990-2006) convective precipitation clouds statistics for
summer & winter
Control RunNN Full
Radiation
Run
NN - ControlControl1 –
Control2
The contour intervals for the ) total clouds fields the cloud fields
are 10% and for the differences – 5%.
Fields
Differences
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 38
JJACTL NN FR
NN - CTL CTL1 – CTL2
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 39
DJFCTL NN FR
NN - CTL CTL1 – CTL2
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 40
The time mean (1990-2006) boundary layer clouds statistics for summer &
winter
Control RunNN Full
Radiation
Run
NN - ControlControl1 –
Control2
The contour intervals for the boundary clouds fields the cloud fields
are 10% and for the differences – 5%.
Fields
Differences
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 41
JJACTL NN FR
NN - CTL CTL1 – CTL2
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 42
DJFCTL NN FR
NN - CTL CTL1 – CTL2
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 43
Some Time Series
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 44
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 45
Temperature at 850 hPa, K
Solid – NN run
Dashed – Control Runs
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 46
Seasonal Predictions
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 47
SST seasonal differences for winter
Control RunNN Full
Radiation
Run
NN - ControlControl1 –
Control2
The contour intervals for the SST fields are 5º K
and for the SST differences are 0.5º K.
Fields
Differences
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 48
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 49
Total precipitation rate (PRATE) seasonal differences for summer
Control RunNN Full
Radiation
Run
NN - ControlControl1 –
Control2
The contour intervals for the PRATE fields are 1 mm/day for the
0 – 6 mm/day range and 2 mm/day for the 6 mm/day and higher;
for the PRATE differences the contour intervals are 1 mm/day
Fields
Differences
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 50
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 51
Total clouds differences for winter
Control RunNN Full
Radiation
Run
NN - ControlControl1 –
Control2
The contour intervals for the ) total clouds fields the cloud fields
are 10% and for the differences – 5%.
Fields
Differences
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 52
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 53
Convective precipitation clouds seasonal differences for summer
Control RunNN Full
Radiation
Run
NN - ControlControl1 –
Control2
The contour intervals for the ) total clouds fields the cloud fields
are 10% and for the differences – 5%.
Fields
Differences
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 54
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 55
NN Emulations of Model RadiationConclusions – 1
• NN is a powerful tool for speeding up calculations of model radiation through developing NN emulations– Accurate and fast NNs emulations have been
successfully developed for:• NCEP LWR & SWR parameterizations • NCAR CAM LWR & SWR parameterizations• NASA LWR parameterization
– The simulated diagnostic and prognostic fields are very close for the parallel climate and seasonal prediction runs performed with NN emulations and the original parameterizations
– NN emulations approach works well for high vertical resolutions L > 60. It provides simultaneously high accuracy and satisfactory speedup.
NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 56
Conclusions – 2 Upcoming Developments
• Developments and improvements for facilitating transition to operational use– Investigation of robustness of NN emulations with respect
to:• Increasing CFS horizontal resolution• Increasing the frequency of radiation calculations in CFS• Changes in the model (e.g., change of other parameterizations
in CFS) • Transition of the NN radiation into GFS
• Developments allowing to reduce probability of larger errors and outliers: – Quality control and compound parameterization – NN ensembles
• Development of dynamically adjustable NN emulations (to climate changes, etc.)
• Using NN emulations for generating ensembles with perturbed physics
• NN emulations can be introduced in DAS (fast calculations + fast adjoint)