Development of Neural Network Emulations of Model Radiation for Improving the Computational...

Development of Neural Network Emulations of Model Radiation for

Improving the Computational Performance of the NCEP Climate

Simulations and Seasonal ForecastsVladimir Krasnopolsky

NOAA/NCEP/SAIC University of Maryland/ESSIC

In collaboration with: M. Fox-Rabinovitz, Y-T. Hou,

S. Lord, and A. Belochitski

Acknowledgements: H. Pan, S. Saha, S. Moorthi, M. Iredell

CTB Seminar Series

NCEP-CTB: 5/27/2009 V. Krasnopolsky, Neural Network Emulations of Model Radiation 2

Outline• Background

– CFS; Motivation for Development of NN Radiation

– Neural Networks

• NN Radiation:– Accurate and Fast NN Emulations of LWR

and SWR Parameterizations– Validation

• Approximation Accuracy• Parallel Runs

– 17 year climate simulation– Seasonal predictions

• Conclusions


CFS Background and

Motivations for Development of NN

Radiation


NCEP Climate Forecast System (CFS) (1)The set of conservation laws (mass, energy,

momentum, water vapor, ozone, etc.)• Deterministic First Principles Models, 3-D Partial Differential Equations on the Sphere:

- a 3-D prognostic/dependent variable, e.g., temperature – x - a 3-D independent variable: x, y, z & t– D - dynamics (spectral)– P - physics or parameterization of physical processes (1-D vertical r.h.s. forcing)

• Continuity Equation• Thermodynamic Equation• Momentum Equations

( , ) ( , )D x P xt


NCEP CFS (2)Physics – P, currently represented by 1-D (vertical) parameterizations

• Major components of P = {R, W, C, T, S, CH}:– R - radiation (long & short wave processes): AER Inc.

rrtm, ncep0, and ncep1– W – convection, and large scale precipitation processes– C - clouds– T – turbulence – S – land (noah), ocean (MOM3/4), ice – air interaction– CH – chemistry (aerosols)

• Components of P are 1-D parameterization of complicated set of multi-scale theoretical and empirical physical process models simplified for computational reasons

• P is the most time consuming part of climate/weather models!


Distribution of NCEP CFS Calculation TimeNCEP CFS T126L64

Radiation Dynamics Other

~60%

~20%

~20%


Motivations• Calculation of model radiation takes usually a

very significant part (> 50%) of the total model computations.

• Calculation of model radiation is always a trade-off between the accuracy and computational efficiency:– NCEP and UKMO reduce the frequency of

calculations– ECMWF:

• reduces horizontal resolution of radiation calculations in climate and NWP models

• uses neural network long wave radiation in DAS

– Canadian Meteorological Service reduces vertical resolution of radiation calculations


Developed Accurate and Fast NN Radiation:

• Allows sufficiently frequent calculations of radiation

• Allows radiation calculations at each grid point of high resolution 3D grid

• NN developed for both long and short wave radiations


NN Background


Mapping and NNs• MAPPING (continuous or almost

continuous) is a relationship between two vectors: a vector of input parameters, X, and a vector of output parameters, Z,

• NN is a generic approximation for any continuous or almost continuous mapping given by a set of its input/output records:

SET = {Xi, Zi}i = 1, …,N

mn ZandXXFZ );(


Linear part Nonlinear part

x1

xn

xi

x2

xn-1

NN - Continuous Input to Output MappingMultilayer Perceptron: Feed Forward, Fully Connected

1x

2x

3x

4x

nx

1y

2y

3y

my

1t

2t

kt

NonlinearNeurons

LinearNeurons

X Y

Input Layer

Output Layer

Hidden Layer

Y = FNN(X)

Jacobian !

Neuron

tj

0 01 1 1

01 1

( )

tanh( ); 1,2, ,

k k n

q q qj j q qj j ji ij j i

k n

q qj j ji ij i

y a a t a a b x

a a b x q m

1

1

( )

tanh( )

n

j j ji ii

n

j ji ii

t b x

b x

jjTj sbX jj ts )(


NN as a Universal Tool for Approximation of Continuous & Almost Continuous Mappings

Some Basic Theorems:

Any function or mapping Z = F (X), continuous on a compact subset, can be approximately represented by a p (p 3) layer NN in the sense of uniform convergence (e.g., Chen & Chen, 1995; Blum and Li, 1991, Hornik, 1991; Funahashi, 1989, etc.) The error bounds for the uniform approximation on compact sets (Attali & Pagès, 1997):

||Z -Y|| = ||F (X) - FNN (X)|| ~ O(1/k) k -number of neurons in the hidden layer


NN{W}

X Training Set Z

ErrorE = ||Z-Y||X

Input

Y

Output

Z DesiredOutput

Weight AdjustmentsW

E No

Yes EndTraining

E

BP

NN TrainingOne Training Iteration

W

E ≤


Major Advantages of NNs:NNs are generic, very accurate and convenient mathematical (statistical) models which are able to emulate complicated nonlinear input/output relationships (continuous or almost continuous mappings ).

NNs are robust with respect to random noise and fault- tolerant.

NNs are analytically differentiable (training, error and sensitivity analyses): almost free Jacobian!

NNs emulations are accurate and fast but NO FREE LUNCH!

Training is complicated and time consuming nonlinear optimization task; however, training should be done only once for a particular application!

NNs are well-suited for parallel and vector processing


Basis for Accurate and Fast NN Emulations of

Model Physics

• Any parameterization of model physics is a continuous or almost continuous mapping

• NN is a generic tool for emulating such mappings


NN Emulations of Model Physics Parameterizations Learning from Data

GCM

X Y

Original Parameterization

F

X Y

NN Emulation

FNN

TrainingSet …, {Xi, Yi}, … Xi Dphys

NN Emulation

FNN


NN for RadiationLong Wave Radiation

• Long Wave Radiative Transfer:

• Absorptivity & Emissivity (optical properties):

4

( ) ( ) ( , ) ( , ) ( )

( ) ( ) ( , ) ( )

( ) ( )

t

s

p

t t t

p

p

s

p

F p B p p p p p dB p

F p B p p p dB p

B p T p the Stefan Boltzman relation

0

0

{ ( ) / ( )} (1 ( , ))

( , )( ) / ( )

( ) (1 ( , ))

( , )( )

( )

t t

tt

dB p dT p p p d

p pdB p dT p

B p p p d

p pB p

B p the Plank function


NN Emulation of Input/Output Dependency:Input/Output Dependency:

The Magic of NN Performance

Xi

OriginalParameterization Yi

Y = F(X)

Xi

NN EmulationYi

YNN = FNN(X)

Mathematical Representation of Physical Processes

4

( ) ( ) ( , ) ( , ) ( )

( ) ( ) ( , ) ( )

( ) ( )

t

s

p

t t t

p

p

s

p

F p B p p p p p dB p

F p B p p p dB p

B p T p the Stefan Boltzman relation

0

0

{ ( ) / ( )} (1 ( , ))

( , )( ) / ( )

( ) (1 ( , ))

( , )( )

( )

t t

tt

dB p dT p p p d

p pdB p dT p

B p p p d

p pB p

B p the Plank function

Numerical Scheme for Solving Equations Input/Output Dependency: {Xi,Yi}I = 1,..N


NCEP LW Radiation and NN Characteristics

• 612 Inputs:– 10 Profiles: temperature, humidity, ozone, pressure, cloudiness, CO2, etc– Relevant surface and scalar characteristics

• 69 Outputs:– Profile of heating rates (64)

– 5 LW radiation fluxes • Hidden Layer: One layer with 50 to 300 neurons • Training: nonlinear optimization in the space with

dimensionality of 15,000 to 100,000– Training Data Set: Subset of about 200,000 instantaneous profiles

simulated by CFS for 17 years– Training time: about 1 to several days– Training iterations: 1,500 to 8,000

• Validation on Independent Data:– Validation Data Set (independent data): about 200,000 instantaneous

profiles simulated by CFS


NCEP SW Radiation and NN Characteristics

• 650 Inputs:– 10 Profiles: pressure, temperature, water vapor, ozone

concentration, cloudiness, CO2, etc– Relevant surface and scalar characteristics

• 73 Outputs:– Profile of heating rates (64)– 9 LW radiation fluxes

• Hidden Layer: One layer with 50 to 200 neurons • Training: nonlinear optimization in the space with

dimensionality of 25,000 to 130,000– Training Data Set: Subset of about 200,000 instantaneous profiles

simulated by CFS for 17 year– Training time: about 1 to several days – Training iterations: 1,500 to 8,000

• Validation on Independent Data:– Validation Data Set (independent data): about 200,000

instantaneous profiles simulated by CFS


NN Approximation Accuracy and Performance vs. Original Parameterization

(on independent data set)Parameter Model Bias RMSE RMSEt RMSEb Performance

LWR(K/day)

NCEP CFSAER rrtm

2. 10-3 0.40 0.09 0.64 12

times faster

NCAR CAMW.D. Collins

3. 10-4 0.28 0.06 0.86 150

times faster

SWR(K/day)

NCEP CFSAER rrtm 5. 10-3 0.20 0.21 0.22

~45

times faster

NCAR CAM W.D. Collins

-4. 10-3 0.19 0.17 0.43 20

times faster


Error Vertical Variability Profiles

LWR – solid line; SWR – dashed line

RMSE profiles in K/day


Individual Profiles (NCEP CFS)


Validation of Full NN Radiation in CFS• The Control CFS run with the original LWR

and SWR parameterizations is run for 17 years.

• The NN Full Radiation run: CFS with LWR and SWR NN emulations is run for 17 years.

• Another Control CFS Run after updates of FORTRAN compiler and libraries

• Validation of the NN Full Radiation run is done against the Control run. The differences/biases are less than/within observation errors and uncertainties of reanalysis

• The differences between two controls (“butterfly”/”round off” differences) have been also calculated and shown for comparison.


Climate Simulation17 years:

1990 – 2006


Zonal and time mean Top of Atmosphere Upward Fluxes (Winter)

The solid line – the difference (the full radiation NN run – the control (CTL)),

the dash line – the background differences (the differences between two

control runs). All in W/m2.

LWR

SWR


Zonal and time annual mean Downward and Upward Surface Long Wave Fluxes

The solid line – the difference (the full radiation NN run – the control (CTL)),

the dash line – the background differences (the differences between two

control runs). All in W/m2.

Downward Upward


The time mean (1990-2006) SST statistics for summer & winter

Control RunNN Full

Radiation

Run

NN - ControlControl1 –

Control2

The contour intervals for the SST fields are 5º K

and for the SST differences are 0.5º K.

Fields

Differences


CTL NN FR

NN - CTL

CTL_O – CTL_N

SST

CTL1 – CTL2


CTL NN FR

NN - CTL CTL1 – CTL2

SST


The time mean (1990-2006) total precipitation rate (PRATE) statistics

for summer & winter

Control RunNN Full

Radiation

Run


Control2

The contour intervals for the PRATE fields are 1 mm/day for the

0 – 6 mm/day range and 2 mm/day for the 6 mm/day and higher;

for the PRATE differences the contour intervals are 1 mm/day

Fields

Differences


CTL NN FR

NN - CTL

PRATE

CTL1 – CTL2


CTL NN FR


PRATE


The time mean (1990-2006) total) total clouds statistics for summer & winter

Control RunNN Full

Radiation

Run


Control2

The contour intervals for the total clouds fields the cloud fields

are 10% and for the differences – 5%.

Fields

Differences


JJACTL NN FR



DJFCTL NN FR



The time mean (1990-2006) convective precipitation clouds statistics for

summer & winter

Control RunNN Full

Radiation

Run


Control2

The contour intervals for the ) total clouds fields the cloud fields


Fields

Differences


JJACTL NN FR



DJFCTL NN FR



The time mean (1990-2006) boundary layer clouds statistics for summer &

winter

Control RunNN Full

Radiation

Run


Control2

The contour intervals for the boundary clouds fields the cloud fields


Fields

Differences


JJACTL NN FR



DJFCTL NN FR



Some Time Series



Temperature at 850 hPa, K

Solid – NN run

Dashed – Control Runs


Seasonal Predictions


SST seasonal differences for winter

Control RunNN Full

Radiation

Run


Control2

The contour intervals for the SST fields are 5º K

and for the SST differences are 0.5º K.

Fields

Differences



Total precipitation rate (PRATE) seasonal differences for summer

Control RunNN Full

Radiation

Run


Control2

The contour intervals for the PRATE fields are 1 mm/day for the

0 – 6 mm/day range and 2 mm/day for the 6 mm/day and higher;

for the PRATE differences the contour intervals are 1 mm/day

Fields

Differences



Total clouds differences for winter

Control RunNN Full

Radiation

Run


Control2



Fields

Differences



Convective precipitation clouds seasonal differences for summer

Control RunNN Full

Radiation

Run


Control2



Fields

Differences



NN Emulations of Model RadiationConclusions – 1

• NN is a powerful tool for speeding up calculations of model radiation through developing NN emulations– Accurate and fast NNs emulations have been

successfully developed for:• NCEP LWR & SWR parameterizations • NCAR CAM LWR & SWR parameterizations• NASA LWR parameterization

– The simulated diagnostic and prognostic fields are very close for the parallel climate and seasonal prediction runs performed with NN emulations and the original parameterizations

– NN emulations approach works well for high vertical resolutions L > 60. It provides simultaneously high accuracy and satisfactory speedup.


Conclusions – 2 Upcoming Developments

• Developments and improvements for facilitating transition to operational use– Investigation of robustness of NN emulations with respect

to:• Increasing CFS horizontal resolution• Increasing the frequency of radiation calculations in CFS• Changes in the model (e.g., change of other parameterizations

in CFS) • Transition of the NN radiation into GFS

• Developments allowing to reduce probability of larger errors and outliers: – Quality control and compound parameterization – NN ensembles

• Development of dynamically adjustable NN emulations (to climate changes, etc.)

• Using NN emulations for generating ensembles with perturbed physics

• NN emulations can be introduced in DAS (fast calculations + fast adjoint)

Date post:	13-Dec-2015
Category:	Documents
Upload:	delilah-page
View:	216 times
Download:	0 times

Development of Neural Network Emulations of Model Radiation for Improving the Computational...

Documents