WRF Project Overview

Joseph B. Klemp, NCAR

COMET WORKSHOP

Boulder, Colorado

31 March 2000

Weather Research and Forecasting (WRF) Model

Weather Research and Forecast (WRF) Model

Promote closer ties between research and operations

Develop an advanced mesoscale forecast and assimilation system

Research:

Design for 1-10 km horizontal grids

Advanced data assimilation and model physics

Accurate and efficient across a broad range of scales

Well-suited for both research and operations

Community model support

Original Partners:

– NCAR Mesoscale and Microscale Meteorology Division– NOAA National Centers for Environmental Prediction– NOAA Forecast Systems Laboratory– OU Center for the Analysis and Prediction of Storms

Additional Collaborators:

– Air Force Weather Agency– NOAA Geophysical Fluid Dynamics Laboratory– NASA GSFC Atmospheric Sciences Division– NOAA National Severe Storms Laboratory– NRL Marine Meteorology Division– EPA Atmospheric Modeling Division– University Community

WRF Project Collaborators

WRF Project Management

WRF OversightBoard

WRF ScienceBoard

WRF Coordinator

WRF Development Teams (5)

Responsible for overall supervision of the WRF Project:

– Monitors plans and progress of the project – Obtains commitments from the heads of participating agencies– Deals with funding requests and budget issues– Provides progress reports to the USWRP IWG and other funding agencies– Appoints the WRF Coordinator and members of the WRF Science Board

Members represent organizations that have made a major commitment of time and resources to the WRF effort

– Steve Lord, chair NOAA/NCEP– Bob Gall NCAR/MMM– Steve Nelson NSF/ATM– Sandy MacDonald NOAA/FSL & GFDL– Col. Charles French USAF/AFWA

WRF Oversight Board (WOB)

Provides technical guidance to the WRF effort to help ensure that WRF will meet the needs of a broad community in both research and operations

– Identifies functional requirements for desired applications– Provides feedback on technical approaches– Promotes active participation in WRF development efforts

Members represent a broad constituency of the research and operational mesoscale forecast community:– Appointed for three-year terms– Communicate via email, web postings, and annual meetings

WRF Science Board (WSB)

Provides overall coordination of the WRF ProjectProvides overall coordination of the WRF Project

Keeps the WOB informed of progress and seeks advice on issues that cannot be resolved at the working level

Appoints WRF Development Teams leaders Works together with Team Leaders to ensure that:

– Overall design goals are achieved– Milestones are accomplished on schedule– Efforts are coordinated among development teams– Technical issue and progress are discussed with the WSB

WRF Coordinator

Numerics and Software

(J. Klemp)

Data Assimilation (T. Schlatter)

Analysis and Validation

(K. Droegemeier)

Community Involvement

(W. Kuo)

Operational Implementation

(G. DiMego)

Dynamic Model Numerics

(W. Skamarock)

Post Processing (L. Wicker)

NCEP Requirements

(G. DiMego)

Wo

rkin

g G

rou

ps

AFWA Requirements

(M. Farrar)

Model Testing and Verification

(C. Davis)

Model Physics (J. Brown)

Web Pages & Workshops (J. Dudhia)

Distribution, Documentation,

and Support (J. Dudhia)

Software Architecture,

Standards, and Implementation (J. Michalakes)

Standard Initialization (J. McGinley)

3-D Var (J. Purser)

4-D Var,Ensemble

Techniques (D. Barker)

WRF Development Teams

Performance-Portable– Performance: scaling and time to solution– Architecture independence– No specification of external packages

Run-Time Configurable– Scenarios, domain sizes, nest configurations– Dynamical-core and physics

Maintainability & Extensibility– Single source code– Modular, hierarchical design, coding standards– Plug compatible physics, dynamical cores

WRF Software Objectives

Model domains are decomposed for parallelism on two-levels

– Patch: section of model domain allocated to a distributed memory node– Tile: section of a patch allocated to a shared-memory processor within a node– Distributed memory parallelism is over patches; shared memory parallelism is over tiles within

patches

Single version of code enabled for efficient execution on:

– Distributed-memory multiprocessors

– Shared-memory multiprocessors– Distributed memory clusters of

SMPs

WRF Multi-Layer Domain Decomposition

Logical domain

1 Patch, divided into multiple tiles

Inter-processor communication

WRF Hierarchical Software Architecture

Top-level “Driver” layer isolates computer architecture concerns– Manages execution over multiple nested domains– Provides top level control over parallelism, including patch-decomposition, inter-

processor communication, shared-memory parallelism, etc.– Controls Input/Output

Low-Level “Model” layer code performs actual model computations– Is written to be callable for calculations within a single tile– Allows scientists to work with clean application code

Intermediate “Mediation” layer mediates between model and driver layers

Fortran90 facilitates hierarchical architecture – Allows dynamic memory allocation, derived data-types, pointers– Streamlines grid management

Parallel Scaling on Compaq Computer

Compaq ES40, 41x81x81

0

5

10

15

20

25

30

35

0 5 10 15 20 25 30 35

processors

spee

du

p

1 (2d)

2 (2d)

4 (2d)

1 (1d)

2 (1d)

4 (1d)

ideal

Penalty for IJK Loop Order

IJK versus KIJ for all patch dimensions X,Y=(21,41,81); 41 levels throughout Penalty for IJK decreases with increased length of minor dimension, X Penalty is most severe for sizes typical of a DM patch IJK is strongly favored by vector for adequate length of X Surprise: vector prefers KIJ for short X; but an unlikely result once full physics

2141

81

21

41

81

0

5

10

15

20

25

30

X tile dimension

Y tile dimension

Alpha workstation (EV56)

2141

81

21

41

81

-80

-60

-40

-20

0

20

40

60

80

100

X tile dimension

Y tiledimension

VPP 5000

Numerical Modeling Issues:

– Equations / variables – Vertical coordinate– Terrain representation– Grid staggering– Time Integration scheme– Advection scheme

Strategy:

– Identify and analyze alternative procedures– Evaluate alternates in idealized simulations– Evaluate in NWP applications as model complexity increases

Numerics for Dynamical Model Solver

Mountain Wave with Step Terrain Coordinate

Split-Explicit Eulerian Model:

– Pressure and temperature diagnosed from thermodynamics– Two time level split-explicit time integration– Flux-form prognostic equations in terms of conserved variables – Accurate shape preserving advection– Both terrain-following height and mass coordinates being tested

Semi-Implicit Semi-Lagrangian Model:

– Unstaggered (A) grid– Forward trajectories with cascade interpolation back to grid– High order compact differencing– Terrain following hybrid coordinate

Prototype Nonhydrostatic Model Solvers

0

z

W

x

U

t

Qz

W

x

U

t

z

Ww

x

Uwg

zR

t

W

z

Wu

x

UufV

xR

t

U

,,, wWvVuU

pcR p

Conservative variables:

Inviscid, 2-Dequations inCartesiancoordinates

Pressure termsdirectly related to

Flux-Form Equations in Height Coordinates

Flux-Form Equations in Mass Coordinates

0

,,

0

p

Rpgw

dt

d

x

U

t

Qx

U

t

w

x

Uwpg

t

W

u

x

Uu

x

p

x

p

t

U

tst ,/Hydrostatic pressure coordinate:

Inviscid, 2-Dequations without rotation:

,,, wWuUConservative variables:

Comparison of Height and Mass Coordinates

Implement and test basic physics in WRF:– Kessler-type (no-ice) microphysics – Lin et al. (graupel included) microphysics – Kain-Fritsch cumulus parameterization – Shortwave radiation (cloud-interactive) from MM5 – Longwave radiation (RRTM) – MRF (Hong and Pan) PBL – Blackadar surface slab ground temperature prediction

Implement a complete suite of research physics packages

Encourage and facilitate community involvement in advanced model physics development and evaluation

Strategy for WRF Model Physics

Essential features of initial 3D-Var system:

– Basic quality control

– Assimilation of conventional observations (surface, radiosonde, aircraft)

– Multivariate analysis

– Adherence to WRF coding standards

Additional features to be added:

– Anisotropic background errors

– Additional observation operators (radar, satellite, wind profiler, etc.)

– Flexible choice of first guess

– Further enhancements

WRF 3D-Var Data-Assimilation System

WRF Model Testing and Verification Strategy

Analytic and converged numerical solutions

– Inviscid dynamics (baroclinic instability, frontogenesis)– Buoyancy driven flow (gravity currents, warm thermals)– Topographic flow (nonhydrostatic, hydrostatic, inertial-gravity mountain waves)– Moist convection (idealized convection with constant eddy mixing)

Regime dependence of nonlinear flows

– Topographic flow (finite amplitude waves, wave overturning, lee vortices)– Moist convection (convective behavior as a function of CAPE and shear)

Observational case studies

– Extratropical cyclones (STORM-FEST case)– Topographic flow (downslope windstorm, orographic precip., cold-air damming)– Moist convection (supercell case, squall-line case, multi-parameter radar case)– PBL-surface physics (1-D dirunal cycle, sea-breeze case, marine inversion and CTD)

Development Task 2000 2001 2002 2003 2004

Basic WRF model (limited physics, standard initialization)

Research quality NWP version of WRF

Model physicsSimple Research suite Advanced

3D-Var assimilation systemBasic Advanced

4D-Var assimilation systemBasic Advanced

Testing for operational use at AFWA

WRF model adapted to NCEP computing environment

Release and support to community Operational deployment

Tentative Timeline for WRF Project

12 January

14 February

29-30 March

23 June

First WRF Oversight Board Meeting

WRF Planning Meeting

WRF Planning Workshop

First Annual WRF Users Workshop

WRF Calendar for 2000

WRF Status & Updates: www.wrf-model.org