ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 1

Norm Phillips’ work in Data Assimilation A.Hollingsworth Seattle Jan 2004

A.Hollingsworth, A.Simmons, W. Zwieflhoefer,

M.Dragosavac, S.Uppala, J.Woollen*, D.Marbouty,

J-N Thepaut, R Engelen, A Dethof,

ECMWF

* NCEP

The Interplay of Computer Power, Computer

Architectureand Numerical Algorithms in the

progress of Numerical Weather Prediction

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 2

Scope of Talk

NWP requirements for resolution

Semi-Lagrangian time-schemes

Implementing efficient schemes on parallel machines

Operational and scientific implications of such

economies

A look back to June 1944

A look to the challenges of the future

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 3

Significance of model and analysis resolution

Model resolution can control the success of forecasts for major rain systems.

The next few pictures illustrate the importance of resolution (40km v 65km) in one of a series of episodes of heavy rains in the Mediterranean

In the 40km model, the forecast successfully stretched, and then rolled up, the streamer of Potential Vorticity, while the 65 km model was a bust on these critical features.

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 4

9 September 2000 - 12 UTC D+6 forecast

AN TL511 D+6 TL319 D+6 AN TL511 D+6 TL319 D+6

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 5

Paying for Model Resolution

The change in 2000 at ECMWF from a 65km (T319) to a 40 km (T511) model had a big positive impact, as was expected from hundreds of days of pre-operational trials.

Dritschel et al. (1999) imply that 15km resolution is needed for a good 5-day forecast of the PV field – realisable about 2010.

Resolution is very costly (computer cost increases as ~cube of resolution).

Several approaches can meet the need

Increase the money stream (v.difficult)

Keep the money-stream constant and rely on Amdahl’s Law.

Keep the money-stream constant, rely on Amdahl’s Law, and use efficient time schemes (semi-implicit, semi-Lagrangian)

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 6

The semi-Lagrangian scheme, integrates the equations of motion by •Calculating, for every point on the grid, the trajectory from the departure point (at t-1)•Interpolating the (t-1) values to the departure points•Advecting the values forward along the trajectories

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 7

Parallellism & Communication on Distributed Memory Machines

•Efficient time schemes require the solution of global equations.

•One cannot get all the data into one processor, soA continuous complex shuffling / re-shuffling of the data is required to do a global calculation in many small steps

•The figure illustrates the many data transpositions needed to pass from physical to Fourier to legendre space and back again

•The implication is that the speed & capacity of the inter-processor communication is as crucial as the processor power.

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 8

1987 1992 1997 1998 20000

200

400

600

800

1000

1200

1400

1600F

orec

ast d

ays

per

com

pute

day

Eulerian scheme

Operational scheme

T106L19CRAY X-MP/4

T213L31CRAY C90/16

T213L31

Fujitsu VPP700/116

TL319L50

Fujitsu VPP700/116

Semi-Lagrangian algorithmic improvements - operational resolutions

TL511L60

Fujitsu VPP5000/100

Factors 1 4 12 72 150Reasons Semi-Lagrangian Reduced Gaussian Linear grid Higher resolutions

two-time levels stratosphere

x 4 x 12 x 72 x 150

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 9

Implications of numerical efficiency for productivity & vendor competition

Current semi-Lagrangian schemes offer very substantial gains in efficiency

Without those gains ECMWF could not afford Deterministic model at 40 km (T511) 51 member Ensemble system at 80km (T255) Advanced 4D-Var assimilation system Assimilation of millions of pieces of data from several

dozen satellite instruments Coupled seasonal forecast systems at 2 deg. (T95)

At best ECMWF could afford a 2-day forecast at T511

We certainly could not plan a 25 km system (T799/L90) in 2005

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 10

Sustained forecast improvements in both hemispheres Convergence of skill

between the hemispheres

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 11

A look to the past

Let us now praise great men -

Day 6 June 1944

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 12

Observations for 12UTC 3 June 1944

423 pilot balloons

Observations supplied by Jack Woollen, NOAA/NWS/NCEP

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 13

Observations for 12UTC 3 June 1944

676 SYNOPS 112 SHIPS

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 14

10m wind and low cloud 00UTC 6 June 1944

T159 3D-Var analysis

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 15

10m wind and low cloud 06UTC 6 June 1944

T159 3D-Var analysis

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 16

10m wind and low cloud 12UTC 6 June 1944

T159 3D-Var analysis

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 17

10m wind and low cloud 18UTC 6 June 1944

T159 3D-Var analysis

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 18

The early morning of June 6 1944: Low cloud and westerly winds off the Normandy beaches

Pictures fromUS NavalHistorical Center

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 19

June 6 1944: Clear skies over the channel and later over the Normandy beaches

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 20

Comparison with contemporary charts

Reproduced from www.meteo.fr

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 21

10m wind and cloud T799 from 00UTC 3 June 1944

H+78H+54

H+84 H+90

06UTC 5 June

12UTC 6 June

06UTC 6 June

18UTC 6 June

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 22

10m wind & cloud with 25km (T799) from 12UTC 3 June

H+66H+42

H+72 H+78

06UTC 5 June

12UTC 6 June

06UTC 6 June

18UTC 6 June

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 23

http://www.ecmwf.int/newsevents/releases/030604.html

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 24

Historical perspective1994 (ten years ago):

Cray C90-16 installed in 1992

16 processors

Sustained performance: 6 gigaflops2004 (one decade later):

IBM Phase 3

~4000 processors (250-fold increase)

Sustained performance: 2 teraflops (~120-fold increase per decade)

Distributed memory systems and a competitive HPC market

2013 (almost another decade): Preparation of 10 km system in the Strategy Review

Unknown number of processors

Sustained performance: 200 teraflops (100-fold increase)

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 25

Climate Monitoring and the Chemical Weather Forecast SystemLong-range transport of air pollutants is a well

established fact:- ~ 40% of the material in Europe arrived from Asia, via N.America

Satellite data provide a vast amount of data on atmospheric composition: reactive gases, smog, aerosol, greenhouse gases.

Extraction of the information on composition requires as the pre-requisite the sophisticated NWP assimilation systems to provide temperature, humidity, cloud, …

The GEMS consortium (10 Science institutes, 10 regional air-quality labs, ECMWF) will provide an operational global composition monitoring capability and a global /regional air-quality operational forecast capability, built around ECMWF system (funding by EU).

A fully interactive chemistry module in the global ECMWF model will pose substantial computational challenges.

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 26

Structure of an Earth-system model - all processes and interactions must be well represented

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 27

Monitor Greenhouse Gases:- CO2, N2O, CH4, CO

CO2 – Stratosphere – May 2003

First analysis of stratospheric CO2 shows Brewer-Dobson type of circulation. Variability is also much smaller than in troposphere.

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 28

CO2 – Troposphere – May 2003

CO2 tropospheric columns are assimilated from AIRS infrared observations. Monthly mean distribution for May 2003

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 29

Monitor & Forecast AEROSOL (& Fires)

Model and assimilate global aerosol informationHeritage: -Instruments: MERIS, MODIS x 2, MISR, SEAWIFS,

POLDERData Mgt tbdR/T “Modelling “Sources/ Sinks “Data Assim. “Validation “

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 30

Monitor & Forecast Reactive GasesOzone Hole 1 Oct 2003

Southern Hemisphere

1 Map of total column O3

2 Cross-section

3 Validation v. Neumayer ozonesonde

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 31

Closing words

Magna Laudatio to JNWP / NMC / NCEP for 50 years of pioneering leadership

Good wishes from ECMWF for your continued success.

The future will be at least as challenging as the past,on the scienceon the political scienceon the operational systems to meet customer

needs

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 32

EndThank you for your attention!

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 33

7 September 2000 - 12 UTC D+4 forecast

ANALYSIS TL511 D+4 TL319 D+4

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 34

8 September 2000 - 12 UTC D+5 forecast

AN TL511 D+5 TL319 D+5 ANALYSIS TL511 D+5 TL319 D+5

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 35

ECMWFJNWP+50 Hollingsworth, Washington DC June 2004 Slide 36

Group Captain J.M. Stagg

Photograph fromwww.metoffice.com

Meteorological advisor to Eisenhower

Responsible for reconciling the forecasts of three teams:

The Met Office The Royal Navy The US Air Force