ESA DA Projects Progress Meeting 2 University of Reading

Advanced Data Assimilation MethodsWP2.1 Perform (ensemble) experiments to quantify model errors.

Stefano Migliorini, Ross Bannister,National Centre for Earth Observation, University of ReadingAli Rudd, Laura BakerDepartment of Meteorology, University of Reading

11 December 2012

Motivation• Satellite observations form the vast majority of the total

number of observations assimilated in NWP models.• To exploit information from satellite (as well as in-situ)

instruments, prior knowledge from NWP forecasts is needed (Bayesian approach).

• Climatology of forecast errors at larger scales reflects well-known balance relationships of atmospheric flow.

• Structure of high-res forecast errors are much more uncertain.• Aim of this work is to provide reliable estimates of forecast

errors at convective scale (new generation models) to improve assimilation of in-situ, radar and satellite data.

Aims of the project• Investigate sources of uncertainty in high-res forecasts:

– Initial and boundary condition errors.– Model errors due to the parameterisation of subgrid-scale

processes.

• Use a convective-scale ensemble prediction system (EPS).• Evaluate the effects of these errors on the forecast error

covariances.• Check reliability of errors using observations.• Improve our knowledge of high-res forecast errors and of

their balance relationships for better high-res DA.

Case Study: 20 September 2011

1200UTC analysis 1800UTC analysis

• DIAMET IOP2 – flight campaign case.

• Frontal wave structure.• SW-NE flow across southern UK.• Interesting banded structure in

radar not captured in the operational 1.5km forecast or our control forecast.

Ensemble systemUK Met Office operational ensemble systems

MOGREPS: Met Office Global and Regional Ensemble Prediction System:

•MOGREPS-G: 60 km grid spacing, 70 vertical levels.•MOGREPS-R: 18 km grid spacing, 70 vertical levels.•23 perturbed members and one control member.•(MOGREPS-UK: 2.2km grid spacing, 12-member ensemble).

MOGREPS-R

MOGREPS-G

Figure source: J.F. Caron

• Domain over southern UK (360 x 288 grid points).• Control member from 3D-Var analysis.• 23 perturbed members: initial condition perturbations and LBCs from MOGREPS-R.• Hourly-cycling ETKF for the first 6 hours.• 6 hour forecast from 12Z.

MOGREPS-R

1.5 km domain

MOGREPS-G

07 08 09 10 11 12 13 14 15 16 17 18

6hr forecast

Figure source: J.F. Caron

Simulating model error: the Random Parameter scheme

Has been used operationally in MOGREPS. Not used previously in a convective-scale EPS. RP treats a set of parameters in various parametrization schemes as stochastic variables. Applies different random perturbations to these parameters for each ensemble member. Based on first-order auto-regression model (P

t is the parameter value at time t):

Pt = μ + r (P

t-1 – μ) + ε

μ is the default value of the parameter. r = 0.95 is the auto-correlation coefficient of P. ε is the stochastic shock term (random value in range ± (P

max – P

min) / 3).

Pmax

, Pmin

for each parameter are estimated by experts. Have studied forecast sensitivities to each parameter.

Options to how RP can be applied: CTL: Parameters set the same between members (ics only). RP-60: Update every 60 minutes. RP-30: Update every 30 minutes. RP-fix: Parameters set at t = 0 only. onlyRP:RP-60\30\fix without ic.

Sensitivity to perturbed parameters

RMS difference between perturbed and control forecasts at T+3 (1500

UTC)

10m u-wind

1.5m temperature

Ensemble experimentsEnsemble name Description Model error

variability IC and LBC variability

Inflation

CTL Control No Yes Yes

IC+BC+RPfix RP scheme with fixed params

Yes Yes Yes

IC+BC+RP30 RP scheme with 30 minute update

Yes Yes Yes

IC+BC+RP60 RP scheme with 60 minute update

Yes Yes Yes

RPfix ME only: RP scheme with fixed

params

Yes No Yes

RP30 ME only: RP scheme with 30 minute update

Yes No Yes

RP60 ME only: RP scheme with 60 minute update

Yes No Yes

How does model error affect the spread?

___ control ensemble___ ensemble with fixed perturbed parameters

1.5m temperature

Hourly rainfall accumulation

10m wind speed

Domain-averaged ensemble spread:

1

1variance

n-points

n points

ii

How does model error affect the spread?

1.5m temperature

Hourly rainfall accumulation

10m wind speed

Domain-averaged ensemble spread:

1

1variance

n-points

n points

ii

___ control ensemble___ ensemble with fixed perturbed parameters___ ensemble with periodic update (30 min)___ ensemble with periodic update (60 min)

How does model error affect the spread?

1.5m temperature

Hourly rainfall accumulation

10m wind speed

___ control ensemble___ ensemble with fixed perturbed parameters___ ensemble with periodic update (30 min)___ ensemble with periodic update (60 min)- - - model error only (fixed parameters)- - - model error only (30 min)- - - model error only (60 min)

How does model error affect the forecast skill?

surface temperature

Bett

er s

kill

u-wind component v-wind component

Bett

er s

kill

CRPS:- Continous ranked probability score.- Comparison of CDF of forecast and obs.

___ control ensemble___ ensemble with fixed perturbed parameters___ ensemble with periodic update (30 min)___ ensemble with periodic update (60 min)

rain accumulation

How does model error affect the forecast skill?

Threshold of 0 mm Threshold of 1.0 mm

Precipitation skill score:ens

enscontrol

1BS

PSSBS

Precipitation skill score for hourly rainfall accumulation

___ control ensemble___ ensemble with fixed perturbed parameters

Threshold of 0.2 mm

Bett

er s

kill

How does model error affect the forecast skill?

___ control ensemble___ ensemble with fixed perturbed parameters___ ensemble with periodic update (30 min)___ ensemble with periodic update (60 min)

Threshold of 0 mm Threshold of 1.0 mm

Precipitation skill score:ens

enscontrol

1BS

PSSBS

Precipitation skill score for hourly rainfall accumulation

Threshold of 0.2 mm

Summary• Developing a method of representing model error in a convective

scale ensemble:– Random parameters scheme.

• How does the additional representation of model error affect the spread of the ensemble?– Temperature and wind speed – applying the RP scheme increases the

spread.– Rainfall rate – the RP scheme has an undesirable peaks in the spread –

this is reduced by keeping parameters fixed.

• How does model error affect the forecast skill?– Small effect on forecast skill.– Skill in rain rate and accumulation is reduced – probably due to

reduction in total rain rate.

Forecast errors in data assimilation

• q-w correlations > 0.• The more buoyant the parcel the wetter.• Descent leads to warmer and drier parcels.

Conclusions

• Including model error variability increases ensemble spread in most quantities, over and above that found from ensembles that include only initial condition and lateral boundary condition variability.

• Including model error variability is not guaranteed to increase ensemble spread in all quantities, e.g. we have found that the spread of rainfall forecasts is actually reduced, although it is not clear why this is so.

• Ensemble forecasts can inform data assimilation studies of the correct structure of forecast error statistics and the balances that are obeyed.

• Future work includes investigations on covariance length scales, study of forecast errors using observations and effects of sampling errors.

Extra slides

Do any of the ensemble members capture the banding in the rain?

Ensemble: RP scheme on - with model error

1500 UTC

“stamp plot”

Parameters in our modified RP scheme• Parameters above

the black line are in the existing scheme; those below are new

• We vary some parameters together where appropriate (eg. ei and eic; x1r, x1i and x1ic) – i.e. we use the same random seed for ei and eic so that they vary together rather than independently

• We have found that the particle size distribution parameters (x1r, etc.) have a larger effect than any others – possibly too large