Ensemble ForecastingLab Activities

Ensemble ForecastingLab Activities

M. Mullusky & J. Demargne

J. Schaake, E. Welles, D.-J. Seo, H. Herr,

L. Wu, X. Fan, and S. Cong

OHD, 04/21/04

ContentContent

2

• Introduction and Ensemble Activities

• Ensemble Pre-Processor Methodology

• Ensemble Pre-Processor Status by Component– Ensemble Generation

– Calibration

– Evaluation & Verification

– Ensemble Product & Visualization

– Papers

• ESP system– Current ESP System: SS-SAC, Ensemble Post-Processor

– Future ESP System: VAR, Processors for other uncertainties

– Verification

– Architecture

• Conclusion

IntroductionIntroduction

• Main goal of ensemble activities:

– Seamless and consistent probabilistic forecasts for all lead times

– Accounts for both meteorological and hydrologic uncertainties

– Verify ESP performance in both space and time

• The time scale is currently tied to the lead times of available meteorological forecasts:– 1 to 5 days: short term– 6 to 14 days: medium range– Two weeks and beyond: long range

3

Ensemble ActivitiesEnsemble Activities

• Main activities for the whole ESP system

Pre-ProcessorProcessor

VARReservoirs*FLDWAV*

Post-Processor

Verification

Architecture Management Product Dissemination

* new options required for specific forecast points 4

Research & Implementation Diagram

5

No assumption of normality for observed and

forecast distributions

Archived data

X

Y

Observed

Fore

cast

0

ZX

ZY

Observed

Fore

cast

zX0

zY0

Joint distribution

NQT

Normal Space

1. Short-Term Calibration: at each time step for the whole year, compute the parameters of the joint distribution of observed andforecast precipitation/temperature values

Ensemble Pre-Processor MethodologyEnsemble Pre-Processor Methodology

Example for PQPF/PQTF

Short Term Parameters

that describe the joint

distribution

6

PQPF (PQTF) given a

QPF (QTF)

Inverse NQT

ZX

ObservedPr

obab

ility

0

1

P ( ZX ≤ zx0 | ZY = zy )z0

Conditional distributionNormal Space

ZX

ZY

Observed

Fore

cast

zX0

zY0

For a given forecast

Joint distribution

NQT

Normal Space

2. Generate Short-Term PQPF/PQTF Distribution: at each time step for the forecast period, compute the parameters of the conditional distribution of future precipitation/temperature values

Ensemble Pre-Processor MethodologyEnsemble Pre-Processor Methodology

Example for PQPF/PQTF

QPF (QTF)

3. Short-Term Distribution Mapping: at each time step of the forecast period, generate ensemble points given the conditional distribution of future precipitation/temperature from climatology time series

3. Short-Term Distribution Mapping: at each time step of the forecast period, generate ensemble points given the conditional distribution of future precipitation/temperature from climatology time series

7

1

Prob

abili

ty

01960 1949 1983

Precipitation Amount

1Pr

obab

ility

01960 1949 1983

Precipitation Amount

1

Prob

abili

ty

01960 19491983

Precipitation Amount

…

T1 T2 T3

Ensemble points incorporate the skill of the single value forecast

Space-time properties are similar to the historical events properties

Climatology

Forecast

Climatology

Forecast

Climatology

Forecast

Ensemble Pre-Processor MethodologyEnsemble Pre-Processor Methodology

4. Distribution Mapping if no QPF/QTF Forecast: at each time step of the forecast period, use the smoothed climatology distribution of historical precipitation/temperature and distribution mapping to generate ensembles

4. Distribution Mapping if no QPF/QTF Forecast: at each time step of the forecast period, use the smoothed climatology distribution of historical precipitation/temperature and distribution mapping to generate ensembles

8

1

Prob

abili

ty

01960 1949 1983

Precipitation Amount

1Pr

obab

ility

01960 1949 1983

Precipitation Amount

1

Prob

abili

ty

01960 19491983

Precipitation Amount

…

T1 T2 T3

Space-time properties are similar to the historical events properties

Smooth Climatology Smooth Climatology Smooth Climatology

Ensemble Pre-Processor MethodologyEnsemble Pre-Processor Methodology

5. Climate adjustments: integrates days 1-365 meteorological forecasts/climate outlooks from NCEP/CPC. The pre-processor adjusts smoothed historical mean areal precipitation (MAP) and temperature (MAT) time series with respect to the current meteorological forecasts/climate outlooks.*Pre-processor will only do climate adjustments if no QPF/QTF forecast

5. Climate adjustments: integrates days 1-365 meteorological forecasts/climate outlooks from NCEP/CPC. The pre-processor adjusts smoothed historical mean areal precipitation (MAP) and temperature (MAT) time series with respect to the current meteorological forecasts/climate outlooks.*Pre-processor will only do climate adjustments if no QPF/QTF forecast

9

Ensemble Pre-Processor MethodologyEnsemble Pre-Processor Methodology

ContentContent

10

• Introduction and Ensemble Activities

• Ensemble Pre-Processor Methodology

• Ensemble Pre-Processor Status by Component

– Ensemble Generation

– Calibration

– Evaluation & Verification

– Ensemble Product & Visualization

– Papers

• ESP system

• Conclusions

Pre-Processor Status: Ensemble GenerationPre-Processor Status: Ensemble Generation

• Delivered enhancements (04/19/04 delivery)– Create one unified pre-processor

– Allow non 12Z forecasts

– Extend the QPF from the control file

• Future enhancements– Allow ingestion of NetCDF data

– Modify the 6-10 day temperature adjustments. Add the 8-14 day temperature and precipitation adjustments

– Compute short term temperature ensembles more efficiently (remove redundant NQT)

– Add Forecaster Control

– Enhance the short term procedure to use the CPC precipitation forecasts for days 2-5 if no RFC forecast is available

– Enhance the short term procedure to use the CPC precipitation and temperature forecasts for days 6-14 if no RFC forecast is available 11

Pre-Processor Status: Calibration

Pre-Processor Status: Calibration

• Delivered enhancements (Dec. 03 delivery)

– Three RFCs are using Linux parameters

• Future Enhancements

– Update parameters

– Combine ens_pre_cp and ens_pre_cp2 into one operationally robust calibration program

– Estimate parameters for days 1-5 from CPC forecasts for ABRFC and MARFC, compare to parameters derived from RFC archive

– Enhance operational calibration program to include the short term calibration procedures

12

Pre-Processor Status: Evaluation

Pre-Processor Status: Evaluation

• Current enhancements:

– Created a research evaluation prototype to evaluate the goodness of fit of the model by comparing a simulated joint distribution with the real forecast-observation distribution

• Future Enhancements:

– Add a bivariate normality test to the evaluation prototype

– Provide analysis to test cases for three RFCs for days 1-5 precipitation and temperature

– Develop a checking technique for the estimate of rho

13

Pre-Processor Status: Verification

Pre-Processor Status: Verification

• Current enhancements:

– Created a verification developmental prototype that aims at assessing the quality of days 1-5 precipitation and temperature ensembles

Includes the ensemble generation component to simulate ensembles

Output: ~20 statistics including Nash-Sutcliffe Efficiency, Brier Skill Score, and Heidke Skill Score

• Future Enhancements:– Integrate other verification statistics (Talagrand diagram,

discrimination diagram)

– Extend lead times14

Pre-Processor Status: Product Analysis & Display

Pre-Processor Status: Product Analysis & Display

• ESPADP

– Delivered Enhancements (04/19/04 delivery)

ESPADP can read in the “PQPT/PQTF” output data cards

Fixed the “OBSOverlayPRD” and “OverlayPRD” feature

– Future Enhancements

???

15

Pre-Processor Status: Papers

Pre-Processor Status: Papers

• Paper 1: motivation for a new methodology

• Paper 2: presentation of the short-term ensemble pre-processor with example of results for daily precipitation and temperature ensembles at CNRFC

• Paper 3: results from applying the short-term ensemble pre-processor at ABRFC, CNRFC and MARFC

16

ContentContent

17

• Introduction and Ensemble Activities

• Ensemble Pre-Processor Methodology

• Ensemble Pre-Processor Status by Component

• ESP system

– Current ESP System: SS-SAC, Ensemble Post-Processor

– Future ESP System: VAR, Processors for other uncertainties

– Verification

– Architecture

• Conclusion

Current ESP SystemCurrent ESP System

State Updating

18

Corrects bias, accounts for hydrologic uncertainty

Reflect both uncertainties

Corrects bias, accounts for meteorological uncertainty

Hydrologic model

Ensemble traces of streamflow

Ensemble Post-Processor

QPF, QTF

Ensemble Pre-Processor

Ensemble traces of future precipitation and temperature

Final ensemble traces of streamflow

Current ESP System: State Updating & Post-Processor

Current ESP System: State Updating & Post-Processor

• SS-SAC (State-Space Sacramento Model): updates state variables through data simulation using latest observed streamflow– Requires to re-calibrate Sacramento Model parameters and

to estimate uncertainty of inputs, state variables and parameters

• Post-Processor: accounts for all hydrologic uncertainties collectively

– Parametric uncertainty & structural uncertainty in hydrologic model, as well as model initial conditions uncertainty

– Corrects for systematic model biases

19

Future ESP SystemFuture ESP System

State Updating

Parametric Uncertainty Processor

Initial Condition Uncertainty Processor

Hydrologic model

20Reflect all uncertainties

Verification & Retrospective

Verification

Ensemble traces of future precipitation and temperature

Ensemble Pre-Processor

Ensemble Post-Processor

Final ensemble traces of streamflow

Ensemble traces of streamflow

QPF, QTF

Structural Uncertainty Processor

Future ESP System: Individual Uncertainty Processors

Future ESP System: Individual Uncertainty Processors

• Goal: to explicitly account for individual sources of hydrologic uncertainties

• Initial Conditions Uncertainty Processor (VAR Project): to reduce and to quantify uncertainty in the initial conditions and to effect automatic run-time modification

Variational assimilation-based technique assimilates streamflow observations at the headwater basin outlet, potential evaporation and precipitation in real time

• Parametric Uncertainty Processor: to capture propagation of long-memory errors and extremely nonlinear errors and to simplify post-processing

• Structural Uncertainty Processor

21

Future ESP System: VerificationFuture ESP System: Verification

• Package to quantify quality of input & output ensembles

• Retrospective verification based on a retrospective simulation of ESP system

– Ensembles of Precipitation, Temperature, & Streamflow

– Needs to integrate the Ensemble Pre-Processor and Post-Processor

• ESP Verification System (ESPVS) currently under redevelopment

– Based on Franz and Sorooshian (2002) and others

– Includes Ranked Probability Score (RPS), Ranked Probability Skill Score (RPSS), discrimination diagram, & reliability diagram 22

Future ESP System: ArchitectureFuture ESP System: Architecture

• Follow a structured development process

– Develop Use Cases to help discover system requirements

– Document requirements to ensure more useable and maintainable software

• Focus on services based architecture to permit faster science infusion

– http://www.nws.noaa.gov/ohd/hrl/hseb/hseb_pdf_links.htm

– Communication between modules with XML

23

HEPEXHydrologic Ensemble Prediction Experiment

HEPEXHydrologic Ensemble Prediction Experiment

• Goal

– Develop “engineering quality” hydrologic ensemble prediction procedures for time scales (flash-flood to 1-yr) and space scales (1-km to continental)

• Organization

– IAHS (PUB), GEWEX (WRAP), WMO

• Initial Workshop: ECMWF, March 2004

– Develop science plan

24

ConclusionConclusion

25

Ensemble Pre-Processor

ProcessorVAR

ReservoirsFLDWAV

Post-Processor

Verification

Architecture Management Product Dissemination

CalibrationEnsemble Generation

Product Analysis & Display

Evaluation & Verification