Date post: | 18-Jan-2018 |
Category: |
Documents |
Upload: | wesley-bond |
View: | 216 times |
Download: | 0 times |
1
How Does NCEP/CPC Make Operational Monthly and Seasonal
Forecasts?
Huug van den Dool (CPC)
ESSIC, February, 23, 2011
2
Assorted Underlying Issues
• Jin Huang - R2O - CTB• Which tools are used…• How are tools combined???• CFSv1 CFSv2• Dynamical vs Empirical Tools• Skill of tools and OFFICIAL• How easily can a new tool be included?• US, yes, but occasional global perspective
3
Menu of CPC predictions:
• 6-10 day (daily)• Week 2 (daily)• Monthly (monthly + update)• Seasonal (monthly)• Other (hazards, drought monitor, drought outlook,
MJO, UV-index, degree days, POE, SST) (some are ‘briefings’)
• Informal forecast tools (too many to list)
• http://www.cpc.ncep.noaa.gov/products/predictions/90day/tools/briefing/index.pri.html
4
5
EXAMPLEPUBLICLY
ISSUED
6From an internal CPC Briefing package
7
EMP EMP EMP EMP
EMPDYN
DYN
CONCON
N/A
88
SMLR CCA OCN
LAN
LFQ
(15 CASES: 1950, 54, 55, 56, 64, 68, 71, 74, 75, 76, 85, 89, 99, 00, 08)
OLD-OTLK
CFSV1
ECPIRI
ECA CON
9
Element US-T US-P SST US-soil moisture
Method:CCA X X XOCN X X CFS X X X XSMLR X XECCA X XConsolidation X X X
Constr Analog X X X XMarkov XENSO Composite X XOther (GCM) models (IRI, ECHAM, NCAR, CDC etc):
X X
CCA = Canonical Correlation AnalysisOCN = Optimal Climate NormalsCFS = Climate Forecast System (Coupled Ocean-Atmosphere Model)SMLR = Stepwise Multiple Linear RegressionCON = Consolidation
10
Long Lead Predictions of US SurfaceTemperature using Canonical CorrelationAnalysis. Barnston(J.Climate, 1994, 1513)
Predictor - Predictand Configuration
Predictors Predictand
* Near-global SSTA
* N.H. 700mb Z * US sfc T
* US sfc T
four predictor “stacked” fields one predictandperiod
4X652=2608 predictors 102 locations
Data Period 1955 - last month
11
12
13
About OCN. Two contrasting views:- Climate = average weather in the past- Climate is the ‘expectation’ of the future
30 year WMO normals: 1961-1990; 1971-2000 etc
OCN = Optimal Climate Normals: Last K year average. All seasons/locations pooled: K=10 is optimal (for US T).
Forecast for Jan 2012 = (Jan02+Jan03+... Jan11)/10. – WMO-normal
plus a skill evaluation for some 50+ years.
Why does OCN work?1) climate is not constant (K would be infinity for constant climate)2) recent averages are better3) somewhat shorter averages are better (for T)see Huang et al 1996. J.Climate. 9, 809-817.
14
OCN has become the bearer of most of the skill, see also EOCN
method (Peng et al)
15
16
GHCN-CAMS
FAN
2008
17
Major Verification Issues
• ‘a-priori’ verification (used to be rare)
• After the fact (fairly normal)
18
Source Peitao Peng
After the fact…..
19
(Seasonal) Forecasts are useless unless accompanied by a reliable a-
priori skill estimate.
Solution: develop a 50+ year track record for each tool. 1950-present.
(Admittedly we need 5000 years)
20
Consolidation
21
--------- OUT TO 1.5 YEARS -------
22
OFFicial Forecast(element, lead, location, initial month) =
a * A + b * B + c * C +…
Honest hindcast required 1950-present. Covariance (A,B), (A,C), (B,C), and
(A, obs), (B, obs), (C, obs) allows solution for a,
b, c (element, lead, location, initial month)
23
CFS skill 1982-2003
24
Fig.7.6: The skill (ACX100) of forecasting NINO34 SST by the CA method for the period 1956-2005. The plot has the target season in the horizontal and the lead in the vertical. Example: NINO34 in rolling seasons
2 and 3 (JFM and FMA) are predicted slightly better than 0.7 at lead 8 months. An 8 month lead JFM forecast is made at the end of April of the previous year. A 1-2-1 smoothing was applied in the vertical to
reduce noise.CA skill 1956-2005
25
M. Peña Mendez and H. van den Dool, 2008: Consolidation of Multi-Method Forecasts at CPC. J. Climate, 21, 6521–6538.
Unger, D., H. van den Dool, E. O’Lenic and D. Collins, 2009: Ensemble Regression.Monthly Weather Review, 137, 2365-2379.
(1) CTB, (2) why do we need ‘consolidation’?
26
27
(Delsole 2007)
28
UR MMA COR
RI RIM RIW
Climo
29
3CVRE
SEC
SEC and CV
30
31
Bayesian Multimodel Strategies
Linear regression leads to unstable weights for small sample sizes.
Methods for producing more stable estimates have been proposed by van den Dool and Rukhovets (1994), Kharin and Zwiers (2002), Yun et al. (2003), and Robertson et al. (2004).
These methods are special cases of a Bayesian method, each distinguished by a different set of prior assumptions (DelSole 2007).
Some reasonable prior assumptions: R:0 Weights centered about 0 and bounded in magnitude
(ridge regression)R:MM Weights centered about 1/K (K = # models) and bounded in magnitudeR:MM+R Weights centered about an optimal value and bounded in magnitudeR:S2N Models with small S2N (signal-to-noise) ratio tend to have small weightsLS Weights are unconstrained (ordinary least squares)
From Jim Kinter (Feb 2009)
32
If the multimodel strategy is carefully cross validated, then the simple mean beats all other investigated multimodel strategies.
Since Bayesian methods involve additional empirical parameters, proper assessment requires a two-deep cross validation procedure. This can change the conclusion about the efficacy of various Bayesian priors.
Traditional cross validation procedures are biased and incorrectly indicate that Bayesian schemes beat a simple mean.
From Jim Kinter (Feb 2009)
33
34
35No SEC, (no CV required), ‘raw’)
36
Good to know:
• If a model/methods has no systematic error (by design), leave it alone, please do not apply systematic error correction where none is needed.
• There is more (need for) statistics in dynamical models and their post-processing than in any statistical method
37
38
39
40
See also:
O’Lenic, E.A., D.A. Unger, M.S. Halpert, and K.S. Pelman, 2008: Developments in Operational Long-Range Prediction at CPC. Wea. Forecasting, 23, 496–515.
41
Empirical tools can be comprehensive! (Thanks to
reanalysis, among other things).
And very economic.
42
SST Z500
PrecipT2m
CA
43
SST Z500
PrecipT2m
CFS
Source: Wanqiu Wang
44
Is CPC in good shape to admit more tools???
Even if these tools come from the outside???
Does the outside have the right expectations???
45
extra
46
Dynamics> Empiricism Symbiosis
• Successive generations of Reanalyses are produced because NWP exists, the desire for initial conditions etc, but empirical prediction methods are one of the main beneficiaries of having all this data.
• Early empirical methods have served as an example (for anybody to follow) to produce (honest) hindcasts, a-priori skill assessment, cross validation etc
• Consolidation (of tools) is “color-blind” relative to questions of dynamical or empirical origin
47
Progress in Empirical Methods
• More data to work with (1 year per year)
• More (and global) data to work with (Re-analyses). Oceans, land, atmosphere(note the symbiotic relationship with modeling here)
• New empirical methods
• New applications Inch-by-inch methods
Revolutionary changes
Always be ready for surprises
48
Why are empirical Methods competitive with dynamical
methods (in seasonal prediction)??
• Linearity (define)• A system with skill in <= 3EDOFs is
functionally linear• See Chapter 10 of book
49
Mix of Dynamical & Empirical
• Day 1 – 14: NWP ‘reigns’• Wk 2 – 6 Dynamics & Empirical• SI: Dynamics & Empirical• Decadal ? • Climate Change ?