Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 1
Roberto Buizza and Martin Leutbecher European Centre for Medium-Range Weather Forecasts
The forecast skill horizon
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 2
L2
Synopsis of my three lectures
§ L1 -‐ Ensembles: why? And how have we designed them at ECMWF?
Ø A complete solu>on of the weather predic>on problem can be stated in terms of an appropriate probability density func>on (PDF)
Ø Ensemble predic>on based on a finite number of integra>ons is the only feasible method to predict the PDF of forecast states
Ø At ECMWF, ini>al uncertain>es are simulated using EDA-‐ and SV-‐based perturba>ons, and model uncertain>es with two stochas>c schemes
§ L2 – Ensemble forecast skill: how could we measure it?
Ø Reliability is an important property that probabilis>c systems must have Ø Different metrics should be used to evaluate the quality of probabilis>c
forecasts (BS and BSS, reliability diagrams, ROCA, V, EFI and its metric) to highlight different aspects of their quality
L1
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 3
Synopsis of my three lectures
§ L3 – Predictability: where is the Forecast Skill Horizon (FiSH)?
Ø There is not a unique defini>on of forecast skill limit: the forecast skill horizon (the FiSH length) depends on the field (scale, variable, region)
Ø The forecast skill horizon is well beyond 2 weeks even for local, instantaneous fields, thus confirming results published in literature that certain phenomena (MJO, NAO, blocking, ..) can be predicted beyond 2 weeks
L3
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 4 © ECMWF
1) Context 2) Ensembles: why? 3) Where is the forecast skill horizon? Why is it there? 4) Conclusions
Outline
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 5 © ECMWF
Context: where is the forecast skill horizon (FiSH)?
The view so far has been that local, daily values can be predicted only up to about 2 weeks. ‘… the range of predictability (is defined as) the >me interval within which the errors in predic>on do not exceed some pre-‐chosen magnitude …’ ‘.. the range of predictability is about 16.8 days ..’ ‘.. these results .. offer liEle hope for those who would extend the two-‐week goal to one month … ’
(Lorenz, 1969)
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 6 © ECMWF
The forecast skill horizon: the view of the 1970s-‐80s
30
25
20
15
5
0
Fc day
FiSH No skill
Forecast skill horizon (@ ̴ 2 weeks)
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 7 © ECMWF
An elongated area with some severe
thunderstorms producing localised flash floods
21 Feb 2013: flash flood in Catania.
Today we can predict severe events few days ahead
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 8 © ECMWF
Day3 Day1
Day5 Day7
PR(TP>10mm/d): a week signal of increased chance for larger precip is present in the area even up to 7 days in advance
21 Feb 2013: flash flood in Catania.
Today we can predict severe events few days ahead
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 9 © ECMWF
Day7
Day5
Day3
Day1
Today we can predict severe events few days ahead
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 10 © ECMWF
0.2
0.3
0.4
0.5
0.6
Cor
rela
tion
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
YEAR
NAO Correlation
MJO in IC NO MJO in IC
The skill of monthly forecasts have been con>nuously improving both in the tropics for the MJO (top leg) and the extra-‐tropics for the NAO (top right). Improvements in the physics have led to beher teleconnec>on between tropics and extra-‐tropics (bohom).
5
6
7
8
9
10
11
12
13
14
15
Fore
cast
Day
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
YEAR
NAO Correlation
0.5 0.6 0.8
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
32
Fore
cast
Day
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
YEAR
MJO Bivariate Correlation
0.5 0.6 0.8
NH – NAO (11d)
TR – MJO (26d)
Large-‐scale events can be predicted weeks ahead
(From Frederic Vitart)
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 11 © ECMWF
Italy's electricity prices jumped in June and power trade volumes improved as a heat wave boosted demand, Italian energy markets operator GME said on Friday (Reuters, 13 July).
2mT 9-‐15 July 2012
5 July +5-‐11 days
21 June +19-‐25 days
Large-‐scale events can be predicted weeks ahead
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 12 © ECMWF
The tropics remain the area where seasonal predic>on has the highest skill, as indicated e.g. by the accuracy of 1-‐year forecasts of SST anomaly in the Nino3.4 area.
1 Nov ‘11 > Nov ‘12
Tropics: El-‐nino SST’ can be predicted months ahead
1 Nov ‘12 > Nov ‘13 1 Nov ‘13 > Nov ‘14
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 13 © ECMWF
Some_mes EU-‐2mT’ also can be predicted months ahead
Est EU MJJ12. Since Feb S4 predicts 75% probability of below normal condi>ons.
S4 SPI(MJJ) – 1Feb+456m S4 SPI(MJJ) – 1May+123m
SPI(MJJ) – ERA-I
(From Fredrik WeEerhall)
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 14 © ECMWF
30
25
20
15
5
0
Fc day
FiSH No skill
How can we conciliate these viewpoints?
Is the forecast skill horizon at 2 weeks as thought in the 1970s-‐80s? Or is it longer? How did we manage to push the limit beyond 2 weeks?
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 15 © ECMWF
How can we conciliate these viewpoints?
We aim to address the following key ques>ons: 1. If we consider local, instantaneous Z500 fcs, where is the forecast skill horizon (how
long is the FiSH)? 2. Does it make sense to talk very generally about a forecast skill horizon or should we
qualify its defini>on? 3. Can we develop a unifying framework that allows us to compare in a clear way the
skill of forecasts of different variables at different scales and over different regions?
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 16 © ECMWF
1) Context 2) Ensembles: why? 3) Where is the forecast skill horizon? Why is it there? 4) Conclusions
Outline
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 17 © ECMWF
fc0
fcj
reality
PDF(0)
PDF(t)
Forecast time
To predict the >me evolu>on of the probability density func>on of forecast states. In other words, to predict the most likely scenario and its uncertainty, expressed e.g. in terms of probabili>es of weather events, to es>mate the forecast confidence, ..
Ensembles: why?
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 18 © ECMWF
ENS51 S451
HRES
EDA25
ORAS45
ERA 4DV
PDF(0) << 4DV+EDA25+ORAS45 PDF(0) << ERA+ORAS45 (the past) PDF(T) << HRES+ENS51/S451
System components simulate the effect of: • Observa_on uncertain_es • Model uncertain_es (2 stochas_c schemes)
2014: ensembles are used in analysis and forecast modes
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 19 © ECMWF
The ECMWF IFS (2013) and the coupled ocean-‐atm ENS
ORTAS45 Real Time Ocean Analysis ~8 hours
HRESTL1279L91 (d0-10)
ENS51
TL639L62 (d 0-10)TL319L62 (d10-32)
Atmospheric model
Wave model
Ocean model (d0)
Atmospheric model
Wave model
EDA11
TL399L91
NB: ver>cal resolu>on was increased in 2013: since June EDA and HRES have been using L137 and ENS L91; EDA members also increased to 25.
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 20 © ECMWF
Background error correla>on length scale for long(pmsl) and pmsl
km
Ensembles are now used to es_mate flow dependent stats
The EDA provides 4DV-‐HRES with flow dependent background error co-‐variances. EDA-‐based perturba>ons have been used to generate ENS ini>al perturba>ons since May 2010.
• Un-‐pert obs • EDA var/co-‐var
4DV00
• Sta>c Jb • Perturbed obs • Model err
EDA00 • Un-‐pert obs • EDA var/co-‐var
4DV12
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 21 © ECMWF
ENS re-‐fc suite to es_mate the model-‐climate
51m ENS is run twice a week up to 32d. A 5m ENS is run for the past 20y to es>mate the M-‐climate (re-‐fc suite). ENS fcs have been bias-‐corrected, with bias computed using 500 ENS re-‐fcs [5w*(5m*20y)]. A reference 100m climatological ensemble (CLI) has been defined by 32d consecu>ve analyses (with the same IC as the ENS refc).
20y
51 T639 L62
51 T319 L62
2013
5 5 5 5
5 5 5 5
5 5
…28 6 13 20 27 March …
2012
5 5 5 5
5 5 5 5
5 5
5 5 5 5
5 5 5 5
5 5
2011
5 5 5 5
5 5 5 5
5 5
2010
1993
…..
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 22 © ECMWF
1) Context 2) Ensembles: why? 3) Where is the forecast skill horizon? Why is it there? 4) Conclusions
Outline
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 23 © ECMWF
The predic_ve skill limit: defini_on
The predic>ve skill limit is the >me when the forecast error crosses a certain threshold. As threshold, we have used m-‐2σ, where m is the average climatological error.
m-‐2σ
Forecast
Forecast steps (days)
error
CLI reference
F
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 24 © ECMWF
CLI single fcs
ENS single fcs (control)
2w 17d
<Z500>180km over NH: local instantaneous skill
Results indicate that for local, instantaneous single fc of Z500 over NH is beyond 2-‐weeks. FiSH is @ ̴22 days!
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 25 © ECMWF
Let’s think ensemble and generalise the problem
Ø ENS fcs: bias-‐corrected forecasts are from the ECMWF 51-‐member ENS, the medium-‐range/monthly forecasts (32km up do d10, 64km agerwards)
Ø Verifica_on: ERA-‐I analyses Ø CLI fcs: 100-‐member climatological ensemble defined by ERA-‐I 32-‐d subsequent
analyses Ø Accuracy metric: Con>nuous Ranked Probability Score (CRPS) Ø Skill: CRPS(ENS) vs CRPS(CLI) Ø Cases: 141 (2 per week, for 16m from 2/7/12 to 4/11/13)
ENS +5d
CLI CLI CLI
ENS +10d ENS + … d?
obs obs obs
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 26 © ECMWF
<Z500>180km over NH: local instantaneous skill
ENS probabilis_c fcs
2w 17d
The same conclusion can be reached if we think in probabilis>c terms. FiSH is @ ̴22 days
CLI ensemble
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 27 © ECMWF
Forecast skill depend on the spa_al-‐temporal scale
Large-‐scale, >me-‐average features can be predicted longer ahead than instantaneous, grid-‐point values. Certain (large-‐scale, low-‐frequency) phenomena can be predicted weeks and months ahead.
0.30.4 0.4
0.4
Local, instantaneous wind-‐speed
Weekly-‐mean, regional
temperature anomaly
Monthly-‐mean, con_nental-‐scale rain anomaly
10 100 1000 10000 km 0.1 1 10 100 days
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 28 © ECMWF
MJO and NAO
• Few days: the >me limit up to which local, instantaneous variables can be predicted
• Few weeks: the >me limit up to which large-‐scales (NAO, MJO, ..) can be predicted
• Few months: the >me limit up to which coupled, very large-‐scales (Nino) can be predicted
MJO over tropics
2 weeks
3-‐4 weeks
6 months
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 29 © ECMWF
Forecast skill depend on the spa_al-‐temporal scale
All forecasts represent average values over a space-‐>me volume: even a instantaneous, local values represents an implicit average. Large-‐scale, t-‐average features are more predictable than instantaneous, local values. Unpredictable “noise” can be removed by averaging to isolate the predictable signal. We have applied the same metric to differently averaged (in 4D) forecasts and asked: a) Does FiSH depend on the spa:al-‐temporal average (and on the variable)? b) Does it make sense to talk very generally about a forecast skill limit?
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 30 © ECMWF
Forecast skill depend on the spa_al-‐temporal scale
Consider increasingly coarser fields, defined by temporally averaged and spectrally truncated fields: • Spa>ally: spectrally
truncated from T120 (180km) to T60 (360km), T15, T7, T3
• Temporally: from instantaneous (H0) to 1, 2, 4 and 8 day averages (H24-‐H192)
H0 -‐ T120 (180km)
H0 -‐ T30 (720km)
H0 -‐ T15 (1500km)
H0 -‐ T7 (3000km)
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 31 © ECMWF
Forecast skill depend on the spa_al-‐temporal scale
Consider increasingly coarser fields, defined by temporally averaged and spectrally truncated fields: • Spa>ally: spectrally
truncated from T120 (180km) to T60 (360km), T15, T7, T3
• Temporally: from instantaneous (H0) to 1, 2, 4 and 8 day averages (H24-‐H192)
H0 – T120 (180km)
2d – T120 (180km)
4d – T120 (180km)
8d – T120 (180km)
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 32 © ECMWF
<Z500>180km over NH: instantaneous, <..>48h and <..>96h
CLI ENS
ENS fc
Instantaneous
4-‐day average
8-‐day average
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 33 © ECMWF
<Z500>180km over NH: instantaneous, <..>48h and <..>96h
CLI ENS
ENS fc
Instantaneous 1-‐day average 2-‐day … 4-‐day … 8-‐day … 16-‐day …
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 34 © ECMWF
<Z500>180km over SH: instantaneous, <..>48h and <..>96h
CLI ENS
ENS fc
Instantaneous
4-‐day average
8-‐day average
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 35 © ECMWF
<T850>180km NH, SH & TR: instantaneous, <..>48h and <..>96h
FiSH depends on the variable, the 4D-‐scale (i.e. 4D volume where average is taken) and the area where accuracy is computed.
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 36 © ECMWF
The forecast skill horizon: results based on ECMWF ENS
30
25
20
15
5
0
F (T850T120,H0)
F(T850T120,H24)
F(T850T120,H96)
FiSH depends on the variable, the 4D-‐scale and the area
Fc day
FiSH
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 37 © ECMWF
The forecast skill horizon: results based on ECMWF ENS
30
25
20
15
5
0
FiSH
Fc day
FiSH is shown here for: § Variables: Z500, T850 and T200 § Time-‐averages: 0, 2-‐days and 8
days § Trunca>on: T120 § Areas: NH, SH and TR
Values are well beyond 2 weeks even for instantaneous, local forecasts.
8d (H48) 2d (H24)
Instantaneous (H0)
FiSH depends on the variable, the 4D-‐scale and the area
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 38 © ECMWF
Suppose that we have a good system that can simulate all scales relevant to predict phenomena with a scale (X,T), and ini>alise them properly. The skill of the phenomena depends on the compe>>on between: • Errors propaga>ng from the smaller scales, i.e. noise destroying the signal, and • Predic>ve signal propaga>ng from the wider, longer-‐range scales
Phenomena slave
External forcing
free
wider longer-‐_me
(XS,TS) (X,T) (XL,TL)
(from Hoskins 2012, QJRMS)
Why is FiSH so long? How can we interpret these results?
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 39 © ECMWF
How can we interpret these results?
Errors propagate from the small to the large scales thus reducing the predic_ve skill
Predictable signals propagate from the large scales to the smallest scales
For example, looking at the atmosphere only:
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 40 © ECMWF
-‐ The MJO can affect extra-‐tropical, low-‐frequency phenomena such as blocking -‐ Diurnal tropical convec>on influences organized convec>on and the MJO -‐ The MJO propagates interac>ng with El Nino -‐ El Nino and the MJO are affected by varia>ons in solar radia>on and greenhouse gases -‐ Blocking influences and is influenced by synop>c scales, fronts -‐ …..
Blocking
fronts
Solar radia_on Greenhouse gases
organiz convec
MJO, El Nino
convec
(XS,TS) (X,T) (XL,TL)
Free smaller scales
An example: blocking over the Euro-‐Atlan_c sector
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 41 © ECMWF
CY31R1 (system-‐3) CY36R1 with system-‐3 convec_on CY36R1 (close to System-‐4) CY36R1 but slightly modified entrainment CY36R1 but slightly modified CAPE adjustment _me scale TRMM observa_ons
PDF of daily (24-‐hour accumulated) precipita_on
Progression in tropical precipita>on modelling as a result of convec>on improvements (entrainment / detrainment and closure formula>ons)
Changes in convec_on improves tropical precipita_on
obs
(From Peter Bechtold)
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 42 © ECMWF
Progression in MJO modelling as a result of convec>on improvements (entrainment / detrainment and closure formula>ons).
Cy 31r1 (as in system-‐3) CY36R1 (close to system-‐4)
Beser physics > more realis_c MJO propaga_on
(From Frederic Vitart)
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 43 © ECMWF
(From B. Wang and J.-‐Y. Lee)
ECMWF ECMWF
We have seen that the ECMWF system can predict ENSO SST up to 1 year ahead. Considering the MJO, the ECMWF system is capable to predict it up to about 25 days.
More realis_c MJO > skilful MJO fcs up to 3 weeks
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 44 © ECMWF
The skill of d19-‐25 PR(2mT>Upp3) forecasts is higher if there is an ac>ve MJO in the Ics. (results based on 45 cases, 1989-‐2008).
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1forecast probability
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
obs f
requ
ency
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1forecast probability
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
obs f
requ
ency
MJO in ICs BSS(EU)=0.03 NO MJO in ICs BSS(EU)=-0.09
MJO in ICs BSS(NH)=0.04 NO MJO in ICs BSS(NH)=-0.06
Beser MJO predic_on > higher skill over Europe
(From Frederic Vitart)
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 45 © ECMWF
1) Context 2) Ensembles: why? 3) Where is the forecast skill horizon? Why is it there? 4) Conclusions
Outline
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 46 © ECMWF
Where is the forecast skill horizon?
Lorenz (1969): ‘… one flap of a sea gull’s wing would forever change the future course of the weather .. Such a change would be realized within about 17 days ..’
We showed that there is not a unique defini>on of forecast skill limit, and that the Forecast Skill Horizon, say the FiSH length, depends on the field (scale, variable, region). The forecast skill horizon is well beyond 2 weeks even for local, instantaneous fields, thus confirming results published in literature that certain phenomena (MJO, NAO, blocking, ..) can be predicted beyond 2 weeks using a unifying, coherent framework.
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 47 © ECMWF
• Reduced ini>al errors • More complete models (coupling to land and ocean) • BeEer models (improved moist processes, ..) • New methods (ensembles, ..) • Understanding of sources of predictability • Scale analysis
2 weeks 3 weeks 4 weeks 5 weeks
1970s <(t)>180km
<.;.>180km,48h
<.;.>180km,96h
<.;.>180km,192h
Z500 over NH
FiSH length
Where is the forecast skill horizon?
Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 48 © ECMWF
In other words .. 1970s: results based on atmosphere-‐only models suggested that a sea-‐gull wing could affect the weather anywhere ager ~ 2 weeks 2010s: results based on more accurate, higher resolu>on coupled ocean-‐atmosphere models indicate that the limit is well beyond 2 weeks and that the forecast skill limit has not yet been reached
forget the sea-‐gulls .. think FiSH!!