Factors affec+ng weather forecast skill in the Arc+c
Gordon McBean and Ken Porter University of Western Ontario
STAR Final Workshop Winnipeg, June 14th 2010
Factors affecting weather forecast skill in the Arctic
• Observational information on the “state” of the weather
• Variability of the weather • Understanding of the weather • And ability to “see ahead” – forecast • Knowledge of the individual forecaster • Location of the forecaster
Inuvik
Iqaluit Rankin Inlet
Cambridge Bay
Resolute
Calgary Winnipeg Regina
The Pas Edmonton
Fort McMurray
Site Lat Long Radar Radiosonde
Edmonton 53.32 113.58 Yes Yes
Calgary 51.11 114.02 Yes
Fort McMurray 56.65 111.22 Yes
Regina 50.43 104.67 Yes
Winnipeg 49.92 97.23 Yes
Saskatoon 52.17 106.72 Yes
The Pas 53.97 101.1 Yes
Site Lat Long Radar Radiosonde
Iqaluit 63.75 68.55 Yes
Inuvik 68.3 133.48 Yes
Cambridge Bay 69.12 105.14 Yes
Resolute 74.72 94.99 Yes
Rankin Inlet 62.82 92.12
Observational information
0
10
20
30
40
50
60
0 50 100 150 200 250
Wea
ther
Sta
tions
reco
gniz
ed b
y En
viro
nmen
t Can
ada
Radial Distance (km)
Spatial Proximity of Weather Stations Recognized by Environment Canada (Prairies vs. Arctic)
Arctic sites AVG
Prairie sites AVG
Observational information
-18.0
-13.0
-8.0
-3.0
2.0
7.0
1940 1950 1960 1970 1980 1990 2000 2010
Mea
n A
nnua
l Air
Tem
pera
ture
(°C
)
Time (Year)
Change in Mean Annual Temperature (Prairies vs. Arctic)
Arctic sites AVG
Prairie sites AVG
Arctic 11 year running mean
Prairie 11 year running mean
Variability of the weather
-5.0
-4.0
-3.0
-2.0
-1.0
0.0
1.0
2.0
3.0
4.0
5.0
1940 1950 1960 1970 1980 1990 2000 2010
Air
Tem
pera
ture
(°C
)
Time (Year)
Departure from 1951-1980 Mean Annual Air Temperature (Prairies vs. Arctic)
Arctic Departure from 1951-1980 mean
Prairie Departure from 1951-1980 mean
Arctic Departure from 1951-1980: 11 year running mean
Prairie Departure from 1951-1980: 11 year running mean
0.0
100.0
200.0
300.0
400.0
500.0
600.0
1940 1950 1960 1970 1980 1990 2000 2010
Tot
al A
nnua
l Pre
cipi
tatio
n (m
m)
Time (Year)
Change in Total Annual Precipitation (Prairies vs. Arctic)
Arctic sites AVG
Prairie sites AVG
Arctic 11 year running mean
Prairie 11 year running mean
-100.0
-50.0
0.0
50.0
100.0
150.0
200.0
1940 1950 1960 1970 1980 1990 2000 2010
Pre
cipi
tatio
n (m
m)
Time (Year)
Departure from 1951-1980 Mean Total Annual Precipitation (Prairies vs. Arctic)
Arctic Departure from 1951-1980 mean
Prairie Departure from 1951-1980 mean
Arctic Departure from 1951-1980: 11 year running mean
Prairie Departure from 1951-1980: 11 year running mean
Time Period Temperature Dew Point Temperature
RH Surface Pressure
01/01/1958-08/31/1961 13.97 °C 13.54 °C 10.88 % 1.05 mb 01/01/1978-08/31/1981 13.62 °C 13.89 °C 12.13 % 1.08 mb 01/01/1998-8/31/2001 14.01 °C 14.70 °C 13.23 % 1.06 mb
RMS values for hourly data from Iqaluit for select 7me periods
Is the weather becoming variable?
Edmonton Prairies
Arctic
Forecasts for period November 1998 to June 2004 Alternate Limits/ Visual Flight Rules (VFR) = for air flight planning
ALT_above = above alternate limits (dependent on station)
ALT_below = below alternate limits (dependent on station)
VFR_above = above VFR limits (a ceiling of 1000 ft. or visibility of 3 statute miles)
POD- Probability of Detection: What fraction of the observed "yes" events were correctly forecast? (Perfect score = 1)
POD = hits / (hits + misses)
FAR- False Alarm Ratio: What fraction of the predicted "yes" events actually did not occur (i.e., were false alarms)? (Perfect score = 0)
FAR = false alarms / (hits + false alarms)
CSI- Critical Success Index: How well did the forecast "yes" events correspond to the observed "yes" events? (Perfect score = 1)
CSI = hits / (hits + misses + false alarms)
HSS – Heidke Skill Score: What was the accuracy of the forecast relative to that of random chance? (Perfect score = 1) HSS = (C-E)/(T-E)
C = correct number of forecasts = hits + correct negatives T = total number of forecasts = hits + misses + false alarms + correct negatives E = expected number of correct forecasts by chance = ((hits + false alarms)(hits +misses)+(false alarms +
correct negatives)(misses + correct negatives)) / T
Edmonton Forecasts
Forecasts
Prairies
Arctic
ALT_above
VFR_above ALT_below
ALT_above ALT_below VFR_above
Nov/98 – Jun/04
Nov/98 – Jun/04
Environment Canada Data 1948 - 2006
POD – Probability of Detection FAR – False Alarm Ratio
Analysis and Comparison
CSI – Critical Success Index (Threat Score) HSS – Heidke Skill Score
Hits Misses Observed Yes
False Alarms
Correct Negatives
Observed No
Forecast Yes
Forecast No
Total
Forecasts
Obs
erve
d Yes No
Yes
No
Total
Tota
l
-40
-30
-20
-10
0
10
20
30
Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08
Prairie Mean Temperature Arctic Mean Temperature
0
20
40
60
80
100
120
Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08
Prairie Mean Total Precipitation Arctic Mean Total Precipitation
Mea
n M
onth
ly T
empe
ratu
re (°
C)
Mea
n M
onth
ly P
reci
pita
tion
(mm
) Mean Monthly Temperatures and Precipitation Levels During Skill Score Period of Nov/98 – Jun/04
0.84
0.86
0.88
0.9
0.92
0.94
0.96
0.98
1
Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08
Prairie POD Arctic POD
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08
Prairie FAR Arctic FAR
Skill
Sco
re (%
) Sk
ill S
core
(%)
Skill Score Data for ALT_(Above Alternate) Limits
Skill
Sco
re (%
) Sk
ill S
core
(%)
0.7
0.75
0.8
0.85
0.9
0.95
1
Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08
Prairie CSI
Arctic CSI
Skill Score Data for ALT_(Above Alternate) Limits
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08
Prairie HSS Arctic HSS
For ALT_Above
Prairie stations were on average • 4.2% higher in Probability of Detection, • 6.0% lower in False Alarm Ratio, • 9% higher in Critical Success Index • 2.2% lower in Heidke Skill Score.
VFR_Above ALT_Above FAR POD CSI HSS FAR POD CSI HSS
Prairie AVG 0.070 0.945 0.883 0.461 0.042 0.967 0.930 0.377 Arctic AVG 0.146 0.886 0.770 0.405 0.102 0.928 0.840 0.399
Seasonal Skill Scores Between 1998 – 2004 (Prairies vs. Arctic) Sk
ill S
core
(%)
0
0.02
0.04
0.06
0.08
0.1
0.12
Spring Summer Fall Winter
Prairie ALT_above_FAR
Arctic ALT_above_FAR
0.8
0.82
0.84
0.86
0.88
0.9
0.92
0.94
0.96
0.98
Spring Summer Fall Winter
Arctic ALT_above_CSI
Prarie ALT_above_CSI
0.3
0.32
0.34
0.36
0.38
0.4
0.42
0.44
0.46
0.48
Spring Summer Fall Winter
Arctic ALT_above_HSS
Prarie ALT_above_HSS
Skill
Sco
re (%
)
Seasonal Skill Scores Between 1998 – 2004 (Prairies vs. Arctic)
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08
Prairie FAR Arctic FAR
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08
Prairie POD Arctic POD
Skill
Sco
re (%
) Sk
ill S
core
(%)
Skill Score Data for ALT_(Below Alternate) Limits
ALT_Below
FAR POD CSI HSS
Prairie AVG 0.539 0.459 0.261 0.377
Arctic AVG 0.463 0.443 0.323 0.399 For ALT_below, Prairie stations were on average • 1.5% higher in Probability of Detection • 7.6% higher in False Alarm Ratio • 6.2% lower in Critical Success Index
FAR is higher for below alternate conditions because it is much safer to forecast below and actually be above as opposed to forecast above and be below; POD and CSI are lower for below alternate conditions because like FAR it is much safer to forecast below and have a higher miss rate for conditions that are above). By deriving accuracy relative to random chance for HSS, ALT above and below will have the same skill score. This means once again Prairie stations had a 2.2% lower HSS.
0.3
0.32
0.34
0.36
0.38
0.4
0.42
0.44
0.46
0.48
0.5
Spring Summer Fall Winter
Prairie ALT_below_POD
Arctic ALT_below_POD
0.3
0.35
0.4
0.45
0.5
0.55
0.6
0.65
0.7
Spring Summer Fall Winter
Prairie ALT_below_FAR
Arctic ALT_below_FAR
Seasonal Skill Scores Between 1998 – 2004 (Prairies vs. Arctic) Sk
ill S
core
(%)
Forecasts for below alternative flight limits - Probability of Detection From 1998-2004, there appears to be an improvement for Edmonton forecasts (positive linear slope = 2E-05) and a slight decline in Iqaluit forecasts (negative linear slope = -2E-06).
Forecasts for below alternative flight limits - Probability of Detection On a seasonal basis, Edmonton forecasts are improving in the spring and summer where as Iqaluit forecasts are improving in the spring and fall. Edmonton is experiencing a decline in forecasting in the winter and fall where as Iqaluit is experiencing a decline in the summer and winter.
0
0.05
0.1
0.15
0.2
0.25
Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08
Prairie FAR Arctic FAR
0.78
0.82
0.86
0.9
0.94
0.98
Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08
Prairie POD Arctic POD
Skill
Sco
re (%
) Sk
ill S
core
(%)
Skill Score Data for VFR_(Above) Limits
VFR_Above ALT_Above FAR POD CSI HSS FAR POD CSI HSS
Prairie AVG 0.070 0.945 0.883 0.461 0.042 0.967 0.930 0.377 Arctic AVG 0.146 0.886 0.770 0.405 0.102 0.928 0.840 0.399
For VFR_Above
Prairie stations were on average • 5.9% higher in Probability of Detection, • 7.7% lower in False Alarm Ratio, • 11.3% higher in Critical Success Index • 5.6% higher in Heidke Skill Score.
0.84
0.86
0.88
0.9
0.92
0.94
0.96
0.98
Spring Summer Fall Winter
Prairie VFR_above_POD
Arctic VFR_above_POD
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
Spring Summer Fall Winter
Prairie VFR_above_FAR
Arctic VFR_above_FAR
Seasonal Skill Scores Between 1998 – 2004 (Prairies vs. Arctic) Sk
ill S
core
(%)
0
10
20
30
40
50
60
0 50 100 150 200 250
Wea
ther
Sta
tions
reco
gniz
ed b
y En
viro
nmen
t Can
ada
Radial Distance (km)
Spatial Proximity of Weather Stations Recognized by Environment Canada (Prairies vs. Arctic)
Arctic sites AVG
Prairie sites AVG
VFR_Above ALT_Above
FAR POD CSI HSS FAR POD CSI HSS
Prairie AVG 0.070 0.945 0.883 0.461 0.042 0.967 0.930 0.377
Arctic AVG 0.146 0.886 0.770 0.405 0.102 0.928 0.840 0.399
ALT_Below
FAR POD CSI HSS
Prairie AVG 0.539 0.459 0.261 0.377
Arctic AVG 0.463 0.443 0.323 0.399
Number of Recognized Environment Canada Surface Weather Sta+ons Within 200 km
Skill Score (%
) Skill Score (%
)
Skill depending on number
of stations
ALT_Below
FAR POD CSI HSS
Prairie AVG 0.539 0.459 0.261 0.377
Arctic AVG 0.463 0.443 0.323 0.399
Comparison VFR_Above ALT_Above
FAR POD CSI HSS FAR POD CSI HSS
Prairie AVG 0.070 0.945 0.883 0.461 0.042 0.967 0.930 0.377
Arctic AVG 0.146 0.886 0.770 0.405 0.102 0.928 0.840 0.399
Factors affecting weather forecast skill in the Arctic
• Observational information on the “state” of the weather
• Variability of the weather • Understanding of the weather • And ability to “see ahead” – forecast • Knowledge of the individual forecaster • Location of the forecaster
Thank you for listening.
Weather Forecasting Skill in the Arctic
BLACK MOUNTAIN • TOUR DATES
06/24/10 Calgary, AB - Republikw/ Lord Beginner
06/25/10 Edmonton, AB - Starlite Roomw/ Lord Beginner
06/26/10 Saskatoon, SK - Amigo'sw/ Lord Beginner
06/27/10 Regina, SK - The Exchangew/ Lord Beginner, Orbital Express
06/28/10 Winnipeg, MB - The Garrick Centrew/ Lord Beginner, Sleepy Sun
07/03/10 London, ON - Call the Office w/ One Hundred Dollars
Europe
• 07/23/10 Toronto, ONT - Horseshoe Tavernw/ Quest for Fire