Date post: | 12-Jan-2016 |
Category: |
Documents |
Upload: | loren-thompson |
View: | 213 times |
Download: | 0 times |
National Weather Service
Model Flip-Flops and Model Flip-Flops and Forecast OpportunitiesForecast Opportunities
Bernard N. MeisnerScientific Services DivisionNWS Southern RegionFort Worth, Texas
National Weather Service
The views expressed herein are those of the author and do not necessarily reflect the position of the National Weather Service.
National Weather Service
Model Flip FlopsModel Flip Flops
Primary Motivation: Primary Motivation:
• Introduction of 1200 UTC MEXMOS guidance caused Introduction of 1200 UTC MEXMOS guidance caused concerns among some forecasters. concerns among some forecasters.
• Perception was that guidance from 1200 UTC model Perception was that guidance from 1200 UTC model runs would be substantially different from theruns would be substantially different from the0000 UTC guidance.0000 UTC guidance.(Flips and Flops)(Flips and Flops)
Secondary Motivation: Secondary Motivation:
• Given substantial improvements in MOS guidance, are Given substantial improvements in MOS guidance, are there still opportunities for forecasters to add value, there still opportunities for forecasters to add value, particularly beyond Days 1 and 2?particularly beyond Days 1 and 2?(Forecast Opportunties)(Forecast Opportunties)
National Weather Service
Model Flips and FlopsModel Flips and FlopsHow to define them?How to define them?
Current model runlies outside envelopeof previous ensemble.
Run-to-run changein MOS max/mintemperatures >10oF.
Run-to-run change in MOSmax/min temperatures exceedsmonthly Mean Absolute Error.
National Weather Service
VerificationVerification
108 Hr108 HrForecastForecast
102 Hr102 HrForecastForecast
96 Hr96 HrForecastForecast
National Weather Service
Model Flip FlopsModel Flip Flops
Data Used:Data Used:MEXMOS Guidance for DFW and PIT Airports.MEXMOS Guidance for DFW and PIT Airports.(Have also looked at a few other sites in NWS Southern Region.)(Have also looked at a few other sites in NWS Southern Region.)
Flip (and Flop) Criterion:Flip (and Flop) Criterion:
Run-to-run change in Max/Min temperature guidance Run-to-run change in Max/Min temperature guidance greater than 10greater than 10ooF.F.
National Weather Service
Flip-FlopSpreadsheet
Model“Flips”
Run-to-run changeexceeds this criterion
All these flipswere in the
proper direction
National Weather Service
Flip-FlopSpreadsheet
Model“Flip/Flop”
Run to run changeexceeds this criterionfor two consecutive
model runs
This flop was inthe wrong direction.
National Weather Service
No discernable differencebetween 0000 and 1200 runs
00 0000 00
00
0000
00
12 12 12
1212
12
12
National Weather Service
October 2005 November 2005
December 2005
Mean Absolute Guidance Errors - KDFWMean Absolute Guidance Errors - KDFW
January 2006
National Weather Service
DFW Max/Min TemperatureDFW Max/Min Temperature1010ooF ThresholdF Threshold
MonthMonth FlipsFlips
Number Number CorrectCorrect
Flip – Flip – FlopsFlops
Number Number CorrectCorrect
Oct 2005Oct 2005 33 33 -- --
NovNov 77 33 -- --
DecDec 99 44 77 44
Jan 2006Jan 2006 88 55 -- --
FebFeb 55 55 11 --
MarMar 1010 55 33 22
AprApr 22 00 -- --
TOTALTOTAL 4242 25 (60%)25 (60%) 1111 6 (55%)6 (55%)
National Weather Service
Model Flips…and FlopsModel Flips…and Flops
Summary:Summary:
• No substantial difference between 0000 and 1200 No substantial difference between 0000 and 1200 UTC model guidance.UTC model guidance.
• Perception of systematic run-to-run changes is not valid.Perception of systematic run-to-run changes is not valid.
• Flips (and flops) occur during the cool season.Flips (and flops) occur during the cool season.
• Typically occur for Typically occur for just one verification timejust one verification time..
• Model flips are rare; flip-flops are very rare.Model flips are rare; flip-flops are very rare.
• 53 Flips; 11 Flops (out of 400+ model runs/7000+ forecasts)53 Flips; 11 Flops (out of 400+ model runs/7000+ forecasts)
• Flips are most common for Days 4-6. Flips are most common for Days 4-6.
• Flip-flops are most common for Days 5-6.Flip-flops are most common for Days 5-6.
National Weather Service
Forecast OpportunitiesForecast Opportunities
National Weather Service
National Weather Service
Forecast OpportunitiesForecast Opportunities
Data Used:Data Used:MEXMOS Guidance for DFW and PIT Airports.MEXMOS Guidance for DFW and PIT Airports.(Have also looked at a few other sites in NWS Southern Region.)(Have also looked at a few other sites in NWS Southern Region.)
Forecast Opportunity Criterion:Forecast Opportunity Criterion:Max/Min temperature guidance error greater than 10Max/Min temperature guidance error greater than 10ooF.F.
Caveat:Caveat:Smaller errors at certain thresholds can be significant!Smaller errors at certain thresholds can be significant!
2828ooF vs 33F vs 33ooF; 99F; 99ooF vs 104F vs 104ooFF
National Weather Service
ForecastOpportunitiesSpreadsheet
Observed temperaturecells turn pink whenthey exceed climate
normal by this amount
Performance of National Weather Service Forecasts Compared to Operational, Consensus, and Weighted Model Output Statistics by Jeffrey A. Baars and Clifford F. Mass
National Weather Service
ForecastOpportunitiesSpreadsheet
These cells turn brownwhen day-to-day change
in observed max/minexceeds this amount
National Weather Service
ForecastOpportunitiesSpreadsheet
These cells turn bluewhen guidance errorexceeds this amount
National Weather Service
ForecastOpportunitiesSpreadsheet
Forecast Opportunitiesoccur for calendar days
with large departuresfrom climatology.
National Weather Service
Percent of Forecasts:14%
October 20058%
November 20059%
December 200514%
Forecast Opportunities - KDFWForecast Opportunities - KDFW
Threshold 10oF
January 200613%
October 200532%
November 200534%
December 200543%
Threshold 5oF
January 200635%
Forecast Opportunities - KDFWForecast Opportunities - KDFW
October 20058%
November 20059%
December 200514%
Threshold 10oF
January 200613%
Forecast Opportunities - KDFWForecast Opportunities - KDFW
February 200614%
March 200616%
April 20066%
Threshold 10oF
May 20064%
Forecast Opportunities - KDFWForecast Opportunities - KDFW
National Weather Service
June 20060%
July 20060%
Threshold 10oF
September 20064%
August 20061%
Forecast Opportunities - KDFWForecast Opportunities - KDFW
National Weather Service
Forecast OpportunitiesForecast Opportunities
Forecast Opportunities:Forecast Opportunities:• Typically occur for Typically occur for calendar dayscalendar days and can be and can be
consistent from one run to the next.consistent from one run to the next.
• Are most common for Days 3-7.Are most common for Days 3-7.
• Frequently occur for days on which the observed Frequently occur for days on which the observed max/min temperature departs substantially from the max/min temperature departs substantially from the climatological norm.climatological norm.• MEXMOS guidance typically underestimates the observed MEXMOS guidance typically underestimates the observed
departure from normal.departure from normal.
• Rarely occur for days when the temperature change Rarely occur for days when the temperature change from the previous day is large.from the previous day is large.• GFS seems to handle these events well.GFS seems to handle these events well.
National Weather Service
Future WorkFuture Work
Flips and Flops:Flips and Flops:• Can one formulate an objective definition of a model Can one formulate an objective definition of a model
flip (and flop)?flip (and flop)?
• Is it possible to anticipate model flips?Is it possible to anticipate model flips?
• When a flip occurs, can we determine whetherWhen a flip occurs, can we determine whetherthe flip is in the correct direction?the flip is in the correct direction?
Forecast Opportunities:Forecast Opportunities:• How can we identify Forecast Opportunities?How can we identify Forecast Opportunities?