1
GOES-R AWG Aviation Team: Convective Initiation
June 14, 2011
Presented By: John R. Walker University of Alabama in Huntsville
In Close Collaboration With: John R. Mecikalski, Christopher Jewett, and Lori Schultz
2
Outline
Executive Summary (1 slide) Algorithm Description (4 slides) ADEB and IPR Response Summary (5 slides) Requirements Specification Evolution (1 slide) Validation Strategy (6 slides) Validation Results (3 slides) Summary (1 slide)
3
Executive Summary
The ABI Convective Initiation (CI) algorithm generates one Option 2 product.
Software Version 5 was just delivered in June. The ATBD (100%) is on track to be delivered in late June 2011.
Algorithm performs spectral and temporal differencing tests using several ABI IR channels to monitor cloud growth properties.
Validation Datasets: Use of the MSG 5-minute temporal resolution dataset is used as proxy along with radar data from within the MSG domain when available. Additionally, model simulated ABI brightness temperatures and model-derived radar reflectivities were used for validation datasets. Finally, the current proxy version of the algorithm has been used with GOES data and WSR-88D radar for validation.
Validation studies indicate spec compliance for the CI product.
4
Algorithm Description
5
Convective Initiation Algorithm Description
INPUT for 2 consecutive image times: INPUT for 2 consecutive image times: -ABI Channels 8,10,11,14,15,16-ABI Channels 8,10,11,14,15,16-AWG Cloud Type product-AWG Cloud Type product
(Temporal overlap method)(Temporal overlap method)
Define “Cloud Objects”Define “Cloud Objects”
Track “Cloud Objects”Track “Cloud Objects”
Perform Spectral/Temporal Perform Spectral/Temporal Differencing Tests on Differencing Tests on Tracked Cloud ObjectsTracked Cloud Objects
Determine Null or Positive Determine Null or Positive Forecast of CI for each Forecast of CI for each Tracked Cloud ObjectTracked Cloud Object
(12 tests to monitor cloud top properties …(12 tests to monitor cloud top properties …height, rate of growth, etc..)height, rate of growth, etc..)
7 of 12 tests must be “passed” in order to 7 of 12 tests must be “passed” in order to produce a “Positive” CI forecast. Then, produce a “Positive” CI forecast. Then, CI CI forecasts are OUTPUT.forecasts are OUTPUT.
11
22
33
44
“Cloud Objects” are defined from the Cloud Typing algorithm output from water cloud, assuming these clouds are immature and pre-convective.
Then, tracking is accomplished by using an overlap detection scheme between two times:» Take all defined objects from two successive times and search for temporal
overlap between the objects. Then, assign each overlap and contributing objects a unique Integer ID number. For both ‘Time 1’ and ‘Time 2’, each object with the same overlap will have the same ID number for tracking.
6
Time 1 Time 2 Time 1 + Time 2
Yellow regions are overlap regions
Yellow regions are overlap regions
Convective Initiation Algorithm Description
Channel brightness temperatures are used to perform spectral analyses on updraft regions of the storm that represent growing cumulus.
Use the coldest 25% of 11 μm channel data pixels associated with each object (potential updraft region) and average the brightness temperature of those pixels for all input IR channels to come away with a representative brightness temperature for each spectral channel for each object.
Compute spectral and temporal differencing for each object using the representative average brightness temperatures.
7
Convective Initiation Algorithm Description
Spectral and Temporal Differencing critical values were developed using the information from Mecikalski and Bedka (2006) on current GOES and MSG interest fields by Mecikalski et al. (2010) and further tested by Siewert et al. (2010).
7 out of 12 critical values must be met for a cloud object to be flagged with a “Positive” CI forecast.
8
Siewert et al. (2009)
Interest Field Physical Basis (Mecikalski et al.. 2009)
Critical Value
6.2-10.8 μm Cloud Depth -35oC to -10oC
6.2-7.3 μm Cloud Depth -25oC to 3oC
10.8 μm Cloud Depth/Glaciation
-20oC to 0oC
8.7-10.8 μm Glaciation -10oC to 0oC
Tri-channel Diff Glaciation -10oC to 0oC
15 min Tri-Channel Glaciation Trend >0oC
15 min 12.0-10.8 μm Cloud Depth >0oC
12.0-10.8 μm Cloud Depth -3oC to 0oC
15 min 10.8 μm Cloud Growth < -4oC
15 min 6.2-7.3 μm Cloud Depth Trend >0oC
15 min 6.2-10.8 μm Cloud Depth Trend >3oC
13.4-10.8 μm Cloud Depth -25o to -5oC
Convective Initiation Algorithm Description
9
ADEB Report Response Summary
10
ADEB Report Response Summary
ADEB Report Comment:“The team needs to demonstrate that it meets spec by further validation. In particular, validation outside the area of radar coverage needs to be conducted under a full range of convective cases, including, for example, convection under cirrus. The board specifically recommends using man-in-the-loop analysis.”
Response:We completely agree that more validation needs to be performed on the product output in order to make sure it meets the required specifications under a variety of convective scenarios. We have already added a great deal of validation cases toward meeting this goal by conducting an extensive evaluation/validation of …
11
ADEB Report Response Summary
Response (continued):… the proxy (current GOES) version of this algorithm, using National Weather Service WSR-88D radar data for validation. Furthermore, we have also finished validation of the latest RAMS model-simulated data made available from CIRA. Together, these efforts have added well over 14,000 new validated CI forecasts to our growing list of statistics, which has produced statistical accuracies well above the required 70% specification.Regarding the issue of seeking validation under cirrus-contaminated scenes, the algorithm is not designed to perform processing in these conditions. The spectral input for the algorithm is based on the AWG Cloud Type algorithm output, and we select from that data only the parts of the domain filled with warm water and mixed phase clouds. Because of the …
12
ADEB Report Response Summary
Response (continued):… thin and varying optical depths of most cirrus clouds, the satellite-retrieved spectral signatures of underlying clouds are distorted, hiding the true spectral signals of potentially growing convective clouds. Nevertheless, we are seeking out as diversified cases as possible in order to further validate the algorithm. Currently, our main focus is to acquire more validation data from radar sites under the MSG domain in Europe on convectively active days. So far, this has proven to be a very arduous task, but we are now working on a few lead ins that we have recently encountered and expect to add to our growing set of statistics from this.
13
ADEB Report Response Summary
ADEB Report Specific Comment:“I note that the CI definition as it stands now- 35 dBZ reflectivity in the base scan- is actually more representative of a mature storm that is already raining, rather than developing convection that has a 35 dBZ echo aloft and is newly glaciated. A radar-based CI discriminator would surely be different. In fact, after the NEXRAD dual-pol upgrade, convective initiation might be better defined as the first occurrence of ice phased precipitation indicated by negative Zdr for example. A more advanced GOES-R algorithm for earlier detection of deep convection might be considered for the post-launch science infusion.”
14
ADEB Report Response Summary
Response:Certainly, there is great debate within the community over what metric defines convective initiation, ranging from the first lightning strike, on one end of the spectrum, all the way up to the first detection of a newly forming cumuliform cloud. For our purposes here, we established and agreed early on that we would follow the metric found most prominently in the literature (Roberts and Rutledge, 2003; Mecikalski and Bedka, 2006, etc..), which is the first instance of a ≥35 dBZ radar echo, specifically, at a low elevation, such as 1,000 meters. Furthermore, the most commonly available radar data is that which originates from the base scan, which is difficult enough to obtain from radar sites existing within the MSG domain.
15
RequirementsConvective Initiation
M – Mesoscale C - CONUS
Nam
e
User &
Priority
Geograp
hic
Coverage
(G, H
, C, M
)
Vertical
Resolution
Horizon
tal
Resolution
Mapp
ing
Accu
racy
Measurem
ents
Ran
ge
Measurem
ents
Accu
racy
Produ
ct Refresh
Rate/C
overage Tim
e
Ven
dor Allocated
Grou
nd
Latency
Produ
ct Measurem
ent P
recision
Tem
poral Coverage Q
ualifiers
Produ
ct Exten
t Qu
alifier
Cloud
Cover C
ondition
s Q
ualifier
Produ
ct Statistics Q
ualifier
Convective Initiation
GOES-R
C N/A 2 km 1 km Binary Yes/No detection
70% Probability of Correct Detection
5 min 159 sec 5% Day and night
Quantitative out to at least 65 degrees LZA and qualitative beyond
Clear conditions down to feature of interest associated with threshold accuracy
Over specified Geographic area
Convective Initiation
GOES-R
M N/A 2 km 1 km Binary Yes/No detection
70% Probability of Correct Detection
5 min 159 sec 5% Day and Night
Quantitative out to at least 70 degrees LZA and qualitative beyond
Clear conditions down to feature of interest associated with threshold accuracy
Over specified Geographic area
C - CONUS M - Mesoscale
16
Validation Strategy
17
CI forecasts generated for cloud objects are subjectively compared to base radar reflectivity.
The first instance of a base ≥35 dBZ radar echo is used to validate the forecasts, per the literature (Roberts and Rutledge, 2003). Both “null” and “positive” CI forecasts are validated and used to fill the Dichotomous Forecast Verification contingency table below.
Validation Strategy
Dichotomous Forecast VerificationDichotomous Forecast Verification
Was CI Forecasted? Was CI Forecasted? YESYESDid CI Occur? Did CI Occur? YESYES
HitsHits
Was CI Forecasted? Was CI Forecasted? YESYESDid CI Occur? Did CI Occur? NONO
False AlarmsFalse Alarms
Was CI Forecasted? Was CI Forecasted? NONODid CI Occur? Did CI Occur? YESYES
MissesMisses
Was CI Forecasted? Was CI Forecasted? NONODid CI Occur? Did CI Occur? NONO
Correct NegativesCorrect Negatives
18
Validation Strategy Validation of individual cloud object forecasts starts from the time
each forecast is generated, and extends out to ~90 minutes.
Cloud motion must be subjectively analyzed and compared to potential downstream CI events, as observed from radar data.
Though the validation is performed subjectively, strict criteria are followed to ensure consistency.
Because each set of forecasts is based on IR imagery from independent sets of two input image times, each set of forecasts is completely independent of the previous set of forecasts, even for the same discernable cloud objects from different times. Therefore, different forecasts for the same cloud objects through time can be included in the contingency table statistics.
19
Validation Strategy Positive forecasts of CI upstream of corresponding radar-detected
CI events are considered as “HITS” in the contingency table.
Null forecasts of CI just prior to and upstream of corresponding radar-detected CI events are considered as “MISSES”.
Positive forecasts of CI upstream of non-existent CI events are considered as “FALSE ALARMS”.
Null forecasts of CI upstream of non-existent CI events are considered as “CORRECT NEGATIVES”.
Diagnostic (aka: “0-Lead time”) and Negative Lead Time forecasts are considered as “MISSES”. The first step of validation is to check if a CI forecast is given for convection that has already initiated.
20
Validation Strategy1702 UTC
1700 UTC 1717 UTC 1758 UTC
Validation example Validation example from AWG CI proxy from AWG CI proxy algorithm:algorithm:
June 6, 2011June 6, 2011North Florida/North Florida/Southern GeorgiaSouthern Georgia
Red= Positive CI FcstRed= Positive CI FcstBlue= Null CI FcstBlue= Null CI Fcst
21
Validation Strategy1702 UTC 1715 UTC
1700 UTC 1717 UTC 1758 UTC
Validation example Validation example from AWG CI proxy from AWG CI proxy algorithm:algorithm:
June 6, 2011June 6, 2011North Florida/North Florida/Southern GeorgiaSouthern Georgia
Red= Positive CI FcstRed= Positive CI FcstBlue= Null CI FcstBlue= Null CI Fcst
22
Validation Strategy1702 UTC 1715 UTC 1732 UTC
1700 UTC 1717 UTC 1758 UTC
Validation example Validation example from AWG CI proxy from AWG CI proxy algorithm:algorithm:
June 6, 2011June 6, 2011North Florida/North Florida/Southern GeorgiaSouthern Georgia
Red= Positive CI FcstRed= Positive CI FcstBlue= Null CI FcstBlue= Null CI Fcst
23
Validation Results
Three unique data set types have been used to run and to validate the AWG CI algorithm:
1) MSG SEVIRI data over Europe with 5-minute temporal resolution and available radar data within the domain.
2) RAMS Model simulated ABI radiances with 5-minute temporal resolution and simulated radar reflectivities (courtesy of CIRA).
3) Current GOES-East data over the CONUS with 15-30 minute temporal resolution and available WSR-88D radar data within the domain (CI forecasts generated by the AWG CI proxy algorithm, which is limited by poor spectral, temporal, and spatial resolutions).
24
Validation Results
Total CI forecasts validated: 14,671
Total validated: 184Accuracy: 80.4%POD: 87.0%FAR: 15.8%POFD: 32.8%
25
Validation Results
HITSHITS427427
False AlarmsFalse Alarms10441044
MISSESMISSES1818
Correct NegativesCorrect Negatives30553055
RAMS Simulated ABI Dataset
MSG SEVIRI Dataset
HitsHits255255
False AlarmsFalse Alarms308308
MISSESMISSES9999
Correct NegativesCorrect Negatives92819281
GOES CI Proxy Dataset
HitsHits107107
False AlarmsFalse Alarms2020
MissesMisses1616
Correct NegativesCorrect Negatives4141
Total validated: 4,544Accuracy: 76.6%POD: 96.0%FAR: 71.0%POFD: 25.5%
Total validated: 9,943Accuracy: 95.9%POD: 72.0%FAR: 54.7%POFD: 3.0%
Validation efforts from all 3 data sets have produced “Statistical Accuracies” above the required 70% specification.
However, more validation studies still need to be performed using our closest available ABI proxy for input data, the MSG SEVIRI data set, since it is neither model-simulated nor limited by antiquated spectral, spatial, and temporal resolutions.
26
Validation Results
27
The Convective Initiation algorithm provides a unique multi-spectral and temporal differencing approach to forecasting the locations of near-term future convection. It takes advantage of the many IR spectral channels available with the ABI.
This product meets all required specifications.
More validation is planned, using data from the MSG SEVIRI instrument.
Version 5 was just delivered, and the 100% ATBD is slated for delivery soon.
Summary