V14.2 12 Jan 2010V14.2 12 Jan 2010
11
SPSRB Project Plan “NPP Microwave Sounder-based Tropical Cyclone
Products (NTCP)”
IPT Lead: Mark DeMaria, STAR/NESDISIPT Backup Lead: Limin Zhao, OSPO/NESDIS
August 8-9, 2012
V14.2 12 Jan 2010V14.2 12 Jan 201022
Outline
• Integrated Product Team (IPT)• Requirements• Capabilities Assessment• IT System Architecture• Plan of Operations• Project Milestones• Funding • Spending Plan and Tasks
V14.2 12 Jan 2010V14.2 12 Jan 201033
Integrated Product Team (IPT)
• IPT Lead: Mark DeMaria (StAR)• IPT Backup Lead: Limin Zhao (OSPO)• NESDIS Team:
─ STAR: Mark DeMaria, John Knaff─ OSPO: Limin Zhao, Liqun Ma (TC products PAL)─ OSD: Tom Schott, Gary Mineart─ NDE: Dylan Powell─ Data Center: Phil Jones─ Others: Andrea Schumacher (CIRA), Jiande Wang (SSAI), Hazari
Syed (SSAI)
• User Team─ Lead: Michael Brennan and Jack Beven (NHC)
• Oversight Panel (OP) lead: ICAPOP• Other OPs involved: Soundings, Precipitation
V14.2 12 Jan 2010V14.2 12 Jan 201044
Requirement(s)
• SPSRB User Request #1009-0017 submitted by NHC─ Requesting the current AMSU-based TC intensity and structure estimates
and mean layer winds be generalized from ATMS during the NPP and JPSS era to support the continuity of operation.
• User community─ National Hurricane Center (NHC) and Central Pacific Hurricane Center
(CPHC)─ Joint Typhoon Warning Center (JTWC) and Dept. of Defense─ NWS Offices
• Benefit to user─ Continuity of TC products that provide guidance to NHC forecasters
• NOAA Mission Goal supported:─ Weather and Water─ Commerce and Transportation
• Mission priority: Optimal
V14.3 11 Arp 20115
Capabilities Assessment
Current Operational Capabilities
Requested Capabilities Proposed Operational Capabilities
Satellite Source (s) NOAA polar-orbiting satellites (AMSU)
NPP and JPSS (ATMS) NPP (ATMS)
Product Name AMSU-based TC products ATMS-based TC products ATMS-based TC products
Accuracy Intensity MAE of 13.5 kt / 9 hPa. Wind radii MAE of 32, 18, 10 nmi for 34-, 50-, 64-kt radii, respectively.
Same or better than current AMSU-based products.
Same as requested
Latency 3 hours after synoptic time
3 hours after synoptic time Same as requested
Timeliness New product generated every 6 hours, updated every 2 hours
New product generated every 6 hours, updated every 2 hours
Same as requested
Coverage Global N. Atlantic and N.E. Pacific tropical cyclone basins
Global
Resolution Point values for intensity and radii, 0.2 lat/lon grid for balanced winds
Point values for intensity and radii, 0.2 lat/lon grid for balanced winds
Same as requested
Other attributes
V14.2 12 Jan 2010V14.2 12 Jan 201066
Capabilities Assessment (Continued)
• Current Operational Shortfalls─ AMSU-based TC intensity, radii and balanced wind products used
by NHC forecasters rely on AMSU soundings on board NOAA POES and Metop satellites. NOAA-19 was the last POES pm orbiting satellite and capabilities from this orbit have a finite lifetime.
• Define how product development addresses shortfalls─ Generalizing the current algorithms to utilize the temperature profile
retrievals from next generation polar orbiting satellite microwave instruments (i.e., NPP ATMS) will ensure data and operation continuity.
• Impact if project is not funded─ Disruptions to AMSU sounding data will result in disruptions of
these TC guidance products used by NHC and JTWC. The improved capabilities ATMS will not be fully utilized by NHC and JTWC operations.
V14.2 12 Jan 2010V14.2 12 Jan 201077
Algorithm Status and Approach (1/2)
• Define algorithm maturity─ NOAA-15/16/17 AMSU-based Intensity and wind radii algorithms have been running
operationally at NCEP since 2003. Algorithm and validation details are provided in: Demuth, J., M. DeMaria, and J.A. Knaff, 2006: Improvement of Advanced Microwave Sounding Unit
Tropical Cyclone Intensity and Size Estimation Algorithms, Journal of Applied Meteorology and Climatology, 45, 1573-1581.
Demuth, J., M. DeMaria, J.A. Knaff and T. H. Vonder Haar, 2004: Validation of an advanced microwave sounder unit (AMSU) tropical cyclone intensity and size estimation algorithm, J. App. Met., 43, 282-296.
─ NOAA-15/16/17 AMSU-based balanced wind algorithms have been running operationally since 2005. Algorithm and validation details are provided in:
Bessho, K., M. DeMaria, and J.A. Knaff, 2006: Tropical cyclone wind retrievals from the Advanced Microwave Sounder Unit: Application to Surface Wind Analysis. J. Appl. Meteor. and Clim., 45,399-415.
─ The algorithm will be updated to use the operational MIRS sounding products instead of the Goldberg/Barnett statistical retrievals. The evaluation of the change is undergoing through a CIRA cal/val project, which should be completed before this project starts. Using the MIRS sounding retrievals will eliminate duplication of effort and improve product reliability
V14.2 12 Jan 2010V14.2 12 Jan 201088
Algorithm Status and Approach (2/2)
• Overview of technical approach of the algorithm and its implementation─ Product will be implemented at NESDIS as part of the MIRS processing to take advantage of
existing polar satellite data flows and 24 x 7 support of satellite retrievals.─ Ancillary data: official TC positions, and 100-mb heights from NCEP GFS.─ Algorithms use hydrostatic balance and statistical analysis to estimate intensity and wind radii.
Mean-layer winds are obtained by solving equations for non-linear balance.─ Concept of operations: The ATMS TC products will utilize the MIRS ATMS products and
implemented as a Level-3 sub-system loosely attached to the operational MIRS products system framework to take advantage of existing polar satellite data flows and 24 x 7 operational monitoring support that have established at NESDIS.
• Briefly describe the validation concept─ ATMS-derived products will be compared to AMSU-derived products. Hurricane flight
reconnaissance intensities and wind radii will be used as ground-truth for validation.─ Performance on ATMS is expected to exceed current AMSU products due to improved horizontal
resolution
9999
IT System Architectures for NPP TC Products – Within NDE (1/2)
Input Data Includes: MIRS ATMS temperature profile retrievals (NDE), ATCF TC positions and GFS analyses (both from outside NDE, brought in via DDS)Red denotes connections that need to be made.
Input Data Includes: MIRS ATMS temperature profile retrievals (NDE), ATCF TC positions and GFS analyses (both from outside NDE, brought in via DDS)Red denotes connections that need to be made.
V2.1 Jun 2010V2.1 Jun 2010
ESPCDDS
ESPCDDS
NDE/ESPC IT EnvironmentNDE/ESPC IT EnvironmentNDE/ESPC IT EnvironmentNDE/ESPC IT Environment
NDE Linux“Production ”
NDE Linux“Production ”
NDE Linux“System Test” NDE Linux“System Test”
NPP ATMS TC productsNPP ATMS TC products
NDE Linux “Development”
NDE Linux “Development”
GFS analyses, global TC positions
GFS analyses, global TC positions
NDE AIX “Production ” NDE AIX
“Production ”
MIRS temperature retrievals
MIRS temperature retrievals
TC“CIRA Linux Server”
TC“CIRA Linux Server”
R2O
Outside of NDE
Environment – See next
slide
Outside of NDE
Environment – See next
slide
10101010
IT System Architecture for NPP TC Products – Outside of NDE (2/2)
Input data that must be sent to NDE via DDS: TC positions (a-decks and b-decks) and GFS analysesRed denotes connections that need to be made.Input data that must be sent to NDE via DDS: TC positions (a-decks and b-decks) and GFS analysesRed denotes connections that need to be made.
V2.1 Jun 2010V2.1 Jun 2010
NCEP NCEP
ESPCESPC
QCMonitoring
QCMonitoring
CLASS/NCDCCLASS/NCDC
ArchiveArchiveArchiveArchive
JTWC JTWC
ESPCDDS
ESPCDDSNPP ATMS TC
productsNPP ATMS TC products
GFS analyses, TC positions
GFS analyses, TC positions
NHC NHC
ESPCESPC
OPSO web/ftpOPSO web/ftp
NPP ATMS TC productsNPP ATMS TC products
NPP ATMS TC products
NPP ATMS TC products
NP
P A
TM
S
TC
pro
du
cts
NP
P A
TM
S
TC
pro
du
cts
TC positions
TC positions
TC positionsTC positions
TC positions and G
FS analyses
TC positions and G
FS analyses
NDE Environment
– See previous
slide
NDE Environment
– See previous
slide
ftp pullftp pull
NP
P A
TM
S T
C
prod
ucts
NP
P A
TM
S T
C
prod
ucts
NPP ATMS TC
products
NPP ATMS TC
productsftp pull
ftp pull
V16.0 July 201011
Research and Operational System Environments
Project Name: NPP Microwave Sounder-based Tropical Cyclone Products (NTCP)
IT Item Research Operations
Agency STAR NDE
Platform(s) and need datesDell Precision 690 with 4 GB RAM (already in
place)NDE production Linux machine (x86_64, 24 CPUs @
2.67 GHz and 36 GB of RAM)
Operating Systems Linux RedHat v6 Linux RedHat v6
Programming languages/compilers *GNU Fortran 90 Compiler GNU Fortran Compilers (v4.1.2)
Scripting languages Bash, Perl, Python (v2.6 or v2.7) Bash, Perl, Python (v2.6 or v2.7)
Graphical/Imaging programs, COTS S/W, other tools, libraries, etc
GrADS, gnuplot, ImageMagick GrADS, gnuplot, ImageMagick
Helpdesk Monitoring Tool (standardized tool or customized tool?)
None. NDE products monitoring tool
Other platforms needed for monitoring/imaging/graphics (specify platform & operating system)
None. Dell Linux boxes with Red Hat for image generation, products QC and web monitoring
tools.
Other (tools, shareware, libraries, critical non-static ancillary data, etc)
Java, JavaScripts, wgrib2
Python Libraries: Numpy, Scipy, Matplotlib, Basemap, netCDF4
Fortran 90 Libraries: zlib (1.2.7), hdf (1.8.9), netcdf (4.1.3)
Ancillary: ATCF TC info (adeck; current,past,forecast positions & intensities),
GFS analyses (GRIB2)
Java, JavaScripts, wgrib2
Python Libraries: Numpy, Scipy, Matplotlib, Basemap, netCDF4
Fortran 90 Libraries: zlib (1.2.7), hdf (1.8.9), netcdf (4.1.3)
Ancillary: ATCF TC info (adeck; current,past,forecast positions & intensities),
GFS analyses (GRIB2)
Red font indicates new IT resource requirements for this projectBlue font indicates existing IT resources will be used for this project
V16.0 July 2010
Operational Agency System Requirements
Project Name: NPP Microwave Sounder-based Tropical Cyclone Products (NTCP)
Back-up Operations
IT Item Development Operations On-Site Off-Site
Agency NDE NDE ESPC
Platform(s) and need dates (include secondary platforms for monitoring, imagery or graphics, if necessary)
NDE Linux NDE Linux Linux - TBD
Storage required on systems
50 GB on dev;
50 GB on test;
15 GB on QC
50 GB on ops
15 GB on QC
50 GB on ops-cip
15 GB on QC-cip
How often does system run (granule time, orbital, daily); event or schedule driven?
Test driven Every 6 hours Every 6 hours
Memory used at run time ~ 100MB ~ 100MB ~ 100MB
Input data volume and input data sources
NCEP GFS, ~ 9MB
MIRS ATMS temp retrieval, ~ 40MB
ATCF TC positions, ~1.0KB
NCEP GFS, ~ 9MB
MIRS ATMS temp retrieval, ~ 40MB
ATCF TC positions, ~1.0KB
NCEP GFS, ~ 9MB
MIRS ATMS temp retrieval, ~ 40MB
ATCF TC positions, ~1.0KB
Data volume for distribution; planned distribution server; specific push users & volumes
NA ESPC Data Distribution Server, FTP;
~200 MB/day
CIP Data Distribution Server, FTP;
~200 MB/day
Communication Requirements/Protocol
ftp, wget ftp, wget ftp, wget
Days to retain input and output data
3 days 3 days 5 days
Red font indicates new IT resource requirements for this projectBlue font indicates existing IT resources will be used for this project
V14.2 12 Jan 2010V14.2 12 Jan 20101313
Plan of Operations (1/2)
• Product Retirement Plan─ No product retirement is planned at this time. This product can run in parallel to AMSU-based
products.
• Archive Plan─ The TC intensity/structure products will be provided to NHC as a text bulletin in Automated
Tropical Cyclone Format (ATCF) and will be archived there (same as current AMSU products). ─ TC balanced winds products will be archived at NCDC, and a Data Submission Agreement will
be developed for it.─ Local archive requires to keep three day worth of data for real-time use
• Long Term Maintenance Plan─ 8 x 5 science support provided by STAR/RAMMB personnel.─ 24 x 7 technical support of input data flow, internet distribution, and basic product operation to be
provided by OSPO personnel.
• Quality Monitoring Plan─ STAR/RAMMB:
Monitoring and validating the products quality The intensity/wind radii estimates will be added to the ATCF system where they can routinely be compared
with similar estimates from other platforms. This can be done by STAR/RAMMB in coordination with SAB. Real time monitoring and feedback can also be obtained from NHC.
The nonlinear balance winds are input to the Multi-Platform Tropical Cyclone Surface Wind Analysis system that will be operational at NESDIS, which includes automated input quality control that can be leveraged to routinely monitor the nonlinear balance winds
V14.2 12 Jan 2010V14.2 12 Jan 20101414
Plan of Operations (2/2)
• Quality Monitoring Plan (cont)─ OSPO
Monitoring the data flow and processing status through NDE GUI interface 24/7 and automated anomaly detection through email notification.
Monitoring the products generation, timeliness and distribution status, and also inspecting imagery during normal business hours to detect anomalies and monitor long term stability/accuracy of products
• Documentation/Metadata Plan
─ STAR will lead the development of the SPSRB required documents, including ATBD.
─ NDE and OSPO will provide assistant on the operational related sections as
indicated in the document templates.
• User Involvement Plan
─ Feedback regarding accuracy, timeliness, and comparisons with AMSU-based
products will be solicited from NHC forecasters.
V14.2 12 Jan 2010V14.2 12 Jan 20101515
Project Milestones and Key Tasks
Product Delivery/Tracking Name: NPP microwave sounder-based tropical cyclone products• MAY 11 (MAY 11) (MAY 11): Development Phase Begins
─ MAY 11: IPT Lead informed to begin product development─ MAY 11: Begin adapting current AMSU algorithms to MIRS retrieval datasets─ OCT 11: NPP launch, begin collecting ATMS radiance data when available for global tropical cyclones─ JUL 12: Begin collecting MIRS ATMS soundings and limb corrected ATMS brightness temperatures, for global
tropical cyclones (collecting sample cases, real-time ingest expected Jul 2013)─ JUL 12: Initial Archive Requirements identified─ JUL 12: Quality Monitoring Concept Defined─ JUL 12: Long-term Maintenance Concept Defined─ JUL 12: Preliminary Design Review─ AUG 12: Development processing system defined─ OCT 12: Test case processed─ MAR 13: Critical Design Review─ MAR 13: Final Archive requirements identified─ APR 13: Develop pre-ops retrieval algorithms (using available NPP ATMS sounding test cases)─ APR 13: Code is prepared for implementation─ MAY13: Software Code Review─ MAY 13: Deliver Algorithm Package to NDE
• JUN 13 (MAR 13) (MAR 13): Pre-operational Phase Begins─ JUN 13: The TC DAP tested and evaluated within NDE SADIE, algorithm adjustments made─ JUL 13: TC products run on the NDE System Integration for 2013 Atlantic hurricane season─ AUG 13: Code transitions to operations; all documentation is complete─ SEP 13: Brief SPSRB Oversight Panel(s) on product status─ SEP 13: Brief SPSRB capability is ready to go operational
• OCT 13 (AUG 13) (AUG 13): Operational Phase Begins─ OCT 13: SPSRB declares product operational
SPSRB Secretaries/manager update the SPSRB product metrics web pages OSD updates Satellite Products database
V14.2 12 Jan 201016
FY13 Product Deliverable Details
Product Type and Number
Fiscal Year (FY)
Product Delivery/Tracking Name
Environmental Observational Parameters
Satellites Sensors N
#
E
#
R
#
T
#
O
#
Tailoring Options or Comments
FY13 NPP ATMS tropical cyclone products
Tropical cyclone maximum wind
NPP ATMS 1 Formats: text
Coverage areas: global
Resolutions: N/A
Update cycle: every 6 h
FY13 NPP ATMS tropical cyclone products
minimum sea level pressure
NPP ATMS 1 Formats: text
Coverage areas: global
Resolutions: N/A
Update cycle: every 6 h
FY13 NPP ATMS tropical cyclone products
radii of 34 kt winds in NE, SE, NW and SW
NPP ATMS 1 Formats: text
Coverage areas: global
Resolutions: N/A
Update cycle: every 6 h
FY13 NPP ATMS tropical cyclone products
radii of 50 kt winds in NE, SE, NW and SW
NPP ATMS 1 Formats: text
Coverage areas: global
Resolutions: N/A
Update cycle: every 6 h
FY13 NPP ATMS tropical cyclone products
radii of 64 kt winds in NE, SE, NW and SW
NPP ATMS 1 Formats: text
Coverage areas: global
Resolutions: N/A
Update cycle: every 6 h
FY13 NPP ATMS tropical cyclone products
balanced horizontal winds at 1000 to 200 mb within 600 km of the storm center
NPP ATMS 1 2 Formats: ASCII, netCDF, gif
Coverage areas: global
Resolutions: 0.2 degree lat/lat
Update cycle: every 6 h
N=New E=Enhanced R=Replacement T=Tailored O=Other
V14.2 12 Jan 2010V14.2 12 Jan 20101717
Funding (K) NPP microwave sounder-based tropical cyclone products
Funding Source Procurement Office Purchase Items FY11 FY12 FY13
OSD Polar Gnd Sys
-- Totals => 75 (0) 0 (0) 0 (0)
STAR CIRA Grant 75 (0) 0 (0) 0 (0)
STAR 0 (0) 0 (0) 0 (0)
JPSS NPP Products
-- Totals => 0 (0) 80 (0) 25 (0)
STAR CIRA Grant 0 (0) 80 (0) 25 (0)
STAR 0 (0) 0 (0) 0 (0)
JPSS NPP Products
-- Totals => 0 (0) 20 (0) 35 (0)
OSPO Contractor 0 (0) 20 (0) 35 (0)
OSPO IT Security 0 (0) 0 (0) 0 (0)
OSPO Base Totals => 0 (0) 0 (0) 0 (0)OSPO S/W Licenses 0 (0) 0 (0) 0 (0)
OSPO H/W Maintenance 0 (0) 0 (0) 0 (0)
OSPO Sys Admin, O&M 0 (0) 0 (0) 0 (0)
V14.2 12 Jan 2010V14.2 12 Jan 20101818
FY11 Purchases and Tasks
• (75K of OSD Gnd Sys – NPP Products): CIRA STAR Grant for 1 research associate at 50% time from MAY 2011 to APR 2012 and 1 IT person at 30% time from MAY 2011 to APR 2012.─ Begin adapting AMSU algorithms to use MIRS temperature profile
retrievals─ NPP launch in Oct 2011 – begin collecting ATMS radiance data
when available for global tropical cyclones─ Coordinate with the MIRS group on the availability of ATMS
temperature profile retrievals (MIRS DAP delivered Nov/Dec 2012, real-time ingest should be available soon thereafter)
─ Collect ATMS soundings and limb corrected ATMS brightness temperatures for global tropical cyclones and adapt retrieval algorithm
V14.2 12 Jan 2010V14.2 12 Jan 20101919
FY12 Purchases and Tasks
• (80K of JPSS NPP Products): CIRA STAR Grant for 1 research associate at 50% time from MAY 2012 to APR 2013 and 1 IT person at 30% time from MAY 2012 to APR 2013.─ Collect ATMS soundings and limb corrected ATMS brightness temperatures for global
tropical cyclones and adapting retrieval algorithm (real-time flow of ATMS MIRS delayed until July 2013, development will proceed using test case data)
─ Perform Preliminary Review─ Critical Design Review─ Operation Code Review─ Develop the NPP TC DAP for integration to the NDE─ Develop final algorithm using ATMS sounding test case data, process test case─ Deliver algorithm package to NDE
• (20K of JPSS NPP Products): OSPO Software Contractor for 1 senior programmer at 15% time (12 months) from MAY 2012 to APR 2013─ Work with the NDE team to conduct the capacity planning─ Work with the NDE team to develop the transition plan─ Provide support to PDR, CDR─ Work with CIRA and NDE to prepare Submission Agreement with NCDC─ Lead the Code Review effort─ Work with the NDE team to prepare for integrating the NPP TC products into the NDE
V14.2 12 Jan 2010V14.2 12 Jan 20102020
FY13 Purchases and Tasks
• (25K of JPSS NPP Products): CIRA STAR Grant for 1 research associate at 50% time from MAY 2013 to AUG 2013 and 1 IT person at 50% time from MAY 2013 to AUG 2013.─ Evaluate preoperational test results and make final adjustments to algorithm to prepare for
operations─ Help with implementation of operational code at NESDIS.─ Prepare SPSRB required documentation.
• (35K of JPSS NPP Products): OSPO Software Contractor for 1 senior programmer at 85% time (4 months) from MAY 2013 to AUG 2013.─ Assistant on implementing the NPP TC products into operation at NDE─ Testing and evaluation of integration─ Operational Readiness Review─ Work with CIRA and NDE to accomplish the required documents─ Develop the operation procedures for monitoring and maintenances─ Develop the monitoring tools for the NPP TC products