Date post: | 16-Dec-2015 |
Category: |
Documents |
Upload: | ralph-bennett |
View: | 218 times |
Download: | 2 times |
Designing a DTC Verification System
Jennifer MahoneyNOAA/ESRL21 Feb 2007
Talk Overview
• Define verification system
• Present the key components that will form the DTC verification system framework
Allow flexibility
Serve a variety of users
• Discuss the complexities of developing the DTC verification system
Acknowledgments
• ESRL/GSD Verification System Engineering TeamSean MadineNick MathesonMissy PettyDan Schaffer
What is a Verification System?
What is a DTC Verification System?
Verification Analysesas a Function of Software Characteristics
DTC Verification Analysis Activities
Software Characteristics
Non-Functional Requirements
• Important Operational Considerations Stability Reliability Security
• Important Development Considerations Support for collaborative interaction Adaptability to other environments Extensibility to other scientific problems Maintainability Portability to other organizations
of the ‘System’
Verification Mechanics
Aggregate/Combine to
Produce SpecificMetrics
Pre-processForecasts
Pre-processObservations
Create Displays
Verification System Components- Historical Perspective
• Good solution for a small set well defined questions
• System structure remains consistent, but can expand forecast dataset
• Solution needs to change when want to add observation datasets, new verification mechanics and assessment techniques, integrate non-meteorological information, and inter-compare a variety of forecast systems
Verification System Components:Data Management Structuring System
Data Warehouse
Finest resolutionForecast/observation pairs+ forecast/obs attributes
Verification System Components:Data Management Structuring System
Data Warehouse
Finest resolutionForecast/observation pairs+ forecast/obs attributes
Data Marts
Verification System Components:Data Management Structuring System
Data Warehouse
Finest resolutionForecast/observation pairs+ forecast/obs attributes
Data Marts
IntercompareForecastSystemsProduceStatisticalInformation
Aggregation driven by specific data relationships
Verification Mechanics
Pre-processForecasts
Pre-processObservations
Displays
DTC Verification System Components
Store FinestGranularityVerificationElements
Apply storageOptimization Strategy Database info
Query interrogationto compute score
Verification System Complexities
• Management of complex interdependent relationships between datasetsEvent-driven component initiation
• Eliminate unnecessary time delays between data execution steps• Handle abnormal delays in data access, processing, and user
access• Account for differences between operational deployments
Multi workflow requests• Incorporate relationships with the addition of new datasets,
verification methodologies, and forecast intercomparisons• Provide only meaningful workflow comparisons (i.e., par down list
of all possible choices to meet user analysis questions)
Verification System Complexities
• Effective access to the resultsProvide tiered optimizations
• Fast turn around of pre-defined set of analyses• Flexibility to define varying set of analyses• Support wide-range of user needs
Allow for wide variety of graphical and standard displays
• Traditional statistical displays• Displays that allow the integration of verification information
Summary
• Verification system will mainly support the forecast intercomparison evaluation (e.g., core tests) function of the DTC
• Scientific conceptualism developed for the toolkit will feed into the DTC verification system
• DTC requirements for the verification system require complex data management and interrogation strategies
• Tiered approaches for data access are required to provide maximum system flexibility to end users
Future Efforts
• Gather verification system requirements from the workshop
• Develop a Functional Requirements document• Begin preliminary development of the system
concepts by the end of the year
Contributors to the DTC System Development
• NCAR – scientific verification concepts and toolkit capabilities
• NCEP – bridge from the current NCEP verification system functionality to the DTC verification system framework
• GSD – verification system framework by extending database management and web access strategies
• DTC users – functional requirements for the verification system
Questions?