GFESuite Technical Review

Post on 30-Dec-2015

24 views 1 download

Tags:

description

GFESuite Technical Review. FSL/MD/EFT Oct 9, 2001. 3. Then, type (fast). 2. Coordinate for consistency. 1. Examine Guidance using AWIPS. TEXT. Public Marine Fire Wx Aviation. What Is GFESuite?. New method of producing forecasts in the NWS. Old Technique. - PowerPoint PPT Presentation

transcript

GFESuite Technical ReviewGFESuite Technical Review

FSL/MD/EFT

Oct 9, 2001

Oct. 9, 2001 GFESuite Technical Review 2

What Is GFESuite?What Is GFESuite?

New method of producing forecasts in the NWS

Old Technique

1. ExamineGuidance using AWIPS

2. Coordinatefor consistency

3. Then, type (fast)...

TEXT

PublicMarineFire WxAviation

Oct. 9, 2001 GFESuite Technical Review 3

What Is GFESuite?What Is GFESuite?

New method of producing forecasts in the NWS

IFP Technique1. ExamineGuidance using AWIPS

3. Then, press a button to generate products

abcGraphical

Text

RadioPhone

Web

2. Edit GuidanceDigitally

Oct. 9, 2001 GFESuite Technical Review 4

What is GFESuite?What is GFESuite?

Before GFESuite, the forecaster had to interpret the guidance, and then type the products– Detail in models was lost– Products could be inconsistent

Numerical Models andGuidance

Forecast Products

Oct. 9, 2001 GFESuite Technical Review 5

What is GFESuite?What is GFESuite?

Interactive Forecast Preparation technique– Bridges gap between guidance and products– Official Forecast is depicted via grids– Grids promote consistent and efficient product

generation, along with preserving model detail – Produces gridded, graphical, and text products

Numerical Models andGuidance

Forecast Products

GFESuite

Oct. 9, 2001 GFESuite Technical Review 6

What is GFESuite?What is GFESuite?

Components– Database server – holds grids/metadata– GFE (Graphical Forecast Editor)– Product Generators– Derivation Algorithms from Models– Misc. supporting programs

• Intersite coordination, daily forecast critique

Oct. 9, 2001 GFESuite Technical Review 7

Project GoalsProject Goals

Development of grid-based forecast tools and supporting system

Development of modernized products

Exploration of grid-based forecast process.

Oct. 9, 2001 GFESuite Technical Review 8

Project Funding/HistoryProject Funding/History

AWIPS funded project– Part of NWS Modernization Effort– Part of NWS Strategic Plan– Collaborative effort with MDL (IFPS)– Staffing Level: 3-8, now 5 ¾

90 91 92 93 94 95 96 97 98 99 00 01 02

RPP

AWIPSConcepts, Prototypes

Prep for AWIPS, IFPS

Oct. 9, 2001 GFESuite Technical Review 9

Project Scope – Two FacetsProject Scope – Two Facets

GFESuite software at all WFOs (~120)– Operational with AWIPS 5.0– Improvements with 5.1.1 through 5.2.2

Rapid Prototyping Project– RPP software now at 50+ sites

• SR/WR/ER plans to use RPP instead of AWIPS s/w

– Software is 1 year ahead of AWIPS software– Interaction with forecasters VERY IMPORTANT– Software is merged into AWIPS software

Oct. 9, 2001 GFESuite Technical Review 10

InteractionsInteractions

E-Team

LDAD

Intl Div.(CWB)

AWIPS(SDD/MD)

FRD(LAPS,

algorithms)

Avia. Div.(AWC)

GFESuiteMSAS

NWSHNWSH

NWS RegNWS Reg

WFOsWFOs

NCEPNCEP

NWS TCNWS TC

Oct. 9, 2001 GFESuite Technical Review 11

AgendaAgenda

Rapid Prototyping Project (Dave)Forecasting on Grids (Tracy)Walkthrough of:

– Derivation of Surface Fields (Mike)– Making a Forecast with GFE (Tom)– GFE from a WFO Perspective (Todd)– Forecast Products (Deb)

Future Plans (Mark)

Rapid Prototyping ProjectRapid Prototyping Project

Dave Howard

Oct. 9, 2001 GFESuite Technical Review 13

Rapid Prototyping Project (RPP)Rapid Prototyping Project (RPP)

BackgroundRelease ProcessInstallationFeedback

Oct. 9, 2001 GFESuite Technical Review 14

RPP Goal:RPP Goal:Enhance Feedback LoopEnhance Feedback Loop

Prototype New Features Rapid Turnaround Complete Forecast Process

Oct. 9, 2001 GFESuite Technical Review 15

RPP ActivityRPP Activity

AFWG/IFWG Successor November 1999 - first release 13 releases to date 15 official WFO sites

– Plus 30 informal sites

Oct. 9, 2001 GFESuite Technical Review 16

RPP SitesRPP Sites

Oct. 9, 2001 GFESuite Technical Review 17

RPP Release Process: CycleRPP Release Process: Cycle

Generally 6-8 weeks 4-6 weeks development, 1-2 weeks

testing RPP releases coordinated with

AWIPS releases

Oct. 9, 2001 GFESuite Technical Review 18

RPP vs. AWIPS ReleasesRPP vs. AWIPS Releases

09/99 01/00 05/00 09/00 01/01 05/01 09/0107/00 11/0103/0011/99 11/00 03/01 07/01

1 2 3 4 5 6 7 1312111098

5.1.15.0 5.1.2 5.2.1

RPP

AWIPS

Oct. 9, 2001 GFESuite Technical Review 19

RPP Release Process: ContentRPP Release Process: Content

Functionality/GUI enhancements– RPP requests– AWIPS requirements

Updated documentation Bug Fixes

– DRs– RPP sites– Internal

Oct. 9, 2001 GFESuite Technical Review 20

RPP ProgressRPP Progress(Cumulative)

Oct. 9, 2001 GFESuite Technical Review 21

RPP Release Process: TestingRPP Release Process: Testing

New Features Regression

– Complete Test Case Suite– Each AWIPS release or every 2-3 RPP releases

Bug Fix Verification RPP as Field Test

– Point Release if needed

Oct. 9, 2001 GFESuite Technical Review 22

RPP InstallationRPP Installation

Media– Package w/ CD and Readme– Web download

Considerations– Fresh Install– Upgrade

• Base/Site/User Hierarchy

– Migration

Easy: 10 minutes with minimal intervention

Oct. 9, 2001 GFESuite Technical Review 23

RPP FeedbackRPP Feedback

Listserver– E-mail based

RPP management calls Workshops Individual visits to FSL

Oct. 9, 2001 GFESuite Technical Review 24

RPP Feedback: RPP Feedback: Major InfluencesMajor Influences

GUI Layout (follows forecast process) GUI Ease-of-Use / Configurability

Forecaster can create/modify any tool– Derivation Algorithms– Smart Tools/Procedures– Text Product Generators

Forecasting on GridsForecasting on Grids

Tracy Hansen

Oct. 9, 2001 GFESuite Technical Review 26

Bridge between Models and ProductsBridge between Models and Products

Graphical Forecast Editor

Local Models Forecast Products

Forecaster Expertise

Digital Data Formatting

Oct. 9, 2001 GFESuite Technical Review 27

Forecast ProcessForecast Process

Set Up

Edit

GenerateProducts

Derive Surface Elements From ModelsDerive Surface Elements From Models

Review Previous ForecastReview Previous Forecast

Simple ToolsSimple Tools

Smart ToolsSmart Tools

Consistency CheckingConsistency Checking

Publish to Official DatabasePublish to Official Database

GraphicsGraphics TextText Digital DataDigital Data

Copy Selected Forecast Grids From DerivedCopy Selected Forecast Grids From Derived

Oct. 9, 2001 GFESuite Technical Review 28

Forecast ProcessForecast Process

Set Up

Edit

GenerateProducts

Derive Surface Elements From Models

Review Previous Forecast

Simple Tools

Smart ToolsSmart Tools

Consistency Checking

Publish to Official Database

Graphics Text Digital Data

Copy Selected Forecast Grids From Derived

Oct. 9, 2001 GFESuite Technical Review 29

Smart ToolsSmart Tools

Forecast GridsForecast Grids

Meteorological AlgorithmsMeteorological Algorithms

Numerical ModelsNumerical ModelsObservationsObservationsTopographyTopography

User InputUser Input

Smart ToolSmart Tool

ModifyModify

Oct. 9, 2001 GFESuite Technical Review 30

Smart ToolsSmart Tools

Forecast GridsForecast GridsNumerical Models

ObservationsTopography

User Input

Smart Tool

ModifyModifydef Fog_SmartTool(T, Td, Wind, Wx):def Fog_SmartTool(T, Td, Wind, Wx): if T if T Td < 2 and Wind < 4: Td < 2 and Wind < 4: Wx = “FogWx = “Fog”” return Wxreturn Wx

Oct. 9, 2001 GFESuite Technical Review 31

Smart ToolsSmart Tools

From a SOO at one of the RPP sites:

“Smart Tools permit science to rule, and guessing to wane.”

“This is hands-on forecasting!”

Derivation of Surface FieldsDerivation of Surface Fields

Mike Romberg

Oct. 9, 2001 GFESuite Technical Review 33

OverviewOverview

What– Provides a first guess for the forecast– Derives surface weather conditions from model

fields

Why– Forecasts require surface elements not

available in model (or not available via AWIPS)– Forecasts require higher spatial/temporal

resolution than available from model

Oct. 9, 2001 GFESuite Technical Review 34

What Can Be DoneWhat Can Be Done

Interpolate low resolution fields to higher– But they will still be smooth at the higher res.

Adjust values using high resolution topography

– 1km topography available– Interpolate to real surface using soundings– Example: Surface T adjusted for topography

Oct. 9, 2001 GFESuite Technical Review 35

Calculating Surface TCalculating Surface T

Interpolate model fields to high res– Boundary Layer Temperatures

• 2 meter, 0-30mb, 30-60mb, 60-90mb, 90-120mb AGL

– topography

Adjust to the real surface.– Calculate the lapse rate using boundary layers– Adjust 2 meter temperature using model/1km

topography differences and lapse rate

Oct. 9, 2001 GFESuite Technical Review 36

Derived Surface TDerived Surface T

Raw modelData at 80km

Sampled to 5kmresolution

Topography adjustments made

2 meter Temperature Surface Temperature

Oct. 9, 2001 GFESuite Technical Review 37

Derived Surface TDerived Surface T

2 meter temperature

Eta Model at available D2D model resolution

80km

Oct. 9, 2001 GFESuite Technical Review 38

Derived Surface TDerived Surface T

Sampled Eta Model to 5km resolution using bi-linear interpolation

Oct. 9, 2001 GFESuite Technical Review 39

Derived Surface TDerived Surface T

Corrected for topography differences between model and actual terrain

Oct. 9, 2001 GFESuite Technical Review 40

Implementation AlternativesImplementation Alternatives

Compiled Language (C++)Interpretative Language (Python)

Oct. 9, 2001 GFESuite Technical Review 41

Compiled Software for Derivation Compiled Software for Derivation AlgorithmsAlgorithms

Written in C, FORTRAN, C++ Fast Not configurable by users

(recompilation) Users can't add new surface fields Long delays between user feedback

and enhancements

Oct. 9, 2001 GFESuite Technical Review 42

Interpretative Software for Interpretative Software for Derivation AlgorithmsDerivation Algorithms

Written in Python Comparably fast as compiled

version. Configurable by users, provides

instant feedback. Users can add new surface fields and

models

Oct. 9, 2001 GFESuite Technical Review 43

Numerical PythonNumerical Python

Fast, compact, multidimensional array extension for Python

Developed at LLNL (open source) Design based on APL languages (Basis,

MATLAB, FORTRAN, S) Used by LLNL, LANL, NASA for large scale

physics codes on massively parallel supercomputers (MPI).

Best of both worlds (interpreted/fast)

Oct. 9, 2001 GFESuite Technical Review 44

FrameworkFramework

Communicates with IFP server– Fetch model data– Store surface grids

Uses Python's introspection to:– Find user supplied derivation methods– Determine dependent parameters

Calculates the order and times to run methods

Oct. 9, 2001 GFESuite Technical Review 45

Example: Derivation ClassExample: Derivation Class

class EtaForecaster(Forecaster): def calcT(self, t_FHAG2, t_BL030, t_BL3060, t_BL6090, t_BL90120, t_BL12015, p_SFC, topo, stopo, gh_c, t_c): def calcTd(self, p_SFC, T, t_FHAG2, stopo, topo, rh_FHAG2): def calcSky(self, gh_c, rh_c, topo): def calcPoP(self, gh_c, rh_c, QPF, topo): def calcFzLevel(self, gh_c, t_c, topo): def calcSnowAmt(self, T, FzLevel, QPF, topo): def calcHaines(self, t_c, rh_c): def calcMixHgt(self, T, topo, t_c, gh_c): def calcWx(self, tp_SFC, cp_SFC, bli_BL0180, T, QPF):

Oct. 9, 2001 GFESuite Technical Review 46

Example Derivation methodsExample Derivation methods

def calcQPF(self, tp_SFC): qpf = tp_SFC / 25.4 return qpf

def calcWind(self, wind_FHAG10): mag = wind_FHAG10[0] dir = wind_FHAG10[1] mag = mag * 1.94 dir = clip(dir, 0, 359.5) return (mag, dir)

Oct. 9, 2001 GFESuite Technical Review 47

A more interesting exampleA more interesting example

def calcT(self, t_FHAG2, t_BL3060, p_SFC, stopo, topo): dpdz = 287.04 * t_FHAG2 / (p_SFC / 100 * 9.8) # 45milibars is halfway between 30 and 60 dpdz = dpdz * 45 # meters between p_SFC and t_BL3060 lapse = (t_FHAG2 - t_BL3060) / dpdz # degrees / meter lapse = clip(lapse, lapse, 0.012) t = t_FHAG2 + lapse * (stopo - topo) return self.KtoF(t)

Oct. 9, 2001 GFESuite Technical Review 48

SamplesSamplesCWR MixHgt TransWind

T Wx Haines

Making a Forecast with GFEMaking a Forecast with GFE

Tom LeFebvre

Forecasting with the GFEForecasting with the GFEfrom a WFO perspectivefrom a WFO perspective

Todd Dankers

Forecast ProductsForecast Products

Deb Miller

Oct. 9, 2001 GFESuite Technical Review 52

Forecast ProductsForecast Products

Official Forecast DatabaseCompressed netCDF GridsGraphics via ifpIMAGE ProgramText Products via Text FormatterIntersite Coordination Grids (internal)

National Digital Forecast Database Grids

Oct. 9, 2001 GFESuite Technical Review 53

Official Forecast DatabaseOfficial Forecast Database

Actual forecast generated by siteForecast is sequence of gridsAll products derived from this

database– Promotes consistency between products

Oct. 9, 2001 GFESuite Technical Review 54

Compressed netCDF GridsCompressed netCDF Grids

Primary route of accessing and disseminating grids

Primary method of grid exchange between sites– Used for intersite coordination of grids

Oct. 9, 2001 GFESuite Technical Review 55

IfpIMAGE ProgramIfpIMAGE Program

Generates imagery product in standard PNG format

Minimal SW written. Reused GFE display capabilities, running in background, writing to PNG instead of Xlib.

Grids

GFE

Data Sys.

Visualizations

PNG

Graphic DrawingRoutines

Xlib

Oct. 9, 2001 GFESuite Technical Review 56

PNG Image and FeaturesPNG Image and FeaturesGraphics and images

can be overlaidOne PNG image for

each time stepAdjustable PNG sizesClipped to specific

geographic regionMap backgrounds may

be specified…

Oct. 9, 2001 GFESuite Technical Review 57

Sacramento: Max/Min HumiditySacramento: Max/Min Humidityhttp://www.wrh.noaa.gov/sacramento/html/expfire.htmlhttp://www.wrh.noaa.gov/sacramento/html/expfire.html

Oct. 9, 2001 GFESuite Technical Review 58

Denver: TemperatureDenver: Temperaturehttp://www.crh.noaa.gov/den/cgi-bin/getgraf.plhttp://www.crh.noaa.gov/den/cgi-bin/getgraf.pl

Oct. 9, 2001 GFESuite Technical Review 59

Tucson: PoPTucson: PoPhttp://www.wrh.noaa.gov/Tucson/gfe/digitest.shtmlhttp://www.wrh.noaa.gov/Tucson/gfe/digitest.shtml

Oct. 9, 2001 GFESuite Technical Review 60

Salt Lake City: Max Clearing IndexSalt Lake City: Max Clearing Indexhttp://www.wrh.noaa.gov/Saltlake/projects/ifp/html/clrindx.htmlhttp://www.wrh.noaa.gov/Saltlake/projects/ifp/html/clrindx.html

Oct. 9, 2001 GFESuite Technical Review 61

Tulsa: Winds Tulsa: Winds http://www.nwstulsa.noaa.gov/cgi-bin/forecast.plhttp://www.nwstulsa.noaa.gov/cgi-bin/forecast.pl

Oct. 9, 2001 GFESuite Technical Review 62

Tulsa: GFE images in LDADTulsa: GFE images in LDAD

Oct. 9, 2001 GFESuite Technical Review 63

Sample ifpIMAGE Configuration FileSample ifpIMAGE Configuration File

Oct. 9, 2001 GFESuite Technical Review 64

Text ProductsText Products

Stand-alone program written in Python and C++

Generates tabular or text phrase summaries of grid data

Oct. 9, 2001 GFESuite Technical Review 65

Oct. 9, 2001 GFESuite Technical Review 66

Sample Tabular Text ProductSample Tabular Text Product

Oct. 9, 2001 GFESuite Technical Review 67

Intersite Coordination GridsIntersite Coordination Grids

PUB

BOU

CYS

PUB

CYS

BOU

Internal NWS Tool to facilitate coordinated forecasts

Oct. 9, 2001 GFESuite Technical Review 68

NDFD Grids NDFD Grids (a.k.a. National Mosaic Grid)(a.k.a. National Mosaic Grid)

NWS plans to issue national gridded products by Sept 2003NWS plans to issue national gridded products by Sept 2003

Oct. 9, 2001 GFESuite Technical Review 69

Intersite Intersite Coordination Coordination GridsGrids

OAX

TOP

TSA

SGF ProbabilityofPrecipitation

Oct. 9, 2001 GFESuite Technical Review 70

SummarySummary

Text and graphic product ideas originated in the field and were developed with field feedback– Products were developed to

accommodate both past and future needs

Intersite Coordination Grids will be essential for a successful NDFD

Future Plans / SummaryFuture Plans / Summary

Mark Mathewson

Oct. 9, 2001 GFESuite Technical Review 72

Project ChallengesProject Challenges

Technical challenges were not the most difficult.– Right language choice (Python/C++)– Right platform choice (Linux/PC)

Field needed a complete system – not just the GFE. Project scope expanded.

GFESuite in IFPS provides conflicting paradigms to forecasters– Point-based vs. grid-based

Oct. 9, 2001 GFESuite Technical Review 73

Project ChallengesProject Challenges

Political– FSL/MDL– Regional vs. NWSH

Forecaster Acceptance:– Paradigm shift– Insufficient Training

Development to Deployment Duration– Too slow with AWIPS, Good with RPP– Still trying to reach the goals

Oct. 9, 2001 GFESuite Technical Review 74

Future WorkFuture Work

Intersite Coordination / NDFD Verification of Algorithms and Forecast Improvements to tools/algorithms Involvement with Training Improved Efficiency of grid editing Improved Forecast Methodology

Oct. 9, 2001 GFESuite Technical Review 75

SummarySummary

GFESuite operational – “minimal level” A lot of progress made towards goals

– But much more time needed to achieve them

RPP is a wonderful way to do business.– Interactions with field most satisfying

Lots of progress, especially since RPP. Staff is very motivated.

But, many challenges remain.