Date post: | 01-Jan-2016 |
Category: |
Documents |
Upload: | lynette-simon |
View: | 231 times |
Download: | 5 times |
NAWIPS Developer Training
Scott Jacobs
Steve Schotz
NCEP/Central Operations
Systems Integration Branch
18-19 April 2006
18-19 April 2006 NAWIPS Developer Training 2
Overall Topics
Introductions and Schedule History / Background Development Process Coding Standards and Common Errors Library Guide and Header Files Tables and Help, PDF and Error Files GEMPAK Fundamentals Test Programs and Other Common Debug Tools Utilities and Access to SIB Systems Upcoming and Long-term projects
18-19 April 2006 NAWIPS Developer Training 3
Schedule
4 sessionsDay 1
9:00-12:001:30-4:30
Day 28:30-11:301:00-3:30
1 or 2 breaks during each session
Ask questions!
18-19 April 2006 NAWIPS Developer Training 4
Day 1 - Morning
Introductions History / Background Development Principles Development Process
18-19 April 2006 NAWIPS Developer Training 5
Introductions
Name Where do you work? What do you do? Why are you here?
18-19 April 2006 NAWIPS Developer Training 6
Topics
N-AWIPS mission N-AWIPS history N-AWIPS customers Overview of N-AWIPS functionality N-AWIPS development principles Overview of N-AWIPS development
process
18-19 April 2006 NAWIPS Developer Training 7
N-AWIPS Mission
Develop robust meteorological workstation software to meet NCEP requirements, NCEP-AWIPS (NAWIPS)
Provide access to high resolution/global datasets
Provide flexible meteorological analysis and display tools
Provide integrated product generation capabilities for current and future modernized products
Provide robust, centrally supported and maintainable software
18-19 April 2006 NAWIPS Developer Training 8
N-AWIPS History
GEMPAK- Developed at NASA during 1980s to support research and academic communities (foundation for N-AWIPS) Provides set of ~ 150 libraries, (reusable code) for decoding, data basing,
performing calculations, displaying, and product formatting Extensive use in university and research communities
Exchange of ideas, algorithms and codes GEMPAK migrated to NMC- Early 1990s
National Centers (NC) given responsibility to address NC specific requirements – Late 1980s Recognition of significant mission differences between WFOs and NCs Decision primarily cost driven
18-19 April 2006 NAWIPS Developer Training 9
N-AWIPS History
Transition Project (TP) formed in Office of Director - 1991 TP mission - address NCs workstation modernization
requirements AWIPS contract awarded – 1992 TP begins deployment of NTRANS into operations – 1993 TP folded into NCO – 1995 AWIPS operational – 1999 New AWIPS contract awarded to Raytheon – 2005
18-19 April 2006 NAWIPS Developer Training 10
N-AWIPS Customers
NCEP service centers Climate Prediction Center; Hydrometeorological Prediction Center; Ocean Prediction Center (Camp Springs, Maryland)Aviation Weather Center (Kansas City, Missouri)Storm Prediction Center (Norman, Oklahoma)Tropical Prediction Center (Miami, Florida)
NCEP supercomputer and Environmental Modeling Center
NWS Alaska & Pacific regions
NWS River forecast centers
Unidata (~200 universities + private industry)
Government labs (MDL, ARL, ...)
18-19 April 2006 NAWIPS Developer Training 11
High Level Requirements
Diverse products, geographic and temporal scales
Somewhat diverse forecast processes
Mostly common/generic software requirements
18-19 April 2006 NAWIPS Developer Training 12
N-AWIPS Fundamental Capabilities
Data visualization
Integrated product generation
18-19 April 2006 NAWIPS Developer Training 13
N-AWIPS Data Visualization Capabilities
Definition: Ingest, decoding, calculation and display of
meteorological data
Meteorological data types include: Radar and satellite imagery Model data: grids, MOS Observations: METAR, marine, aircraft, upper-air,
lightning, pilot reports, profilers, … Forecast products: flash flood, Sigmets, Airmets,
watches, warnings, ….
18-19 April 2006 NAWIPS Developer Training 14
N-AWIPS Data Visualization Capabilities
Critical requirements include: Flexible user-defined geographic and temporal scales and map projections Wide variety of user-specified, “on the fly” calculations
Grid Diagnostics ~170 built in grid operators that allow user specified combinations of calculations, e.g., adv((vor(geo),thrm(hght@300:700))
Variety of display options, e.g., station plotting models, contour attributes, etc. User-defined data overlay options
Key display functions: Multiple animation loops: max of 16 loops with total of 120 frames Automatic data time matching to frame time to support data overlay
comparisons Procedures for automatic loading of data into multiple loops Continuous roaming to allow detailed views of data over large geographic
regions
18-19 April 2006 NAWIPS Developer Training 15
Integrated Product Generation
Definition: Creation/Editing of graphical products overlaid on meteorological data
displays
Critical requirements: Drawing and editing tools using meteorological objects, e.g., fronts, weather
symbols, etc. Graphical objects navigated to account for multiple product sectors and
projections, wind rotation, … Facility to import first guess fields, e.g., model fields and other centers’
graphical products Product formatting to support GIF, TIFF, PostScript, GRIB, BUFR,
Redbook… Product layering to support multi-component or multi-time concurrent
editing Object grouping
18-19 April 2006 NAWIPS Developer Training 16
N-AWIPS Development Principles
Maximize Code Maintainability Software must be written to be understood/maintained/enhanced by someone
other than original developer(s) Must be operationally robust
Maximize Code Reuse Employ functional libraries with test programs Never write code more than once to perform a particular function
Maximize Functional Flexibility/Adaptability Generic functionality to meet broad spectrum of current and anticipated needs,
e.g., flexible calculation tools, display options Maximize Hardware Independence - Unix (IRIX, SUN, HPUX, AIX), Linux, single
or multi-headed workstations Multiple hardware platform support improves software robustness Enables user platform/configuration choices Employ industry standards, e.g., ANSI C, no extensions
18-19 April 2006 NAWIPS Developer Training 17
N-AWIPS Development Principles
Minimize dependence on commercial external software systems Keep the system as simple as possibleFacilitate installation and testingReduce user costs of N-AWIPS
18-19 April 2006 NAWIPS Developer Training 18
N-AWIPS Development Process Overview
Use an evolutionary development strategy Build in small frequent increments, quarterly release
cycle Refine requirements based upon forecaster feedback Continue to improve system to support new products,
refinements of existing products and increase forecaster productivity
18-19 April 2006 NAWIPS Developer Training 19
N-AWIPS Development Process Overview
Employ disciplined development process covering entire software development life cycle Repeatable process Process must be efficient to allow changes to the
system to be cost effective Traceable process and document trail with peer
reviewed check points
18-19 April 2006 NAWIPS Developer Training 20
N-AWIPS Development Process Road Map
Document Creator Reviewer(s)
Requirements Centers Centers and N-AWIPS Team Lead (NTL), N-AWIPS Leads
Release Task List NTL N-AWIPS Leads
Task Proposal N-AWIPS Lead NTL, Lead SW Engineer (LSE), [Center]
Task Design(s) N-AWIPS Lead Lead SW Engineer (LSE) [Leads+NTL]
Config Items (Code, tables, help etc)
Coder N-AWIPS Lead(s), LSE
Delivery Form Coder N-AWIPS Lead, Configuration Manager (CM)
Code check & logs (nawips.log, changes.log, tables.log)
CM
Delivery test report Independent tester N-AWIPS Lead, NTL
Release Note Entry N-AWIPS Lead NTL
Integration Test report N-AWIPS Team N-AWIPS Lead, NTL, LSE
18-19 April 2006 NAWIPS Developer Training 21
Development Process Topics
Requirements Analysis Proposal Detailed Design Coding and Developer Testing Independent Testing Release Notes Approvals
18-19 April 2006 NAWIPS Developer Training 22
Requirements Analysis
Attempt to understand the user needs Must ask a lot of questions to make sure the needs are
well understood Don’t assume anything
Think about what the users are saying Ask the users to explain anything that is not clear
Don’t skip this step It always takes longer to fix something than it would
have to learn the requirements first Even for small tasks, spend some time on requirements
18-19 April 2006 NAWIPS Developer Training 23
Proposals and Designs
Purpose is several foldReduce costly downstream errorsCreates a repeatable processDocuments the approach and design for future
referenceTies the requirement to the coding and testing
18-19 April 2006 NAWIPS Developer Training 24
Proposals
Task Goal1-2 sentence statement of what you want to
accomplish Identify the requirement number addressed
BackgroundExplain user requirementsReview current functionality
The length of the proposal is dependent on the complexity of the task
18-19 April 2006 NAWIPS Developer Training 25
Proposals
Proposed Solution or OptionsDescribe any algorithms If there is more than one solution, list them for
review and discussionAlso include any pros/cons to each option
Areas of Code Affected Identify the libraries or applications
E.g., All GD programs; Only GDPLOT2; DG library; PGEN
18-19 April 2006 NAWIPS Developer Training 26
Proposals
Specific Tasks If this will be a multi-part task, list the subtasks For example
Part 1: Create new library routines and add to test programPart 2: Incorporate new routines in applications
Each subtask should be able to be tested alone Test Plan
Create a preliminary test plan Will help you make sure that you have covered
everything This will also help the reviewer(s)
18-19 April 2006 NAWIPS Developer Training 27
Proposals
Proposals are reviewed by Lead Software Engineer (LSE) NAWIPS Team Lead (NTL) Other designers, as needed In some cases, users or their representatives
Discussion If needed, a meeting will be arranged to discuss various aspects of
the proposal Approval
A proposal is approved when either the LSE or the NTL initials the document
Approval may be contingent upon changes indicated in the body of the document
See example document
18-19 April 2006 NAWIPS Developer Training 28
Detailed Design
DescriptionOverview of the changesSummarize the proposalLength is dependent on the complexity of the
taskDesigns are also done for bug fixes
In this case reference the bug or problem report
18-19 April 2006 NAWIPS Developer Training 29
Detailed Design
Function List Include any calling sequences changes (CSC)
Include the entire sequence with the changes If no CSC, only indicate “No CSC”
Describe the changes needed for each function Include enough detail so that someone other than you
could do the coding Also include any functions to be deleted
18-19 April 2006 NAWIPS Developer Training 30
Detailed Design
Library Check This is important when changing GEMLIB/CGEMLIB functions Many applications call these library functions Changing a library routine for one task may have unintended
consequences in some other application This check attempts to find all locations where a particular
function is called in order to assess the impact This is also needed to ensure that all necessary testing is included
in the test plan The utility “whocalls” can be used to help fill this in
Utilities will be discussed more fully later
18-19 April 2006 NAWIPS Developer Training 31
Detailed Design
Header Files, Tables, etc. List any files, other than source code, that also need to
be modified or created Test Plan
Include all tests that must be run Test the intended changes Test any applications found from the Library Check to
ensure no change in functionality
18-19 April 2006 NAWIPS Developer Training 32
Detailed Design
Author Make sure to put your name in as the author
Update the Level of Effort This should be a number of days to finish the coding,
unit testing and independent testing Update the start and end dates
These dates are for when you start and finish editing the detailed design
If the design is returned without approval, and you modify the document, indicate that as “Version 2” with the new dates
18-19 April 2006 NAWIPS Developer Training 33
Detailed Design
Designs are reviewed by Lead Software Engineer (LSE) Others as needed
NAWIPS Team Lead (NTL) Other designers
Discussion If needed, a meeting will be arranged to discuss various aspects of
the design Approval
A design is approved when the LSE initials the document Approval may be contingent upon changes indicated in the body
of the document
See example document
18-19 April 2006 NAWIPS Developer Training 34
Coding and Developer Testing
Copy existing files or create new files in your local account Do not work in the software account Make sure you get all necessary tables, etc. Also get the link scripts
Compile scripts are all in $GEMPAK/buildLink scripts are in the directories with the application source
code
Be careful with file names if working on GPLT functions since the same name is used in APPL and GPLT; and GPLTDEV and DEVICE
18-19 April 2006 NAWIPS Developer Training 35
Coding and Developer Testing
Proceed with the development as outlined in the detailed design If you find that the design is incomplete or incorrect,
update the design and make a note in the document header If you are not the original author of the design, discuss it with
that person first Make sure that you update the log in each source file
Compile on as many platforms as you can There should be no compiler errors nor warnings when
you are ready to deliver Access to SIB systems will be covered later
18-19 April 2006 NAWIPS Developer Training 36
Coding and Developer Testing
Conduct the developer unit tests Test the changes as outlined in the test plan Collect any special data sets for later testing Test unusual situations
That is try to break the code with strange inputThis is easiest if the new routines have been added to the
appropriate test program
18-19 April 2006 NAWIPS Developer Training 37
Coding and Developer Testing
Delivery Create a directory tree for your changes that mimic the
software accountStart at the “gempak” directoryPut all files in the appropriate subdirectories
Enter your name and the dates you started and finished coding on the form
Enter the path to the delivery tree you createdAccess to SIB systems will be discussed later
Describe the changes in a few sentencesThis description will become the log entry in “nawips.log”
18-19 April 2006 NAWIPS Developer Training 38
Coding and Developer Testing
Delivery (cont.) List all modified files
Put those with calling sequence changes first
List all new files List all files to be deleted All files listed will be compiled by the configuration
manager (CM) If a header file is changed, make sure to indicate all source
files with dependencies on that header fileThese files will also be compiled by the CM
List all applications that must be linked
18-19 April 2006 NAWIPS Developer Training 39
Coding and Developer Testing
Delivery (cont.) Describe the tests conducted and any significant results
Indicate any platform specific information
Provide a path to a set of executables for all platformsThis is for the code reviewer
Update the testing section of the Design with the followingDescribe any special data sets and their locationProvide any special testing instructions not covered in the
original design test plan
18-19 April 2006 NAWIPS Developer Training 40
Coding and Developer Testing
Code is reviewed by either The task owner
This is the person who wrote the design, if someone else did the coding
Another lead designer Lead Software Engineer (LSE)
Approval A delivery is approved when the reviewer initials the
document It is then given to the CM to add to the baseline
software accountThe CM enters the delivery number on the form
See example document
18-19 April 2006 NAWIPS Developer Training 41
Independent Testing
Conducted by someone not already associated with the task Must test on all platforms List all tests done
Must do all tests in the test planAnything else the person can think of to break it
List any problems encounteredThese would indicate a serious condition and would need to be
addressed as soon as possible
Also list any observations and suggestionsThese are less serious, and may be addressed later
18-19 April 2006 NAWIPS Developer Training 42
Independent Testing
The task owner reviews the test report Writes responses to all problems
If there are serious problems, the task goes back to coding
Writes responses to all observations and suggestions If any warrant immediate fixes, the task goes back to coding
If there are no issues or delayed issues, the task is DONE!
Approval The task owner initials and dates the test report after
review
See example document
18-19 April 2006 NAWIPS Developer Training 43
Release Testing
Test scenarios to emulate operations List of commands and button clicks Product creation tests Modified as needed to accommodate different data sets
Batch scripts NTRANS metafile generation New scripts to test all programs
Allow comparison between versions
Can and should be run at various times throughout the release cycle
18-19 April 2006 NAWIPS Developer Training 44
Release Notes When are Release Notes required
The user interface changesResults or output changePublic calling sequences changeAt the discretion of the NTL
Should be written as early as possible in the development process
See example document
18-19 April 2006 NAWIPS Developer Training 45
Release Notes Should sufficiently describe the changes to
functionalityRanges from a few sentences to a few
paragraphs
Will be incorporated into the full Release Notes document and the “What’s New” help file in NMAP2
The NTL reviews and edits the release notes for consistency with the design
18-19 April 2006 NAWIPS Developer Training 46
Day 1 - Afternoon
Coding Standards Common Errors Library Guide
18-19 April 2006 NAWIPS Developer Training 47
Coding Standards
Principles Make the system extensible so that additional needs can
be metDon’t need to serve all needs at once
Isolate complexity as much as possible Code must be able to be maintained by someone other
than the original developerCode should be readable and consistentFunctions should be as short as possible
Utilize functional libraries to maximize code reuse Routines/functions should be independently testable
Use of test programs will be discussed later
18-19 April 2006 NAWIPS Developer Training 48
Coding Standards
Fortran [77]All files should consist of only one subroutine
or functionLibrary routines should be well defined with a
single purpose
Lines should be less than 72 charactersDo not use the GOTO statement
18-19 April 2006 NAWIPS Developer Training 49
Coding Standards
Fortran [77] Routine Names
GEMLIB routines are 2 character library name + underscore + 4 characters to describe routine
Program routines are 2 character program class + 1 character program id + 3 characters to describe routine
GPLT routines all start with “G” and have up to 6 charactersDEVICE routines all start with “D” and have up to 6
charactersDriver routines all start with “H” and have up to 6 characters
18-19 April 2006 NAWIPS Developer Training 50
Coding Standards
Fortran [77] Uppercase
All Fortran keywordsSubroutine and function names Parameters defined by the PARAMETER statementThis is for readability since Fortran is case independent
VariablesLower caseShould not exceed 6 characters Implicit typing
I-N are Integers, only All others are Reals All Characters must be declared
18-19 April 2006 NAWIPS Developer Training 51
Coding Standards
Fortran [77] Indentations
First should be a full tab (8 spaces)Second should be 4 spaces after the tab (12 spaces)Third should be 2 tabs (16 spaces)Etc.
Continuation lines should begin with 5 spaces and a plus sign (+) or ampersand (&)
All mathematical symbols should have one leading space and one trailing space, when feasibleA set of assignments should have their equal signs lined up
18-19 April 2006 NAWIPS Developer Training 52
Coding Standards
Fortran [77] Common Blocks
In GEMPAK, there are specific uses for commonsCommons are not used to just bypass calling sequencesCommons are stored in include filesCommons group related variablesUse is discouraged for main programsMay be used for libraries – hidden from the applications
Parameter definitionsGeneral use parameters are defined in GEMPRM.PRMLocal parameters may be defined in the routine or local
include file Include files will be discussed further later
18-19 April 2006 NAWIPS Developer Training 53
Coding Standards
Fortran [77] Comments
Should be aligned with the codeLines with text should start with “C*”Should have a blank comment before and after any textShould be as descriptive as possible, while remaining brief
See code examples
18-19 April 2006 NAWIPS Developer Training 54
Coding Standards
C Generally the same standards as for Fortran Indentation starts in the first column
Each indent is 4 spaces thereafter
There are no continuation characters Comments are indented with the corresponding code
Open the comment with “/*” on one lineText on subsequent lines beginning with a “*”End the comment with “*/” only, on one lineThe asterisks are lined up
18-19 April 2006 NAWIPS Developer Training 55
Coding Standards
C Groups of related functions may be collected into a
single file, called a module Every function must have a prototype
This will be discussed further with header filesFortran functions called by C must also have prototypes
C is case sensitive, case is used to distinguish contentMacros in #define statements are all uppercaseLocal variables are lower case or mixed with the first character
as lower caseLocal-global variables start with underscore and may have
mixed caseGlobal variables, from header files, do not currently follow a
standard for case, most are all lower case
18-19 April 2006 NAWIPS Developer Training 56
Coding Standards
C All control statements (while, for, if, do) should have
the executable statements enclosed by “{“ and “}”This applies even if there is only 1 statementMakes it easier to add functionality later
Including adding diagnostic print statements for debugging
Makes it easier to understand exactly what is included in the control statement
The { should appear on the same line as the control The } should be indented the same as the corresponding
controlSee code examples
18-19 April 2006 NAWIPS Developer Training 57
Coding Standards
General Each file should begin with a prolog to document the
routine The chronological log in the prolog should be updated
every time the routine is modified If a routine is running out of indent space, then the
routine is trying to do too muchConsider breaking it up into smaller functions
In control and math statements, do not assume order of operation, use parentheses to be explicit
18-19 April 2006 NAWIPS Developer Training 58
Common Errors
Case sensitivitySince C is case sensitive and Fortran is not
some errors can ariseGEMPAK uses case in Fortran for readability
Even so, everything is really lower case after compilation
In C Fortran functions must be lower case Any C function called by Fortran must be only lower case
18-19 April 2006 NAWIPS Developer Training 59
Common Errors
Value AssignmentsDon’t forget to set the value of variables before
using the valueIn C, it is preferable to do the assignment in a
separate statement from the declaration In this way the assignments can be grouped together
18-19 April 2006 NAWIPS Developer Training 60
Common Errors
Check all cases for a variableDon’t forget 0, nor the outer limitsWhen checking values in a control statement
Make sure that all possible values are accounted for in the code
A common error is to check for positive or negative, but forget about what happens when the value is zero
Is there an upper/lower limit that makes sense for the value
Does a line width of 1,000,000,000 make sense, and can it happen in the code?
18-19 April 2006 NAWIPS Developer Training 61
Common Errors
Underscore at the end of function name By default, without the user’s knowledge, some Fortran
compilers put an underscore at the end of function names IRIX, SunOS, Linux In fact, Linux puts 2 underscores by default
A compiler flag set for GEMPAK limits it to 1
Any Fortran function called by C, or C function called by Fortran, must have an underscore added to the function name in the C source
This is accomplished with a header file (uscore.h) and a compiler flag (-DUNDERSCORE)
18-19 April 2006 NAWIPS Developer Training 62
Common Errors
Return codes and Error processing If a function needs to return after a problem, the return
code should be set a unique valueAll libraries and programs have error tablesWhen adding a return code, check this table for the next
available value
Errors return a negative valueErrors should cause the processing to stop and return to a
reasonable point in the application
Warnings return a positive valueProcessing can continue, but it may not be what the user really
wanted
18-19 April 2006 NAWIPS Developer Training 63
Common Errors
Exceeding array sizesMake sure that all array references are within
the bounds of the arrayCan the value of the index ever be negative?
In Fortran, this may be ok depending on the array declaration
Can the value of the index ever be larger than the allocated size of the array?
18-19 April 2006 NAWIPS Developer Training 64
Common Errors
Use of the qualifier “const” in CEnsures data protection
Important when passing strings or pointers that should not be changed in the function
Constant parameters in a function will generate a compiler error if there is an attempt to modify the value
18-19 April 2006 NAWIPS Developer Training 65
Common Errors
Memory managementWhen using dynamic memory allocation in C
Make sure to free all local memory before returningIf the allocated memory is global to a library
Make sure that a function is provided to free all global memory
18-19 April 2006 NAWIPS Developer Training 66
Common Errors
C vs. Fortran Variable types
C has variable types that are not supported by Fortran “unsigned” has no meaning in Fortran int and long may be different sizes in C
There are also types that must be declared differently by Fortran
short == INTEGER*2
C array counting starts at 0, Fortran starts at 1 Two dimensional arrays are reversed
vararr[10][20] == vararr(20,10)
18-19 April 2006 NAWIPS Developer Training 67
Common Errors
C vs. Fortran Fortran passes all variables by reference
The equivalent in C is a pointer to a variable typeConstant values should not be used in calling sequences
StringsArrays of strings are different in C and FortranCannot pass an array of strings between the two languagesFortran automatically passes the length of all strings in a call,
hidden at the end of the list When calling Fortran from C with a string in the call, add a
“strlen(strng)” for each string after the last real sequence parameter
18-19 April 2006 NAWIPS Developer Training 68
Common Errors
C vs. FortranPointers
If you are used to Fortran the concept of pointers can be difficult to follow
In C, you can access both the value and the memory address of a variable
A pointer “points” to the memory locationExample
int *ivar;*ivar is the value (dereferenced from the pointer)ivar is the memory address
18-19 April 2006 NAWIPS Developer Training 69
Library Guide
GEMPAK External
18-19 April 2006 NAWIPS Developer Training 70
GEMPAK Libraries
The source is organized by function General library functions (C and Fortran) Decoder libraries Message queue utilities GPLT Device drivers Specific program code
18-19 April 2006 NAWIPS Developer Training 71
GEMPAK Libraries
Some libraries have their own global variables not shared with other libraries
Fortran does not allow for private functions Therefore, it is by convention that some functions are not called
from other functions outside of a particular library
In C, defining a function with the “static” qualifier will make it local to a module Therefore, it cannot be called from any functions outside of the
module
18-19 April 2006 NAWIPS Developer Training 72
GEMPAK Libraries
build data
error help
include maps
tables pdf
parm txt
utilities
appl bridge
cgemlib contrib
device driver
gemlib gplt
nmaplib nxmlib
programs syslib
textlib
source
gempak bin doc
icons os
resource extlibs
nawips
18-19 April 2006 NAWIPS Developer Training 73
GEMPAK Libraries General Libraries (C and Fortran)
The GEMPAK libraries handle a number of functions that are needed by all programs
String manipulations File access Table access Time calculations Data packing and data file management
There are also libraries that are used by certain classes of programs Grid diagnostics Surface data access Upper air data access Image data access and display GRIB processing
18-19 April 2006 NAWIPS Developer Training 74
GEMPAK Libraries
aw cv
dg, de, df, dl, dv, pd dm
dp er
fl gb
gd gdu
gg gh
gp gr
im in
ip jb
lc lv
lw mr
mv na
oa pc, pr, ps, pt
sf sn
ss st
tb tg
ti tm
vf
gemlib
18-19 April 2006 NAWIPS Developer Training 75
GEMPAK LibrariesLibrary Name
Function Usage
AW Display Redbook graphics files Used by GPMAP
CV Curve functions
DG, DE, DF, DL, DV, PD
Grid diagnostics Public functions called by applications; Private functions called internally to compute diagnostics
DM GEMPAK data management Called by SF, SN and GD
DP Data packing Called by DM
ER Error message processing and display
FL Low-level Fortran file access
GB GRIB data processing Used by NAGRIB
GD Grid data interpretation Public functions to access grid data
GDU Grid data utilities for matching wild cards Used by GDINFO
GG Graphical display functions for maps, lat/lon, misc data
GH Hurricane product creation
GP Generic functions pulled from GPLT Used to transform wind directions
18-19 April 2006 NAWIPS Developer Training 76
GEMPAK LibrariesLibrary Name
Function Usage
GR Grid data utility functions Functions that don’t fit other groups
IM Image data processing
IN Input string parsing Called by applications
IP GEMPAK text interface Get the user input for all applications; NTS files
JB Jack Woolen BUFR functions Previous version of NCEPBUFR library
LC Location functions Processes AREA and GAREA parameters
LV Level utilities Used in the upper air applications
LW Reads a “long-word” format file Only lightning data is stored in this type of file
MR Upper air merge functions Used in the upper air applications
MV Byte swapping and manipulation When the file was saved on a different machine
NA NAGRIB library
OA Objective analysis
PC, PR, PS, PT
Parameter conversions Public functions to set up later computations; Controlled by table: pcconv.tbl
18-19 April 2006 NAWIPS Developer Training 77
GEMPAK LibrariesLibrary Name
Function Usage
SF Surface data interpretation Public functions to access surface data
SN Upper air data interpretation Public functions to access upper air data
SS System functions Before C libraries, allowed access to system level functionality
ST String manipulation
TB Table reading Various specific table access functions
TG Grid time calculations
TI Time calculations
TM Terminal input General functions to get input from the terminal
VF Create SPC watch products Works on the “verification” file to create products; Written in C, but part of GEMLIB
18-19 April 2006 NAWIPS Developer Training 78
GEMPAK Libraries
cas cbf
cds ces
cfl cgr
clo cpg
crg css
cst ctb
cvg cvq
g2 shp
spf utf
xml
cgemlib
18-19 April 2006 NAWIPS Developer Training 79
GEMPAK LibrariesLibrary Name
Function Usage
CAS Reads intermediate ASCII files for SIG WX charts VG to BUFR and BUFR to VG processing
CBF Accesses BUFR files for SIG WX charts
CDS Display of VG objects PGEN and display in GPMAP
CES VG object attribute management PGEN
CFL Low-level C file access
CGR Miscellaneous graphical utilities Many for PGEN and other polygon manipulation
CLO Locator functions Location searching in station or bound files
CPG Plot 6-bit FAX files Used by GPFAX
CRG Range record management PGEN
CSS System functions Similar to Fortran SS
CST String manipulation
CTB Table reading
18-19 April 2006 NAWIPS Developer Training 80
GEMPAK LibrariesLibrary Name
Function Usage
CVG VG file access
CVQ Query information about VG files and objects
G2 GRIB2 data processing Used by NAGRIB2
SHP Shape file processing Used to convert Shape files to GEMPAK map and bounds files
SPF Stored Procedure File processing Used by NMAP2 for saving and restoring lists of data sources
UTF AFOS graphics display Used by GPMAP
XML Interface functions to XML libraries
18-19 April 2006 NAWIPS Developer Training 81
GEMPAK Libraries
Decoder libraries Each decoder has a library of functions that is unique to that
decoder METAR Synoptic Upper Air MOS Etc.
There is also a library of routines that are used by all decoders to handle common functionality, e.g., command line input, bulletin reading
18-19 April 2006 NAWIPS Developer Training 82
GEMPAK Libraries
Message Queue Library Communication between GEMPAK processes is done using
message queues There is a library of routines that is linked with the application
programs that construct the information for the message queue This APPL information is sent to the GPLT process Then GPLT has a library of routines that creates messages for a
second message queue The GPLT information is sent to the user specified device driver
18-19 April 2006 NAWIPS Developer Training 83
GEMPAK Libraries
GPLT Mainly responsible for coordinate transformation from Map
coordinates to the specific Device coordinates Grid – Map – Plot – View – Normal – Device
Other responsibilities Setting and storing the current map navigation Satellite navigation computations Contour and streamline algorithms Wind rotation Setting plotting margins Setting and storing object plotting attributes
For example -- Lines: Dash pattern and width
18-19 April 2006 NAWIPS Developer Training 84
GEMPAK Libraries
access attribute
bounds color
contour control
coord curve
device graph
grid main
map plot
satellite transform
utility
gplt
18-19 April 2006 NAWIPS Developer Training 85
GEMPAK LibrariesLibrary Name Function
ACCESS Start and close GPLT
ATTRIBUTE Set and query attributes of drawables (e.g., lines, fronts)
BOUNDS Set and query coordinate boundaries
COLOR Set and query color information
CONTOUR Compute and display contour lines and fills
CONTROL Control frame information (e.g., clear, roam, looping, windows)
COORD Private functions to compute transformation information
CURVE Sets and computes the type of contour curves (e.g., cubic splines)
18-19 April 2006 NAWIPS Developer Training 86
GEMPAK LibrariesLibrary Name Function
GRAPH Set and query information for plotting graphs
GRID Set and query information for setting grid projections on maps and graphs
MAIN The GPLT main program
MAP Set and query information for plotting maps
PLOT Plot the drawables
SATELLITE Native satellite navigation transformations
TRANSFORM Coordinate transformation functions
UTILITY Draw maps, lat/lon lines, streamlines and polygon boundaries
18-19 April 2006 NAWIPS Developer Training 87
GEMPAK Libraries
access attribute
color control
curve draw
main plot
satellite window
device
18-19 April 2006 NAWIPS Developer Training 88
GEMPAK LibrariesLibrary Name Function
ACCESS Start and close GPLT
ATTRIBUTE Set and query attributes of drawables (e.g., lines, fronts)
COLOR Set and query color information
CONTROL Control frame information (e.g., clear, roam, looping, windows)
CURVE Sets and computes the type of contour curves (e.g., cubic splines)
DRAW Device independent code to interface to the device drivers
MAIN The DEVICE main program
PLOT Plot the drawables
SATELLITE Plot images to the drivers
WINDOW Set the “window” dimensions and the clipping boundaries
18-19 April 2006 NAWIPS Developer Training 89
GEMPAK Libraries
Device Drivers Current device drivers
FAX NCEP 6-bit Fax GF GIF file created using the X server GIF GIF file created without using the X server NC NCEP Computer Graphics Metafile PS PostScript RBK AWIPS Redbook graphics file TIFF TIFF graphics file VG Vector Graphics file XW X window
18-19 April 2006 NAWIPS Developer Training 90
GEMPAK Libraries
Application Programs Decoders Grid data plotting Surface data plotting Upper Air data plotting Objective Analysis Graphic utilities Graphical User Interface programs
NMAP2 combines display of image, grid, surface, upper air, misc
18-19 April 2006 NAWIPS Developer Training 91
External Libraries
Collection of libraries from disparate sources Collaborative efforts
Automated Objective Dvorak Technique (AODT)GAMET
Specific functionalityGeneric Polygon Clipper (GPC)MELBUFR / NCEPBUFRCREWSSNetCDF
General utilitiesXML / XSLT / DOMPNGJASPER
18-19 April 2006 NAWIPS Developer Training 92
External Libraries
The decision was made to pull these libraries out of the core GEMPAK directory treeMade more sense to keep them separate from
locally developed code
The installation of some of these libraries forced the recent change in the location of executable, library and include files to the “os” subdirectories
18-19 April 2006 NAWIPS Developer Training 93
Day 2 - Morning
Header Files Tables Maps and Bound Files Help, PDF and Error Files Build Scripts GEMPAK Fundamentals Test Programs
18-19 April 2006 NAWIPS Developer Training 94
Header and Include Files
Global C header files and Fortran Include files for GEMPAK are in $GEMINC
System specific files and all header files from the external libraries are in $OS_INC
Library specific files are found in the directory with the source code
See file listing
18-19 April 2006 NAWIPS Developer Training 95
Header and Include Files
Fortran Include files Common block files
DGCMN.CMNsfcmn.cmn
Parameter filesSets constant values used to declare arrays or string lengthsGEMPRM.PRMFUNCCODE.PRM
Files in $GEMINC are those needed by functional groups of code that are not in the same source directoryDGCMN is used by DG, DF, DV, DE, DL which are all
related to grid diagnostics
18-19 April 2006 NAWIPS Developer Training 96
Header and Include Files
C header files Macro definitions Structure definitions Global variable declarations Public prototypes Files in $GEMINC are those needed by functional
groups of code that are not in the same source directorygeminc.h gemprm.h, proto.h, uscore.h, vgstruct.h
Always use the “file.h” construct for local files, not <file.h>This allows a local copy to override the global copy, during
development
18-19 April 2006 NAWIPS Developer Training 97
Tables
Tables are all in $GEMTBL Directories are an attempt at categorizing
the tablesOrganized by function or type
bufr,colors, config, draw, grid, loop, luts, pack, parms, pgen, rad, sat, stns, txtprd, xsd, xslt
Organized by program or librarygrphgd, hcnadv, melbufr, nmap, nsharp, nwx
See file listing
18-19 April 2006 NAWIPS Developer Training 98
Tables
There is a hierarchy used when searching for a table ./table_name (local directory) $NCDESK/type/table_name $NCSITE/type/table_name $GEMTBL/type/table_name
NCSITE and NCDESK are used to hold site-wide or user specific tables
Copy the tables from $GEMTBL to these locations before modifying
Having the local directory first facilitates testing since it will override the other locations
18-19 April 2006 NAWIPS Developer Training 99
Tables
Key tables In the config directory
prefs.tbl Settings that are used to control default values Most of the tags refer to the user interface or product generation
in NMAP Can be used for any program
datatype.tbl Allows the use of aliases for data sources Equates the aliases to paths and file name templates Sets other attributes for the data source
clo.tbl Sets up aliases for the bounds and stations tables used in the
locator library
18-19 April 2006 NAWIPS Developer Training 100
Tables
Key tables In the config directory
forecasters.tbl List of forecasters to display in the user interface for selection
mapfil.tbl List of map aliases
miscset.tbl Sets the attributes and flags for the miscellaneous data sources Sets default values
printer.tbl List of printers for selection in the user interface
prmlst.tbl Parameter list aliases Sets the list of parameters to display in a station model Sets the colors and other attributes
18-19 April 2006 NAWIPS Developer Training 101
Maps
The map files can be found in $GEMMAPS GEMPAK uses a internally developed direct access file
format $GEMMAPS/ascii/mapfiles.doc gives specifics
The majority of the maps come from the NWS Shape file database and are converted to GEMPAK map files
Some maps came from the CIA map database Some maps have been created from VG files Most maps have multiple resolutions
TP, HI, ME, LO
18-19 April 2006 NAWIPS Developer Training 102
Maps
The maps can be displayed in any GEMPAK plotting application or NMAP2
To allow NMAP2 to access the maps Edit the file $GEMTBL/config/mapfil.tbl to add an
alias for the particular map Edit the file $GEMTBL/nmap/mapovl.nmap to add the
alias to the Map control window The GEMPAK programs can use the alias or the
actual file name Multiple maps can be overlaid by listing them in the
parameter $MAPFIL separated by a “+”
18-19 April 2006 NAWIPS Developer Training 103
Maps
The map utility programs are in $GEMMAPS/programs There are programs to convert from the binary format
to ascii and vice versagsfssf and ssfgsf If a map needs to be fixed: convert it to ascii, fix the problem,
then convert back to binary
The “nam” programs work with the CIA map data “reducebnd” is used when creating boundary tables
18-19 April 2006 NAWIPS Developer Training 104
Bounds Files
These files are located in $GEMTBL/bounds They are not really tables, but…
The files are plain ascii files There is an index file that is created by the utility
“createbinfo” Any time that the bound file is modified, the index file
must be updated Bounds files are closed polygons that represent
geographical regions or forecast areas A few of the NWS Shape files are also converted to
bounds files Some files are created from VG files using “vgftobnd”
18-19 April 2006 NAWIPS Developer Training 105
Bounds Files
The bounds files are used in numerous functionsThe NMAP2 locator (bottom right of the
window)GFA processing for the AWCWatch product creation for the SPCUnified Surface Analysis VG file clipping Ice extent display for the OPCDisplay of filled areas from GPMAP using the
BND parameter
18-19 April 2006 NAWIPS Developer Training 106
Help, PDF and Error Files
All help files are in $GEMHLPThe program files are in the “hlp” subdirectoryThe parameter file are in “hlx”
Users can display the content of the help files from the various programsFrom a GEMPAK program
Enter any program or parameter name after the word “help”
All GUI programs have a “Help” button that will display the file(s) related to that application
18-19 April 2006 NAWIPS Developer Training 107
Help, PDF and Error Files
Parameters require 2 files .hl2 files contain the content displayed to the
user .hl1 files are a one-line description used in the
program help files and the PDF files Program help files are generated from the
TXT filesThe text files are in $GEMPAK/txt/programsThe help files are generated by the utility
program “hlpbld”
18-19 April 2006 NAWIPS Developer Training 108
Help, PDF and Error Files
PDF files are unique to GEMPAKNOT the Portable Document Format
Cannot use Acrobat to display the contentsJust an ascii file
Left over from the time that GEMPAK used the TAE text interface developed at NASA
Format was changed from the original TAE to just a collection of the all of the .hl1 files for a program
Created using the utility “pdfbld”
18-19 April 2006 NAWIPS Developer Training 109
Help, PDF and Error Files
Error files Found in $GEMERR One for every GEMPAK application and library Contain a list of code numbers and descriptions
Negative values are errorsPositive values are warnings
Errors are displayed to the user using the function ER_WMSGThe function uses the category, the code number and an
optional string to locate and display the messageThe optional string replaces “!AS” in the description in the
error file
18-19 April 2006 NAWIPS Developer Training 110
Build Scripts
Compile scriptsLocated in $GEMPAK/buildUsed to compile librariesAlso compiles application librariesAdds all compiled files to the appropriate
library archive (.a) file
18-19 April 2006 NAWIPS Developer Training 111
Build Scripts
Link scriptsUsed to link all applicationsDriver scripts are in $GEMPAK/build
These scripts go to the individual program directories to link the applications
The link scripts are located in the program directories
18-19 April 2006 NAWIPS Developer Training 112
Build Scripts
Things to considerAs the name implies, nmaplib should only be
used when linking NMAP2Many link scripts have GEMLIB or CGEMLIB
listed more than onceThere is no way to predict when or how to add more
than one reference to a libraryWhen in doubt, follow the link script for a similar
programStill no luck, send e-mail or ask
18-19 April 2006 NAWIPS Developer Training 113
Dot Files
Used to set environment variables useful in the execution of the applications
Also set system aliases and the exec path Many environment variables, and the path, are set
based on the current operating system In csh, the user sources the .cshrc In sh, bash or ksh, the user sources the .profile
Changes are rare, but very important to keep up to date Modifications are highlighted in the release notes
18-19 April 2006 NAWIPS Developer Training 114
GEMPAK Fundamentals
Functional Layers Applications GPLT Device Drivers Decoders GEMPAK Data Files Product Generation and VG files
18-19 April 2006 NAWIPS Developer Training 115
Functional Layers of GEMPAK
Application AP
PL
Message Queue
Message Queue
Normal execution of an application, that plots data, starts GPLT and a Device Driver
The processes communicate using Message Queues SYSLIB is used to create and use the Message Queues
Allows user to change Device Driver during execution of a program Allows user to overlay the output of many apps to 1 Device Driver
GP
LT
GP
LT
DE
V
DE
VIC
E
DR
IVE
R S
peci
fic
H-l
evel
DR
IVE
R
18-19 April 2006 NAWIPS Developer Training 116
Example: Drawing a Line
Application AP
PL
Message Queue
Message Queue
Same function names in APPL/GPLT and GPLTDEV/DEVICE APPL sends message through MQ GPLT main program receives message and calls function Similar from GPLT to DEVICE
GP
LT
GP
LT
DE
V
DE
VIC
E
DR
IVE
R S
peci
fic
H-l
evel
DR
IVE
R
GSLINE
GLINE
GSLINE DSLINE DSLINE
GLINE DLINE DLINE
ILINE
HLINE
XLINE
18-19 April 2006 NAWIPS Developer Training 117
Example: Without MQs
Application
Executable file is larger Can only output to one device driver Done for better performance in batch processing Done for GUI programs, since the input mechanism is the same as the
output device
GP
LT
DE
VIC
E
DR
IVE
R S
peci
fic
H-l
evel
DR
IVE
R
GSLINE DSLINE
GLINE DLINE
ILINE
HLINE
XLINE
18-19 April 2006 NAWIPS Developer Training 118
GEMPAK Applications
Most applications are divided based on the type of GEMPAK data file they work with GD – Grid data processing SF – Surface data processing SN – Sounding (Upper Air) data processing
Others are grouped by functionality OA – Objective analysis GP – Miscellaneous graphics NA – Binary data decoding (GRIB) DC – Decoders GUI – All graphical user interface programs
18-19 April 2006 NAWIPS Developer Training 119
GPLT
Responsible for the following functionalitySetting MAP and GRID projectionsCoordinate transformationsContouring and Streamline algorithmsNative satellite navigation computationsSetting marginsSetting and saving plotting attributesFront-end to plot/draw functions
18-19 April 2006 NAWIPS Developer Training 120
GPLT
Coordinate Systems Grid
IntermediateQ – rotated
MapW – rotatedLinear
Plot View Normal Device
Units Grid is grid point
based on kx and ky Map is lat/lon in
degrees P, V, N are
normalized and range from 0 to 1
Device is pixels in the output – i and j
18-19 April 2006 NAWIPS Developer Training 121
Device Drivers
Device independent routines plot the following Symbols and markers Winds as barbs or arrows Text Pattern lines Filled polygons Fronts
All of these types of elements are converted to line segments for output to the specific driver
Clipping to the plot boundaries is also done in the device independent code
18-19 April 2006 NAWIPS Developer Training 122
Device Drivers 3 common blocks are used to set and store
drawing attributes DEVREQ
Attributes requested by the userUsed by GPLT
DEVSETAttributes used in the deviceUsed by both GPLT and DEVICE
DEVACTAttributes actually set in the deviceUsed by both GPLT and DEVICE
For example: There are 32 colors. If the user requests color 40, the actual color used is 8.
18-19 April 2006 NAWIPS Developer Training 123
Current Active Device Drivers FAX – NCEP 6-bit Fax format GF – GIF image (extension of XW driver) GIF – GIF image (no X server needed) GN – Generic test driver NC – NCEP Computer Graphics Metafile PS – PostScript RBK – Redbook AWIPS graphics TIFF – Tagged Image File Format VG – Vector Graphics File XW – X Window XWP – X Window Plus (combines XW and PS)
18-19 April 2006 NAWIPS Developer Training 124
Device Driver Differences
FAX, RBK and TIFF are monochrome only PS can be monochrome, gray shades or full color The origin for most drivers is the bottom left
corner The XW/XWP origin is the top left corner
VG does not have a DEVICE coordinate All locations are saved to the file as lat/lon values
VG also does not draw the map to the output file
18-19 April 2006 NAWIPS Developer Training 125
Decoders
In general, a decoder is a program which reads data in one format and converts it to another formatText report to GEMPAK data file
GEMPAK decoders also take care of opening output files and writing data to the files
18-19 April 2006 NAWIPS Developer Training 126
Decoders
The decoders take command line options and input
Help is displayed with the “-h” option The output file name may use a template for
the date and timeThe actual file name will change depending on
the input data
18-19 April 2006 NAWIPS Developer Training 127
Decoder OptionsOption Description
-v N Level of verbosity for the logs
-c curtim Set the “current” time
-b nhours Number of hours to decode prior to “current” time
-d decoder_log Decoder log file name
-t time_out Interval for the timeout in seconds
-n Flag to NOT save the text report as data
-h Print the help file, then exit
-p prmfil Set the parameter packing table
-s stntbl Set the station table
-a iadstn Set the number of additional stations
-m maxtim Set the maximum number of times
18-19 April 2006 NAWIPS Developer Training 128
Current Decoders
DCACFT – Aircraft DCAIRM – AWC AIRMET DCCSIG – Convective SIGMET DCFFA – Flash flood watch DCFFG – Flash flood guidance DCGMOS – GFS MOS DCHRCN – TPC forecast advisory DCISIG – International SIGMET DCLSFC – Surface land synoptic
18-19 April 2006 NAWIPS Developer Training 129
Current Decoders
DCMETR – Hourly surface observations DCMSFC – Surface marine synoptic DCNCON – Non-convective SIGMET DCNMOS – NGM MOS DCRDF – Regional Digital Forecast DCSCD – Supplemental Climatological DCSVRL – Severe local storm DCTAF – Terminal aerodrome forecast DCUAIR – Upper air soundings DCWARN – Local WFO warnings
18-19 April 2006 NAWIPS Developer Training 130
Current Decoders
DCWCN – Watch county notification DCWCP – SPC Watch corner points DCWOU – SPC Watch outline DCWSTM – Winter storm reports DCWTCH – SPC watches DCXMOS – Extended GFS MOS
NAGRIB, NAGRIB2 – GRIB NAMSND – BUFR soundings
18-19 April 2006 NAWIPS Developer Training 131
GEMPAK Data Files
GEMPAK Data Management Files Applications do not access the data management functions directly Applications use the interface routines for the type of data being
accessed Grid (GD) programs use the GD access functions Surface (SF) programs use the SF access functions Upper Air (SN for Sounding) programs use the SN access functions
Clean interfaces That is, all information is passed in the calling sequences of the
routines, no information is shared between levels through global variables
The data management library does not interpret the data stored in the files
The data access routines interpret the information for the data type
18-19 April 2006 NAWIPS Developer Training 132
GEMPAK Data Files
Responsibilities of the data management functions Read data from the files Write data to the files Data packing
Set up at file creation Packing, based on a table, into as few integers as necessary DM handles conversion from Real values to packed integers, and vice
versa Searches through the data headers Machine type conversions
Differences in workstations and operating systems is isolated to the DM routines
Any necessary byte swapping is handled by the DM routines only
18-19 April 2006 NAWIPS Developer Training 133
GEMPAK Data Files
Label:
Version number, number of rows, columns and parts, file type, missing value, machine type (all set at file creation)
File Info:
File headers, free data space, part and header names, packing info for SF and SN, navigation for GD files
Row and Column Headers:
Store “key” information, Keys are used for searching
Data Pointers
Data Records
Data Management File Structure
18-19 April 2006 NAWIPS Developer Training 134
GEMPAK Data Files
18-19 April 2006 NAWIPS Developer Training 135
GEMPAK Data Files
Missing Data Value For packed data, the missing value has all bits turned on
Min: 0; Max: 3; Interval: 1 Offset: 0; Scale: 0; Bits: 3 Valid data values: 0, 1, 2, 3, Missing
The missing value is stored in the file header at creation
Machine Type The machine type is written to the file header When reading and writing data, the value is checked against the
current machine If necessary, the value is used to determine byte order and
swapping
18-19 April 2006 NAWIPS Developer Training 136
Vector Graphics Files
Used for interactive product generation Used to store georeferenced objects
All locations and points in the objects are latitude/longitude
Binary format fileThe format is defined in vgstruct.hThe format was created by NAWIPS designers It is NOT a standard format known outside of
the NAWIPS system
18-19 April 2006 NAWIPS Developer Training 137
Vector Graphics Files
Structure of a VG fileFollows the definition of the element structure
in vgstruct.hActually a union of structuresAll individual elements are defined in their own
structuresFront, Line, Text, Symbol, Watch, GFA, etc.The structures contain the attributes for the element
and the location(s) used to plot on a map
18-19 April 2006 NAWIPS Developer Training 138
Vector Graphics Files
For exampletypedef struct frontinfo { int numpts; int fcode; int fpipsz; int fpipst; int fpipdr; int fwidth; char frtlbl[4];} FrontInfo;
typedef struct fronttype { FrontInfo info; float latlon[MAXPTS*2];} FrontType;
typedef struct lineinfo { int numpts; int lintyp; int lthw; int width; int lwhw;} LineInfo;
typedef struct linetype { LineInfo info; float latlon[MAXPTS*2];} LineType;
typedef struct fileheadtype { char version[128]; char notes[256];} FileHeadType;
18-19 April 2006 NAWIPS Developer Training 139
Vector Graphics Files
For exampletypedef struct vg_hdrstruct{ char delete; char vg_type; char vg_class; signed char filled; char closed; char smooth; /* smooth level of VG elements */ char version; /* version number of VG elements */ char grptyp; int grpnum; int maj_col; int min_col; int recsz; float range_min_lat; float range_min_lon; float range_max_lat; float range_max_lon; } VG_HdrStruct;
18-19 April 2006 NAWIPS Developer Training 140
Vector Graphics Files
For exampletypedef struct vg_dbstruct { VG_HdrStruct hdr; union { FileHeadType fhed; FrontType frt; LineType lin; } elem;} VG_DBStruct;
18-19 April 2006 NAWIPS Developer Training 141
Product Generation
Challenges of manual product generation Initial drawing of elementsSetting element attributesSubsequent editing of the elements or the
attributesEfficient file formatMinimize element search times
18-19 April 2006 NAWIPS Developer Training 142
Day 2 - Afternoon
Test Programs Common Debug Tools Utilities Points of Contact Access to SIB Systems Upcoming and Long-term Projects
18-19 April 2006 NAWIPS Developer Training 143
Test Programs
Many GEMLIB and CGEMLIB libraries have test programs
The test programs were created to test the individual functionality of each routine
The goal is to have every function included in at least one test program
When adding a new function, make sure to add it to the appropriate test program
18-19 April 2006 NAWIPS Developer Training 144
Test Programs
The test programs have a simple user interfaceAll input is queried from the user at the
terminalThis allows for easily testing all paths through
the codeA script can be created to run the program
many times with various input The output from the functions is printed to
the terminal
18-19 April 2006 NAWIPS Developer Training 145
Common Debug Tools
Built-in tools for Linuxgdb is the GNU debuggerddd is a graphical front-end to gdbvalgrind is run similar to a debugger
Tracks memory usageChecks for uninitialized variablesChecks for out-of-bounds conditions for arrays
mtrace is used to trace calls to malloc and the associated memory
strace records system calls from a program
18-19 April 2006 NAWIPS Developer Training 146
Common Debug Tools
PurifyPurchased from Rational (now part of IBM)Only for HPUXSimilar to valgrind
Checks for memory issues
18-19 April 2006 NAWIPS Developer Training 147
Common Debug Tools
PRINT/PRINTFA very powerful toolAllows very specific control over what is being
monitoredDon’t overlook the simple solution
18-19 April 2006 NAWIPS Developer Training 148
Utilities
Many utilities have been developed by the NAWIPS team to accomplish tasks searcher
Searches a directory tree for the requested stringCan also take action on the files found to contain the search
string whocalls
Generates a list of functions that call the requested function whatcalls
Shows a list of functions called by the specified function deliv.perl
Creates a list of functions to be deliveredThe list is suitable for inclusion on the delivery form
18-19 April 2006 NAWIPS Developer Training 149
Utilities
Other utilities have been downloaded or are part of Linux mgdiff
Graphical difference between two filesUsed to check or reconcile changes made to files
mpagePrinting utilityCreates virtual pages so that more than one page is printed per
sheet of paper
18-19 April 2006 NAWIPS Developer Training 150
Points of Contact
Your local NAWIPS liaison Your branch chief Steve Schotz - NAWIPS Team Leader
Scott Jacobs - SIB Technical [email protected]
NAWIPS [email protected] tags on the subject line to differentiate issues
BUG, DEV-QUESTION, USER-QUESTION
18-19 April 2006 NAWIPS Developer Training 151
Access to SIB Systems
Currently, one system at each center can access HP1
New accounts have been made for each center to log into SIB systemsFirst, log into HP1, then you can log into other
platformsBy using these accounts, you will have access
to all SIB utilities These accounts will access the development
version of NAWIPS (nawdev)
18-19 April 2006 NAWIPS Developer Training 152
Access to SIB Systems
Current platformsHPUX - hp1AIX - ibm2 IRIX - sgi82Linux
RHEL 3 (kernel 2.4el) - lnx64RHEL 4 (kernel 2.6el) - lnx365
18-19 April 2006 NAWIPS Developer Training 153
Access to SIB Systems
New accountsdevawcdevcpcdevhpcdevopcdevspcdevtpc
Will provide passwords
18-19 April 2006 NAWIPS Developer Training 154
Upcoming and Long-term Tasks
Grid Toolkit Graphical Object Toolkit
18-19 April 2006 NAWIPS Developer Training 155
Grid Toolkit Topics
Objectives Benefits Stakeholders Risks Strategy FY06 Activities Preliminary FY07+ Plans
18-19 April 2006 NAWIPS Developer Training 156
Project Objective
Develop a common grid calculator toolkit/service that can be accessed by any hydrometeorological application.
18-19 April 2006 NAWIPS Developer Training 157
Benefits
The toolkit will: Contain a clean API and shareable architecture Be grid size independent Be independent of data format and data access
Enable a distributed data access paradigmEnable access to GRIB files directly
Allow the entire NWS to benefit from NCEP expertise in model grid algorithms Include all current GEMPAK ‘on the fly’ diagnostic
capabilities
18-19 April 2006 NAWIPS Developer Training 158
18-19 April 2006 NAWIPS Developer Training 159
Stakeholders
NCEP Service Centers NCO EMC AWIPS Unidata
18-19 April 2006 NAWIPS Developer Training 160
Risks
N-AWIPS senior software analysts availability
Other NCEP requirement priorities Uncertain LOE for later phases Uncertainty in API requirements beyond N-
AWIPS
18-19 April 2006 NAWIPS Developer Training 161
Strategy
Overlapping multi-phase – Initial goal of integration into N-AWIPS Analysis Design Development
Test and integrate after each phase in N-AWIPS initially
Plan for eventual integration into AWIPS-Next
18-19 April 2006 NAWIPS Developer Training 162
FY06 Activities
Remove grid size limitation for calculationsAnalyze codeRewrite grid diagnostic libraries in c
~ 400 diagnostic library routines in 7 librariesRegression test scripts writtenAdd dynamic memory management for internal
grids Integrate into operational deadline – target
5.9.4 (August 2006)
18-19 April 2006 NAWIPS Developer Training 163
Preliminary FY07+ Schedule
FY06 –FY07 High-Level activities API definition to decouple grid calculator from
GEMPAK grid file structureApplication interfaceData access interface
Design APIs Develop test plans Code APIs Modify N-AWIPS applications to use new APIs Integrate and test in N-AWIPS
18-19 April 2006 NAWIPS Developer Training 164
Graphical Object Toolkit Topics
Background Objectives Benefits Stakeholders Risks Strategy FY06 Activities Preliminary FY07+ Plans
18-19 April 2006 NAWIPS Developer Training 165
Background
N-AWIPS Vector Graphics (VG) files are key component of NCEP graphical product generation
VG files attributes Contain meta information and earth locations for large suite of
meteorological objects and graphical products Fronts, jets, meteorological symbols, clouds, Sigmets, jets, watches,
etc. VG files are used as basis to generate other product formats
Text Products, GIF, BUFR, etc. VG files are currently in-house developed binary file format
18-19 April 2006 NAWIPS Developer Training 166
Project Objective
Redesign the VG file format to use Extensible Mark Up Language (XML) based format so that NCEP graphical products can be more readily ingested by external systems.
Develop a toolkit that provides a simple set of APIs so that applications beyond the NAWIPS suite can easily access and store meteorological graphical objects.
18-19 April 2006 NAWIPS Developer Training 167
Benefits
NCEP graphical products more easily converted to other formats for use by outside users GIS systems proprietary formats Possible web displays
SVG (a W3C standard)X3d (a W3C standard)
Facilitate conversion to BUFR Greatly facilitate conversion to text products
18-19 April 2006 NAWIPS Developer Training 168
Benefits- Continued
XML file format is a standard Internationally recognized World Wide Web
Consortium (W3C) standard U.S. Government accepted standard Platform independent
Extensible solution New attributes/objects easily incorporated Backward file compatibility more easily managed
18-19 April 2006 NAWIPS Developer Training 169
Benefits- Continued
Toolkit provides APIs so that outside groups benefit from N-AWIPS team expertise in managing meteorological objectsEnable intelligent retrieval and storage of
meteorological objects and attributesBy object type, location or group, e.g, retrieve all
cold fronts or closest cold front to a location Interprets objects into XML files and vice versa
18-19 April 2006 NAWIPS Developer Training 170
Stakeholders
NCEP Service Centers NCO AWIPS Unidata External Users and Systems
18-19 April 2006 NAWIPS Developer Training 171
Risks
N-AWIPS senior software analysts availability Other NCEP requirement priorities Staged implementation into NMAP is difficult;
risk mitigation strategy includes: Initial analysis and prototyping phase Development done in parallel with incremental testing
using NCEP graphical products Integrate into N-AWIPS with extensive parallel testing
18-19 April 2006 NAWIPS Developer Training 172
FY06 Activities
Preliminary Analysis Conducted Federal Geographic Data Committee (FGDC)
specification for geospatial meta data (file header) investigated
Preliminary XML schema analysis conducted XML libraries and tools installed XML training conducted for N-AWIPS team Project suspended due to resource limitations
XML approach is being used for components of GFA task
18-19 April 2006 NAWIPS Developer Training 173
Preliminary FY07+ Plans
Develop XML schema for VG files Possibly adopt or extend Geography Mark Up
Language (GML) W3C XML specification Define toolkit interface Prototype toolkit and test using NCEP graphical
products Convert N-AWIPS to use toolkit
Perform parallel testing
Implement toolkit in N-AWIPS