11

Unidata Policy Committee Unidata Policy Committee 5 February5 February

Level II Data Access & DistributionLevel II Data Access & DistributionLinda MillerLinda Miller

RootsRoots Why Level II Data?Why Level II Data? CRAFTCRAFT ApplicationsApplications NWS distribution NWS distribution UPC involvementUPC involvement Next StepsNext Steps

22

RootsRoots 1998 discussions with reps from USWRP, 1998 discussions with reps from USWRP,

OU, Unidata, NCAR, FSL, et alOU, Unidata, NCAR, FSL, et al• Use of LDM to distribute Level II data for Use of LDM to distribute Level II data for

universities and to NCDC for archivaluniversities and to NCDC for archival Unanimous endorsement for UPC to Unanimous endorsement for UPC to

proceed with collaboration at 1998 Policy proceed with collaboration at 1998 Policy Committee meeting in R.I. Committee meeting in R.I.

33

Roots-cont.Roots-cont. Collaborative Radar Acquisition Field Collaborative Radar Acquisition Field

Test (CRAFT) was bornTest (CRAFT) was born Main goal: “..to demonstrate real-Main goal: “..to demonstrate real-

time compression and data time compression and data distribution of multiple radars distribution of multiple radars delivering Level II data with a view delivering Level II data with a view toward nationwide implementation” – toward nationwide implementation” – technology transfer!technology transfer!

44

Why Level II Data?Why Level II Data? Level III (NIDS-type) are digital products Level III (NIDS-type) are digital products

(pictures)(pictures) Level II provides full resolution digital data; Level II provides full resolution digital data;

all elevation angles; full data precisionall elevation angles; full data precision Level I is too voluminous-includes I&Q in Level I is too voluminous-includes I&Q in

time series from reflectivity, velocity, time series from reflectivity, velocity, spectrum width (nuts and bolts of data)spectrum width (nuts and bolts of data)

Level II is research quality dataLevel II is research quality data

55

CRAFTCRAFT Successful collaboration with stakeholders Successful collaboration with stakeholders

from universities, private sector, governmentfrom universities, private sector, government Testing and refinements with LDM and Testing and refinements with LDM and

NSSL’s compression (Bzip2) with initial 6 NSSL’s compression (Bzip2) with initial 6 radarsradars

Use of LDM to move data from radars to Use of LDM to move data from radars to NCDC for archiving requirementNCDC for archiving requirement• Allowed transition from 8mm Exabyte Allowed transition from 8mm Exabyte

tapes tapes • Huge cost savings and reliability realizedHuge cost savings and reliability realized

66

CRAFT-contCRAFT-cont Currently 102 radars Currently 102 radars 35 sites receiving data from OU-CAPS35 sites receiving data from OU-CAPS

• 17 universities17 universities• 12 government 12 government • 6 private industry6 private industry

1 March- NWS will have 121 NWS and 3 1 March- NWS will have 121 NWS and 3 DOD sites on “CRAFT“/NWS NetworkDOD sites on “CRAFT“/NWS Network

NWS will announce options for distributing NWS will announce options for distributing Level II data by end of Feb 2004Level II data by end of Feb 2004

77

ApplicationsApplications Use in courses to expose students to full suite of radar Use in courses to expose students to full suite of radar

information-future plans include mesoscale modelinginformation-future plans include mesoscale modeling Cloud and wind analysis & mesoscale data assimilationCloud and wind analysis & mesoscale data assimilation Initializing forecastsInitializing forecasts Support NASA’s Tropical Rainfall measuring Mission (TRMM) Support NASA’s Tropical Rainfall measuring Mission (TRMM)

Ground Validation ProgramGround Validation Program Real time storm trackingReal time storm tracking Spotter operationsSpotter operations Experimental correction of echo tops researchExperimental correction of echo tops research Storm-scale data assimilation and numerical predictionStorm-scale data assimilation and numerical prediction Visualization with IDV and IRASVisualization with IDV and IRAS Storm analysis and radar data quality improvementsStorm analysis and radar data quality improvements Creating special products for customers Creating special products for customers Data archivalData archival

88

NWS DistributionNWS Distribution

ROC/NWS active CRAFT stakeholders CRAFT deemed an appropriate project for technology

transfer to NWS operations• NOAA/NWS became Abilene collaborator• Use of Internet2 High Speed Network to support distribution

of Level II data to NCEP, NCDC, and academic community• Private sector has 10 sec latency requirement• NCEP has 60 sec latency requirement• NWS held meeting in June, 2003, to discuss proposed method

of data distribution using Internet• Private sector responded by recommending FOS option

99

• Family of Services Proposed Radar Products Service II Estimated Costs

- One Time Cost (OTC) for processors (servers), routers, router I/O cards, cables, matrix switch I/O cards, etc. ($68K)

- OTC will be spread out over a period of 5 years to minimize the cost to external users ($13.6K)

- Other Direct Cost (Includes OTC) ($15K)

- Direct Labor Cost ($42K)

- Distributed (Indirect) Costs ($25K)

- Total Product/Service Cost ($83K/YR)

- Estimated Quantity (4), (5), (6), (7), (8)

- Final Product/Service Price ($21K), ($17K), ($14K), ($12K), ($10K)

1010

NWS Distribution StatusNWS Distribution Status Solicitation sent 12/05Solicitation sent 12/05 Evaluations conducted by:Evaluations conducted by:

• 3 Unidata technical staff3 Unidata technical staff• 3 NWS technical staff3 NWS technical staff

Point system to each proposal-Miller collected all Point system to each proposal-Miller collected all evaluations and tallied outcome – sent to NWSevaluations and tallied outcome – sent to NWS

Awards will be announced after MOAs have been Awards will be announced after MOAs have been signed by selecteessigned by selectees

1111

NWS Distribution & UPC Involvement NWS Distribution & UPC Involvement

UPC involvement UPC involvement • LDM 6LDM 6• Coordination with NWS Hdqtrs for a “Call for Coordination with NWS Hdqtrs for a “Call for

Participation” for top level sites to provide Level II data Participation” for top level sites to provide Level II data distribution to universities, government, private industry distribution to universities, government, private industry

Terms of “Call” – freely available to univ, gov, and on “cost Terms of “Call” – freely available to univ, gov, and on “cost recovery” basis for private industryrecovery” basis for private industry

Sign an MOA with NOAA/NWS with terms associated with Sign an MOA with NOAA/NWS with terms associated with the solicitation.the solicitation.

Solicitation-not FAR (Federal Acquisition Regulation) Solicitation-not FAR (Federal Acquisition Regulation) • UPC to provide LDM training to NWS regional hdqtrs staff UPC to provide LDM training to NWS regional hdqtrs staff

1212

NWS NWS 4 sites will provide LDM data 4 sites will provide LDM data

distribution of Level II datadistribution of Level II data 2 sites were pre-selected by the NWS;2 sites were pre-selected by the NWS;

• Mid-Atlantic Crossroads (Max) GigaPop Mid-Atlantic Crossroads (Max) GigaPop (collocated with Univ of Maryland)(collocated with Univ of Maryland)

• ERC GigaPop (collocated with NCDC-ERC GigaPop (collocated with NCDC-Asheville)Asheville)

2 remaining sites to be selected2 remaining sites to be selected

1313

INTERNET2(Abilene)

NEX

RA

DS

NWSNet

NWSNet

NWSNet

NWSNet

Eastern RegionLDM Server

Central RegionLDM Server

Southern RegionLDM Server

Western RegionLDM Server

WFO

WFO

WFO

WFO

WFO

WFO

WFO

WFO

WFO

WFO

WFO

WFO

Utah EducationNetwork GigapoP

NOAA NWSSilver Spring

DS3 Gigabit

DS3

DS3

OC3

Mid Atlantic Crossroads(MAX) GigaPoP

North Texas GigaPoP

Great PlainsNetwork GigaPoP

Ethernet

ONENetGigaPoP

(Oklahoma City)

OU LDM Server

Private SectorClients

I2 GigaPoP

Unidata ClientUniversity LDM

Server

I2 GigaPoP

Unidata ClientUniversity LDM

Server

TOC LDMServer

ERC GigaPoP(NCDC Asheville, NC)

ERC LDMServer

Backup Capability

LDM Feeds to Univ ersities

LDM Feeds to TOC Serv er

LDM Feeds to ERCWC Serv er

LDM Feeds to OU Serv er

Distribution of NEXRAD Level II Archive DataTo Research/Private Sector Users

(11/13/2003)

NCEP LDMServer

NCDC LDMServer

Private SectorClients

GigabitEthernet

OC3 ATM

1414

Next StepsNext Steps Unidata to train regional hdqtr sitesUnidata to train regional hdqtr sites

• Salaries, benefits, & travel covered by Salaries, benefits, & travel covered by NWSNWS

Assist selected top level sites with Assist selected top level sites with IDD topology, if desiredIDD topology, if desired• Provide support to top level <.edu> Provide support to top level <.edu>

sitessites• Provide support for <.edu> recipients of Provide support for <.edu> recipients of

datadata

1515

More Information?More Information?

Contact:Contact:support@unidata.ucar.edusupport@unidata.ucar.edu

ororLinda Miller Linda Miller

External Liaison, UnidataExternal Liaison, Unidatalmiller@unidata.ucar.edulmiller@unidata.ucar.edu

303-497-8646 303-497-8646