Sponsored by the National Science Foundation

GENI Alpha Demonstration

Nowcasting: UMass/CASA Weather Radar Demonstration

David IrwinNovember 3, 2010

http://geni.cs.umass.edu/visehttp://geni.cs.umass.edu/dicloud

http://www.geni.net

Sponsored by the National Science Foundation 2November 3, 2010

Problem

• CASA (an NSF ERC) is studying experimental networks of small controllable weather radars– Better data is the foundation of better hazardous

weather detection and earlier warnings

– Complex modeling to detect inclement weather requires many resources: sensors, bandwidth, storage, and computation

• Costly to dedicate resources for rare events– How do we generate accurate, short-term “nowcasts”

using these new distributed radar systems?

Sponsored by the National Science Foundation 3November 3, 2010

Solution

• Today: only a few large NEXRAD radars (100s)

• Tomorrow: many (1000s) smaller, less expensive radars produce data close to the ground where weather happens

• Requires a flexible infrastructure for coordinated provisioning of shared sensing, networking, storage, and computing resources on-demand

Sponsored by the National Science Foundation 4November 3, 2010

Example: Puerto Rico Testbed

• UPRM Student Testbed – Led by Jorge Trabal, Prof. Sandra Cruz-Pol,

and Prof. Jose Colom– http://www.youtube.com/watch?v=7TR64BhwMlI

Sponsored by the National Science Foundation 5November 3, 2010

Demo Background

• Dynamic end-to-end Nowcasting on GENI– Use GENI/Orca Control Framework (RENCI/Duke)

• https://geni-orca.renci.org/trac/• http://geni-ben.renci.org:11080/orca/

– Reserve heterogeneous slice of resources• Sensing Slice: UMass ViSE radars• Networking Slice: NLR, BEN-RENCI• Computation Slice: Amazon EC2 + UMass and RENCI VMs• Storage Slice: Amazon S3

Sponsored by the National Science Foundation 6November 3, 2010

Demo Data Flow

• Dynamic end-to-end Nowcasting – Mapping Nowcast Workflows onto GENI

Archival Storage

Radar Nodes

“raw” live data

Upstream LDM feed

archived netcdf data

Nowcast Processing

aggregatedmulti-radar data

Post to Web

Nowcast images for display

Sponsored by the National Science Foundation 7November 3, 2010

Multi-radar NetCDF Data

Nowcast Processing

1. DiCloud Archival Service (S3)2. LDM Data Feed (EC2)

“raw” live data

Generate “raw” live dataViSE/CASA radar nodes

http://stb.ece.uprm.edu/current.jsp

Use proxy to track usage-based spending on Amazon and enforce quotas and limits

http://geni.cs.umass.edu/vise/dicloud.php

1. Ingest mulit-radar data feeds2. Merge and grid multi-radar data2. Generate 1min, 5min, and 10min Nowcasts3. Send results over NLR to Umass4. Repeat

ViSE views steerable radars as shared, virtualized resources

http://geni.cs.umass.edu/vise

Nowcast images for display

Sponsored by the National Science Foundation 8November 3, 2010

GENI Technologies and Credits

• UMass-Amherst– ViSE and DiCloud projects

• University of Puerto Rico, Mayaguez– Jorge Trabal, Prof. Cruz-Pol, and Prof. Colom– OTG Radars

• Colorado State University– Prof. V. Chandrasekar– Nowcasting Software

• RENCI/Duke– Orca Control Framework– BEN network

• Starlight

Sponsored by the National Science Foundation 9November 3, 2010

Conclusion

• GENI is critical for next-generation applications– Enable nowcasting in experimental radar systems– GENI capabilities: “sliceability”/virtualization, federation,

network programmability

• Provide domain scientists a new platform– Experiment with tightly integrated systems combining

sensing, storage, networking, computing– Engage domain scientists in CASA and elsewhere

• Extend GENI network to Puerto Rico

Sponsored by the National Science Foundation 10November 3, 2010

Wrap-up

Sponsored by the National Science Foundation 11November 3, 2010

Demo Data Flow

• Dynamic end-to-end Nowcasting on GENI

Archival Storage

Radar Nodes

Generate “raw” dataViSE/CASA radar nodes

“raw” live data

http://stb.ece.uprm.edu/current.jsp

DiCloud ArchivalService on Amazon S3

Upstream LDM feed

archived netcdf data

Data publicly availableto downstream nodes

Use proxy to track usage-based spending on Amazon and enforce quotas and limits

http://geni.cs.umass.edu/vise/dicloud.php

ViSE views steerable radars as shared, virtualized resources

http://geni.cs.umass.edu/vise

Nowcast Processing

aggregatedmulti-radar data

GenerateNowcasts

1. Ingest data feeds from multiple radars2. Merge multi-radar data3. Generate 1min, 5min, and 10min

Nowcasts4. Repeat

http://vise-testbed.cs.umass.edu/nowcast/nowcast.html

Post to Web

Nowcast images for display

Sponsored by the National Science Foundation 12November 3, 2010

Thank you

UMass:Emmanuel

Cecchet

Prashant Shenoy

Jim Kurose

Eric Lyons

UPRM:Gianni Pablos

José Ortiz

Wilson Castellanos

Melissa Acosta

José Cordero

Benjamín De Jesús

Sandra Cruz-Pol

José ColomCSU:

V. Chandrasekar

Evan Ruzanski

Yanting Wang RENCI:Ilia Baldine

Jeff Chase

Anirban Mandel

Sponsored by the National Science Foundation 13November 3, 2010

Demo Overview

• Dynamic end-to-end Nowcasting on GENI – Slice of sensing, networking, computing, and storage

Archival Storage

Radar Nodes

Upstream LDM feed

Nowcast Processing

Post to Web

Generate “raw” dataViSE/CASA radars

Archive radar dataAmazon S3

“raw” live data

Archived data availableto downstream nodes

archived netcdf data

aggregatedmulti-radar data

Nowcast images for display

GenerateNowcasts

Sponsored by the National Science Foundation 14November 3, 2010

Demo Resource Listing in Orca/GENI