Slide 1/20RCAC – Rosen Center For Advanced Computing

Real-time Data Delivery and Remote Visualization through Multi-layer

Interfaces

Vinaitheerthan Sundaram, Lan Zhao, Bedřich Beneš, Carol X. Song, Rakesh Veeramacheneni, Peter Kristof

Rosen Center For Advanced ComputingDepartment of Computer Graphics Technology

Purdue University

Work supported by:National Science Foundation

Slide 2/20RCAC – Rosen Center For Advanced Computing

NEXRAD II Data

• Next Generation Radar (NEXRAD) Level II Data (OR) Weather

Surveillance Radar (WSR-88D) Level II Data– This data contains a very fine temporal and spatial resolution of three

attributes: reflectivity, Doppler radial velocity and spectrum width

– These attributes are vital to understanding, monitoring and predicting severe

weather conditions

– There are 158 Radar Stations in the country

Acknowledgment: Figures are downloaded from websites

www.CCSU.edu and www.answers.com.

Slide 3/20RCAC – Rosen Center For Advanced Computing

NEXRAD II Data Generation• 3D structure in Radar Data

– Radars go through a programmed set of movements, which involve a continuous rotation over 360° in azimuth and a simultaneous increase in elevation by 1° to 3° per complete sweep

• Continuous NEXRAD Level II radar data stream– The radar data files vary in size from a few megabytes to tens of megabytes each,

depending on the weather conditions. The files are compressed using modified bzip2

– The temporal resolution is 4-5 minutes in severe weather vs. 9-10 minutes in calm weather

Slide 4/20RCAC – Rosen Center For Advanced Computing

Availability of NEXRAD II Data in Near Real-time

• NEXRAD II Data is available in real-time on the

TeraGrid through Purdue resource provider.

• Opportunity: – The real-time availability of high-resolution radar data provides

an exciting opportunity for a wide spectrum of users ranging

from basic ( students) to expert (researchers) if the radar data

can be accessed and visualized in 3D in a timely manner.

• However, catering to wide spectrum of users presents

unique challenges as the requirements for each user

differ.

Slide 5/20RCAC – Rosen Center For Advanced Computing

Talk Outline

• Motivation– Providing real-time data access and remote visualization for a

wide spectrum of users

• Challenges– A review of challenges in the state-of-the-art systems

• A Unique and Versatile System Design– Multi-layer Interfaces

– Multiple Service access points

• Back-end Architecture – The enabler– Parallel Data Pre-Processing and partial-volume caching

• Summary and Future Work

Slide 6/20RCAC – Rosen Center For Advanced Computing

Challenges

• Limitations in the State-of-the-art– Not handle large amounts of data from many stations over a

long time

– No direct interaction with the data for users

– Not accessible to general public because of complicated interfaces

– No access points to third party applications

• Challenges– Data management issues: Radar data streams at 50 MB/secs

– Native compression format of radar data

– Data processing: Computationally expensive processing

– Special-purpose hardware (GPU) required

Slide 7/20RCAC – Rosen Center For Advanced Computing

Different User Levels and their requirements

• Expert Users ( small group )– Perform in-depth investigation– Examples: Researchers, Emergency Management Personnel

• Learners/Casual Users ( large group )– Access and visualize data for educational or personal purposes– Examples: K12 and College Students, Public

• Advanced Users ( small group )– Explore and evaluate data but don’t have resources– Examples: Graduate Students, Researchers from other domains

• Users levels are NOT mutually exclusive– Expert User can be an advanced user when attending

conferences and can be a casual user when teaching a class

Slide 8/20RCAC – Rosen Center For Advanced Computing

System Design

Expert Users

Casual Users

Advanced Users

Slide 9/20RCAC – Rosen Center For Advanced Computing

LiveRadar3D Gadget• Web 2.0 technologies

– AJAX, Google Gadgets, Social Networking Applications

– Allows rapid dissemination of scientific tools to wide audience

• Live Radar 3D– Shows animated Flash movie of

3D visualization of the region near user’s zip

– Scalable because movies are pre-generated and stored at the webserver

– Granularity: 7 Regions (midwest, south, southwest .. )

Slide 10/20RCAC – Rosen Center For Advanced Computing

LiveRadar3D Desktop• Desktop client

– Written in C++– Runs on Linux / Windows – Can be run on standard GPU

cards– Uses pre-processed volumes– Leverages Teragrid processing

power and local GPU

• Advantages– Fast interactive 3D

manipulation– Scalable: Supports large number

of stations and large intervals of time

User selects Radar- stations and time period

• Usage Scenario

Tool connects to data-access interface and fetch

processed volumes

Tool renders on the local GPU

Slide 11/20RCAC – Rosen Center For Advanced Computing

LiveRadar3D VNC

• For users who – Don’t want to download a client

– Don’t have the resources such as GPU

• Uses VirtualGL/TurboVNC to enable remote 3D visualization

• A convenient way to do advanced interactive and collaborative visualization remotely

• Browser accessible• Similar to LiveRadar3D Desktop in functionality

– Allows full capability available to expert users

• Disadvantage – Needs server farm to scale

Slide 12/20RCAC – Rosen Center For Advanced Computing

3rd Party Applications Access

• Our architecture is modular and supports fine-grained service access points

• Enables developing interesting 3rd party applications such as– Weather prediction application can connect to data access

interface

– Custom 3D visualizations can be built on pre-processed volumes

Slide 13/20RCAC – Rosen Center For Advanced Computing

Services and Backend Data Architecture

• In our earlier work, we presented a system that – Accesses NEXRAD II data

– Processes it into render-able 3D volumes using Teragrid

– Visualizes using Texture-based volume rendering

• Disadvantages– On-demand processing => Slow for large amounts of data

– Single access point targeted at expert users

• Extensions:– Multiple services and access points

– Preprocessing data to improve response time and scalability

– Volume Caching for easy access and reuse

Slide 14/20RCAC – Rosen Center For Advanced Computing

Backend Data Flow Diagram

Slide 15/20RCAC – Rosen Center For Advanced Computing

Parallel Data Pre-Processing

• Partial 3D volumes – efficient data structure using Hash-maps

– spatial/temporal independence property => parallel generation

– can be quickly merged to form full 3D volumes that can be rendered

– two orders of magnitude smaller than actual data and much smaller than generating full 3D volumes

• Generation of partial volumes on Teragrid– Monitors the arrival of new data and pre-processes them and

stores the partial-volumes

– SRB archives the past-year partial volumes

Slide 16/20RCAC – Rosen Center For Advanced Computing

The fully Interactive 3D Viz. Tool for NEXRAD II Data

Slide 17/20RCAC – Rosen Center For Advanced Computing

Visualization Images Generated Using Our Tool3D Visualization of 120 NEXRAD II Stations

Scaling – High Resolution Visualization of 3 Radar Stations at different scales ( 1 and 5 )

Rotation of April 10 Tornado3D Visualization of 120 NEXRAD II Stations

Scaling – High Resolution Visualization of 3 Radar Stations at different scales ( 1 and 5 )

Rotation of April 10 Tornado

Slide 18/20RCAC – Rosen Center For Advanced Computing

Hurricane Ike Images (September 14 2008 )

Slide 19/20RCAC – Rosen Center For Advanced Computing

Related Work

• NEXRAD Data Visualization– Integrated Data Viewer (IDV) by Unidata

– National Climate Data Center Java NEXRAD Viewer and Data Exporter

– CRAFT Interactive Radar Analysis System ( Java Viewer )

– LEAD Project ( Gateway )

• Remote Visualization– NanoHub – very small data

– Insley et al. Parallel RayTracing– slow for 3D interactions

• Web Gadgets– Weather.com/Floen.com – simple 2D visualization and

animation of radar data only for one station or whole nation

Slide 20/20RCAC – Rosen Center For Advanced Computing

Summary and Future Work

• Summary of our contribution– A hierarchical and user-oriented design

• rich and easy access to NEXRAD II data for a broad range of users.

– Improved response time and scalability• parallel data pre-processing and partial volume caching

– An integrated end-to-end backend system • radar data retrieval, pre-processing, remote rendering and 3D data

visualization.

• Future Work– Developing optimized data structures by exploiting the spatial

and temporal characteristics of the data

Slide 21/20RCAC – Rosen Center For Advanced Computing

Thank you for your attention!

Q & A