Interactive Scientific Visualization on the GridDAS3 Symposium 3 Enabling Grids for E-sciencE...

transcript

Interactive Scientific Visualization on the Grid

Dieter Kranzlmüllerkranzlmueller@gup.jku.at

GUP, Joh. Kepler Univ. Linz, Austria

DAS3 Symposium 2

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Example: WISDOM• Grid-enabled drug discovery process for neglected

diseases– In silico docking

compute probability that potential drugs dock with target protein

– To speed up and reduce cost to develop new drugs

• WISDOM (World-wide In Silico Docking On Malaria)– First biomedical data challenge – 46 million ligands docked in 6 weeks

Target proteins from malaria parasiteMolecular docking applications: Autodock and FlexX~1 million virtual ligands selected

– 1TB of data produced – 1000 computers in 15 countries

Equivalent to 80 CPU years

• Significant results– Best hits to be re-ranked using Molecular Dynamics

DAS3 Symposium 3

Example: Avian Flu• Avian Flu H5N1

– H5 and N1 = proteins on virus surface

• Biological goal of data challenge– Study in silico the impact of selected point

mutations on the efficiency of existing drugs – Find new potential drugs

• Data challenge parameters:– 5 Grid projects: Auvergrid, BioinfoGrid, EGEE,

Embrace, TWGrid– 1 docking software: autodock– 8 conformations of the target (N1)– 300 000 selected compounds

>100 CPU years to dock all configurations on all compounds

• Timescale: – First contacts established: 1 March 2006– Data Challenge kick-off: 1 April 2006– Duration: 4 weeks

Credit: Y-T Wu

DAS3 Symposium 4

Example: Earthquakes• Seismic software application determines epicentre,

magnitude, mechanism

• Analysis of Indonesian earthquake (28 March 2005)– Seismic data within 12 hours after the earthquake– Solution found within 30 hours after earthquake occurred

10 times faster on the Grid than on local computers– Results

Not an aftershock of December 2004 earthquakeDifferent location (different part of fault line further south)Different mechanism

Rapid analysis of earthquakes important for relief efforts

Peru, June 23, 2001Mw=8.4

Sumatra, March 28, 2005Mw=8.5

DAS3 Symposium 5

Grid Applications

• World-wide In-Silico Docking on Malaria (WISDOM)

• Studies on Avian Flu (H5N1)

• Earthquake Analysis

The Grid is “only” another tool for scientists

to solve their scientific problems

D. Kranzlmüller DAS3 Symposium 6

Definition of Grids

Computational grids are ...– large-scale distributed computing

infrastructures, that– offer ubiquitous access to networked resources– for integrated and collaborative use by multiple

organizations

Most “valuable” resources: computing/storage

Services of the Grid

Grid middleware services (e.g. Globus Toolkit, …):– Security– Communication– Fault detection– Resource and data management– …

Batch processingThroughput computing

Grid Computing

3 Steps of current grid applications:

1. Submit job to the grid2. Receive results from the grid3. Analyze results Visualization

Example: Biomedical Investigation

1. Obtain data from medical scanner

2. Perform simulation on medical data on the grid

3. Analyze results from simulation

Influencing the Simulation

Change parameters and see how they influence the results of the simulation

Figure Courtesy University Amsterdam (UvA)

“Putting the User into the Loop”

„Into the Grid“

Workernode

WorkernodeGatekeeperGatekeeper

on the Grid

I need to solve a“big” problem

Job Submit

Assign Work

“Use the Grid”

Steering

Workernode

on the Grid

What is my program doing?

Steering

BatchSystem

Shell Access

Workernode

on the Grid

Why is myprogram broken?

Shell Access

Workernode

on the Grid

glogingloginInteractive Grid

Service

gloginglogin should …

• … enable online communication between nodes on the Grid and off the Grid

• … provide shell functionality for access to Grid nodes

• … be a standard (lightweight) Grid job• … be easy to install and use• … be secure

Initialize Connection

Workernode

GatekeeperGatekeeper

on the Grid

ClientClient

glogin Point ofContact

glogin’’socket

interactivebidirectional connection

gloginglogin Security

• Authentication using X509 certificates• Encryption of traffic (GSSAPI/rfc2744)• Session hijacking prevention• Specific or arbitrary host selection• Dynamic or static port selection (using

GLOBUS_TCP_PORT_RANGE)• Proxy-certificate delegation

Initialize Connection

Workernode

GatekeeperGatekeeper

on the Grid

ClientClient

glogin Point ofContact

glogin’’

interactivebidirectional connection

trafficforwarding

glogin Shell – Interactive access to grid nodes

• Authentication via grid certificates• Tunneling of arbitrary traffic

“Putting the User into the Loop”

gloginglogin

Grid Visualization Kernel

Goal:Visualization services for the grid

GVK = „Middleware-Extensionfor Visualization“

Objectives of GVK 1/2

Simulation ...• is decoupled from the visualization

(temporal/spatial)• is running „somewhere on the grid“• is a „moving target“ in a heterogeneous

environment

Objectives of GVK 2/2

Visualization ...• may be invoked

anytime, anywhere, anyhow• may be invoked by multiple, cooperating

people (multiplexing)• should be as fast/good as possible

(via any available network)

Exploitation of the Grid

Side effect:• Grid delivers computational power which

can be used for visualization.• Parallelization of visualization algorithms

– Triangle mesh partitioning (Metis)– Raytracing parallelization on pixel level– Reference image rendering on different nodes– ...

“Immediate” Goals of GVK

• Integration of GVK:– Interfaces for existing visualization toolkits– Visualization on different devices (CAVE, PDA, …)

• Visualization Pipeline:– Setup and processing of visualization data on the grid

• Network Transportation Optimizations: – Decrease communication latency– Increase system throughput

Integration of GVK

• GVK services are called from existing visualization toolkits

• GVK provides modules with grid-enabled functionality

• GVK transparently performs data conversion, multiplexing, ...

GVK Integration with existing tools

ExampleOpenDX

flow graph

GVK Integrationwith existing tools

ExampleOpenDX

flow graphusingGVK

Integration into Visualization Toolkits

Possibilities:• GVK as output device

(remote visualization)• GVK performs main part

(data input, visualization output)• Particular visualization modules are

replaced by GVK functionality

1. Obtain data from medical scanner

2. Perform simulation on medical data on the grid

3. Analyze results from simulation

• Parallel simulationof blood flowon the Grid

• Online visualizationof simulationresults on thedesktop

• Interactive steeringof simulation

• Grid is „invisible“

Cooperation with University Amsterdam

GVK Displayon different devices

• Simulation of floodingon the Grid

• Visualizationof results in the CAVE

• Grid is„invisible“

Cooperation with Slowak Academy of Sciences

VisualizationMapping

Rendering

Data EnrichmentEnhancement

Reduction

VisualizationPipeline

Simulation

OutputDevice

SimulationData

DerivedData

AbstractVis-Object

DisplayableImage

VisualizationPipeline

on the Grid

Simulation

OutputDevice

SimulationData

AbstractVis-Object

DisplayableImage

DerivedData

AbstractVis-Object

Simulation

OutputDevice

SimulationData

AbstractVis-Object

DisplayableImage

GVK Extension

Grid-enabled Video streaming• Generate video stream at data origin using

off-screen rendering and video capturingData remains where it is produced!

• Transport video stream to output device• Display video stream on output device• Manage interactive input on output device

GVid Extension to GVK

Workernode

GVidEncodeGVid

Encode

Workernode

GridVisualization

Kernel

GridVisualization

Kernel

on the Grid

ClientClient

glogin

glogin’video

streamvideo

stream WO

inter-action

interaction interaction

Example: GVid

Example: GVid Output Device

Sony Playstation Portable (PSP):• CPU: MIPS R-4000• Memory Stick PRO Duo (32 MB-1 GB)• Wi-Fi (802.11b)

• MPEG-4 VideoCodec

http://en.wikipedia.org/wiki/PlayStation_Portable

GVid Output on PSP

Level-of-Detail

33 tetrahedrons 264 tetrahedrons 36452 tetrahedrons

GVK Reductionof data transport

Occlusion culling

Conclusions

GVK …• … is a grid middleware extension

for interactive visualization• … can be used with different visualization

front-ends and output devices• … exploits the grid for increased

performance and includes various optimizations

Conclusions

• Today‘s grid services provide basic (low-level) functionality for grid applications

• Higher-level grid middleware services are needed for future applications

DAS3 Symposium 50

MiddlewareGlobus GT4 CondorAPST

PlatformInfrastructure Unix Windows JVM TCP/IP MPI .Net Runtime

Environmental Sciences

Life & Pharmaceutical

Sciences

ApplicationsGeo Sciences

Building Software for the Grid

VPN SSH

Courtesy IBM

Slide Courtesy David Abramson

DAS3 Symposium 51

MiddlewareGlobus GT4 CondorAPST

PlatformInfrastructure Unix Windows JVM TCP/IP MPI .Net Runtime

Environmental Sciences

Life & Pharmaceutical

Sciences

ApplicationsGeo Sciences

Building Software for the Grid

VPN SSH

Courtesy IBM,Lower Middleware

Upper Middleware & Tools

Slide Courtesy David Abramson

National Grid Initiativessupporting the EGI Vision

Conclusions

• Today‘s grid services provide basic (low-level) functionality for grid applications

• Higher-level grid middleware services are needed for future applications:– Interactive User Interfaces– Visualization Services

Increases demand for novel low-level grid services, too

GVK/GVid Future Work

• Additional application domains• Additional visualization output devices• Adaptive performance optimizations:

requires dedicated monitoring support• Enhanced Interactivity support• Cooperation and Collaboration support

within Virtual Organizations

TeamDieter Kranzlmüller,

Martin Polak, Thomas Köckerbauer,Paul Heinzlreiter, Herbert Rosmanith,

Hans-Peter Baumgartner, Peter Praxmarer,Andreas Wasserbauer, Gerhard Kurka,

Jens Volkert

More InformationGVK: http://www.gup.jku.at/gvkGVid: http://www.gup.jku.at/gvk

glogin: http://www.gup.jku.at/glogin

E-Mail: kranzlmueller@gup.jku.at

Interactive Scientific Visualization on the GridDAS3 Symposium 3 Enabling Grids for E-sciencE...

Documents