Integration of Multidiscipline Applications in Grid-computing Environments

transcript

Integration of Multidiscipline Applications Integration of Multidiscipline Applications in Grid-computing Environmentsin Grid-computing Environments

NGUYEN G.T., J. BLACHON, C. PLUMEJEAUD

PARA’02, Espoo, June 16th, 2002

« OPALE » PROJECT

Conference on Applied Parallel Computing

• Topics

• Located Sophia-Antipolis & Grenoble

• Follow up SINUS project

• New INRIA project since January 1st, 2002

OPALEOPALE

NUMERIC OPTIMISATION (genetic, hybrid, …)

MODEL REDUCTION (hierarchic, multi-grids, …)

INTEGRATION PLATFORMS

Coupling, distribution, parallelism, grids, clusters, ...

APPLICATIONS : aerospace, electromagnetics, …

• APPLICATION DEVELOPER ’S POINT OF VIEW

• SOFTWARE ENGINEERING POINT OF VIEW

• USER POINT OF VIEW

• CONCLUSIONS

OVERVIEWOVERVIEW

• THEORETICAL POINT OF VIEW

• DEVELOPER ’S POINT OF VIEW

• CONCLUSIONS

OVERVIEWOVERVIEW

WHERE WE ARE TODAYWHERE WE ARE TODAY

• 1980 : one year CPU time

• 1992 : one month « »

• 1997 : four days « »

• 2002 : one hour « »

• ASCI White (LLNL) : 8192 IBM SP procs

• ASCI Red (Sandia) : 9632 Intel procs

• ASCI Blue Mountain (LANL) : 6144 SGI procs

Bits and pieces….

TEST CASETEST CASEWING PROFILE OPTIMISATION

TEST CASETEST CASE

• SHOCK-WAVE INDUCED DRAG REDUCTION

• WING PROFILE OPTIMISATION (RAE2822)

• Euler eqns (0,84 Mach, i = 2°) + BCGA (100 gen.)

• 2D MESH : 14747 nodes, 29054 triangles

• 4.5 hours CPU time (SUN Micro SPARC 5, Solaris 2.5)

• 2.5 minutes CPU time (PC cluster 40 bi-procs PIII, Linux)

““CAST” INTEGRATION PLATFORMCAST” INTEGRATION PLATFORM

• TEST CASES IMPLEMENTATION

• “DECISION” CORBA INTEGRATION PLATFORM

• DESIGN FUTURE HPCN OPTIMISATION PLATFORMS

COLLABORATIVE MULTI-DISCIPLINE OPTIMISATION

GENETIC & PARALLEL OPTIMISATION ALGORITHMS

CODE COUPLING FOR CFD, CSM SOLVERS & OPTIMISERS

COLLABORATIVE APPLICATIONS SPECIFICATION TOOL

The front stage….

CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORMUser interface

• CONCLUSIONS

OVERVIEWOVERVIEW

• DISTRIBUTED : LAN, WAN, HSN...

• CODE-COUPLING FOR HETEROGENEOUS SOFTWARE

• COLLABORATIVE APPLICATIONS

• COMMON DEFINITION, CONFIGURATION, DEPLOYMENT, EXECUTION & MONITORING ENVIRONMENTS

• TARGET HARDWARE : NOW, COW, PC-clusters, grids, ...

• TARGET APPLICATIONS : multidiscipline engineering, ...

INTEGRATIONINTEGRATION PLATFORMSPLATFORMS

Distributed tasks interacting dynamically in controlled and formally provable way

What they are...

DISTRIBUTED SIMULATION

• MULTI-DISCIPLINE PROBLEM SOLVING ENVIRONMENTS

• HIGH-PERFORMANCE & TRANSPARENT DISTRIBUTION

• USING CURRENT COMMUNICATION STANDARDS

• USING CURRENT PROGRAMMING STANDARDS

• WEB LEVEL USER INTERFACES

• OPTIMIZED LOAD BALANCING & COMMUNICATION FLOW

What is required...

DESIGN ALTERNATIVESDESIGN ALTERNATIVES

• HARWARE & SOFTWARE ENVIRONMENTS

• EXISTING PLATFORMS

• LEGACY APPLICATION SOFTWARE

• PROBLEM REQUIREMENTS

Optimize specific pbs & solutions : ReMAP

System evolution & development : PARIS

Globus, Condor, NetSOLVE, Legion, ….

How to integrate them into new PSE ?

INRIA PROJECTS ALTERNATIVESINRIA PROJECTS ALTERNATIVES

• HARWARE & SOFTWARE ENVIRONMENTS

• OTHER EXISTING PLATFORMS

• LEGACY & NEW APPLICATION SOFTWARE

• PROBLEM REQUIREMENTS

Optimize specific pbs : ReMAP, ATHAPASCAN

System development : PARIS, OASIS

Globus, Condor, NetSOLVE, Legion, ….

How to integrate them into new PSE ?

• VISUAL PROGRAMMING

• COMPONENTS PROGRAMMING

• OBJECT-ORIENTED TECHNOLOGY

ADVANCES IN SOFTWAREADVANCES IN SOFTWARE

PROGRAMMING : C++, JAVA, C#, ...

APPLICATION MODELING : UML

REUSABILITY

MODULARITYINTEROPERABILITY

DISTRIBUTED OBJECT ARCHITECTUREDISTRIBUTED OBJECT ARCHITECTURE

• TRANSPARENT DISTRIBUTED OBJECT COMPUTING

• CORBA COMPLIANT

• SIMPLE SOFTWARE MODEL

• COMPONENTS PLUG-IN (e.g., optimizers, solvers)

- COMPONENTS

- CONNECTORS

DISTRIBUTED OBJECTS ARCHITECTUREDISTRIBUTED OBJECTS ARCHITECTURE

SOFTWARE COMPONENTS

• COMPONENTS ARE DISTRIBUTED OBJECTS

• WRAPPERS AUTOMATICALLY (?) GENERATED

• COMPONENTS ENCAPSULATE CODES

• DISTRIBUTED PLUG & PLAY

CAST PROTOTYPECAST PROTOTYPE

CAST OPTIMIZERS

SOLVERS

Server Wrapper Wrapper

Modules Modules

SOFTWARE COMPONENTSSOFTWARE COMPONENTS

• BUSINESS COMPONENTS

LEGACY SOFTWARE

• OBJECT-ORIENTED COMPONENTS

• DISTRIBUTED OBJECTS COMPONENTS

• METACOMPUTING COMPONENTS ?

C++, PACKAGES, ...

Java RMI, EJB, CCM, ...

DISTRIBUTED OBJECT ARCHITECTUREDISTRIBUTED OBJECT ARCHITECTURESOFTWARE CONNECTORS

• CONNECTORS ARE SYNCHRONISATION CHANNELS

• SEVERAL PROTOCOLS

• CONNECTORS = DATA COMMUNICATION CHANNELS

- SYNCHRONOUS METHOD INVOCATION

- ASYNCHRONOUS EVENT BROADCAST

• COMPONENTS COMMUNICATE THROUGH SOFTWARE CONNECTORS

• APPLICATION DEVELOPER POINT OF VIEW

• CONCLUSIONS

OVERVIEWOVERVIEW

• SUPPORT FOR NEW APPROACHES

• // SOFTWARE LIBRARIES : MPI, PVM, SciLab //, ...

• PARALLEL and/or DISTRIBUTED HARDWARE

• SUPPORT SEVERAL DEGREES PARALLELISM

PARALLEL APPLICATIONSPARALLEL APPLICATIONS

DOMAIN DECOMPOSITION

GENETIC ALGORITHMS

GAME THEORY

HIERARCHIC MULTI-GRIDS

The good news….

• Lays the ground for GRID and METACOMPUTING

• PC & Multiprocs CLUSTERS : thousands GHz procs...

• HIGH-SPEED NETWORKS : ATM, FIBER OPTICS...

ADVANCES IN HARDWAREADVANCES IN HARDWARE

GLOBUS, LEGION

CONDOR, NETSOLVE

Gigabits/sec networks available (2.5, 10, …)

The best news….

CLUSTER COMPUTINGCLUSTER COMPUTINGPC-cluster at INRIA Rhône-Alpes (216 Linux Pentium III procs.)

CLUSTER COMPUTINGCLUSTER COMPUTINGPC-cluster at INRIA Rhône-Alpes (216 Pentium III procs.)

-200-1000100200300400500600

Volet sortiBec sorti

Corps principal

AIRFOIL OPTIMISATIONAIRFOIL OPTIMISATION

The results...

CLUSTER COMPUTINGCLUSTER COMPUTINGPC-cluster at INRIA Rhône-Alpes

Multi-airfoil optimization : game theory + multi-grids hierarchic algo.

CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORM

RENNES

GRENOBLE

PC cluster

PC clustern CFD solvers

GA optimiser

PC clustersoftware

VTHD Gbits/s network

GRID computing...

July 2001...

Check for syntaxe of request

Event channell,i1, i2, i3, ….

Algogen.idl

AlgoGeni1,i2, i3, …, in

CfdSolvercfd1

CfdSolver cfd2

CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORMBehind the stage, again...

Event channel,i1, i2, i3, …, in

CfdSolverCfd1

ProcessorP0

ProcessorP1

ProcessorP3

ProcessorP2

CfdSolverCfd2

ProcessorP0

ProcessorP1

ProcessorP3

ProcessorP2

CfdSolverCfd3

ProcessorP0

ProcessorP1

ProcessorP3

ProcessorP2

Genetic Algorithm

i1, i2 ,i3, …, in

Parallelized with MPI on p processors

Genetic algorithm based on selection, mutation, crossover

CORBA server implemented in C++

CORBA client implemented in C++

THREE LEVELS of PARALLELISMTHREE LEVELS of PARALLELISM

Ag2DWithCorba

0100200300400500600700800

1 2 3 4 5 6 7

Nb CfdSolvers

s) SOPHIA

RENNES

GRENOBLE

CfdSolvers at Sophia, CAST at Grenoble

0 200 400 600 800 1000

Ag atGrenoble

Ag atSophia

Ag atRennes

Time (s)

* Curves quasi-parallels

=> same speed up, whatever the place.

* Join an horizontal asymptote:

time = 200 s

CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORM

The game : load balancing,...

• CONCLUSIONS

OVERVIEWOVERVIEW

BCGA FUN

InitB HYBRID

PROCESS FORMULAEPROCESS FORMULAEMILNER ’S SCCS PROCESS ALGEBRA

InitBCGA:InitHybrid:BGGA:(TRUE:(END)+FALSE:(FUN:(TRUE:(HYBRID:

(TRUE: (=>InitHybrid)+FALSE:(=>FUN)))+FALSE:(=>BGGA))))

OPERATORSOPERATORS

• SYNCHRONIZATION

• PARALLEL EXECUTION

• SERIAL EXECUTION

• ITERATIONS

• COMPLEX EXPRESSIONS : process formulae

• CHOICE

IC simulation : several coupled models

STRONG POINTSSTRONG POINTS

• STRONG THEORETICAL FOUNDATIONS

• SPECIFICATION & VERIFICATION OF COMPLEX APPS

Process algebra for asynchronous systems

• FORMAL SPECIFICATION SYSTEM

• EASY TO USE

Intuitive interface : simple component modelNo theoretical background knowledge requiredTransparent distribution using CORBA

Milner ’s SCCS algebra

• CONCLUSIONS

OVERVIEWOVERVIEW

• GRID COMPUTING

• DISTRIBUTED INTEGRATION PLATFORMS

• MULTIDISCIPLINE SIMULATION

TODAY ’S FUTURETODAY ’S FUTURE

e.g., DIGITAL DYNAMIC AIRCRAFT

CAST, JACO3, CCAT, ProACTIVE ...

GLOBUS, LEGION, CONDOR, ...

• DYNAMIC LOAD BALANCING & RESSOURCE ALLOC

• « COTS » PROGRAMMING

• METACOMPUTING

TOMORROW’S FUTURETOMORROW’S FUTURE

COMPONENTS OFF THE SHELF

POWER SUPPLY PARADIGM APPLIED TOCOMPUTING RESOURCES WORLDWIDE

Behind the stage, again...

OBSERVE, START, SUSPEND, RESUME, STOP, MIGRATE

REMOTE PROCESSES DYNAMICALLY

CONCLUSIONCONCLUSION

• INTEGRATION PLATFORMS PROVIDE

• GRID COMPUTING

• FULLY CORBA COMPLIANT

• ALSO ALLOWS CORBA & non-CORBA COMPONENTS

• SMOOTH TRANSITION FROM EXISTING CODE-COUPLING ENVIRONMENTS

DEFINE, CONFIGURE, DEPLOY, EXECUTE & MONITORCOLLABORATIVE APPLICATIONS

• ALLOWS SEQUENTIAL & PARALLEL COMPONENTS

DOCUMENTATIONDOCUMENTATION

• http://www.inrialpes.fr/opale

Toan.Nguyen@inrialpes.fr

• http://cast.sourceforge.net/manuel

Integration of Multidiscipline Applications in Grid-computing Environments

Documents