All-in-one graphical tool for grid middleware management

Eddy Caron, Abdelkader Amar, Frédéric Desprez, David Loureiro

LIP ENS Lyon, INRIA Rhône-Alpes, GRAAL project

http://graal.ens-lyon.fr

Join work with A. Cedeyn, R. Bolze, Y. Caniou, A. Chis, B. Depardon, J.-S. Gay, G. Lemahec …

Summary

• Introduction• DIET short presentation• DIET DashBoard

DIET tools Workflow tools Grid tool : GRUDU

• Experiments• Conclusion and Future Work

• Context : Development of a toolbox for deploying ap-plication services providers with a hierarchical archi-tecture for scalability

LA

MA

LA

LALA

Server front end

Master Agent

Local Agent

Client

MA

MA

MA

MA

Corba or JXTA

http://graal.ens-lyon.fr/DIET

• Main research issues: scheduling, heterogeneity, automatic deployment, interoperability, high performance data transfer and management, monitoring, fault tolerance, genericity of solutions for various applications, static and dynamic analysis of performance, …

• Validation: Large validation over Grid’5000.• Interoperability: DIET is compliant with the Grid-RPC

standardization for OGF • DIET used case: The Decrypthon project

- DIET was selected by IBM - • Collaborations: RNTL GASP, ACI GRID ASP, TLSE,

ACI MD GDS, ANR LEGO, ANR GWENDIA, Grid’5000

DIET Dashboard

• DIET hierarchies can be deployed on grid/cluster of nodes• In Grid Environments, users need several and complex tools for the

management of resources and client/server applications.• Can be tougth to deploy a DIET hierarchy in a such a grid environment

by hand without high level tools

• The DIET DashBoard is a set of tools written in Java

• Provides to the DIET end-user, friendly-user interfaces to design, deploy and monitor the execution of client/server applications

• Also provides to the grid user tools for the allocation and monitoring resources on Grid'5000

DIET Dashboard

• Extensible set of tools for the DIET community

• Based on seven tools: DIET designer DIET Mapping tool DIET Deployment tool XML GoDIETGenerator Workflow designer Workflow log service DIET resource tool aka GRUDU

DIET Tools

Workflow Tools

Grid Tools

DIET Tools

Allows the user to design graphically a DIET hierarchy.

Allow the user to map DIET components on the allocated Grid'5000 resources

To help the end-user creating hierarchies from

existing frameworks based on the reserved

ressources.

It provides the basic GoDIET operations: open, launch, stop and monitoring for DIET application elements.

Workflow Tools

• Applications viewed as services within DIET• Compose services to get a complete application workflow in a drag’&’drop fashion

To monitor workflows execution by displaying the DAG nodes of each workflow and their states. Three states are available: waiting, running and done. Two modes:local and remote

• Context : Grid'5000 a highly reconfigurable, controlable and monitorable experimental Grid platform. Needed all-in-one GUI client-side tool for the monitoring of the Grid'5000 platform.

Grid'5000 Reservation Utility for Deployment Grid'5000 Reservation Utility for Deployment UsageUsage

• Main goals :

– Displaying the status of the platform (grid/site/job level)

– Resources allocation through the use of OAR (oar v1 & v2 are supported)

– Resources monitoring through the use of the Ganglia (site/job nodes)

– Deployment management with a GUI for KaDeploy (multiple sites at a time)

– A terminal emulator (access frontale/ site frontale/job main node connection)

– and a file transfer manager (local/remote and synchronization features)

• Web: http://graal.ens-lyon.fr/GRUDU

Exp 1: Goals and Protocol of the Experiment

• Validation of the DIET architecture at large scale over different administrative domains

• Protocol DIET deployment over a maximum of processors Large number of clients Comparison of the DIET execution times with average local execution times 1 MA, 8 LA, 540 SeDs 2 requests/SeD 1120 clients on 140 machines DGEMM requests (2000x2000 matrices) Simple round-robin scheduling using

time_since_last_solve

Grid’5000

Paravent : 9 s

Lilles : 34 s Paraci : 11 s

Bordeaux : 33 s Parasol : 33 s

Sophia : 40 s Toulouse : 33 s

Lyon : 38 s Orsay : 40 s

Exp 1 : Results Grid’5000

Exp 2: Goals and Protocol of the Experiment• Validation of the DIET architecture at large scale over different

administrative domains in the framework of the LEGO project (ANR CICG05-11)

Grid’5000

Protocol Get the largest machine reservation

on the largest number of Grid'5000 clusters

Realize a maximum number of Ramses execution (application developed at the DAPNIA/CEA)

Stress DIET over a large number of machine and in a large period of time

But also stress Grid'5000 ... KaDeploy image with DIET and all

the mandatory tools 12 clusters on 7 sites : 979

machines for 48 hours 1 MA, 12 LA, 29 SeDs 816 machines Ramses-only

Exp 2 : Results

• Comparing DAPNIA/CEA to DIET/DIET DashBoard/Grid'5000

DAPNIA/CEA Grid'5000Simulation time (h) 4464 (6 months) 28Processor number 32 1824Simulation number 50 33

Simulation Speed (simu/h) 0,0112 1,18

Ramses Example Output

Conclusion and Future Work

• Conclusion : DIET is a grid middleware designed for scheduling application tasks

with a hierarchical architecture The DIET DashBoard provides to DIET users a full-featured framework

for experiments on Grid'5000 GRUDU provides to the Grid'5000 community a powerful tool

dedicated to the interaction with the grid (monitoring/reservation/deployment etc.)

The experiments show how DIET DashBoard/GRUDU can simplify the use of a grid middleware

• Future work: Support of other Batch Schedulers (like LoadLeveler or SGE) Interaction with other grids (EGEE, DAS3, etc.)

Questions ...

Questions ?

Demoat the Exhibition Hall

16:00 - 20:00