ANR CIGC LEGO (ANR-CICG-05-11)Bordeaux, 2006, December 11th
Automatic Application Deployment on Grids
Landry Breuil, Boris Daix,Sébastien Lacour, Christian Pérez
PARIS Research TeamINRIA/IRISA
Rennes
How to Deploy my Applicationon Grid Resources?
Visualization
Homogeneous cluster
SAN
SAN
Homogeneous cluster
LAN
WAN
Supercomputer
Flowcode
Transport code
Controller
velocity
scalarsscalars
concentration
Manual Deployment
Discover available grid resources Select grid resources for execution
OS, architecture compatibility Map the application onto the selected
resources MPI processes Components
Select compatible compiled executables Upload and install executables, stage input
files in Launch processes on remote computers Set configuration parameters of the application
Components' attributes Network topology information
Too complex!
Automatic Deployment Automatic
Resource discovery Execution node selection File installation Process launch Application configuration
Hide application complexity
Hide grid complexity
stop reading your e-mails!
Grid
Generic Application Description
Translator From specific to generic application description Straightforward to write
CCM Appl. Descriptio
n
MPI Appl. Descriptio
n
GridCCM Appl.
Description
CCM Appl. Descriptio
n
MPI Appl. Descriptio
n
GridCCM Appl.
Description
CCM Planner
MPI Planner
GridCCM Planner
Plan Execution
Plan Execution
Plan Execution
Generic Application Description
Deployment Planner
Deployment Plan Execution
Identification of the Stepsof Automatic Deployment
MPI Application Description
CCM Application Description
Resource Description
Generic Application Description
Control Parameters
Deployment Planning
Deployment Plan Execution
Application Configuration
Stat
ic
Applic
atio
ns
Deployme
nt Tool
ADAGE Feature List MPI, JXTA, CCM application descriptions Network topology description Generic application description Simple control parameters, simple planner Deployment plan execution (RSH/SSH) Basic file transfer support
Aware of file visibility (~NFS) Does not check for already available files
Redeployment support Either re-play or add/remove elements
How Easy is it, in Practice?
As simple as A-B-C adage-deploy –resource http://host/file.xml
–application my_appl.zip–ctrl_param
my_control_parameters Grid resource description
Written once for all by grid admins Optional control_parameters constraints
Keep control on the deployment process
ADAGE & Grid5000
ADAGE is not able to directly talk with OAR PERL script to automatically generate
resource description from OARgrid id oarstat2resources.pl –g 6332 –outputfile
r.xml adage-deploy –inres r.xml …
Some ADAGE results JXTA
1 peer == 1 process M. Jan deployed
~ 29.000 peers on ~ 145 machines in ~ 95 seconds
Most of the time take by ssh commands
CCM 1 component == 1 process H. Bouziane deployed
4000 components on 974 processors machines 432 machines on several G5K sites
Enable to CCM-plugin optimization
Towards ADAGE v2
Complete rewrite of ADAGE By L. Breuil
Objectives of ADAGE v2 Provide a clean code architecture Ease the additions of
Programming model (GridCCM), ie plugins Planner
Better back-end support GAT, Taktuk, …
Support of “dynamic” applications In cooperation with B. Daix, PhD
Discussion ADAGE: a model to support automatic application
deployment Ongoing/Future work
Finish ADAGE v2 (~ begining 2007 ) Stabilize plugins API Stabilize planner API wrt to resources Dynamic application management PadicoTM support? Adapt application description in function of resources
DIET support Not directly targeted
Fault tolerance Application monitoring
