© Geodise Project, University of Southampton, 2003.

Geodise Middleware& Optimisation

Graeme Pound, Hakki Eres, Gang Xue

& Matthew Fairman

Summer 2003

© Geodise Project, University of Southampton, 2003.

Computational Toolbox

• Access to Globus compute resources from Matlab

• Proxy certificate management, job submission and file transfer

• Complex grid-enabled workflows may be coordinated from Matlab

• Uses the Java CoG 1.1

© Geodise Project, University of Southampton, 2003.

Computational Toolbox

gd_createproxy.m Creates a Globus proxy certificate for the user's credentials

gd_destroyproxy.m Destroys the local copy of the user's Globus proxy certificate

gd_certinfo.m Returns information about the user's certificate

gd_proxyinfo.m Returns information about the user's proxy certificate

gd_proxyquery.m Queries whether a valid proxy certificate exists

gd_jobsubmit.m Submits a compute job to a Globus GRAM job manager

gd_jobstatus.m Gets the status of a Globus GRAM job

gd_jobpoll.m Queries the status of a Globus GRAM job until complete

gd_jobkill.m Kills a Globus GRAM specified by job handle

gd_putfile.m Puts a remote file using GridFtp

gd_getfile.m Retrieves a remote file using GridFtp

gd_rmfile.m Deletes a remote file using GridFtp

gd_makedir.m Creates a remote directory using GridFtp

gd_rmdir.m Deletes a remote directory using GridFtp

© Geodise Project, University of Southampton, 2003.

Jython Computational Toolbox

• Python interface to functionality of Matlab Computational Toolbox

• Jython is a 100% Java implementation of the Python scripting language

• Lightweight alternative to the Matlab Computational Toolbox

© Geodise Project, University of Southampton, 2003.

Job Submission Service Client

• Matlab client to job submission web service • User detached from compute endpoint – Condor pool• Machines publish available resources:

– HasProEngineer = TRUE– ProEngineerVersion = "2001400"– ProEngineerPath = "C:\Program Files\proe2001\bin

• DIME file transfer over http• MATLAB Client Functions:

– grid_platform – Describes the platform requirements of the job– grid_submit – Submits the job to the web service, returns a job ID– grid_poll – Polls job ID– grid_status – Queries job status– grid_results – Retrieves the output files

© Geodise Project, University of Southampton, 2003.

OptionsMatlab

• Matlab interface to the Options design exploration system

• State of the art design search and optimisation algorithms

• Objective and constraint functions exposed as Matlab functions

• Grid-enabled job brokers easily incorporated

© Geodise Project, University of Southampton, 2003.

OptionsMatlab Parallel Architecture

• optjob.m allows evaluation of multiple design points

• Concurrent evaluation of optfun.m and optcon.m using Objective Function Service Client

• Single evaluation via optjob.m from the optfun and optcon subroutines

Matlab environment

OptionsMatlab.dll

optjob.m

optjob optfun optcon

optfun.moptcon.m

© Geodise Project, University of Southampton, 2003.

Objective Function Client

• Grid-enabled objective function evaluation• Enables multiple concurrent evaluations• Matlab functions provided:

– gd_objsubmit - Transfers required files, submits job and returns handle

– gd_objvalue - Retrieves function evaluation, prints standard output of failed jobs and cleans up files

– gd_objcleanup - Removes remote files of specified jobs

• Multiple Matlab processes, one Matlab licence • Configuration file defines available objective function

servers

© Geodise Project, University of Southampton, 2003.

Objective Function Client

Matlab

Geodisefile archive

Globus server

gd_archive.m

gd_objsubmit.m

gd_jobpoll.m

gd_objvalue.m

Matlab

objfun.m

1

2

3

4

5

© Geodise Project, University of Southampton, 2003.

A Level 2 Grid Application

• The aim is to find enhanced vibration isolation performance for satellite designs

• Matlab is the scripting environment• Geodise toolkit is utilized• The GA toolbox for Matlab is used for

optimization• UK e-Science Grid resources used

– Cambridge e-Science Centre– CLRC e-Science Centre– London e-Science Centre– Oxford e-Science Centre– Southampton e-Science Centre– Southampton Information Systems Services

© Geodise Project, University of Southampton, 2003.

Parallel Job Submission (11x11)

© Geodise Project, University of Southampton, 2003.

Results (Sample points = 11x11)

© Geodise Project, University of Southampton, 2003.

Conclusions

• Submitting compute jobs is the easy bit.• Complexity of grid computing includes:

– applying for authorisation at a numerous locations– managing the transfer of files– keeping track of heterogeneous compute resources

• Requirements for:– Grid enabled data management– Transparent Grid-enabled tools– Consistent computing environments– Reservation and co-allocation of resources– Grid debugging