The gLite API – PART II

Giuseppe LA ROCCA INFN Cataniagiuseppe.larocca@ct.infn.it

Master Class for Life Science,4-6 May 2010 Singapore

W M P r o x y J a v a A P I ver 3.1.0

Software RequirementsJava Class org.glite.wms.wmproxy.WMProxyAPI

Support to cog-jglobus-1.7.0-binHands-on

Installation guide API JAVA /1

0. Start from a fresh install of Scientific Linux CERN– other platforms are also compatible but

yet not properly tested1. Axis (http://ws.apache.org/axis/) where the needed jars

available are: – axis-1.4-2006-04-22.jar– axis2-1.0-2006-05-05.jar– axis-jaxrpc-1.2-2005-05-03.jar– axis-wsdl4j-1.2-2005-04-12.jar

2. Geronimo J2EE libraries:– geronimo-activation_1.0.2_spec-1.1-2006-06-14.jar– geronimo-j2ee_1.4_spec-1.1-2006-06-14.jar

3. Crypto libraries:– cryptix-asn1.jar– cryptix32.jar– bcprov.jar– puretls.jar

4. gLite libraries:– glite-jdl-api-java.jar– glite-wms-wmproxy-api-java.jar– glite-security-delegation-java.jar– glite-security-trustmanager.jar– glite-security-util-java.ja

5. GLOBUS and other libraries:– cog-jglobus.jar– classad.jar– commons-discovery-0.2-2004-03-23.jar– commons-logging-api-1.1.jar– log4j-1.2.14.jar

W A R N I N G Make sure all the listed

jar(s) are included in your CLASSPATH

environment variable !!

Installation guide API JAVA /2

6. Before to start, please check if the following package(s) have been installed in your machine

– glite-wms-wmproxy-api-java-3.1.3-1.noarch– glite-wms-ui-api-java-3.1.1-1.i386.rpm– glite-jdl-api-java-3.1.2-1.i386.rpm– glite-security-util-java-1.4.0-1.noarch– glite-security-trustmanager-1.8.15-1.noarch– glite-security-delegation-java-1.4.1-1.noarch

• All the needed gLite packages can be found here http://eticssoft.web.cern.ch/eticssoft/repository/org.glite

• All the needed external packages can be found here http://eticssoft.web.cern.ch/eticssoft/repository/externals

Installation guide API JAVA /3

• The JDL to be matched must contains some mandatory Requirements and Rank attributes;

– when using command line interface these are added by default by the UI, but in our case user has to take care to check if his/her JDL file contains them:

Requirements = (other.GlueCEStateStatus == "Production");

Rank = (-other.GlueCEStateEstimatedResponseTime);

Basic Requirement

• The main Java class of this API is: org.glite.wms.wmproxy.WMProxyAPI;

• A client object can be created using one of these following constructor(s):

public WMProxyAPI(String url, String proxyFile)

public WMProxyAPI(String url, String proxyFile, String CAcerts)

public WMProxyAPI(String url, InputStream proxyStream)

public WMProxyAPI(String url, InputStream proxyStream, String CAcerts)

org.glite.wms.wmproxy.WMProxyAPI

Therefore, using one of the four constructors

WMProxyAPI Client = new WMProxyAPI( … );

a new instance is created. This instance will be created to invoke any other WMProxy services.

For example:

WMProxyAPI Client = new WMProxyAPI(“https://wms-gilda-01.ct.infn.it:7443/glite_wms_wmproxy_server”,“/tmp/x509up_u504”);

• Before calling submission or list match services, client proxy credentials are requested to be transferred from the client to the server through a delegation process.

• The following Java code performs this delegation process:

The obtained delegation identifier can be reused for several

calls to the WMProxy server;

Delegation of user credential

• The following Java code submits a job and returns a JobID

Submit a job to WMProxy

• Submission of simple jobs, DAGs, collections and parametric jobs to the WMProxy service requires as input a job description file in which job characteristics and requirements are expressed by means of Condor class-ad-like expressions.

• In this description the users can specify some files that are needed by the submitted jobs during the execution on the remote Computing Elements. These files are listed in the JDL InputSandbox attribute specifying for each of them the complete URI of its location.

• If the job does NOT have any file in the InputSandbox to be transferred from the submitting machine to the WMProxy node, the submission can be performed also calling the jobSubmit service:

jobId = client.jobSubmit(jdlString, delegationID);

Difference between submission and registration /1

Otherwise, these following steps are needed:

• a preliminary server registration:

jobId = client.jobRegister(jdlString, delegationID);

• transfer of files in the InputSandbox from the client machine to the WMS node;

• call the jobStart service to trigger the submission:

client.jobStart(jobId);

Difference between submission and registration /2

• In order to allow grid user to invoke, within Java code, the UrlCopy class to copy files from the WMS Server to the UI and vice versa the following Java package is requested

cog-jglobus-1.7.0-bin.tar.gz

The above package, with all the needed documentation, can be downloaded from here

For any further information visit the official web site

Software Requirements

Executable = "/bin/sh";Arguments = "start_hostname.sh";StdOutput = "hostname.out";StdError = "hostname.err";InputSandbox = {"start_hostname.sh"};OutputSandbox = {"hostname.err","hostname.out"};Requirements = (other.GlueCEStateStatus ==

“Production”);Rank = (-other.GlueCEStateEstimatedResponseTime);RetryCount = 3;

#!/bin/shsleep 5hostname -f

hostname.jdl

start_hostname.sh

$ java WMProxyGetProxyAndSubmit \ https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \hostname.jdl start_hostname.sh | grep -v "ERROR"

Connecting to the service https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server

====================== glite-wms-job-submission Success ======================

The job has been successfully submitted to the WMProxy Your job identifier is: https://gilda-lb.ct.infn.it:9000/P-H1wp0RpTnos2qGyQ2ETQ

===============================================================================

Submit a new request to WMProxy /1

Usage :

java WMProxyGetProxyAndSubmit -h[elp]

java WMProxyGetProxyAndSubmit <wmproxy_server> <jdl_file > [ <InputSandboxFiles> ]

where:

<wmproxy_server> ... the entry point of the WMProxy Server to contact

(e.g. : https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server)

(e.g. : https://gilda-wms-02.ct.infn.it:7443/glite_wms_wmproxy_server)

<jdl_file> ... the jdl file to submit to the grid

<InputSandboxFiles> ... The list of file(s) to transfer

The script, thanks to the UrlCopy Class, performs the copy of the InputSandbox files to the reduced path of the WMS as you can see:

cd /var/glite/SandboxDir/P-/ll https_3a_2f_2fgilda-wms-01.ct.infn.it_3a9000_2fP-H1wp0RpTnos2qGyQ2ETQ/input/

-rwxrwxr-x 1 gilda001 glite 30 Jan 11 09:05 start_hostname.sh

Submit a new request to WMProxy /2

When the job finishs you can retrieve the output file(s) as follow:

$ java WMProxyGetOutputAndPurge \ https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \ https://gilda-lb.ct.infn.it:9000/P-H1wp0RpTnos2qGyQ2ETQ \$PWD/ | grep -v "ERROR"

Connecting to the service https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-output Success ======================

JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://gilda-lb.ct.infn.it:9000/P-H1wp0RpTnos2qGyQ2ETQ have been successfully retrieved and stored in the directory: /home/larocca/Api-Java-WMProxy/ ===========================================================================

Retrieve output file(s)

The following Java code cancels a list of jobIDs

Cancelling a job

A job can be canceled before it ends using the following Java class:

$ java WMProxyCancelJobs \ https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \https://gilda-lb.ct.infn.it:9000/dwgBZqwoPKkl7v_OK6kiQw \| grep -v "ERROR"

Connecting to the service https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server======================== glite-wms-job-cancel Success =============================

The cancellation request has been successfully submitted for the following job(s):

- https://gilda-lb.ct.infn.it:9000/dwgBZqwoPKkl7v_OK6kiQw

====================================================================================

Cancelling a job

The following Java code returns a list of CEs available for job submission

Listing CE ID(s) that match a job

It is possible to see which CEs are eligible to run a job described by a given JDL using the following Java API:

$ java WMProxyGetProxyAndListMatch \ https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \hostname.jdl start_hostname.sh | grep -v "ERROR"

Connecting to the service https://gilda-lb.ct.infn.it:7443/glite_wms_wmproxy_server

COMPUTING ELEMENT IDs LIST ==========================================================================

The following CE(s) matching your job requirements have been found: - gilda-01.pd.infn.it:2119/jobmanager-lcgpbs-gilda - gn0.hpcc.sztaki.hu:2119/jobmanager-lcgpbs-gilda - grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite - grid010.ct.infn.it:2119/jobmanager-lcgpbs-long - grid010.ct.infn.it:2119/jobmanager-lcgpbs-short - grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-gilda - iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-infinite - iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-long - iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-short

- ce.hpc.iit.bme.hu:2119/jobmanager-lcgpbs-gilda==========================================================================

Listing CE ID(s) that match a job

Java API Documentation

http://trinity.datamat.it/projects/EGEE/wiki/apidoc/3.1/htmljava/index.html

Links and References

Datamat – WMProxy quickstart

http://trinity.datamat.it/projects/EGEE/wiki/wiki.php?n=WMProxyClient.QuickStart

JDL Attributes guide for WMProxyhttps://edms.cern.ch/document/590869/1

CoG jglobushttp://dev.globus.org/wiki/CoG_jglobus

Other Java API(s) in EUAsiaGrid wikihttp://www.euasiagrid.org/wiki/index.php/L%26B_Java_APIhttp://www.euasiagrid.org/wiki/index.php/Data_Management_Java_APIhttp://www.euasiagrid.org/wiki/index.php/MyProxy_Java_API

Hands-on

• Connect to the training infrastructure using the information reported in the tutorial sheet

• Run the hands-on available in this web link: http://www.euasiagrid.org/wiki/index.php/WMProxy_Java_API

User Interface: glite-tutor.ct.infn.itAccounts:

– Username: singapore02-singapore39– Password: GridSIN02-GridSIN39– PassPhrase: SINGAPORE

$ cd ${HOME}/Api-Java-WMProxyTo compile Java code run:$ source wmproxy.shSubmit and collect results with the two Java classes:WMProxyGetProxyAndSubmitWMProxyGetOutputAndPurge