WS-VLAM Tutorial: Part III (part3): Learn how to WS-VLAM JAVA API

Adam Belloum

Outline

• Part I: Hands on the User Graphical Interface• Part III: Port a Legacy application to WS-VLAM• Part III: Learn how to use WS-VLAM API

• Notation (Title background):– Generic step – Tutorial specific step

• Part I: Hands on the User Graphical Interface• Part III: Port a Legacy application to WS-VLAM• Part III: Learn how to use WS-VLAM API

Outline (Part III): Learn how to use WS-VLAM API

Step 1: Define an Interface for your componentStep 2: Create a description for your componentStep 3: Define your component dependencies Step 4: JAVA Tutorial (create a WS-VLAM JAVA components)

Step 6: C/C++ Tutorial (create a WS-VLAM C/C++ components)

Step 5: PYTHON Tutorial (create a WS-VLAM PYTHON components)

Step 6: Deploy the component/dependencies libStep 7: Compose a workflow using the componentStep 8: execute the workflow / monitor

Step 1: Define the component interface

Step 1: Example File Reader

• The file reader “fileReader” opens a file specified as a URI using the parameter Interface “filename” and sends the content through an output port “port_out”.

– fileReader support many protocols: GSIFTP, ftp, http, file

Step 2.1: Create a description

• To create a description of your component: use CLAM workflow component configuration tool

Action: • Click on CLAM

Button in the WS-VLAM menubar

Step 2.2: Create a description

• CLAM workflow component configuration tool allows you to define: the name of the component, default host, ports, parameters, classification, and category

Action: • Enter the name of the

component

• Entre the name of the default host “fs2.das3.science.uva.nl”

Step 2.3: Create a description

• CLAM workflow component configuration tool allows you to define: the name of the component, default host, ports, parameters, classification, and category

Action: • Click on ports tab

• Click + to add new input or output port

• Type the name of the ports exactly as you will define them the source code.

Step 2.4: Create a description

• To add a port a pop up window appears “Add input Port” or “Add output port”

Action: • Type the name of the

ports exactly as you did it in source code

• Specify the type of the port either a simple data type, a file type, or Complex data type (see documentation for more details)

Step 2.5: Create a description

• Create a description of your component: use CLAM workflow component configuration tool

Action: • Click on save to

created the description file “component-name.xml”

Step 2.6: Create a description

• Create a description of your component: use CLAM workflow component configuration tool

Action: • You will see in the

filed file path the full path to the create

Component-name

Step 2.7: Example File Reader

Action: • Perform the tasks described in step 2.1 to step 2.6 to

create the fileReader.xml

Step 3.1: Define the component dependencies

Step 3.2: Create a pseudo-module

# ############ NOTE ##################### In the profile.sh export all the environment # variables needed to execute your package## bash environment# # export VARIABLE_NAME=VARIABLE_VALUE## Add your VARIABLES bellow the line######################################

Step 4.1: YOUR JAVA Module

• From a developer’s point of view your components are applications that use a special library provided by the WS-VLAM framework: the “vlport” library

Step 4.2: JAVA modules (pre-requisite)

Step 4.3: Extend your JAVA code

Step 4.4: API for reading from input port

Step 4.4: API for reading from input port

Step 4.5: API for writing to output port

Step 4.5: API for writing to output port

Step 4.6: API for reading/writing to/from parameter

• getParameter (String param−id)– Reads the value of “param−id”

• setParameter (String param−id, String para−value) – Writes the value of “para−value” to “param−id”

Step 4.7: JAVA component• XXXXXXXX:

// YOU MUST IMPORT THE JWRAPPER PACKAGEimport nl.wtcw.vlamg.jwrapper. ;∗

VLOPort outputPortRef = null;VLIPort inputPortRef = null; … // IT IS ASSUMED THAT REFERENCES TO THE PORTS ARE ALREADY TAKEN try {

double rndn = 12345.12345; // WRITING TO THE WS −VLAM Output Port outputPortRef.writeDouble(rndn); ...

// READ FROM THE WS −VLAM INPUT PORT rndn = inputPortRef.readDouble(); ...

} catch(VLAPPException e){

// HANDLE EXCEPTION ... } ...

Step 4.8: JAVA code of the File Reader• The Example:

import nl.wtcw.vlamg.jvlport.VLAppFactory;import nl.wtcw.vlamg.jvlport.Vlport;import nl.wtcw.vlamg.jvlport.OutputStream;

import java.io.*;

/** * Created by IntelliJ IDEA. * User: dvasunin * Date: Jul 29, 2005 * Time: 3:49:34 PM * To change this template use File | Settings | File Templates. */public class JFileReader { static public void main(String args[]){ String line = null; String[] outputPorts; try { outputPorts = new String[] {"port_out"}; Vlport vlport = (vlport.getOutputPort("port_out")));

VLAppFactory.activate(null, outputPorts); PrintWriter pw = new PrintWriter( new OutputStreamBufferedReader reader = new BufferedReader( new FileReader("/etc/passwd")); while((line = reader.readLine()) != null){ System.out.println(line); pw.println(line); } pw.flush(); } catch (IOException e) { e.printStackTrace(); } finally{ VLAppFactory.deactivate(); System.exit(0); } }}

Step 4.9: build your JAVA component• build.xml for FileReader ( ½ )

<project name="WSVLAM-java-modules" default="dist" basedir="."><description>

Project : VLE-wsvlamComponent : JFileReader source

Info : Ant file to call 'make'. Builds wsvlam java modules.

This component can be used as standalone. In that case the Makefile should be called directly.

</description>

<path id="compile.classpath"/> <property name="src.home" value="${basedir}/src"/>

<target name="init" description="making build directory"> <mkdir dir="build/java"/></target>

Step 4.9: build your JAVA component• build.xml for FileReader (2/2)

<target name="build" depends="init" description="compile the source "> <!-- Compile the java code from the srclist } into ${build} --> <javac srcdir="src" destdir="build/java"> <classpath refid="compile.classpath"/> </javac> </target>

<!-- use custom 'invoke' target to pass build parameters & environment -->

<target name="dist" depends="build" description="generate the distribution"><copy todir="dist/JFileReader/bin"><fileset dir="${basedir}/build/java" excludes="**/*.java"/>

</copy><copy todir="dist/JFileReader">

<fileset dir="${basedir}" includes="JFileReader.xml,main.sh"/></copy>

</target></project>

Step 4.10: package your FileReader

(1) Compile the fileReader

$ cd fileReader$ ant

# this will created java classes in # fitleReader/bin

(2) Create a tar file of your component:

$ tar cf fileReader.tar fileReader

(3) Compress the tar file you have created

$ gzip fileReader.tar

Step 5: Deploy the fileReader

• Test mode: you can test your component

– Perform the steps described in the blue box

• Shared mode: you can share your module with other users:

– by moving the tar file you have created in step 4 to a component repository

# On a machine where both GT4 & WS-VLAM# runtime system are installed # Create in your HOME directory called “.wsvlam”

(1) save fileReader.tar.gz to the default location

$ mkdir .wsvlam$ cd .wsvlam/$ mkdir modules$ cd modules

#untar the component tar file you have created # in step 4(2) untar fileReader.tar.gz

$ tar –zxvf fileReader.gz

Step 6: Complete examples in JAVA components

Step ?: Template for component in JAVA

• Is this needed

• ????????

• Dmitry do we still need this??????

<?xml version="1.0" encoding="ISO-8859-1" ?> <config> <ports> <!-- define here the list of input and output ports --> </ports> <java PropertyName="PropertyValue">

<!-- Here we can define System properties for jvm --> <!-- Here an entry point to your program is defined --> <main> your-java-main-class </main>

<options> <!-- Here we define additional options to jvm --> <option> -server </option> </options>

<!-- optional classpath --> <classpath> /path/to/something.jar </classpath> </java> </config>

Step 6.1: Compose a workflow fileReader

• In the component list window you will see the created FileREader components as well as other components

Action: • Click on the

fileReader in the list and drag and drop in the composition panel

Step 6.2: Compose a workflow using LA

• An instance of fileReader will be instantiated

Action: • Click on the fileReader

in the list and drag and drop in the composition panel

Step 6.3: Compose a workflow using LA

• Instantiate the fileWriter component:• This component will move the file produced by fileReader

to any remote location the user specifies

Step 6.4: Compose a workflow using LA

• Create a communication channel:• This component will move the file produced by FileReader

to any remote location the user specifies

Step 7.1: customize the composed workflow

• In the step you should check if all the default values of the parameters of the workflow components composing your workflow are set correctly

Step 7.1: customize the composed workflow

• In the step you should check if all the default values of the parameters of the workflow components composing your workflow are set correctly

Step 7.1: customize the composed workflow

• Input file: fileReader parameter

• fs2.das2.science.uva.nl: Type in the following URI for parameter in each fileReader

• gsiftp://fs2.das.3science.uva.nl/home2/<user-account>/<filename-you-want-to-cp>

• output file: fileWriter parameter

• fs0.das3.cs.vu.nl: Type in the following URI for parameter in each fileReader

• gsiftp://fs0.das3.cs.vu.nl/home0/<user-account>/<filename-you-want-to-created>

Step 8: execute the composed workflow

• Now you are ready to execute the omnimatch workflow, which automatically stage in the data file needed for the omnimatch application, and stage out the result files

Some References

1. Science Z Zhao, A Belloum, M Bubak Editorial: Special section on workflow systems and applications in e-Future Generation Computer Systems 25 (5), 525-527

2. A. Wibisono, D. Vasyunin, V. Korkhov, AS.Z. Belloum WS-VLAM: a GT4 based workflow management system , The 2nd International Workshop on Scientific Workflows, In conjunction with ICCS 2007, Beijing, China, 2007

3. A.S.Z. Belloum, V. Korkhov, S. Koulouzis, M. A Inda, and M. Bubak Collaborative e-Science experiments: from scientific workflow to knowledge sharing JULY/AUGUST, IEEE Internet Computing, 2011

4. Ilkay Altintas, Manish Kumar Anand, Daniel Crawl, Shawn Bowers, Adam Belloum, Paolo Missier, Bertram Ludascher, Carole A. Goble, Peter M.A. Sloot, Understanding Collaborative Studies Through Interoperable Workflow Provenance, IPAW2010, Troy, NY, USA

Some References

5. A. Belloum, Z. Zhao, and M. Bubak Workflow systems and applications , Future Generation Comp. Syst. 25 (5): 525-527 (2009)

6.  Z. Zhao, A.S.Z. Belloum, et al., Distributed execution of aggregated multi domain workflows using an agent framework The 1st IEEE International Workshop on Scientific Workflows, Salt Lake City, U.SA, 2007

7. Zhiming Zhao, Adam Belloum, Cees De Laat, Pieter Adriaans, Bob Hertzberger Using Jade agent framework to prototype an e-Science workflow bus Authors Cluster Computing and the Grid, 2007. CCGRID 2007