WS-VLAM Introduction presentation WS-VLAM Workflow Engine System and Network Engineering group...

WS-VLAM Introduction presentation

WS-VLAM Workflow Engine

System and Network Engineering group

Institute of informatics

University of Amsterdam


Introduction

• This presentation contains information related to WS-VLAM engine

• It helps the reader understand how the workflow engine works, describes the current state and present the ongoing activities and planned ones.

• NOTE: We are currently still working on the User’s and developer’s

Guide, they are not available at this time. The presentations do not replace the documentation it just give a quick introduction.

Do not hesitate to contact us for further information

Thanks for taking some of your time to read about the WS-VLAM work, any comments are more than welcome

WS-VLAM developers [email protected]


Needed background …

• To be able to follow the content of this presentation you will need to know about:

WSRF= Web Service Reference Framework

GT4= Globus Toolkit version 4

• More detailed description of the presented work can be found in the section of publication of the WS-VLAM home page


Objective of this presentation…

• Why you should look at this presentations You want to know how WS-VLAM works and

interacts with the Grid middleware.

• Why you don’t have to worry if you do not understand the content on this presentation: You are an end-user: it will be enough for you to

read the User ’s guide or the WS-VLAM composer presentation

You are an application developer: it will be enough for you to read the Developer ’s guide & the WS-VLAM developers presentation


Outline

• Introduction • Architecture of the WS-VLAM• Hierarchical workflow compositions &

execution • Conclusions

UvA, Amsterdam June 2007 WS-VLAM Introduction presentation

Process Management Stack

Objectives: A WSRF enabled workflow engine

Data Management Stack

Network & Computing ResourcesNetwork & storage Resources

Grid Middleware: Process & resource management

Grid Middleware: Data management

Workflow execution

Workflow composition

Workflow Management system

GridServices

GridServices

GridServicesApplication Templates

Web service

Web Service Interface

OGSA DAI

WorkflowEngine

Web service

WorkflowEngine

Web service

Web Service Interface

Application web service

Distributed Workflow


WS-VLAM Engine characteristics

• Implemented as GT4 WSRF service

• Uses GT4 delegation service for security

• Uses GSI enabled, private VNC for Graphical modules

• Uses notifications mechanisms provides by GT4 for monitoring the execution


GRAMservices

GT4 Java Container

RTSMFactory

Delegationservice

Worker nodes

pre-ws-GRAM

Cli

ent

Job

functions

Delegate

Service host(s) and compute element(s)

WS-VLAM Engine: architecture

RTSM Instance

Workflowcomponents


A three step execution process

Client Delegation Service

RTSM Factory

GRAM RTSM Instance

Step 1

Step 2

• Step 1: Create Delegated Grid Credentials

• Step 2: Instantiate the workflow components

• Step 3: Monitor the workflow execution

Workflowcomponents

Step 3


Sequence-diagram

WS-vlamClient

GT4 DelegationService

RTSM Instance

RTSM Factory

1. Create: delegation credential

2. Submit workflow execution plan

GT4GRAM

Get the delegation credential EPR

Get the RTSM instance EPR 4. Create: RTSM instance

3. Create: RTSM instance

5. subscribe: to notification events

Get the notification events

Step 1

Step 2

Step 3


Current Setting…

Not yet available

DAS-3

Sara

SRB

VU

TUDELFTUvA

Leiden University

Server host Where the WS-VLAM engine

is running


Step1: Create Delegated Credential

VLAMClient RTSM

Factory.

proxy

•Initialize user proxy cert.

•Create delegated credential resource

•Credential EPR returned

EPRGRAM

Worker nodes

Delegation

Credential

RP

GT4 Service Container

EPR = End Point Reference


Step 2: instantiates the workflow components and Create a RTSM instance

VLAMClient

Delegation

RTSMFactory.

Credential

RP

• Request the creation RTSM instance• Pass delegated credential EPR

•RTSM instance EPR returned

EPR

•Access delegated credential resource

•Contact GRAM to instantiates the workflow components

GRAM

RTSMInstance

Worker nodes

•GRAM instantiates the workflow components

Workflowcomponents

Workflowcomponents

EPR = End Point Reference



Step3: monitors the application workflow

VLAMClient

RTSMFactory.

•Contact RTSM instance and subscribe for notification events

GRAM

RTSMInstance

Worker nodes

•RTSM instance subscribes for notification events from GRAM and Workflow components

Workflowcomponents

Workflowcomponents

•GRAM and Workflow components generate events

Notificationevents

Delegation

Credential

RP



Workflow components with graphical output

VLAMClient

RTSMFactory.

GRAM

RTSMInstance

Delegation

Credential

RP

Worker nodes

Workflowcomponents

Workflowcomponents

RTSM Instance

GUIproxy


vnc server

GSI enabled connection


Support hierarchical workflows …

• workflow components can be composed of a collection of other components

• The advantage is now both the composition & the execution of complex workflows become easier



is running


is running

Setting for distributed hierarchical workflows …

DAS-3

Sara

SRB

VU

TUDELFTUvA

Leiden University


Step2: instantiates the distributed workflow components & Create a RTSM instances


VLAMClient

RTSMFactory

EPRGRAM

RTSM Instance

Worker nodes

Workflowcomponents

Workflowcomponents

GRAM

Worker nodes

Workflowcomponents

Workflowcomponents


RTSM Instance

GUIproxy

Dataproxy

GUIproxy

Dataproxy

Distributed RTSM

RTSMFactory

Distributed RTSM


Step2: instantiates the distributed workflow components & Create a RTSM instances


VLAMClient

RTSMFactory

EPRGRAM

RTSM Instance

Worker nodes

Workflowcomponents

Workflowcomponents

GRAM

Worker nodes

Workflowcomponents

Workflowcomponents


RTSM Instance

GUIproxy

Dataproxy

Dataproxy

GUIproxy

Distributed RTSM

RTSMFactory

Distributed RTSM


Conclusions

• WSRF compliant middleware such as GT4 offers useful basic services to develop a truly grid enabled workflow engine:

Delegation service & ws-notification mechanisms proved to be very useful

• Separating the workflow composition from execution suits better the Grid environments

Some References

1. Science Z Zhao, A Belloum, M Bubak Editorial: Special section on workflow systems and applications in e-Future Generation Computer Systems 25 (5), 525-527

2. A. Wibisono, D. Vasyunin, V. Korkhov, AS.Z. Belloum WS-VLAM: a GT4 based workflow management system , The 2nd International Workshop on Scientific Workflows, In conjunction with ICCS 2007, Beijing, China, 2007

3. A.S.Z. Belloum, V. Korkhov, S. Koulouzis, M. A Inda, and M. Bubak Collaborative e-Science experiments: from scientific workflow to knowledge sharing JULY/AUGUST, IEEE Internet Computing, 2011

4. Ilkay Altintas, Manish Kumar Anand, Daniel Crawl, Shawn Bowers, Adam Belloum, Paolo Missier, Bertram Ludascher, Carole A. Goble, Peter M.A. Sloot, Understanding Collaborative Studies Through Interoperable Workflow Provenance, IPAW2010, Troy, NY, USA


Some References

5. A. Belloum, Z. Zhao, and M. Bubak Workflow systems and applications , Future Generation Comp. Syst. 25 (5): 525-527 (2009)

6. Z. Zhao, A.S.Z. Belloum, et al., Distributed execution of aggregated multi domain workflows using an agent framework The 1st IEEE International Workshop on Scientific Workflows, Salt Lake City, U.SA, 2007

7. Zhiming Zhao, Adam Belloum, Cees De Laat, Pieter Adriaans, Bob Hertzberger Using Jade agent framework to prototype an e-Science workflow bus Authors Cluster Computing and the Grid, 2007. CCGRID 2007



http://www.vl-e.nl/

Date post:	18-Dec-2015
Category:	Documents
Upload:	oswin-mclaughlin
View:	218 times
Download:	2 times

WS-VLAM Introduction presentation WS-VLAM Workflow Engine System and Network Engineering group...

Documents