Workflow in Grid Systems Workshop

Dave Berry, Research ManagerUK National e-Science Centre

GGF10, Mar 2004

Outline: Welcome

WelcomeAimsProgramme

“E-Science Workflow Services”BackgroundStructureIssues arising

AimsReport on current work

Find areas of agreementIdentify open issues

Look for opportunitiesResearchDevelopmentCollaboration

OutcomesSample workflowsCP&E special issueInput to RGs and WGs ?

Morning Session

TalksArchitectureApplications

Possible panel topics:ScriptingSecurityDebuggingConstraint Modelling

DiscussionIssues

Afternoon session

TalksLanguages (esp. BPEL)Tools & Enactment

Possible panel topics:Adaptive enactmentWorkflow inferenceEvent-driven enactmentIncorporating devices & human input

DiscussionOutcomes

Outline:“E-Science Workflow Services”

WelcomeAimsProgramme

“E-Science Workflow Services”BackgroundStructureIssues arising

E-Science Workflow Services

> 90 participantsIndustryUK e-ScienceInternational e-Science

OrganisersDave Berry (NeSC)Savas Parastatidis (NEReSC)

Written report In progressUK e-Science Series

December 3-5, 2003

e-Science Institute

e-Science Institute

Speakers

IndustryWfMC, WS-Choreography

UK e-ScienceMyGrid/Taverna, GeoDise, DiscoveryNet, DAME, ICENI, Planning, RealityGrid, JIGSA, OGSA-DAI, Triana, AstroGrid

International e-ScienceChimera/Pegasus, BIRN, Kepler/Ptolemy, Thetis, Narada

ResearchOperational research, Workflow and VO’s

Breakout Sessions

Scientific Workflow Requirements

Carole Goble

Protocols in Scientific Workflows

John Brooke

Workflow Languages and Engines

Matthew Addis

User requirementsReflect the modelling paradigm of the scientist.

Varies between experiments, disciplines

Which user would that be?Creators, users, auditors, validators (I know if its right when I see it)Biologists cf. bioinformaticians, and transitioning between

Different users, different environmentsAppropriate levels of abstraction.

User models -> workflow models

Simple to use & intuitive creation, deployment, execution and debugging environments

A Scientist Writes…

“Work in my problem solving environment so that I don’t need to change the way I work.”

Scientific Workflow lifecycles

Incrementally exploratory prototypesGot the data, now get the Nature paper before the next guy

Large scale productionGot the idea, now get the data for many experiments, teams, communities

Migration from one to the otherCapture of prototype for later non-interactive replay in a parameterised fashion

Different parts of the lifecycle May use different environments and policiesDifferent sorts of users will interact

User interaction

Creation & DiscoveryBy example, plagiarism, drag and drop

Collaborative multi-user interaction in creationReusing workflows -> modularisationReusing workflows with different parameters and dataComposing workflows from different areas, disciplines and scales“eXtreme team workflow creation”

Single User interaction with workflow executionChoice between paths of execution in specific statesParameter modification mid-run

Collaborative multi-user interaction during execution?

Characterising Scientific Workflows

Very large amounts of dataFiles, streams, database queriesGridFTP, http, ftp, sockets …Sometimes it's the computation that needs to be moved to the data

Data model and typesMetadataProvenance

DriversScientific questions, outcomes and vanityMore creators than users in science?

Enactment Stack

Goals

Abstract flows

Concrete flows

Process execution

Service descriptions

Messages

Communications

Po

licies a

nd

secu

rity

On

tolo

gie

s, me

tad

ata

, de

scriptio

ns,

info

rma

tion

service

s

Workflow vs. Service“Perform” document

Transformed result of query 2

Stored procedure

Result of query 1 OGSA-DAI

Service

Questions?

daveb@nesc.ac.uk

Presentations from the workshop: http://www.nesc.ac.uk/action/esi/con

tribution.cfm?Title=303