Grid Workflow Research update in SIG
Dr. ZhenChun Huang Tsinghua Univ.
NRSCC/RSGS/SIG TeamSep. 8th, 2004
Grid, Application Grid and SIG
• Grid– connects distributed resources by network– co-operates resources by standard protocols– creates virtual organizations dynamically and resolves application
problems by the support of grid software.
• Application Grid– grid for a special application area– based on the general grid technology– provides many features to fit the application area additionally
• Spatial Information Grid– Application Grid in Spatial Information Application area– provides features for the Spatial Information Applications
Grid Architecture
Grid Infrastructure
Resources as Services
Applications
ToolsGrid Middleware
Changes in infrastructure
Application for the area
special resources (services)
Tools for the areaThe
application grid
Workflow
Object of grid workflow
• Regard a grid application as a workflow based on resources in grid
• Provide a easy-to-use environment to develop, deploy and execute complex grid applications
• 3R –––– Object of workflow• Choose the Right resource(person or machine)• Do the Right thing• At the Right time
• Grid WorkFlow Language, shortly GWFL• Based on the open-source workflow engine osworkflow 2.6• Its language is a bit different from that designed by WfMC.• Uses finite automaton model
Roadmap
• First half year of 2002– No workflow support in our grid system– Develop application by Java/JSP directly
• Last half year of 2002—first half year of 2003– Introduce BPEL to our grid system and develop demo applicatio
ns based on it– (thank for He Chuan and Gu Lei, they led us to use workflow in g
rid)
• 2003 last half year — now – Give up BPEL and adopt GWFL based on OSWorkflow 2.6 inste
ad– And develop some down-to-earth applications
Why GWFL instead
• BPEL • GWFLCan Invoke Web Service only
Invoke almost anything by “plug-in”
No user/permission Provides interfaces for user/permission
Very difficult to interact with user in workflow
Interact with user by “view” in workflow
No source code and copyright reserved
Open-source and can be modified freely (OSWorkflow)
Workflow Definition of GWFL
A workflow
Startup step
Finish step (optionally)
Step 2Step …
Step …
Step 1
• A workflow has a “startup” step, a “finish” step (optionally) and several “internal” steps• A step has several “status”, a step and its current status make up of a status of finite
automaton
“Step” in GWFL• Each step has several actions, and a action can perform a
state-transition between steps.
A Step
ActionAction
ActionAction
Other Step
Other Step
Other Step
Other Step
Other Step
Other Step
“Action” in GWFL• An action includes:
– 1 view(usually Java Server Page) to show for the user– 1 restricted condition to restrict availability of this action, and it can
be a complex condition linked by AND, OR or NOT operator– 0..n validators to validate the input data– 0..n pre-function & post-function to perform before or after the acti
on– 0..n results to transit the status of finite automaton. One of the resu
lts should be “unconditional result”, and the others are “conditional results” with conditions
– Or Split into several status, or join several status into one.– Results can have pre-function or post-function, too– All the components except view should be “plug-ins”, they may be:
• Java class, BeanShell, and BSF(support perlscript、 vbscript、 javascript)
An Action• An action includes:
An Action
View
Restricted Condition
Validators
Post-function
Conditional Results &
Un-Conditional Result
Split
Join
Pre-function
Condition Plug-in
Validator Plug-in
Function Plug-in
Function Plug-in
Condition Plug-ins
Function Plug-ins
A Sample workflow
• A sample workflow with 4 internal steps, a finish step, and a hidden startup step
The hidden startup step
Internal steps
The finish step
Actions in a step
• A Step can have several actions (in the sample, it has 8 actions)
An Action
• An action with restrict-to conditions, pre-functions and 2 results.
View of action
Restrict-to condition
• A complex restrict-to condition to restrict when the action is available
• Here, the restrict-to condition is: the status is “Underway” and the user is owner of this workflow
• The “condition” tag indicates a condition plug-in, it is a Java class here
Pre-functions and post-functions
• pre-functions or post-functions plug-in can perform some operation before or after the state-transition
The function plug-in is Java class: org.caf.sig.workflow.function.Registry.SearchAnd it needs 2 arguments named “serviceURL” and “type”
Results and state-transition
• This action has 2 results: a conditional result and a un-conditional result
Result tag indicates a conditional result, its condition is a simple condition which plug-in is written in JavaAn unconditional-result tag can be regarded as a “default” result The next step
State-transition
Develop function plug-in by Java
• A function plug-in class in java should implement the interface org.thgrid.gwfl.workflow.FunctionProvider and do any thing you want to do in function execute()
• The following is a example
public class NewFunction implements FunctionProvider { public void execute(Map transientVars, Map args, PropertySet ps) throws WorkflowException { WorkflowContext context = (WorkflowContext) transientVars.get("context"); String argName = ParamUtil.getValue(transientVars, args, "name");
ps.setString("error", "ok"); }}
Develop condition plug-in by Java
• A condition plug-in class in java should implement the interface org.thgrid.gwfl.workflow.Condition and return whether condition is true in function execute()
• The following is a example
public class CustomSelect implements Condition { public boolean passesCondition(Map transientVars, Map args, PropertySet ps) throws WorkflowException { WorkflowContext context = (WorkflowContext) transientVars.get("context"); String argError = ParamUtil.getValue(transientVars, args, “error");
return “ok".equalsIgnoreCase(argError ); }}
View of action
• A view of action is a special JSP fragment
<%@ page contentType="text/html; charset=gb2312" %><%@ taglib prefix="c" uri="http://java.sun.com/jstl/core" %><%@ taglib uri="/WEB-INF/tld/gwfl-propertySet.tld" prefix="gwfl" %>
<form action="workflow.do"> <input type="hidden" name="action" value="formsubmit"> <input type="hidden" name="id" value="<c:out value='${param.id}'/>"> <input type="hidden" name="do" value="<c:out value='${param.do}'/>">
<gwfl:getString entryId="${param.id}" property="statusString" /> <gwfl:getObject entryId="${param.id}" var="retList" property="retList“/>
<input type=“submit” value=“Next Step”></form>
These codes must be included for the execution of workflowBy gwfl custom tags, properties in PropertySet can be accessed
Update in SIG
• Workflow in SIG is the same as it in general grid essentially
• Because the SIG workflow is used for EO information application, there should be some additional features and functions in SIG workflow. And these should be our work– Develop plug-in library for Spatial Information applications in SIG– Make up a “workflow library” framework so that typical workflows
can be stored for SIG– More powerful toolkits for developing, deploying and executing
workflows in SIG– And so on
Plug-in library
• Plug-ins for the functions(pre- & post-), conditions and validators often be used,– Typical functions such as RS image processing, GIS da
ta processing, RS & GIS data discovery, etc.– Typical conditions such as RS meta-data conditions, R
S data & GIS data feature conditions, etc.– Typical validators such as RS meta-data & image valida
tors, GIS data validators (for format, area, precision, etc)
Workflow library & toolkits
• Workflow library framework– Workflow design & test– Workflow register & discovery– User management & authorization– Co-operate with other parts of SIG, such as user
interface, resource registry, etc.• Some typical workflow to put in the library• Workflow toolkits should include
– Workflow editor with a friendly UI (beta version available )
– Workflow debugger to debug the workflow (on the road)– And so on
Applications
• Used in the 863 project: “Application Grid on Forest Resources and Forestry Ecology” (or often shortly, National Digital Forestry Grid, DFG)– We develop almost all the applications in this project
by GWFL– The example we showed is one of these
applications, too
• And it will be open-source (under GPL or LGPL), too– We are arranging source codes for release
Thank you