Ian FosterArgonne National Laboratory

University of Chicago

Univa Corporation

Service-Oriented ScienceScaling eScience Application & Impact

2

Acknowledgements

Carl Kesselman, with whom I developed many ideas & slides

Bill Allcock, Charlie Catlett, Kate Keahey, Jennifer Schopf, Frank Siebenlist, Mike Wilde @ ANL/UC

Ann Chervenak, Ewa Deelman, Laura Pearlman @ USC/ISI

Karl Czajkowski, Steve Tuecke @ Univa Numerous other fine colleagues NSF, DOE, IBM for research support

3

Context:System-Level Science

Problems too large &/or complex to tackle alone …

4

Seismic Hazard Analysis (Southern Calif. Earthquake Center)

InSAR Image of theHector Mine Earthquake

A satellitegeneratedInterferometricSynthetic Radar(InSAR) image ofthe 1999 HectorMine earthquake.

Shows thedisplacement fieldin the direction ofradar imaging

Each fringe (e.g.,from red to red)corresponds to afew centimeters ofdisplacement.

SeismicHazardModel

Seismicity Paleoseismology Local site effects Geologic structure

Faults

Stresstransfer

Crustal motion Crustal deformation Seismic velocity structure

Rupturedynamics

5

SCEC Community Model

IntensityIntensityMeasuresMeasures

Earthquake Earthquake Forecast ModelForecast Model

AttenuationAttenuationRelationshipRelationship

11

Standardized Seismic Hazard Analysis

Ground motion simulation

Physics-based earthquake forecasting

Ground-motion inverse problem

Structural Simulation

AWMAWMGroundGroundMotionsMotionsSRMSRM

Unified Structural RepresentationUnified Structural RepresentationFaults Motions Stresses Anelastic modelFaults Motions Stresses Anelastic model

22

AWP = Anelastic Wave Propagation

SRM = = Site Response Model

RDRDMM

FSMFSM

33

FSM = Fault System Model

RDM = Rupture Dynamics Model

InvertInvert

Other DataOther DataGeologyGeologyGeodesyGeodesy

44

22

33

11

44

55

55

7

Science Takes a Village …

Teams organized around common goals People, resource, software, data, instruments…

With diverse membership & capabilities Expertise in multiple areas required

And geographic and political distribution No location/organization possesses all required

skills and resources Must adapt as a function of the situation

Adjust membership, reallocate responsibilities, renegotiate resources

8

Virtual Organizations From organizational behavior/management:

"a group of people who interact through interdependent tasks guided by common purpose [that] works across space, time, and organizational boundaries with links strengthened by webs of communication technologies" (Lipnack & Stamps, 1997)

The impact of cyberinfrastructure People computational agents & services Communication technologies IT

infrastructure, i.e. Grid

“The Anatomy of the Grid”, Foster, Kesselman, Tuecke, 2001

9

Beyond Science Silos:Service-Oriented Architecture

Decompose across network Clients integrate dynamically

Select & compose services Select “best of breed” providers Publish result as a new service

Decouple resource & service providers

Function

Resource

Data Archives

Analysis tools

Discovery toolsUsers

Fig: S. G. Djorgovski

10

Provisioning

Service-Oriented Systems:The Role of Grid Infrastructure

Service-oriented Gridinfrastructure Provision physical

resources to support application workloads

ApplnService

ApplnService

Users

Workflows

Composition

Invocation

Service-oriented applications Wrap applications as

services Compose applications

into workflows

“The Many Faces of IT as Service”, Foster, Tuecke, 2005

11

Forming & Operating (Scientific) Communities

Define VO membership and roles, & enforce laws and community standards I.e., policy for service-oriented architecture

Build, buy, operate, & share community infrastructure Data, programs, services, computing, storage,

instruments Service-oriented infrastructure

Define and perform collaborative work Use shared infrastructure, roles, & policy Manage community workflow

12

Forming & Operating (Scientific) Communities

Define VO membership and roles, & enforce laws and community standards I.e., policy for service-oriented architecture

Build, buy, operate, & share community infrastructure Data, programs, services, computing,

storage, instruments Service-oriented infrastructure

Define and perform collaborative work Use shared infrastructure, roles, & policy Manage community workflow

13

Defining Community: Membership and Laws

Identify VO participants and roles For people and services

Specify and control actions of members Empower members delegation Enforce restrictions federate policy

A

1 2

B

1 2

A B

1

10

1

10

1

16

Access granted by community

to user

Site admission-

control policies

EffectiveAccess

Policy of site to

community

14

Policy Challenges in VOs

Restrict VO operations based on requestor characteristics VO dynamics create challenges

Intra-VO VO-specific roles Mechanisms to specify/enforce policy at VO

level Inter-VO

Entities/roles in one VO not necessarily defined in another VO

15

Core Security Mechanisms Attribute Assertions

C asserts that S has attribute A with value V Authentication and digital signature

Allows signer to assert attributes Delegation

C asserts that S can perform O on behalf of C Attribute mapping

{A1, A2… An}vo1 {A’1, A’2… A’m}vo2 Policy

Entity with attributes A asserted by C may perform operation O on resource R

16

Trust in VOs

Do I “believe” an attribute assertion? Used to evaluate cost vs. benefit of

performing an operation E.g., perform untrusted operation with extra

auditing Look at attributes of assertion signer Rooting trust

Externally recognized source, e.g., CA Dynamically via VO structure delegation Dynamically via alternative sources, e.g.,

reputation

17

Security Services for VO Policy Attribute Authority (ATA)

Issue signed attribute assertions (incl. identity, delegation & mapping)

Authorization Authority (AZA) Decisions based on assertions & policy

Use with message- or transport-level security

VO AService

VOATA

VOAZA

MappingATA

VO BService

VOUser A

Delegation AssertionUser B can use Service A

VO-A Attr VO-B Attr

VOUser B

Resource AdminAttribute

VO MemberAttribute

VO Member Attribute

18

Closing the Loop

VO

RightsUsers

Rights’

ComputeCenter

Access

Services (runningon user’s behalf)

Rights

Local policyon VO identityor attributeauthority

CAS or VOMSissuing SAMLor X.509 ACs

SSL/WS-Securitywith ProxyCertificates

Authz Callout:SAML, XACML

KCA

MyProxy

19

Forming & Operating Scientific Communities

Define VO membership and roles, & enforce laws and community standards I.e., policy for service-oriented architecture

Build, buy, operate, & share community infrastructure Data, programs, services, computing, storage,

instruments Service-oriented infrastructure

Define and perform collaborative work Use shared infrastructure, roles, & policy Manage community workflow

20

Community

Services Provider

Content

Services

Capacity

Bootstrapping a VOby Assembling Services

1) Integrate services from other sources Virtualize external services as VO services

2) Coordinate & compose Create new services from existing ones

Capacity Provider

“Service-Oriented Science”, Foster, 2005

21

Providing VO Services:(1) Integration from Other Sources

Negotiate servicelevel agreements

Delegate and deploy capabilities/services

Provision to deliver defined capability

Configure environment Host layered functions

CommunityA

CommunityZ…

22

Virtualizing Existing Services into a VO

Establish service agreement with service E.g., WS-Agreement

Delegate use to VO user

UserA

VO Admin

UserBVO User

ExistingServices

23

Deploying New Services

Policy

Client

Environment

Activity

Allocate/provisionConfigure

Initiate activityMonitor activityControl activity

Interface Resource provider

WS-Resource Framework, Globus GRAM, Virtual Workspaces

24

Activities Can Be Nested

Policy

Client

Environment

Interface Resource provider

ClientClient

25

www.opensciencegrid.org

Jobs (2004)

Open Science Grid 50 sites (15,000 CPUs) & growing 400 to >1000 concurrent jobs Many applications + CS experiments;

includes long-running production operations Up since October 2003; few FTEs central ops

26

VOUser

EmbeddedResource Management

ClusterResourceManager

GRAM

ClusterResourceManager

GRAM

• VO admin delegates credentials to be used by downstream VO services.• VO admin starts the required services.• VO jobs comes in directly from the upstream VO Users• VO job gets forwarded to the appropriate resource using the VO credentials• Computational job started for VO

Client-side

VO Scheduler Other Services

VO Admin

. . .

Monitoring and control

HeadnodeResourceManager

GRAM

Deleg Deleg

Deleg

VOUser

VO Job

VO Job

27

Providing VO Services:(2) Coordination & Composition

Take a set of provisioned services …

… & compose to synthesize new behaviors

This is traditional service composition But must also be concerned with emergent

behaviors, autonomous interactions See the work of the agent & PlanetLab

communities

“Brain vs. Brawn: Why Grids and Agents Need Each Other," Foster, Kesselman, Jennings, 2004.

28

Birmingham•

The Globus-BasedLIGO Data Grid

Replicating >1 Terabyte/day to 8 sites>40 million replicas so farMTBF = 1 month

LIGO Gravitational Wave Observatory

www.globus.org/solutions

Cardiff

AEI/Golm

29

Pull “missing” files to a storage system

List of required

Files

GridFTPLocal

ReplicaCatalog

ReplicaLocation

Index

Data Replication

Service

Reliable File

Transfer Service Local

ReplicaCatalog

GridFTP

Data Replication Service

“Design and Implementation of a Data Replication Service Based on the Lightweight Data Replicator System,” Chervenak et al., 2005

ReplicaLocation

Index

Data MovementData Location

Data Replication

30

Hypervisor/OS Deploy hypervisor/OS

Composing Resources …Composing Services

Physical machineProcure hardware

VM VM Deploy virtual machine

Provisioning, management, and monitoring at all levels

JVM Deploy container

DRS Deploy service GridFTP LRC

VO Services

GridFTP

31

Decomposition EnablesSeparation of Concerns & Roles

User

ServiceProvider

“Provide access to data D at S1, S2, S3 with performance P”

ResourceProvider

“Provide storage with performance P1, network with P2, …”

D

S1

S2

S3

D

S1

S2

S3Replica catalog,User-level multicast, …

D

S1

S2

S3

32

Community Commons

What capabilities are available to VO? Membership changes, state changes

Require mechanisms to aggregate and update VO information

VO-specific indexes

S

S

S SInformation

AA

A

FRESH

MOREThe age of

information

33

GT4 Container

Monitoring and Discovery Services

MDS-Index

GT4 Cont.

RFT

MDS-Index

GT4 Container

MDS-Index

Registration &WSRF/WSN Access

GridFTP

adapter

Custom protocolsfor non-WSRF entities

Clients (e.g., WebMDS)

GRAM User

Automatedregistrationin container

WS-ServiceGroup

35

Forming & Operating Scientific Communities

Define VO membership and roles, & enforce laws and community standards I.e., policy for service-oriented architecture

Build, buy, operate, & share community infrastructure Data, programs, services, computing, storage,

instruments Service-oriented infrastructure

Define and perform collaborative work Use shared infrastructure, roles, & policy Manage community workflow

36

Collaborative WorkExecuted

Executing

Executable

Not yet executable

Query

Edit

ScheduleExecution environment

What I Did

What I Want to Do

What I Am Doing

…

Time

37

Managing Collaborative Work Process as “workflow,” at different scales, e.g.:

Run 3-stage pipeline Process data flowing from expt over a year Engage in interactive analysis

Need to keep track of: What I want to do (will evolve with new knowledge) What I am doing now (evolve with system config.) What I did (persistent; a source of information)

AbstractWorfklow

Workflow with executable

nodes

Jobs

TemplateGeneration

WorkflowRefinement

ExecutionEnvironment

38The GriPhyNVirtual Data System

Abstractworkflow

Local planner

DAGmanDAG

StaticallyPartitioned

DAG

DAGman &Condor-GDynamically

PlannedDAG

VDLProgram

Virtual Datacatalog

Virtual DataWorkflowGenerator

JobPlanner

JobCleanup

Workflow spec Create Execution Plan Grid Workflow Execution

39

Functional MRI Analysis3a.h

align_warp/1

3a.i

3a.s.h

softmean/9

3a.s.i

3a.w

reslice/2

4a.h

align_warp/3

4a.i

4a.s.h 4a.s.i

4a.w

reslice/4

5a.h

align_warp/5

5a.i

5a.s.h 5a.s.i

5a.w

reslice/6

6a.h

align_warp/7

6a.i

6a.s.h 6a.s.i

6a.w

reslice/8

ref.h ref.i

atlas.h atlas.i

slicer/10 slicer/12 slicer/14

atlas_x.jpg

atlas_x.ppm

convert/11

atlas_y.jpg

atlas_y.ppm

convert/13

atlas_z.jpg

atlas_z.ppm

convert/15

Workflow courtesy James Dobson, Dartmouth Brain Imaging Center

40

Functional MRI – Mapping Brain Function using Grid Workflows

             <>

41

Functional MRI Virtual Data Queries

Which transformations can process a “subject image”? Q: xsearchvdc -q tr_meta dataType

subject_image input A: fMRIDC.AIR::align_warp

List anonymized subject-images for young subjects: Q: xsearchvdc -q lfn_meta dataType subject_image privacy anonymized subjectType young A: 3472-4_anonymized.img

Show files that were derived from patient image 3472-3: Q: xsearchvdc -q lfn_tree 3472-3_anonymized.img A: 3472-3_anonymized.img

3472-3_anonymized.sliced.hdr atlas.hdr atlas.img … atlas_z.jpg 3472-3_anonymized.sliced.img

42

QuarkNet: Leveraging Trident for Science Education

43PUMA:Analysis of Metabolism

PUMA Knowledge Base

Information about proteins analyzed against ~2 million gene sequences

Analysis on Grid

Involves millions of BLAST, BLOCKS, and

other processesNatalia Maltsev et al.http://compbio.mcs.anl.gov/puma2

44

Astronomy:A Small Montage Workflow

~1200 node workflow, 7 levels Mosaic of M42 created on

TeraGrid

45

Summary (1):Community Services

Community roll, city hall, permits, licensing & police force Assertions, policy, attribute & authorization services

Directories, maps Information services

City services: power, water, sewer Deployed services

Shops, businesses Composed services

Day-to-day activities Workflows, visualization

Tax board, fees, economic considerations Barter, planned economy, eventually markets

46

Summary (2) Community based science will be the norm

Requires collaborations across sciences— including computer science

Many different types of communities Differ in coupling, membership, lifetime, size

Must think beyond science stovepipes Community infrastructure will increasingly become the

scientific observatory Scaling requires a separation of concerns

Providers of resources, services, content Small set of fundamental mechanisms required to build

communities

47

For More Information Globus Alliance

www.globus.org NMI and GRIDS Center

www.nsf-middleware.org www.grids-center.org

Infrastructure www.opensciencegrid.org www.teragrid.org

Background www.mcs.anl.gov/~foster

2nd Editionwww.mkp.com/grid2