The Anatomy of the Grid: An Integrated View of Grid
Architecture
Ian Foster, Steve TueckeArgonne National Laboratory
The University of Chicago
Carl KesselmanUSC/Information Sciences Institute
Outline and Goals
Define what is (and is not) a Grid in terms of a layered protocol architecture
Understand the role of protocols, services, APIs and tools in the deployment and use of Grids
Illustrate how a layered protocol architecture works with specific examples
Propose a view of where the Grid Forum can play a particularly important role
What is a Grid?
Grid language has been driven by genesis from metacomputing, but…
In practice, the Grid is about resource sharing and coordinated problem solving in dynamic, multi-institutional virtual organizations
Focus on how to enable, maintain and control the sharing of resources to achieve a common goal
Some Useful Definitions
Network Protocol A formal description of message formats and
a set of rules for exchange of messages Rules define sequences of message
exchange, and potentially resulting behavior Protocol may define state-change in endpoint
Network Enabled Services Defines a set of capabilities Protocol defines interaction with service
All services require protocols, although not all protocols are to services
More definitions
Resource Entity that is to be shared Provides some capabilities, that can be
accessed via interface (API) or protocol Application Programmer Interface (API) Software Development Kit (SDK)
Package that enables application development, consisting of one or more APIs, and programming tools
Protocols Make the Grid
Protocols vs. APIs Protocols enable interoperability APIs enable portability
Sharing is about interoperability, so … Grid architecture should be about protocols
By extension, a major role for the Grid Forum is to define protocols where standards do not exist
Grid Services Architecture: Previous Perspective
Grid-enabled archives, networks, computers, display devices, etc.; associated local services
Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc.
GridFabric
GridServices
ApplnToolkits
Applns
...
… a rich variety of applications ...
Remoteviz
toolkit
Remotecomp.toolkit
Remotedata
toolkit
Remotesensorstoolkit
Async.collab.toolkit
Characteristics of Grid Services Architecture
Identifies separation of concerns Isolates Grids from languages and specific
programming environments Makes provisions for generic and
application specific functionality Protocols not explicit in architecture
Fails to make clear distinction between language, service, and networking issues
Grid Services Architecture v2.0:Layered Grid Protocol Architecture
Connectivity
Fabric
Resource
Grid
Application
User
Important Points
Being Grid-enabled requires speaking appropriate protocols Protocol only requirement, not reachability Protocols can be used to bridge local resources
or “local Grids” Talk about Intergrid (analog to Internet)
Built on Internet protocols Independent of language and implementation
Focus on interaction over network Services exist at each level
Grid Services Architecture v2.0:Protocols, Services, and Interfaces
Languages/Frameworks
Fabric Layer
Applications
Local Access APIs and protocols
Grid Service APIs and SDKs
Grid ServicesGrid Service Protocols
Resource APIs and SDKs
Resource ServicesResource Service Protocols
User Service ProtocolsUser Service APIs and SDKs
User Services
Connectivity APIs
Connectivity Protocols
The Integrated Grid Architecturein Practice
We now proceed to illustrate the general concepts outlined above by reference to specific examples First, we analyze some extant systems in
terms of this architecture Then, we review example protocols,
services, APIs, SDKs in each layer Apology: We will necessarily need to refer
to some specific protocols and systems
Example: A User Portal
This to be filled in showing the different services protocols APIs etc. used at each layer
Example:Network Weather Service
ditto
Example: A Data Grid
ditto
Layer 1: Fabric
Local access to logical resource May be real component, e.g. CPU, software
module, filesystem May be logical component, e.g. Condor pool
Protocol or API mediated Fabric elements include
SSP, ASP, peer-to-peer, Entropia-like, and enterprise level solutions
Layer 2: Connectivity Protocols
Two classes of connectivity protocols underlie all other components
Internet communication Application, transport and internet layer
protocols I.e., transport, routing, DNS, etc.
Security Authentication and delegation Discussed below
Connectivity Layer Element:Grid Security Protocols
Protocols TLS with delegation
Services K5ssl, Globus Authorization Service
APIs GSS-API, GAA, SASL, gss_assist
SDKs GlobusIO
Layer 3: Resource Protocols
Resource management Storage system access Network quality of service Data transport Resource information
Resource Management
Protocols GRAM+GARA (on HTTP)
Resource services Gatekeeper, JobManager, SlotManager
APIs and SDKs GRAM API, Java CoG Kit Client, DUROC
Data Transport
Protocols Grid FTP, LDAP for replica catalog
Services FTP, LDAP replica catalog
APIs and SDKs GridFTP client library, copy URL API, replica
catalog access, replica selection
Resource Information
Protocol LDAP V3, Registration/Discovery protocol
Service GRIS
APIs & SDKs C API; JNDI, PerlLDAP, ….
Layer 4: Grid Protocols
Grid Information Index Services LDAP and Service registration protocol, … GIIS service LDAP APIs and specialized information API
Co-allocation and brokering GRAM (HTTP+RSL) DUROC service DUROC client API, end-to-end reservation
API
Grid Protocols (cont)
Online authentication, authorization services HTTP MyProxy, Group policy servers MyProxy API, GAA API,
Many others (e.g.): Resource discovery (Matchmaker) Fault recovery
Layer 5: User Protocols
In general, there are many of these, they tend to be one off and not standardized
Examples: Portal toolkits (e.g., Hotpage) Netsolve Cactus framework
Role of Standards
Standard APIs: code reuse, portability Certainly a good thing!
Standard protocols enable resource sharing via interoperability of infrastructure The fundamental problem
Note that Grid Forum is already doing this! Scheduling: remote access protocol Data: Grid FTP Security: Grid Security protocols Information: Standard info representations
Summary
Grids are about sharing, not making big MPI jobs
Well-defined protocol architecture is essential to understanding and progress Provides a framework for figuring out where
the pieces fit Grid Forum can play an important role by
helping to fill in missing pieces But note that protocol design is not for the
nonspecialist (or fainthearted)!
