1Alexander Reinefeld, ZIB Berlin
Zukünftige Dienste im D-Grid:
Neue Anforderungen an die Rechenzentren?
Alexander Reinefeld
Zuse-Institut Berlin
Humboldt Universität zu Berlin
ZKI Herbsttagung in Heilbronn, 29.09.2004
2Alexander Reinefeld, ZIB Berlin
Contents
1 What is the Grid?
2 Grid Concepts
3 Grid Standards
4 Challenges for Computer Services
3Alexander Reinefeld, ZIB Berlin
1 What is the Grid?
4Alexander Reinefeld, ZIB Berlin
The Three Grids
Service Grid
OGSA
Resource Grid
Information Grid
SOAP, WSDL, UDDI
XML
network computingstorage
HTML
file sharing
web
access, usage publication of meta information
sear
chen
gin
es…
…
…
5Alexander Reinefeld, ZIB Berlin
What is “The Grid”?
[The Grid] “intends to make access to computing power,
scientific data repositories and experimental facilities
as easy as the Web makes access to information.“
Tony Blair, 2002
Web = HTMLGrid = ???
6Alexander Reinefeld, ZIB Berlin
What is “The Web”?
“It is fundamentally a decentralized thing
and when we really use it practically,
it becomes a fractal thing.”
Tim Berners-Lee
A fractal thing?
7Alexander Reinefeld, ZIB Berlin
A Fractal Thing?
Fractals (Latin fractus = broken) combine structure and irregularity.
Features:
• Self-similarity
• Infinite detail regardless of magnification
In the eyes of computer scientists:
• ideal scalability (top down)
• no global state (difficult to control bottom up)
8Alexander Reinefeld, ZIB Berlin
More Definitions of “The Grid”
• You're able to get what you want, when you want it.
• You don't have to concern yourself with the infrastructure, the resources simply appear on demand.
• You pay only for what you use, as reflected on your monthly bill.
Hype versus reality
9Alexander Reinefeld, ZIB Berlin
Grid Characteristics
local autonomy
dynamicsscalability
heterogeneity
Cannot affect site policies.Difficult orchestration, coordination. Resources may differ significantly.
Grid may be local or worldwide.Communication latency, bottlenecks due to hierarchy,synchronization.
Form and properties of resources may change during
the lifespan of the application.
10Alexander Reinefeld, ZIB Berlin
2Grid Concepts
11Alexander Reinefeld, ZIB Berlin
Transparency
Hide whether a (software) resource is in memory, on disk, in an archivePersistence
Hide the failure and recovery of a resourceFailure
Hide that a resource may be shared by several competitive usersConcurrency
Hide that a resource may be replicated for concurrent accessReplication
Hide that a resource may be moved to another location while in useRelocation
Hide that a resource may move to another locationMigration
Hide where a resource is locatedLocation
Hide differences in data representation and how a resource is accessedAccess
DescriptionTransparency
12Alexander Reinefeld, ZIB Berlin
Transparency by Virtualization
Supercomputer
PC-ClusterArchives
Analysis
Experiments
PC-Cluster SupercomputerScientist/user
Archives
Today: Monolithic, vertically integrated, proprietary solutions.
Middleware
Supercomputer
PC-ClusterArchive Analysis
Experiment
PC-Cluster Supercomputer
Scientist/user
Archive
Analysis
Tomorrow:Flexible, adaptable, interchangeable “one-stop-shop” solutions via standard interfaces.
J. T
aylo
r, m
odifi
ed b
y H
offm
ann,
Put
zer,
Rei
nefe
ld
13Alexander Reinefeld, ZIB Berlin
Virtualization
… hides the complexity of distribution,
… provides a user-centric view,
… provides synergy,
… and complicates the system architecture!… and complicates the system architecture!
QoS?
14Alexander Reinefeld, ZIB Berlin
3Grid Standards:
WS, OGSA, OGSI, WSRF
15Alexander Reinefeld, ZIB Berlin
Globus Toolkit - Timeline
1995 Globus project proposal
1998 GT 1.0: GRAM, MDS, …
2001 GT 2.0: GridFTP, packaging, reliability, …
(toolkit approach)
2002 GT 3.0 “Technology Preview”(tracking OGSI definition, substantial extensions to web services)
6/03 GT 3.0: OGSI-based, GT2 functionality
1/05 GT 4.0: WSRF
(back to web services, family of standards)
OGSA v0.01: 9/02
v0.17: 6/04
OGSI 1st spec. 07/03
WSRF intro. 01/2004
Standards
16Alexander Reinefeld, ZIB Berlin
Web Services
The famous XML-family
• WSDL (Web Services Description Language)
• UDDI (Universal Description, Discovery and Integration)
• SOAP (Simple Object Access protocol)
• … and other members
17Alexander Reinefeld, ZIB Berlin
18Alexander Reinefeld, ZIB Berlin
Web Services vs. Grid Services
OGSAOpen Grid Service Architecture
proposed at GGF 2002,
„physiology “ paper
Significant implications for how services are managed, named, discovered, and used!
Web Services address discovery & invocation of persistent, stateless
services.
Grid Servicessupport
transient, statefulservice instances.
19Alexander Reinefeld, ZIB Berlin
OGSA - Open Grid Services Architecture
• Refactor Globus protocol suite to enable common base
• Service orientation to virtualize resources, services, information
• Embrace Web Service technologies for standard IDL
à Result: Standard interfaces & behaviors: “Grid Service”.
20Alexander Reinefeld, ZIB Berlin
OGSI Components and their Interplay
Registry
Handle Resolver
ClientGrid Service
Instanz(Zustand)
1. Factorysuchen
nicht in OGSI spezifiziert
dauerhafte Handles
3. Eintrag eines neuen Handle
4. Neue Dienstinstanzmeldet sich an5. Aufruf
kurzlebige Handles
2. Auftrag zur
Instanziierung
Factory
21Alexander Reinefeld, ZIB Berlin
Three Major Concerns about OGSI
• Too much stuff in one specification
• Does not work well with existing Web services tooling
• Too “object oriented”
àWSRF tones down the usage of XML schema.
àWSRF distinguishes between a service and the stateful resources acted upon by that service.
… and how they are addressed by WSRF
àWSRF is a family of composable specifications.
22Alexander Reinefeld, ZIB Berlin
From OGSI to WSRF
WS-Addressing Endpoint ReferenceGrid Service Reference
WS-Addressing Endpoint ReferenceGrid Service Handle
WS-BaseFaultsBase fault type
WS-NotificationNotification portTypes
WS-ServiceGroupServiceGroup portTypes
WS-ResourceLifetimeGridService lifetime mgmt
WS-ResourcePropertiesService data definition & access
WS-RenewableReferencesHandleResolver portType
WSRFOGSI
WSRF Specs
otherSpecs
23Alexander Reinefeld, ZIB Berlin
WSRF Components and their Interplay
Web
Service
Interface
WS Resource
runtime environment
R
WS Resource
context A
context B
discovery,interaction
introspection status notification
status:
Sstatus:
initial endpoint reference
24Alexander Reinefeld, ZIB Berlin
Convergence (?)
Grid
Web
“The definition of WSRF means that Grid and Web communities can move forward on a common base.” [Ian Foster 1/2004]
WSRF
Started far apart in apps & tech
OGSI
GT2GT1
HTTPWSDL,
WS-*
WSDL 2,
WSDM
Have beenconverging
25Alexander Reinefeld, ZIB Berlin
3 Challenges for Computer
Services
26Alexander Reinefeld, ZIB Berlin
Challenges
• dynamic formation & management of virtual organizations
• discovery & online negotiation of access to services: who, what, why, when, how
• configuration of applications and systems that are able to deliver multiple QoS
• autonomic management of distributed infrastructures, services and applications
• management of distributed state
• open, extensible infrastructure
27Alexander Reinefeld, ZIB Berlin
Challenges for Computer Services
• Service Virtualization will cause trouble!
o How to guarantee QoS for a service that I do not provide locally?
• Dynamic behavior means more trouble!
o No self-configuration yet. Not even policies.
• The world is your customer – not your local campus student.
A gentle warning to our politicians: Grid Resources must be paid for!
Grids will improve system utilization, but there will be more demand.
28Alexander Reinefeld, ZIB Berlin
Computer services will need more qualified staff, not less.
à increasing complexityà increasing remote accesses
29Alexander Reinefeld, ZIB Berlin
4 Summary
30Alexander Reinefeld, ZIB Berlin
Lessons Learned From Each Chapter
1 What is the Grid?
Web (and Grid) are “fractal things” [Berners-Lee]. Current Grid implementations are not.
2 Grid Concepts
Transparency through virtualization. à Difficult system design.
3 Grid Standards: WS, OGSA, OGSI, WSRF
Will probably stabilize on WS. But it really doesn’t matter.
4 Challenges for Computer Services
More work ahead. Need more qualified staff. Many open problems.
31Alexander Reinefeld, ZIB Berlin
Information
• A. Reinefeld, F. Schintke. Dienste und Standards für das Grid Computing. In: J. von Knop, W. Haferkamp (Hrsg.), 18. DFN Arbeitstagung über Kommunikationsnetze, Düsseldorf, Lecture Notes in Informatics, 2004, vol. P-55, pp. 293 - 304.
• NGG2 Expert Group, Next Generation Grids 2 – Requirements and Options for European Grids: Research 2005-2010 and Beyond. NGG2 Report, July 2004, http://www.cordis.lu/ist/grids.
• e-Science in Deutschland: F&E-Rahmenprogramm 2005 bis 2009. Vorgelegt von der D-Grid-Initiative, 6. Juli 2004. http://www.d-grid.de
www.zib.de/csr
