PDCEnabling Science

Globus – status & perspective

Olle Mulmo <?????@pdc.kth.se>

PDCEnabling Science

What this talk will cover

• Background: Why we are Where we are today• Where we are today• Where we will go tomorrow

[Insert proper acknowledgement to everyoneI have stolen slides from here]

PDCEnabling Science

Metacomputing...

• A concept conceived of in the (late?) ’80s• 1990’s: Dramatic increase in network bandwitdh

• Opens up for entire new problem areas• Solving ”larger-than-a-supercomputer” problems• Remote visualization• Remote instrumentation• Collaboration (VR)

• 1995:• I-WAY, proof-of-concept demo/project

• 1996:• The Globus Project sees the day

PDCEnabling Science

The Globus approach

• Middleware approach• A set of orthogonal APIs

that applications can use• Command-line tools that

use the APIs (for scripting)• IP hourglass model

• Portability requirements• No perl on the Cray at the time• Poor C++ support• -> Implementation using bourne shell

(sh) and ANSI C only

Diverse global services

Coreservices

Local OS

A p p l i c a t i o n s

PDCEnabling Science

Hot technologies half a decade ago

• Network communication• Raw ATM vs TCP/IP?• Use Nexus to avoid technology lock-in• Already ported to several UNIX flavors• Automatic data conversion• Optimization for local communication (AAL5,TCP,MPI,SHMEM)

• Information Discovery• Queryable, Extensible (”give me a computer with 128M RAM”)• Use LDAP (Lightweight Directory Access Protocol)• Add TTL on all published data• Add heart beats to know the test bed status

PDCEnabling Science

Hot technologies half a decade ago (cont.)

• Security• Hot potato: Local control a must• -> mapfiles (what users running on what accounts)• PKI evolving, big push from SSL• -> let’s use it

• Resource Allocation• No standard: myriads of different local schedulers• -> RSL (Resource Specification Language) + wrappers

PDCEnabling Science

First round: the GUSTO testbed

• 1997: The Globus Metacomputing Toolkit• MDS (Metacomputing Directory Service)• GRAM (Globus Resource Allocation Manager)• DUROC (multi-site coordinated startup)

• No security (plain-text passwords)• No file transfers

PDCEnabling Science

The GUSTO testbed (cont.)

• 1998: Globus release 1.0• Security added: GSI (Globus Security Infrastructure)

• PKI (or Kerberos) below GSS-API• SSL handshakes• Proxy certificate delegation• No encryption due to export laws

• File transfer added:• GASS (fetch files over https:// or file://)

PDCEnabling Science

The GUSTO testbed (cont.)

• ... not really that heavily used, though...

                                                 

PDCEnabling Science

Lessons learned from Globus v 1.x

• Severe performance problems with MDS• A centralized LDAP server didn’t scale very well• -> MDS-2 framework (GRIS and GIIS)

• Complaints on configuration and build times• You got everything and all the ”flavors”• debug,nodebug, nothreads,pthreads,native threads, with MPI,

without MPI, ...• -> Packaging technology

• One of the multi-site jobs often got stuck in a queue / poor network performance• -> GARA (Advanced reservation)

PDCEnabling Science

Lessons learned from Globus v 1.x

• Users don’t want to hack their favorite applications• Easy access

• They have NT on their desktops• Mass storage accessibility is important • -> Data and replica management work begins

PDCEnabling Science

1999: The year of the Grid

• The Concept of a Grid evolves• Resource sharing & coordinated problem solving in dynamic,

multi-institutional virtual organizations

PDCEnabling Science

1999-now: The Grid boom...

• The Grid book covers a lot of groundin the community

• Lots of grid projects spring to life• Increase in funding• Commercial interests (Microsoft,Sun,IBM,...)

• Grid Forum is founded• ”The place to standardize Grid-related stuff”• Many of the protocols and approaches tried

out by Globus are adopted as a first shot

PDCEnabling Science

Some renaming is required...

• GSI -> Grid Security Infrastructure• GRAM -> Grid Resource Allocation Management• MDS -> Monitoring and Discovery Service

• Never say ”Metacomputing” again...

PDCEnabling Science

Back to the drawing board: Globus v 2.0

• New packaging technology (not RPM)• Precompiled binaries• GridFTP• MDS-2: decentralized LDAP framework• Security on all communications channels

• Repackaged with add-ons by several DataGrid projects• GriPhyN, EDG, NMI, ...

PDCEnabling Science

Where we are today

• The Globus Toolkit™ centers around four key protocols• Connectivity layer:

• Security: Grid Security Infrastructure (GSI)

• Resource layer:• Resource Management: Grid Resource Allocation Management

(GRAM)

• Information Services: Grid Resource Information Protocol (GRIP) and Index Information Protocol (GIIP)

• Data Transfer: Grid File Transfer Protocol (GridFTP)

PDCEnabling Science

A second look

• Many different protocols and methods are used• LDAP, HTTP, FTP, ...

• Missing or lack of support for• Databases• Notification systems• Error propagation

• The security part needs more work• Dynamic, Multi-VO Authorization schemes• Audit• Proxy delegation (IETF draft not likely to become an RFC)

PDCEnabling Science

Let’s go shopping: Hot technologies of today

• XML & Web services• Generic and extensible• Many tools available• Backed up by industry

• Attribute Certificates• Give a certificate holder certain privilegies (access to files,

resources,...)• NMI is pursuing this path• -> Globus ships CAS (similar concept)

PDCEnabling Science

Web services means generic encapsulation

• ”Corba rewritten in XML” (and then some)• WSDL: Web Services Definition Language

• Defines a service and what methods that can be invoked• Stub compilers exist for several programming languages• -> Easy to wrap your old application

• SOAP: Simple Object Access Protocol• Invocation protocol, operating on an instance of a WSDL target

• WS-Inspection and UDDI• Conventions (naming etc.) for querying and locating WSDL

documents with certain properties

PDCEnabling Science

Open Grid Services Architecture (OGSA)

• Web Services for the Grid• WSDLs for some key components

• Factory, Registry, Grid Service• Dynamic properties (lifetime, version compatibility) are added• Open-source implementation in Java, .NET & C

• Guidelines, conventions and best practices• Global naming• Lifetime managment• Notification framework• ...

PDCEnabling Science

OGSA

• Still a draft (continously revised)• Alpha release of Core implementation (Java)

• The Factory and generic Grid Service part

• Base implementation next• Current Globus functionality refactored

as Grid Services• Alpha at the end of this summer

• Globus 3.0 (Q1 ’03?)• Old APIs will still work (or so they say)• ...but the wire protocols are new GT3 Core

GT3 Base Services

Other GridServicesGT3

DataServices

PDCEnabling Science

Next steps: your guess is as good as mine...

• Experiences learned with v2 and v3• Access to tertiary storage and databases• How much can be gotten ”for free” from other WS

initiatives?• Other forms of security infrastructures?

• Passport / Liberty Alliance technologies• ”MyProxy” effort

• Continued work on authorization & delegation