InternationalGrid Computing
Enabled by HPIIS NetworksIan Foster
Mathematics and Computer Science DivisionArgonne National Laboratory
andDepartment of Computer Science
The University of Chicago
[email protected] ARGONNE CHICAGO
Overview Grid computing Importance of international cooperation My personal experiences with intl activities The emergence of an intl Grid community Emerging international Grid applications Future directions
[email protected] ARGONNE CHICAGO
Grid Computing Enabling new approaches to collaborative
problem solving based on resource sharing in geographically distributed communities– Much as the Internet enabled new approaches
based on remote collaboration– No central control, omniscience, strong trust
Two principal elements of an R&D agenda– Understanding these new approaches– Defining and creating the infrastructure required
to support resource sharing
[email protected] ARGONNE CHICAGO
Grid Communities and Applications:Natl Earthquake Engineering Simulation
NEESgrid: national infrastructure to couple earthquake engineers with experimental facilities, databases, computers, & each other
On-demand access to experiments, data streams, computing, archives, collaboration
NEESgrid: Argonne, Michigan, NCSA, UIUC, USC
[email protected] ARGONNE CHICAGO
What do we Mean by “Infrastructure”?
High-speed networks– Sine qua non for many applications
Protocols and services, e.g.– Authentication and authorization– Secure remote resource access– Resource discovery
Tools Community
Different elements closely interrelated
[email protected] ARGONNE CHICAGO
U.S. Experience: Infrastructure Builds Community Builds Infrastructure … Early 90s
– Gigabit testbeds, metacomputing Mid to late 90s
– Early experiments (e.g., I-WAY), academic software projects (e.g., Globus), application experiments
2000– Major application communities emerging– Major infrastructure deployments– Rich technology base– Grid Forum: >300 people, >90 orgs, 11 countries
[email protected] ARGONNE CHICAGO
International Grid Computing International cooperation is important
– Applications are often international in scope– Technology expertise is distributed– Cost of non-cooperation is high
International cooperation will not just happen but must be encouraged– Provide infrastructure– Engage application groups and computer
scientists, demonstrate feasibility– Build community
[email protected] ARGONNE CHICAGO
Creation of anInternational Grid Community
1995 I-WAY a defining event in the U.S. NSF PACIs, NASA Info Power Grid follow GUSTO launches large-scale intl cooperation HPIIS networks and iGrid events promote
international applications 1998-2000: Increasing international contacts 1999: Grid Forum and eGrid formed 2000: European Data Grid, GriPhyN, PPDG March 2001: 1st Global Grid Forum meeting
[email protected] ARGONNE CHICAGO
International Grid Computing:Mathematicians Solve NUG30
Community=an informal collaboration of mathematicians and computer scientists
Condor-G delivers 3.46E8 CPU seconds in 7 days (peak 1009 processors) in U.S. and Italy (8 sites)
Solves NUG30 quadratic assignment problem
14,5,28,24,1,3,16,15,10,9,21,2,4,29,25,22,13,26,17,30,6,20,19,8,18,7,27,12,11,23
MetaNEOS: Argonne, Iowa, Northwestern, Wisconsin
[email protected] ARGONNE CHICAGO
Grid Physics Network (GriPhyN) Enabling R&D for advanced data grid systems,
focusing in particular on Virtual Data concept
Virtual Data Tools Request Planning and Scheduling Tools
Request Execution Management Tools
Transforms
Distributed resources(code, storage,computers, and network)
Resource Management
Services
Resource Management
Services
Security and Policy
Services
Security and Policy
Services
Other Grid Services
Other Grid Services
Interactive User Tools
Production Team
Individual Investigator Other Users
Raw data source
ATLASCMSLIGOSDSS
[email protected] ARGONNE CHICAGO
Grid Forum History[Participants – Institutions – Countries]
November 1998: BOF at SC98– Chairs Ian Foster (ANL), Bill Johnston (LBL/NAS)
June 1999: GF1 (San Jose — NASA Ames) [150 – 50 – 4]– Initial structure discussions; formation of working groups
October 1999: GF2 (Chicago — iCAIR) [130 – 60 – 4]– Focused on working group progress and Structure
March 2000: GF3 (San Diego — SDSC) [160 – 70 – 7]– Finalized charters, begin detailed work
June 2000: GF4 (Redmond — Microsoft) [120 – 60 – 6] – Increase in industry participation, first Grid Working Drafts
October 2000: GF5 (Boston — Sun) [190 – 95 – 11]– New WG, increased industry participation
March 2001: GF6/Global Grid Forum 1 (Amsterdam)– Merging efforts with European Grid Forum (eGrid)
[email protected] ARGONNE CHICAGO
Current Grid Forum Groups
Grid Forum eGrid
ApplicationsAccounting
Adv. Prog. Models
Grid Performance
Scheduling & Rsrc Mgmt
Grid User Services
Grid Info Services
Remote Data Access
Security
Applications & Testbeds
Programming Models
Data Access
Resource Management
Performance Analysis
Security
Object Oriented
Steering GrpSteering Grp
[email protected] ARGONNE CHICAGO
Proposed Global Grid Forum
Global Grid Forum
Applications
Adv. Prog. Models
Grid Performance
Scheduling & Rsrc Mgmt
Remote Data Access
Security
Applications & Testbeds
Programming Models
Data Access Resource Management
Performance Analysis Security
Accounting Grid User Services
Grid Info Services
Object Oriented
Steering Group
Advisory Board
• Single leadership team• Annual Global Grid Forum Meeting alternating among regions• Challenge: working group coordination
[email protected] ARGONNE CHICAGO
The Future Current HPIIS networks have only served to
stimulate interest! Rapid evolution of Grid technologies and
application concepts is increasing appetite for network capabilities
We are ready for far more aggressive R&D activities– “TransOceanic Gigabit->Terabit Testbed”– Production applications in numerous scientific
applications—and humanities?
[email protected] ARGONNE CHICAGO
Summary International cooperation is particularly
important for Grid computing HPIIS has helped to stimulate the creation of an
international Grid community– Dozens of international partnerships– Global Grid Forum: >300 people, >90 orgs, 11
countries A tremendous success! Timely to start thinking about how to build on
this success in the future