www.cactuscode.org www.gridlab.org

Introduction to Grid Computing

Ed SeidelMax Planck Institute for

Gravitational Physicseseidel@aei.mpg.de

Grid Intro 2

OutlineOutline

What is the Grid. What does the Grid mean for application

users? What is a Virtual Organization? Why is the Grid different from the Internet? What should application users and

developers be doing right now to be ready for the Grid?

High level overview of today’s testbeds.

Grid Intro 3

History and MotivationHistory and Motivation

1970s - 80s: Internet Email, ftp, etc

1980s - 90s: Remote-, Distributed-, Meta-computing Beginning to think of ways to exploit distributed resources US SC Centers Programs

– Had to access remotely– Think of harnessing together

See examples later SC95: I-Way

First large scale experiments Most failed, but I-Way a success!

Post I-Way: The Grid Explosion of activity, but still immature. Get prepared for

future

Grid Intro 4

Grid Computing: A New ParadigmGrid Computing: A New Paradigm

Computational Resources Scattered Across the World Compute servers (double each 18 months) Handhelds File servers Networks (double each 9 months) Playstations, cell phones etc…

How to take advantage of this for scientific/engineering simulations?

Harness multiple sites and devices Simulations at new level of complexity and scale, interacting with data

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture.

QuickTime™ and a Photo decompressor are needed to see this picture.

Grid Intro 5

Grid vs. Internet/Web Services?Grid vs. Internet/Web Services?

We’ve had computers connected by networks for 20 years

We’ve had web services for 5-10 years The Grid combines these things, and brings

additional notions Virtual Organizations Infrastructure to enable computation to be carried

out across these – Authentication, monitoring, information, resource

discovery, status, coordination, etc

Can I just plug my application into the Grid? No! Much work to do to get there!

Grid Intro 6

All have to work for real applicationsCommunities Teams with a need to share data, or develop common code Communities can be very diverse

Experimentalists, computer scientists, simulation scientists, network researchers

All may need to work together to solve a specific problem May co-develop a code, or contribute modules that should work

together These communities may have very different resources

Within company or university Across the world

Grids aim to bring them together, allow them to harness their resources

These communities may form a type of (virtual) organization

Components for Grid ComputingComponents for Grid Computing

Grid Intro 7

Virtual OrganizationsVirtual OrganizationsResources Machines, networks, archives, file systems, etc at different sites What is a Virtual Organization (VO)?

groups of organizations that use the Grid to share resources for specific purposes

– EU DataGrid, Alliance, TeraGrid, SC02 Global Grid Testbed, etc Typically deploy same technology Deploy directory service: resources registered, and may be

“discovered”. Globus has MDS (Monitoring and Discovery Service)– GIIS (Grid Index Information Service) provides info for entire VO– GRIS (Grid Resource Information Service), installed locally, reports to

GIIS so people or applications can search GIIS for info Authentication: Certificates and Gridmap file

– Sometimes a VO offers a “certificate” for individuals and resources– Sometimes they use other VO’s certificates

So far, there is no Global GIIS that links together all VOs

Grid Intro 8

Software InfrastructureSoftware Infrastructure

Infrastructure: Globus Metacomputing Toolkit Low Level

Fundamental technologies needed to build computational grids.  Security: logins, data transfer Communication Information (GRIS, GIIS) More generally: Grid Services

Middleware Data movers Resource Brokers Portals Application Monitoring systems

High Level Application Toolkits

We focus today on Globus, Cactus, and GridLab Other: Legion, Unicore, Juxta…

Grid Intro 9

Grid Aware Applications (Cactus example): Grid Enabled Modular Toolkits for Parallel

Computation: Provide to Scientist/Engineer, etc.. Plug your Science/Eng. Applications in! Must Provide, Register as Grid Services

Ease of Use: automatically find resources, given need! Distributed simulations: use as many machines as

needed! Remote Viz and Steering, tracking: watch what happens! Take advantage of infrastructure

Collaborations of groups with different expertise: no single group can do it! Grid is natural for this…

Components for Grid ComputingComponents for Grid Computing

Grid Intro 10

Example GridsExample Grids

GridLab Testbed Ten machines in Europe for developers of Grid tools

SC2001 ARG Testbed & Global Grid Testbed Collaboration Hastily assembled loose federation of world machines for SC2001

and now SC2002 demonstrations NCSA Virtual Machine Room and PACI Grid

Production resources TeraGrid (www.teragrid.org)

USA distributed terascale facility at 4 sites for open scientific research

Information Power Grid (www.ipg.nasa.gov) NASAs high performance computing grid

Grid Intro 11

Example GridsExample Grids

NSF PACI Grid

NorduGrid

Grid Intro 12

The NSF TeraGridThe NSF TeraGrid

Some Grids are tightly coupled facilities, like TeraGrid Some are testbeds, like Global Grid Testbed Collaboration Some are long term, loosely coupled: PACI Grid (includes

TeraGrid) All try to provide organized access to distributed resources Most are depending on Globus as the base infrastructure

2624

85

NCSASDSC

Caltech Argonne

13.6 Tflop facility

Grid Intro 13

What does this mean now for users and developers?What does this mean now for users and developers?

There is a grand vision of the future Collecting resources around the world into Vos Seamless access to them, with a single signon NEW applications to exploit of them in unique ways! Today we want to help you prepare for this

There is a frustrating reality of the present These technologies are not yet fully mature Not fully deployed Not consistent across even single Vos

But centers and funding agencies worldwide are pushing this very, very hard Better get ready now You can help! Work with your centers to get this deployed

Grid Intro 14

Getting Ready for the GridGetting Ready for the Grid

Need to start imagining how your applications can exploit the Grid (simplify use of HPC, provide more processing power, better access to data, allow new scenarios to forward your science, better collaborations).

Applications codes will need modifications for the Grid, depending on much of it you want to be able to exploit!

Write new codes with the Grid in mind. Consider using frameworks which are

already Grid-compatible.