INFSO-RI-508833
Enabling Grids for E-sciencE
www.eu-egee.org
Introduction to GRID computing
Introduction GRID TutorialJules Wolfrat
SARA
Grid Tutorial, Groningen, September 2006 2
Enabling Grids for E-sciencE
INFSO-RI-508833
Definition of Grid
• From an EU brochure:– It doesn’t matter if your team is modeling the Earth’s
atmosphere, designing cars, creating animated films or finding new medicines, the basic principle is the same: your Grid supplies all the computing power, software, data and knowledge you need in one integrated package, and helps project teams work more closely together
• The analogy with the power grid:– Like you can plug in anywhere to the power grid without knowing
where your energy is coming from you can plug into the grid without knowing where your (computing) resources are coming from.
Grid Tutorial, Groningen, September 2006 3
Enabling Grids for E-sciencE
INFSO-RI-508833
History (1)
• From a news item in 1991– “Smarr describes the metacomputer as a network of
heterogeneous, computational resources linked by software in such a way that they can be used as easily as a personal computer”
– So the concept was introduced already in the early 90s, known as metacomputing.
– Motivation was the emergence of computer networks.
Grid Tutorial, Groningen, September 2006 4
Enabling Grids for E-sciencE
INFSO-RI-508833
Example (1)
Following is an example of the kind of initiatives started in those years from close by:
In 1996 a project was started in Amsterdam:
The Amsterdam Metacomputing project is an ongoing effort from the University of Amsterdam (UvA), the Free University (VU) and "Academic Computing Services Amsterdam" (SARA) to develop a Metacomputer environment on the Amsterdam campus.Important components of this environment will be: automatic distribution and monitoring of jobs over a network of computer systems, uniform access to files of other users from each place to work and to each computer system incorporated in the environment, distributed storage of data on various fileservers, automatic backup, migration and archiving, general availability of both commercial and public domain software on software servers, and a minimum of system management tasks. In this way scientists will be able to devote all of their time to their actual task: science.
Grid Tutorial, Groningen, September 2006 5
Enabling Grids for E-sciencE
INFSO-RI-508833
Example (2)
An extensive package of services will gradually be implemented and finally include the following components:
• fileservers and distributed, transparent file-systems; • backup, migration and archiving services; • batch-queueing systems, designed for efficient use of local systems, and if desired, of computational servers supplied by SARA;• public domain and specialist (commercial) software servers.
All components will be accessible from the scientist's desktop. A client-server architecture will play an important role. Combining components will be a relatively easy task, enhancing efficiency in terms of man-hours needed to accomplish a given task. These pages, as well as the Metacomputer are still in a development stage ……..
Grid Tutorial, Groningen, September 2006 6
Enabling Grids for E-sciencE
INFSO-RI-508833
Example (3)
Parsytec CC56 CPUsIBM SP2
76 CPUs
CRAY YMP Vector system
Systems available at SARA in 1996
Grid Tutorial, Groningen, September 2006 7
Enabling Grids for E-sciencE
INFSO-RI-508833
Example (4)
• SARA news item on 16-6-1998– Basis voor meta-omgeving gelegd.
– Sinds 4 mei maakt SARA's IBM RS/6000 SP parallelle supercomputer gebruik van de DCE/DFS omgeving, een filesysteem dat een transparante computeromgeving mogelijk maakt. Met het nieuwe filesysteem zijn bestanden van DCE/DFS gebruikers wereldwijd toegankelijk met andere computersystemen die beschikken over DCE/DFS, waarmee een belangrijke basis is gelegd voor de meta-omgeving.
– Gebruikers aan de VU science faculty hebben nu op een uniforme manier toegang tot hun bestanden, ongeacht of ze werken op de RS/6000 SP of een lokaal workstation. Hetzelfde geldt voor gebruikers van het Parsytec CC systeem bij SARA: vanaf zowel de Parsytec als de RS/6000 SP zijn alle bestanden voor de gebruiker direct toegankelijk.
Grid Tutorial, Groningen, September 2006 8
Enabling Grids for E-sciencE
INFSO-RI-508833
Example (5)
• A web interface was developed for submitting jobs to the metacomputing environment, also a meta job language was used.
• Also job migration between systems and mpi over two systems was investigated– First time we heard about globus, one of the well known building
blocks now for grid infrastructures.
– Network link between systems was a problem, only FE link, Gbit not available, HiPPI (800 Mbps) not available for Parsytec.
Grid Tutorial, Groningen, September 2006 9
Enabling Grids for E-sciencE
INFSO-RI-508833
Today (1)
• So what is new today? – Scale! Grid infrastructures operate worldwide
• International infrastructures - EGEE, DEISA, Nordugrid, OSG, TeraGrid
• National – NAREGI (Japan), UK-eScience, D-Grid, NLGrid
– Interoperability – availability of middleware – Globus toolkit, UNICORE, NAREGI, schedulers
Grid Tutorial, Groningen, September 2006 10
Enabling Grids for E-sciencE
INFSO-RI-508833
Today (2)
• Some basic requirements for a grid infrastructure– Transparent user administration – single sign on (single
grid identity), authorisation and accounting based on grid identity – AAA facilities
– Job scheduling – which can handle different environments
– Global data access
– Global information services – job information, data information, resource information
• Interoperability!– Standards needed for federation of infrastructures – GGF,
IETF….
Grid Tutorial, Groningen, September 2006 11
Enabling Grids for E-sciencE
INFSO-RI-508833
Networking (1)
• Developments in network connectivity (high bandwidths) and tools play an important role
– 10 Gbps WAN links available today, both shared links and dedicated lightpaths (based on lambda technology)
– 1 Gbps network adapters are commodity items on systems today and 10GE adapters available
Grid Tutorial, Groningen, September 2006 12
Enabling Grids for E-sciencE
INFSO-RI-508833
Networking (2)
– GridFTP can use multiple streams in order to take full advantage of available bandwidth
– Parallel files systems can take full advantage of underlying high speed networks - throughput can be in the order of 100MByte/s and more
– Tuning of WAN TCP must get attention, e.g. latencies are in the order of milliseconds (~20 in Europe), defaults on systems mostly not suited for bulk data transports.
Grid Tutorial, Groningen, September 2006 13
Enabling Grids for E-sciencE
INFSO-RI-508833
SURFnet6 DWDM on dark fiber
Dordrecht1
Breda1
Tilburg1
DenHaag
NLR
BT
BT NLR
BT
Zutphen1
Lelystad1
Subnetwork 4: Purple
Subnetwork 3: Red
Subnetwork 1: Green
Subnetwork 2: Dark blue
Subnetwork 5: Grey
Emmeloord
Zwolle1
Venlo1
Enschede1
Groningen1
LeeuwardenHarlingen
Den Helder
Alkmaar1
Haarlem1
Leiden1
Assen1
Beilen1
Meppel1
Emmen1
Arnhem
Apeldoorn1
Bergen-op-ZoomZierikzee
Middelburg
Vlissingen Krabbendijke
Breukelen1
Ede
Heerlen2Geleen1
DLO
Schiphol-Rijk
Wageningen1 Nijmegen1
Hilversum1
Hoogeveen1
Lelystad2
Amsterdam1
Dwingeloo1
Amsterdam2
Den Bosch1
Utrecht1
Beilen1
Nieuwegein1Rotterdam1
Delft1
Heerlen1
Heerlen1
Maastricht1
Eindhoven1
Maasbracht1
Rotterdam4
3XLSOP
IBG1 & IBG2Middenmeer1
Muenster
SURFnet 6 infrastructure
Grid Tutorial, Groningen, September 2006 14
Enabling Grids for E-sciencE
INFSO-RI-508833
NetherLight – Lightpath connections to the Netherlands
3rd quarter 2005
622M GLORIAD
GLORIAD-RU @NIKHEF
GE
Grid Tutorial, Groningen, September 2006 15
Enabling Grids for E-sciencE
INFSO-RI-508833
Global Lambda Integrated Facility (GLIF)
World Map
www.glif.isVisualization courtesy of Bob Patterson, NCSA/University of Illinois at Urbana-Champaign.Data compilation by Maxine Brown, University of Illinois at Chicago. Earth texture from NASA.
Grid Tutorial, Groningen, September 2006 16
Enabling Grids for E-sciencE
INFSO-RI-508833
GEANT2 topology
Grid Tutorial, Groningen, September 2006 17
Enabling Grids for E-sciencE
INFSO-RI-508833
The EGEE project
• EGEE– 1 April 2004 – 31 March 2006
– 71 partners in 27 countries, federated in regional Grids
– Operation of a pan European production Grid
• EGEE-II– 1 April 2006 – 31 March 2008
– Expanded consortium• 91 partners
• 11 Joint Research Units
– Natural continuation of EGEE
– Emphasis on providing production-level infrastructure• increased support for applications
• interoperation with other Grid infrastructures
• more involvement from Industry
Grid Tutorial, Groningen, September 2006 18
Enabling Grids for E-sciencE
INFSO-RI-508833
The EGEE infrastructure
• Mission– Manage and operate production e-Infrastructure open to all
user communities and service providers
– Contribute to Grid standardisation and policy efforts
• Infrastructure operation– Currently include ~200 sites across 39 countries
– Continuous monitoring of Grid services in a distributed global infrastructure
– Automated site configuration/management
• Future– Expand on interoperability with related infrastructures
Grid Tutorial, Groningen, September 2006 19
Enabling Grids for E-sciencE
INFSO-RI-508833
EGEE-II Federations and Countries
Grid Tutorial, Groningen, September 2006 20
Enabling Grids for E-sciencE
INFSO-RI-508833
Operational Organisation
Grid Tutorial, Groningen, September 2006 21
Enabling Grids for E-sciencE
INFSO-RI-508833
User Support Activities
Grid Tutorial, Groningen, September 2006 22
Enabling Grids for E-sciencE
INFSO-RI-508833
User support in NE region
• NE website: http://www.egee-ne.org/operations
• User support: contact user support at local site or mail to [email protected] – NE uses a ticketing system monitored by different
partners from our region. In NL NIKHEF, RC-RuG, SARA responsible.
– Tickets from GGUS are also imported in the NE system
• Application support – NA4 activity. In NL RC-RuG, SARA
Grid Tutorial, Groningen, September 2006
Enabling Grids for E-sciencE
INFSO-RI-508833
A Selection of Monitoring tools
1. GIIS Monitor 2. GIIS Monitor graphs 3. GOC Data Base
4. Scheduled Downtimes6. Live Job Monitor
5. GridIce – VO view
Grid Tutorial, Groningen, September 2006 24
Enabling Grids for E-sciencE
INFSO-RI-508833
Grid Tutorial, Groningen, September 2006 25
The DEISA project
• Objective: To enable Europe’s terascale science by the integration of Europe’s most powerful supercomputing systems.
• DEISA is an European Supercomputing Service built on top of existing national services. This service is based on the deployment and operation of a persistent, production quality, distributed supercomputing environment with continental scope.
Grid Tutorial, Groningen, September 2006 26GEAN
T
AIX distributedsuper-cluster
Vector systems(NEC, …)
Linux systems(SGI, IBM, …)
THE DEISA SUPERCOMPUTING GRID
Grid Tutorial, Groningen, September 2006 27
BSC Barcelona Supercomputing Centre Spain
CINECA Consortio Interuniversitario per il Calcolo Automatico Italy
CSC Finnish Information Technology Centre for Science Finland
EPCC/HPCx University of Edinburgh and CCLRC UK
ECMWF European Centre for Medium-Range Weather Forecast UK (int)
FZJ Research Centre Juelich Germany
HLRS High Performance Computing Centre Stuttgart Germany
IDRIS Institut du Développement et des Ressources France
en Informatique Scientifique - CNRS
LRZ Leibniz Rechenzentrum Munich Germany
RZG Rechenzentrum Garching of the Max Planck Society Germany
SARA Dutch National High Performance Computing The Netherlands
and Networking centre
Participating Sites
Grid Tutorial, Groningen, September 2006 28
DEISA technologies
• GPFS – parallel filesystem for transparent file access from all systems – dedicated European network used for high throughput
• Loadleveler-MC for job submission on AIX systems• UNICORE for job submission to all systems• Common Programming Environment (CPE) on all systems for
DEISA users• Single username on all systems
Grid Tutorial, Groningen, September 2006 29
Access
• Users can submit proposals for access to DEISA resources through DECI (DEISA Extreme Computing Initiative) calls
• Proposals are evaluated by national committees and depending on ranking get access to resources
• Most partners contribute about 10% of their resources for DEISA applications
• URL: www.deisa.org