Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 1

GridPP

Steve LloydQueen Mary, University of LondonGrid Brokering Meeting December

2006

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 2

19 UK Universities, CERN and

CCLRC (RAL & Daresbury) Funded by PPARC:

GridPP1 2001-2004 “From Web to Grid”

GridPP2 2004-2007 “From Prototype to Production”

GridPP3 2007-2011 (proposed)“From Production to

Exploitation”

Who are GridPP?

Developed a working, highly functional Grid

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 3

Why? - The CERN Large Hadron Collider - LHC

4 Large Experiments

The world’s most powerful particle accelerator – Starting 2007/8

• ~100,000,000 electronic channels

• 800,000,000 proton-proton interactions per second

• 0.0002 Higgs per second• 10 PBytes of data a year • (10 Million GBytes = 14 Million

CDs)

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 4

International Context

LHC Computing Grid (LCG)Grid Deployment Project for

the Large Hadron Collider (LHC)

EU Enabling Grids for E-SciencE (EGEE) 2004-2008Grid Deployment Project for all disciplines

GridPP LCG

EGEE

GridPP is part of EGEE and LCG (currently the largest Grid in the world)

UK National Grid ServiceUK’s core production computational and data Grid

Open Science Grid (USA)Science applications from HEP to biochemistry

Nordugrid (Scandinavia)Grid Research and Development collaboration

UK part of LCG

PP part of EGEE

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 5

What is (gLite) Middleware?

MIDDLEWARE

CPUDisks, CPU etc

PROGRAMS

OPERATING SYSTEM

Word/Excel

Email/Web

Your Progra

mGames

CPUCluste

r

UserInterfac

eMachine

CPUCluste

r

CPUCluste

r

Resource Broker

Information Service

Single PC

Grid

DiskServer

Your Progra

m

Middleware is the Operating System of a distributed computing system

Replica CatalogueBookkeepin

g Service

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 6

GridPP Middleware Development

Workload Management

Storage Interfaces

Network Monitoring

SecurityInformation Services

Grid Data Management

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 7

The EGEE Grid Status Worldwide

207 Sites

48 Countries

32,823 CPUs

13.1 PB Disk

6,946 Years of CPU time

UK

21 Sites

6306 CPUs

131 TB Disk

1,996 Years of CPU time

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 8

UK Tier-1 and Tier-2 SitesMostly funded by HEFCE

Tier-1

  CPU (KSI2K) Disk (TB)Brunel 116 1.4Imperial 368 8.4QMUL 1209 18.0RHUL 163 8.8UCL 121 1.1Lancaster 484 72.0Liverpool 592 2.8Manchester 1152 145.0Sheffield 183 2.0Durham 99 4.0Edinburgh 11 60.0Glasgow 47 1.6Birmingham 223 9.3Bristol 12 3.8Cambridge 40 4.4Oxford 150 27.0RAL PPD 320 6.4London 1977 37.7NorthGrid 2411 221.8ScotGrid 157 65.6SouthGrid 745 50.9Total Tier-2 5290 376.0Tier-1 1034 180.0

Normally most recent gLite versions.

Still running the LCG RB and CE (not gLite).

SEs being upgraded to SRM 2.2.

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 9

What you need to use the Grid

1. Get a digital certificate (UK Certificate Authority)

2. Join a Virtual Organisation (VO)

3. Get access to a local User Interface Machine (UI) and copy your files and certificate there

Authentication – who you are

Authorisation – what you are allowed to do

4. Write some Job Description Language (JDL) and scripts to wrap your programs

############# HelloWorld.jdl #################Executable = "/bin/echo";Arguments = "Hello welcome to the Grid ";StdOutput = "hello.out";StdError = "hello.err";OutputSandbox = {"hello.out","hello.err"};#########################################

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 10

How it works

Job Descriptio

n

UI Machine

Resource Broker

Input SandboxScript you

want to run

Other files (Job

Options, Source...)

Storage Element

Compute Element

Storage Element

Grid

Proxy Certificate

Output SandboxOutput

files (Plots, Logs...)

Input Data

Output Data

Job

Output Sandbo

x

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 11

What is it good for?

Problems that are highly parallelizable

Problem

Grid

Solution

Input data is independent e.g.

Images:

A=2B=3 A=3

B=3 A=2B=4

Simulation using different

parameters:

Not so good for closely coupled

problems

These pieces may be independent

These pieces will have to interact

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 12

GridPP OutreachCurrent GridPP Outreach Team:

0.5 FTE Dissemination Officer

1.0 FTE Events Officer

Proposed GridPP Outreach Team (GridPP3):

1.0 FTE Dissemination Officer0.5 FTE Industrial Liaison Officer0.5 FTE (User) Documentation

GridPP WebsiteEvents and DemonstrationsPress Releases and News ItemsDissemination Awards (e.g. Schools)Posters, Brochures, Papers

Expand Industrial LiaisonTechnology Transfer Steering GroupImproved User DocumentationTraining e.g. Use of Portal

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 13

GridPP and Industry

• Use of our Grid• Access to the Certificate

Authority• Security tools• GridSite Secure Web Toolkit• R-GMA Information System• APEL Accounting tools• Help getting PPARC funding

What We Have to Offer:

Current Involvement:

• GridPP has a collaboration with IBM through ScotGrid and R-GMA.

• Specific sites also have close relationships with various industrial suppliers.

• One PPARC MiniPPIPS with a startup company.

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 14

Cost Recovery Survey GridPP attitudes to allowing commercial access to

resourcesBirmingham Keen to share if not too time consuming

CambridgeConcerned about organisational issues but happy for limited access

Glasgow Will allow access for limited time

Oxford Concerned about organisational issues

Queen MaryPrefer to offer full technical and research support rather than ‘dumb’ access

Royal Holloway Concerned about diversion of effort

Generally positive as long as not too time consuming and (University) bureaucracy can be overcome

Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 15

Further Information …

http://www.gridpp.ac.uk

RSS News feed