Tony Doyle [email protected] Overview of UK Development and Deployment Programme, LCG PEB...

Tony [email protected]

“Overview of UK Development and Deployment Programme”, LCG PEB Meeting, CERN, 16 September 2003

Tony Doyle - University of Glasgow

OutlineOutline

• Management

• The Project Map

• GridPP Status

• UK Grid Users

• Deployment – LCG and UK perspective

• Current Resources

• EDG 2.0/LCG 1.0 Deployment Status

• Accounting

• Today’s Operations

• Future Operation Planning

• Middleware Status

• Middleware Evolution

• GridPP2 Planning Status


Institutes

GridPP

GridPP in ContextGridPP in Context

Core e-Science

Programme

GridPP CERNLCGTier-1/A

Middleware

Experiments

Tier-2

GridSupportCentre

EGEE

Not to scale!

Apps DevApps

Int


GridPP ManagementGridPP Management

CB (20 members) meets half-yearly to provide Institute overview

PMB (12 members) meets weekly [via VC] to provide management of project

TB (10 members) meet as required in response to technical needs and regularly via phone

EB (14 members) meet quarterly to provide experiments input


GridPP Project OverviewGridPP Project Overview

1. 1 2. 1 3. 1 4. 1 5. 1 6. 1 7. 1

1. 2 2. 2 3. 2 4. 2 5. 2 6. 2 7. 2

1. 3 2. 3 3. 3 4. 3 5. 3 6. 3 7. 3

1. 4 2. 4 3. 4 4. 4 5. 4

1. 5 2. 5 3. 5 4. 5

Navigate down

External link 2. 6 3. 6 4. 6 Link to goals

2. 7 3. 7

2. 8 3. 8

System

UK Grid Rollout

Data Challenges

Dissemination

Tier-1 Centre

Tier-A Centre

Applications

CERN DataGrid Applications

LCG Creation WP1

WP2

ResourcesInfrastructure

Presentation Deployment of

related areasGANGA/GaudiATLAS/LHCb

resources

Participation in

of GridPP

Monitoring of

UK e-Science

Computing Fabric

Grid Technology

Tier-2 Centres

CMSMonte Carlo

WP3

WP4

LHCb

Standards

Open SourceImplementation

Grid Deployment

resourcesEngagementof UK groups

Attract newParticle Physics

UK Testbed

resources

WP6

WP7

WP5

SAM Framework

UKQCD

BaBar

Integration

Integration

Applications

QCD Application

Other

Data Analysis

CDF/D0

GridPP Goal

ATLAS

in the UK for the use of the Particle Physics communityTo develop and deploy a large scale science Grid

5Interoperability

International

WP8

6 71 2 3 4

Worldwide


Financial BreakdownFinancial Breakdown

6/May/2003

£3.8m

£5.67m

£3.67m

£2.08m£1.78m

CERN

DataGrid

Tier - 1/A

ApplicationsOperations

• Five components– Tier-1/A = Hardware + 10 CLRC e-Science Staff

– DataGrid = 25 DataGrid Posts inc. CLRC PPD Staff

– Applications = 17 Experiments Posts (to interface middleware)

– Operations = Travel (~100 people)+ Management + Early Investment

– CERN = 25 LCG posts + Tier-0 + LTA


Quarterly ReportingQuarterly Reporting

Q9 Q10 Q11 Q12

Actual Planned Actual Planned Actual Planned Actual Planned

1.50 0.60 1.30 0.60 0.60 0.60

2.25 1.50 2.50 1.50 1.50 1.50

0.75 1.50 0.50 1.50 1.50 1.50

3.00 3.00 3.00 3.00 3.00 3.00

1.50 0.54 1.60 2.40 2.40 2.40

0.38 1.20 0.60 0.00 0.00 0.00

1.28 0.99 1.00 0.90 0.90 0.90

0.39 0.00 0.00 0.00 0.00

2.40 2.40 2.40 2.40 2.40 2.40

0.90 1.20 0.00 1.20 1.20 1.20

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

1.80 0.00 2.00 1.50 1.50 1.50

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00

0.00 3.00 3.00 2.00 0.00

1.50 1.50 1.00 1.50 1.50 1.50

1.50 1.50 1.50 1.50 1.50 1.50

3.00 3.00 2.00 3.00 3.00 1.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

1.50 1.50 1.50 1.50 1.50 1.50

1.75 1.50 1.50 1.50 1.50 1.50

1.75 1.50 1.50 1.50 1.50 1.50

1.50 1.50 1.50 1.50 1.50 1.50

1.50 1.50 1.50 1.50 1.50 1.50

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 2.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

1.50 1.50 1.50 1.50 1.50 1.50

1.90 1.50 1.40 1.50 1.50 1.50

1.50 1.50 1.50 1.50 1.50 1.50

1.50 1.50 1.50 1.50 1.50 1.50

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

3.00 3.00 3.00 3.00 3.00 3.00

0.00 0.00 1.50 1.50

1-Jul-031-Jan-03 1-Apr-03 1-Oct-03Q Start Date

Initial Surname WG ExptPost/DescriptionInstituteRoute

J Gordon WP5 eSC RALL Cornwall WP3 PPD RALL Cornwall WP7 PPD RALS Fisher WP3 PPD RALD Kelsey WP7 PPD RALD Kelsey WP6 PPD RALB Saunders WP6 PPD RALT Folkes WP5 eSC RALA Sansum WP6 eSC RALR Tasker WP7 BITD RALR Tam WP5 PPD RALS Traylen WP6 PPD RALJ Jensen WP5 eSC RALG Kuznetsov LHCb PPD RALS Burke WP8 EUDG - WP8 topupRALT Shah WP5 PPD RALD Colling WP1 DGPP: WP6/WP1ImperialD Colling WP6 DGPP: WP6/WP1ImperialW Bell WP2 DGPP: WP2GlasgowR Cordenonsi WP3 DGPP: WP3QMULA McNab WP6 DGPP: WP6ManchesterC Cioffi WP8 LHCb DGPP: WP8OxfordM Gardner WP8 Atlas DGPP: WP8RHULH Talini WP8 CMS DGPP: WP8ImperialF Brochu WP8 Atlas DGPP: WP8CambridgeA Washbrook WP4 DGPP: WP5/WP4LiverpoolA Washbrook WP5 DGPP: WP5/WP4LiverpoolO Moroney WP6 DGPP: WP6BristolO Moroney WP8 CMS DGPP: WP8BristolP Mealor WP7 DGPP: WP7UCLG Fairey WP7 DGPP: WP7 + spptManchesterG McCance WP2 DGPP: WP2GlasgowA Holt WP4 DGPP: WP4EdinburghA Soroko ATLAS GANGA1OxfordK Harrison LHCb GANGA2CambridgeA Tan ATLAS ATLAS BirminghamB MacEvoy CMS CMS ImperialJ Nebrensky CMS CMS BrunelT Barrass CMS CMS BristolT Barrass BaBar Babar DBBristolJ Martyniak BaBar Babar JSImperialA Forti BaBar Babar MDManchesterR Walker D0 SAM 1 ImperialA Flavell CDF SAM 2 GlasgowS Stonjek CDF SAM 3 OxfordD Evans D0 D0 LancasterJ Perry UKQCD UKQCD EdinburghM Egbert UKQCD UKQCD Edinburgh

Quarterly reporting allows comparison of delivered effort with expected effort

Feedback loop as issues arise


Funded Effort Breakdown Funded Effort Breakdown (Snapshot 2003Q3)(Snapshot 2003Q3)

Applications18%

LCG35%

EDG30%

Tier-1/A11%

Management6%

LCG effort is largest single area of GridPP

Future project priorities focussed on LCG and EGEE


GridPP Status: The Project MapGridPP Status: The Project Map

1 . 1 2 . 1 3 . 1 4 . 1 5 . 1 6 . 1 7 . 1

1 . 1 . 1 1 . 1 . 2 1 . 1 . 3 1 . 1 . 4 2 . 1 . 1 2 . 1 . 2 2 . 1 . 3 2 . 1 . 4 3 . 1 . 1 3 . 1 . 2 3 . 1 . 3 3 . 1 . 4 4 . 1 . 1 4 . 1 . 2 4 . 1 . 3 4 . 1 . 4 5 . 1 . 1 5 . 1 . 2 5 . 1 . 3 6 . 1 . 1 6 . 1 . 2 6 . 1 . 3 6 . 1 . 4 7 . 1 . 1 7 . 1 . 2 7 . 1 . 3 7 . 1 . 41 . 1 . 5 2 . 1 . 5 2 . 1 . 6 2 . 1 . 7 2 . 1 . 8 3 . 1 . 5 3 . 1 . 6 3 . 1 . 7 3 . 1 . 8 4 . 1 . 5 4 . 1 . 6 4 . 1 . 7 4 . 1 . 8 6 . 1 . 5

2 . 1 . 9 3 . 1 . 9 3 . 1 . 1 0 4 . 1 . 9

1 . 2 2 . 2 3 . 2 4 . 2 5 . 2 6 . 2 7 . 2

1 . 2 . 1 1 . 2 . 2 1 . 2 . 3 1 . 2 . 4 2 . 2 . 1 2 . 2 . 2 2 . 2 . 3 2 . 2 . 4 3 . 2 . 1 3 . 2 . 2 3 . 2 . 3 3 . 2 . 4 4 . 2 . 1 4 . 2 . 2 4 . 2 . 3 4 . 2 . 4 5 . 2 . 1 5 . 2 . 2 5 . 2 . 3 6 . 2 . 1 6 . 2 . 2 6 . 2 . 3 7 . 2 . 1 7 . 2 . 2 7 . 2 . 31 . 2 . 5 1 . 2 . 6 2 . 2 . 5 2 . 2 . 6 2 . 2 . 7 3 . 2 . 5 3 . 2 . 6 3 . 2 . 7 3 . 2 . 8 4 . 2 . 5 4 . 2 . 6 4 . 2 . 7

3 . 2 . 9

1 . 3 2 . 3 3 . 3 4 . 3 5 . 3 6 . 3 7 . 3

1 . 3 . 1 1 . 3 . 2 1 . 3 . 3 1 . 3 . 4 2 . 3 . 1 2 . 3 . 2 2 . 3 . 3 2 . 3 . 4 3 . 3 . 1 3 . 3 . 2 3 . 3 . 3 3 . 3 . 4 4 . 3 . 1 4 . 3 . 2 4 . 3 . 3 4 . 3 . 4 5 . 3 . 1 5 . 3 . 2 5 . 3 . 3 6 . 3 . 1 6 . 3 . 2 6 . 3 . 3 6 . 3 . 4 7 . 3 . 1 7 . 3 . 2 7 . 3 . 3 7 . 3 . 41 . 3 . 5 1 . 3 . 6 2 . 3 . 5 2 . 3 . 6 2 . 3 . 7 3 . 3 . 5 3 . 3 . 6 4 . 3 . 5

1 . 4 2 . 4 3 . 4 4 . 4 5 . 4

1 . 4 . 1 1 . 4 . 2 1 . 4 . 3 1 . 4 . 4 2 . 4 . 1 2 . 4 . 2 2 . 4 . 3 2 . 4 . 4 3 . 4 . 1 3 . 4 . 2 3 . 4 . 3 3 . 4 . 4 4 . 4 . 1 4 . 4 . 2 4 . 4 . 3 4 . 4 . 4 5 . 4 . 1 5 . 4 . 2 5 . 4 . 3 5 . 4 . 41 . 4 . 5 1 . 4 . 6 1 . 4 . 7 1 . 4 . 8 2 . 4 . 5 2 . 4 . 6 2 . 4 . 7 3 . 4 . 5 3 . 4 . 6 3 . 4 . 7 3 . 4 . 8 4 . 4 . 5 4 . 4 . 6 5 . 4 . 51 . 4 . 9 3 . 4 . 9 3 . 4 . 1 0 M e t r ic O K 1 . 1 . 1

M e t r ic n o t O K 1 . 1 . 1 1 . 5 2 . 5 3 . 5 4 . 5 T a s k c o m p le t e 1 . 1 . 1

1 . 5 . 1 1 . 5 . 2 1 . 5 . 3 1 . 5 . 4 2 . 5 . 1 2 . 5 . 2 2 . 5 . 3 2 . 5 . 4 3 . 5 . 1 3 . 5 . 2 3 . 5 . 3 3 . 5 . 4 4 . 5 . 1 4 . 5 . 2 4 . 5 . 3 4 . 5 . 4 T a s k o v e r d u e 1 . 1 . 11 . 5 . 5 1 . 5 . 6 1 . 5 . 7 1 . 5 . 8 2 . 5 . 5 2 . 5 . 6 2 . 5 . 7 3 . 5 . 5 3 . 5 . 6 3 . 5 . 7 6 0 d a y s 1 . 1 . 11 . 5 . 9 1 . 5 . 1 0 T a s k n o t d u e s o o n 1 . 1 . 1

N o l o n g e r a c t i v e 1 . 1 . 1 2 . 6 3 . 6 4 . 6 N o T a s k o r m e t r i c

2 . 6 . 1 2 . 6 . 2 2 . 6 . 3 2 . 6 . 4 3 . 6 . 1 3 . 6 . 2 3 . 6 . 3 3 . 6 . 4 4 . 6 . 1 4 . 6 . 2 4 . 6 . 32 . 6 . 5 2 . 6 . 6 2 . 6 . 7 2 . 6 . 8 3 . 6 . 5 3 . 6 . 6 3 . 6 . 7 3 . 6 . 8 N a v i g a t e u p

2 . 6 . 9 3 . 6 . 9 3 . 6 . 1 0 3 . 6 . 1 1 3 . 6 . 1 2 N a v i g a t e d o w n

E x t e r n a l li n k 2 . 7 3 . 7 L i n k t o g o a l s

2 . 7 . 1 2 . 7 . 2 2 . 7 . 3 2 . 7 . 4 3 . 7 . 1 3 . 7 . 2 3 . 7 . 3 3 . 7 . 42 . 7 . 5 2 . 7 . 6 2 . 7 . 7 2 . 7 . 8 3 . 7 . 5 3 . 7 . 6

2 . 8 3 . 8

2 . 8 . 1 2 . 8 . 2 2 . 8 . 3 2 . 8 . 4 3 . 8 . 1 3 . 8 . 2 3 . 8 . 32 . 8 . 5

T o d e v e l o p a n d d e p l o y a l a r g e s c a l e s c i e n c e G r i di n t h e U K f o r t h e u s e o f t h e P a r t i c l e P h y s i c s c o m m u n i t y

P r e s e n t a t i o n D e p lo y m e n t

5 6 74

1 1 - S e p - 0 3S t a t u s D a t e

I n t . S t a n d a r d s

O p e n S o u r c e

W o r l d w i d e I n t e g r a t i o n

U K I n t e g r a t i o n

M o n i t o r i n g

D e v e lo p in gE n g a g e m e n t

P a r t i c i p a t i o n

O t h e r

D a t a C h a l l e n g e s

R o l l o u t

T e s t b e d

W P 7

A T L A S / L H C b

C M S

B a B a r

C D F / D O

U K Q C D

W P 1

W P 2

W P 3

L C G C r e a t i o n

A p p li c a t io n s

C E R N D a t a G r i d A p p l i c a t i o n s I n f r a s t r u c t u r e

D u e w it h in

A T L A S

G r i d P P G o a l

R e s o u r c e sI n t e r o p e r a b i l i t y D i s s e m i n a t i o n

T i e r - 1

T i e r - A

L H C b T i e r - 2

W P 8

1 2 3

D e p lo y m e n t

W P 4

W P 5

F a b r i c

T e c h n o lo g y

W P 6

U p d a t e

C l e a r


GridPP Status: SummaryGridPP Status: Summary

• GridPP1 has now completed 2 ex 3 years

• All metrics are currently satisfied

• 103 of 182 tasks are complete

• 70 tasks not yet complete or overdue

• 9 tasks are overdue:– 6 are associated with LCG

• 2 of these are trivial (definition of future milestones)• 4 of these are related to the delay in LCG-1

– 2 are associated with applications (CMS and D0)– 1 is associated with the UK infrastructure (test of a

heterogeneous testbed)


Risk Register (Status April 03)Risk Register (Status April 03)

ID NameLi Im Risk Li Im Risk Li Im Risk Li Im Risk Li Im Risk Li Im Risk

R1 Recruitment/retention difficulties 1 2 2 2 2 4 2 2 4 2 2 4R2 Sudden loss of key staff 1 4 4 1 3 3 1 3 3 1 3 3R3 Minimal Contingency 1 2 2 2 2 4 1 2 2R4 GridPP deliverables late 1 3 3 4 2 8R5 Sub-components not delivered to project 2 4 8 2 3 6 2 3 6 2 3 6R6 Non take-up of project results 2 1 2 1 4 4 1 4 4R7 Change in project scope 1 1 1 2 2 4 1 1 1R8 Bad publicity 2 2 4 1 3 3 1 3 3R9 No publicity 2 1 2

R10 External software dependence 2 4 8 1 4 4 2 3 6R11 Lack of monitoring of staff 1 2 2 2 2 4 1 3 3 1 2 2R12 Withdrawal of an experiment 1 4 4 1 4 4 1 2 2R13 Lack of cooperation between Tier centres 1 4 4 1 2 2R14 Scalablity problems 3 3 9 2 3 6 1 3 3R15 Software maintainability problems 3 2 6 1 3 3R16 Technology shifts 1 3 3 1 4 4 1 3 3R17 Repitition of research 3 2 6R18 Lack of funding to meet LCG PH-1 goals 1 3 3R19 Adequate persistency solution not ready 1 3 3R20 Conflicting software requirements 4 1 4 2 2 4R21 Tier-A hardware fails to meet requirements 1 3 3R22 Other Hardware fails to meet requirements 1 1 1R23 Hardware physical risk (large scale) 1 4 4R24 Hardware physical risk (small scale) 2 2 4R25 Hardware procurement problems 2 2 4R26 LAN Bottlenecks 2 1 2R27 Tier-2 organisation fails 1 2 2R28 Tier-2 hardware not used as planned 2 1 2R29 SYSMAN effort inadequate 3 3 9R30 Firewalls interfere with Grid 1 3 3R31 Inablility to establish trust relationships 2 2 4R32 Security inadequate to operate Grid 1 3 3R33 GGF does not establish standards 1 3 3R34 Minimal open source code development 2 2 4R35 Failure of international cooperation 1 4 4 1 4 4R36 e-Science and GridPP divergence 2 2 4R37 Institutes do not embrace Grid 1 3 3R38 Grid is not stable enough for use 4 2 8R39 Delay of the LHC 2 2 4R40 Lack of future funding 2 4 8 2 2 4R41 Network backbone failure 0 4 0R42 Network backbone bottleneck 1 2 2

Alt-i-r

Interop.GridPP LCG EDG Apps Infrast.

Scaling up to aproduction system(LCG-1 deployment)

System managementeffort at UK Tier-2 sites(being addressed as part of GridPP2)


UK Certificates andUK Certificates andVO membership VO membership

1. UK e-Science CA now used in production EDG testbed

2. PP “users” engaged from many institutes

3. UK participating in 6 ex 9 EDG VOs

0 20 40 60 80 100 120

BaBar

Eobs

Iteam

LHCb

Alice

BioMe

CMS

Atlas

WP6

Members

UK Members

1.

2. 3.


UK Deployment OverviewUK Deployment Overview

• Significant resources within EDG. Currently being upgraded to EDG2.

• Integrating EDG on farms has been repeated many times but it is difficult.

• Sites are keen to take part within EDG2 currently, with LCG1 deployment after this.

• By the end of the year many HEP farms plan to be contributing to LCG1 resources.

• Basis of Deployment Input to LCG Plan.

• Input from Tier-1 (~50%) initially and four distributed Tier-2’s (50%) on ~Q1 2004 timescale.

CPU(kSI2K)

Disk TB

Support FTE

Tape TB

CERN 700 160 10.0 1000

Czech Repub 60 5 2.5 5

France 420 81 10.2 540

Germany 207 40 9.0 62

Holland 124 3 4.0 12

Italy 507 60 16.0 100

Japan 220 45 5.0 100

Poland 86 9 5.0 28

Russia 120 30 10.0 40

Taiwan 220 30 4.0 120

Spain 150 30 4.0 100

Sweden 179 40 2.0 40

Switzerland 26 5 2.0 40

UK 1656 226 17.3 295USA 801 176 15.5 1741

Total 5600 1169 120.0 4223


Tier-1 @ RALTier-1 @ RAL

CE SE

LCG 1.0/EDG 2.0

5xWN

LCG Testbed

CE

LCG 1.0/EDG 2.0

230xWN

Tier1/a

CE SE

EDG 2.0

WP3 Testbed

MON

CE SE

EDG 2.0

1xWN

EDG Dev Testbed

MON SE

ADS

•UI within CSF.

•NM for EDG2.

•Top level MDS for EDG.

•Various WP3 and WP5 dev nodes.

•VOMS for DEV TB.

•http://ganglia.gridpp.rl.ac.uk/

SE

LCG0 Testbed

CE 1xWN


London Grid:London Grid:Imperial CollegeImperial College

CE SE

EDG 2.0

EDG Testbed

WNs

CE

EDG 2.0

WNs

BaBar Farm

CE SE

CMS-LCG0

CMS-LCG0

WN

CE SE

EDG 2.0

1xWN

WP3 Testbed

MON

•RB for EDG 2.0.

•Plan to be in LCG1 and other testbeds.


London Grid:London Grid:Queen Mary and UCLQueen Mary and UCL

CE SE

EDG 1.4

1xWN

EDG Testbed

32xWN

• Queen Mary CE also feeds EDG jobs to 32 node e-Science farm.

•Plan to have LCG1/EDG2 running for the end of the year.•Expansion with SRIF grants.(64WN+2TB in Jan 2004, 100WN + 8TB in Dec 2004.)

•http://194.36.10.1/ganglia-webfrontend

CE SE

EDG 1.4

1xWN

EDG Testbed

•UCL Network Monitors for WP7 development.

•SRIF bid in place for 200 CPUs for the end of the year to join LCG1.


Southern Grid:Southern Grid:BristolBristol

CE SE

EDG 2.0

1xWN

EDG Testbed

CE SE

EDG 2.0

1xWN

WP3 Testbed

MON

CE SE

CMS-LCG0

CMS/LHCb Farm

24xWN

CE SE

EDG 1.4

BaBar Farm

78xWN

•GridPP RC.

•Plan to join LCG1


Southern Grid:Southern Grid:Cambridge and OxfordCambridge and Oxford

CE SE

EDG 1.4

15xWN

EDG Testbed •Cambridge farm shared with local NA-48, GANGA users.

•Some RH73 WNs for ongoing ATLAS challenge.

•3TB GridFTP-SE.

•Plan to join LCG1/EDG2 later in the year with an extra 50 CPUs.

•EDG jobs will be fed into local e-Science farm.

•http://farm002.hep.phy.cam.ac.uk/cavendish/

CE SE

EDG 1.4

2xWN

EDG Testbed•Oxford: Plan to join EDG2/LCG1.

•Nagios monitoring has been set up.

•(RAL is also evaluating Nagios)

•Planning to send EDG jobs into 10 WN CDF farm.

•128 node cluster being ordered now.


Southern Grid:Southern Grid:RAL PPD and BirminghamRAL PPD and Birmingham

CE SE

EDG 2.0

9xWN

EDG Testbed

CE SE

EDG 2.0

1xWN

WP3 Testbed

MON

•PPD User Interface

•Part of Southern Tier2 Centre within LCG1.

•50 CPUs and 5TB of disk expected for the end of year.

CE SE

EDG 1.4

1xWN

EDG Testbed•Birmingham: Expansion to 60 CPUs and 4TBs.

•Expect to participate within LCG1/EDG2


NorthGrid: NorthGrid: Manchester and LiverpoolManchester and Liverpool

CE SE(1.5TB)

EDG 1.4

80xWN

CE SE(5TB)

EDG 1.4

60xWN

CE SE

EDG 1.4

9xWN

EDG Testbed BaBar Farm DZero Farm

•GridPP and BaBar VO Servers.

•User Interface

•Plan that DZero farm will join LCG.

•SRIF bid in place for significant HEP resources.

CE SE

EDG 1.4

1xWN

EDG Testbed

•Liverpool plan to follow EDG 2, possibly integrating newly installed Dell (funded by NW Development Agency) and BaBar farm. Largest single Tier-2 resource.


ScotGrid: Glasgow, Edinburgh ScotGrid: Glasgow, Edinburgh and Durhamand Durham

CE SE

EDG 1.4

ScotGRID

59xWN

• WNs on a private network with outbound NAT in place.

• Various WP2 development boxes.

• 34 dual blade servers just arrived. 5TB FastT500 expected soon.

• Shared resources (CDF and Bioinformatics)

CE SE

EDG 2.0

WP3 Testbed

MON

• Edinburgh: 24TB FastT700 and 8-way server just arrived.

• Durham: existing farm available.

• Plan to be part of LCG.

CDF

LHC BIO


EDG 2.0 Deployment Status EDG 2.0 Deployment Status 12/9/0312/9/03

• RAL (Tier1A): Up and running with 2.0.1. UI gppui04 available (as part of CSF) and offer to give access to LCFGng node to help people compare with their own LCFGng setup.

• IC: Existing WP3 testbed site is at 2.0.0. Standard 2.0 RB available• UCL: Trying to go to 2.0: SE up so far.• QMUL: 2.0 installation ongoing.• RAL (PPD): 2.0.0 site up and running.• Oxford: wait until October for 2.0. • Birmingham: Working on getting a 2.0 site up next week• Bristol: WP3 testbed site at 2.0.0. Also doing a new 2.0 site install.

UI and MON up, still doing CE, SE and WN.• Cambridge to follow.• Manchester: Trying to get 2.0.1 set up.• Glasgow: Concentrating on commissioning new hardware during

the next month. Wait until then before going to 2.0.• Edinburgh to follow.


Meeting Current LHC Requirements: Meeting Current LHC Requirements: Experiment AccountingExperiment Accounting

Number of Normalised Processors per country48

9

765

900

0

136

105

332

15

154

297

218

326

0

256

348

127

1156

758

1 2 3 4 5 6 7 8 9 10 11 12 13

14 15 16 17 18 19

Regional Centre Simulation Hits No Pile Up 2x1033 2x1044 NassIDBristol/RAL 0.55 0.33 0.04 0.06 0.02 20Caltech 0.17 0.15 0.00 0.15 0.00 6CERN 0.89 2.20 1.40 2.66 2.25 300Fermilab 0.35 0.41 0.00 0.25 0.33 70ICST&M 0.88 0.59 0.50 0.15 0.12 84IN2P3 0.20 0.00 0.00 0.00 0.00 1INFN 1.55 1.18 0.40 0.72 0.71 99Moscow 0.43 0.14 0.14 0.00 0.00 41UCSD 0.34 0.30 0.00 0.29 0.30 80UFL 0.54 0.04 0.00 0.04 0.04 11USMOP 0.00 0.00 0.00 0.00 0.00 1Wisconsin 0.07 0.08 0.00 0.06 0.00 12

TOTAL 5.94 5.40 2.47 4.36 3.77

Experiment-driven project.Priorities determined by Experiments Board.


Tier-1/A AccountingTier-1/A Accounting

LHCb

ATLAS

CMS

CMS

Monthly accounting:Online Ganglia-based monitoring, see:http://www.gridpp.ac.uk/tier1a/Last month: CMS (and BaBar) jobs.

Annual accounting:ATLAS, CMS and LHCb jobs. Generally dominated by BaBar since January.


Today’s OperationsToday’s Operations

1. Support Team • built from sysadmins. 4 funded by

GridPP to work on EDG WP6, the rest are site sysadmins.

2. Methods• Email list, phone meetings,

personal visits, job submission monitoring

• RB, VO, RC for UK use to support non-EDG use

3. Rollout• Experience from RAL in EDG dev

testbeds and IC and Bristol in CMS testbeds

• 10 sites have been part of EDG app testbed at one time


0

1

2

3

4

5

6

7

8

9

10

FT

EOperations Manager

Tier-2 Expert

Tier-2 Expert

Tier-2 Expert

Tier-2 Expert

Tier-1 Expert

Tier-1 Expert

Applications Expert

University WP6 Posts

RAL WP6 Post

WP8 Post

Testbed Team Production Team

GridPP1 GridPP2

GridPP2 OperationsGridPP2 Operations

• To move from testbed to production, GridPP plans a bigger team with a full-time Operations Manager

• Manpower will be from the Tier-1 and Tier-2 Centres who will contribute to the Production Team

• The team will run a UK Grid which will belong to various grids (EDG, LCG,..) and also support other experiments

• RAL is also leading the LCG Security Group– written 4 documents setting out

procedures and User Rules

– working with GOC task force on Security Policy

– Risk Analysis and further planning for LCG in 2004


LCG OperationsLCG Operations

• RAL has led project to develop an Operations Centre for LCG1– Applied GridPP and MapCenter monitoring to LCG1– Dashboard combining several types of monitoring – Set up a web site with contact information– Developing Security Plan– Accounting (the current priority, building upon

resource centre and experiment accounting)




EGEEEGEE

Tier1 (16.5 FTE)

UK Team (8 FTE)

UK GSC (2 FTE)

(2FTE)

EGEE ROC(5 FTE)

EGEE CIC(4.5 FTE)

• The UK Production Team will be expanded as part of EGEE ROC and CIC posts to meet EGEE requirements

• To deliver an EGEE grid infrastructure that must also deliver to other communities and projects

• Could do this just within PP (matching funding available) but also want to engage fully with UK Core programme


Tier-1/A Services [FTE]Tier-1/A Services [FTE]

• High quality data services

• National and International Role• UK Focus for International Grid development

• Highest single priority within GridPP2

Regained Programme

CPU 2.0

Disk 1.5

AFS 0.0

Tape 2.5

Core Services 2.0

Operations 2.5

Networking 0.5

Security 0.0

Deployment 2.0

Experiments 2.0

Management 1.5

Total 16.5


Tier-2 Services [FTE]Tier-2 Services [FTE]

• Four Regional Tier-2 Centres • London: Brunel, Imperial College, QMUL, RHUL, UCL.• SouthGrid: Birmingham, Bristol, Cambridge, Oxford, RAL PPD.• NorthGrid: CLRC Daresbury, Lancaster, Liverpool, Manchester, Sheffield.• ScotGrid: Durham, Edinburgh, Glasgow.

• Hardware provided by Institutes

• GridPP provides added manpower

Role FTE

Tier Centre Experts 4 (taken from those below)

Hardware Support 8

Core Services 4

User Support 2

Specialist Support 6

Total 20

Current Planning

Y 1 Y 2 Y 3

Hardware Support 4.0 8.0 8.0

Core Services 4.0 4.0 4.0

User Support 1.0 2.0 2.0

Specialist Services

Security 1.0 1.0 1.0

Resource Broker 1.0 1.0 1.0

Network 0.5 0.5 0.5

Data Management 2.0 2.0 2.0

VO Management 0.5 0.5 0.5

14.0 19.0 19.0

Existing Staff -4.0 -4.0 -4.0

GridPP2 10.0 15.0 15.0

Total SY 40.0


Operational RolesOperational Roles

• Core Infrastructure Services (CIC) – Grid information services

– Monitoring services

– Resource brokering

– Allocation and scheduling services

– Replica data catalogues

– Authorisation services

– Accounting services

• Still to be defined fully in EGEE

• Core Operational Tasks (ROC)– Monitor infrastructure, components

and services

– Troubleshooting

– Verification of new sites joining Grid

– Acceptance tests of new middleware releases

– Verify suppliers are meeting SLA

– Performance tuning and optimisation

– Publishing use figures and accounts


SRB for CMSSRB for CMS

• UK eScience has been interested in SRB for several years.

• CCLRC has gained expertise for other projects and is collaborating with SDSC

• Now hosting MCAT for worldwide CMS pre-DC04

• Interfaced to RAL Datastore– Service Started 1 July 2003

• 183,000 files registered

• 10 TB of data stored in system

• Used across 13 sites worldwide including CERN and Fermilab

• 30 Storage resources managed across the sites

MCATDatabase

MCATServer

SRB AServer

SRB BServer

SRBClient

a

b

c d

e

f

g


EDG StorageElementEDG StorageElement

• Not initially adopted by LCG1

• Since then limited SRM functionality has been added to support GFAL – available for test by LCG

• Full SRMv1 functionality has been developed and is currently being integrated on internal testbed

• GACLs being integrated


RGMA - StatusRGMA - Status

• Running on WP3, EDG-development and EDG-application testbeds• Application Deployment: 29 CEs, 11 SEs, 10 sites in 6 countries

– RGMA browser access in < 1sec

• Monitoring scripts being run on the testbeds and results linked from the WP3 web page– http://hepunx.rl.ac.uk/edg/wp3/

• Registry replication is being tested on WP3 testbed– Better performance & higher reliability required

• Authentication successfully tested on WP3 testbed• Two known bugs remain

– Excessive threads requiring GOUT machine restart• New code has been developed with extensive unit tests. Now

being tested on WP3 testbed• This new code will support at least 90 sites

– Latest Producer choosing algorithm failing to reject bad LPs – shows up intermittent absence of information

• Revised algorithm needs coding (localised change)


RGMA - UsersRGMA - Users

• Users and Interfaces to other systems: – Resource Broker– CMS (Boss)– Service and Service Status for all EDG services – Network Monitoring & Network Cost Function– MapCenter– Logging & Bookkeeping– UK e-Science, CrossGrid and BaBar evaluating– Replica Manager– MDS (GIN/GOUT) – Nagios – Ganglia (Ranglia)

• Future: RB direct use of RGMA (no GOUT)– Better performance and reliability


Middleware, Security and Network Middleware, Security and Network Service EvolutionService Evolution

0

5

10

15

20

25

30

FT

E

GridPP1 GridPP2

Security

Networking

Data Management

Information Services

Workload ManagementWP1

WP2

WP3

WP4

WP5

WP70

5

1 0

1 5

1 2 3 4

GridPP

Non GridPP

• Information Services [5+5 FTE] and Networking [1.5+1.5 FTE]: strategic roles within EGEE

• Security expands to meet reqts.

• Data and Workload Management continue

• No further configuration management development

• programme defined by – mission criticality

(experiment requirements driven)

– International/UK-wide lead – leverage of EGEE, UK core

and LCG developments

Activity Current Planning

Security 3.5

Info-Mon. 4.0

Data & Storage 4.0

Workload 1.5

Networking 3.0

TOTAL 16.0

SecurityMiddleware

Networking


GridPP2 ProposalGridPP2 Proposal

1. Executive Summary ........................................................................................................................4

2. Outline ............................................................................................................................................5

3. Introduction .....................................................................................................................................6 3.1 Experimental Motivation .......................................................................................................6 3.2 GridPP1: From Web to Grid..................................................................................................7 3.3 GridPP2: From Prototype to Production..............................................................................10

4. Experimental Requirements for the Production Grid .....................................................................17 4.1 CPU Requirements.............................................................................................................17 4.2 Storage Requirements........................................................................................................18 4.3 Networking Requirements ..................................................................................................19 4.4 Total Hardware Requirements............................................................................................19

5. The Grid Computing Hierarchy......................................................................................................19 5.1 Tier-0..................................................................................................................................21 5.2 Tier-1..................................................................................................................................23

A National and International UK Role............................................................................................25 5.3 Tier-2..................................................................................................................................25 5.4 Summary of Tier Centre Production Services .....................................................................27

6. Meeting the Experiments’ Hardware Requirements ......................................................................29

7. Grid Development .........................................................................................................................30 7.1 Middleware, Security and Network Development................................................................30

7.1.1 Data and Storage Management ......................................................................................30 7.1.2 Workload Management...................................................................................................32 7.1.3 Information & Monitoring .................................................................................................32 7.1.4 Security...........................................................................................................................33 7.1.5 The Network Sector ........................................................................................................34

7.2 Summary of Middleware, Security and Network Services ...................................................36 7.3 Application Interfaces .........................................................................................................37

8. Management.................................................................................................................................39

9. Dissemination and Technology Transfer .......................................................................................40 9.1 Dissemination.....................................................................................................................40 9.2 Technology Transfer...........................................................................................................41

10. Resource Request ........................................................................................................................41 10.1 Overview ............................................................................................................................41 10.2 Tier-0 Resources................................................................................................................42 10.3 Tier-1 Resources................................................................................................................43 10.4 Tier-2 Resources................................................................................................................43 10.5 Application Interfaces .........................................................................................................43 10.6 Middleware, Security and Networking.................................................................................44 10.7 Management, Dissemination and Operations .....................................................................44 10.8 Travel and Consumables....................................................................................................44 10.9 Resource Request Details and Interface Issues .................................................................44 10.10 Resource Request Summary..............................................................................................46

11. Conclusion....................................................................................................................................48

12. Appendix.......................................................................................................................................48 12.1 Tier-0 Planning Document ..................................................................................................48 12.2 Tier-1 Planning Document ..................................................................................................48 12.3 Tier-2 Planning Document ..................................................................................................48 12.4 Middleware Planning Document .........................................................................................48 12.5 Applications Planning Document ........................................................................................48 12.6 Management Planning Document.......................................................................................49 12.7 Travel Planning Document .................................................................................................49 12.8 Hardware Requirements Planning Document .....................................................................49

http://www.gridpp.ac.uk/docs/gridpp2/

~30 page proposal + figures/tables

+ 11 planning documents:15. Tier-0

16. Tier-1

17. Tier-2

18. The Network Sector

19. Middleware

20. Applications

21. Hardware Requirements

22. Management

23. Travel

24. Dissemination

25. From Testbed to Production


£0.24m

£2.21m

£2.41m

£4.01m

£3.05m

£3.14m

£4.82m

£2.25m

£3.16m

£m £1m

£2m

£3m

£4m

£5m

£6m

£7m

Tier-0 Hardware

Tier-0 Staff

Tier-1 Hardware

Tier-1 Staff

Tier-2 Hardware

Tier-2 Staff

App. Integration

LHC Application Dev.

Non LHC Application Dev.

Middleware/Security/Network

Operations/Management/Dissem.

23% Institutes

27% EGEE + Others

Experiment Collaboration Bids

98% non-PPARC

?50% of £5mSRIF-1 SRIF-2

18% CLRC

Current planning based uponCurrent planning based upon£19.6m Funding Scenario£19.6m Funding Scenario

£0m

£2m

£4m

£6m

£8m

£10m

£12m

£14m

2003 2004 2005 2006 2007 2008

Application Development

Tier-1 and 2 staff


Tier-2 Hardware

Tier-2 Staff

Tier-1 Hardware

Tier-1 Staff

Tier-0 Hardware

Tier-0 Staff

Application Integration


Dissemination

Travel and Operations

Management

GridPP2Proposal

ExternallyFunded

PPPARC Review Timeline:

Projects Peer Review Panel(14-15/7/03)

Grid Steering Committee(28-29/7/03)

Science Committee(October 03)


TimelineTimeline

2003 2004 2005 2006 2007 2008

Middleware Deployment Operations

ResourceProvider

UserCommunity

EDG

LCG

LCG

EGEE

EGEE

HEP/BIOMED/…

Prototype-1EDG 1.x

(GT2, Condor-G)

Prototype-2LCG-1 (EDG-2)

Prototype-3LCG-2 Releases

Production-1LCG-3

Production-2 Production-3

Migrate to OGSA/F. Follow GGF Standards


GridPP2 Project MapGridPP2 Project Map

• Need to build this in: to identify progress…

1 . 1 2 . 1 3 . 1 4 . 1 5 . 1 6 . 1 7 . 1

1 . 1 . 1 1 . 1 . 2 1 . 1 . 3 1 . 1 . 4 2 . 1 . 1 2 . 1 . 2 2 . 1 . 3 2 . 1 . 4 3 . 1 . 1 3 . 1 . 2 3 . 1 . 3 3 . 1 . 4 4 . 1 . 1 4 . 1 . 2 4 . 1 . 3 4 . 1 . 4 5 . 1 . 1 5 . 1 . 2 5 . 1 . 3 6 . 1 . 1 6 . 1 . 2 6 . 1 . 3 6 . 1 . 4 7 . 1 . 1 7 . 1 . 2 7 . 1 . 3 7 . 1 . 41 . 1 . 5 2 . 1 . 5 2 . 1 . 6 2 . 1 . 7 2 . 1 . 8 3 . 1 . 5 3 . 1 . 6 3 . 1 . 7 3 . 1 . 8 4 . 1 . 5 4 . 1 . 6 4 . 1 . 7 4 . 1 . 8 6 . 1 . 5

2 . 1 . 9 3 . 1 . 9 3 . 1 . 1 0 4 . 1 . 9

1 . 2 2 . 2 3 . 2 4 . 2 5 . 2 6 . 2 7 . 2

1 . 2 . 1 1 . 2 . 2 1 . 2 . 3 1 . 2 . 4 2 . 2 . 1 2 . 2 . 2 2 . 2 . 3 2 . 2 . 4 3 . 2 . 1 3 . 2 . 2 3 . 2 . 3 3 . 2 . 4 4 . 2 . 1 4 . 2 . 2 4 . 2 . 3 4 . 2 . 4 5 . 2 . 1 5 . 2 . 2 5 . 2 . 3 6 . 2 . 1 6 . 2 . 2 6 . 2 . 3 7 . 2 . 1 7 . 2 . 2 7 . 2 . 31 . 2 . 5 1 . 2 . 6 2 . 2 . 5 2 . 2 . 6 2 . 2 . 7 3 . 2 . 5 3 . 2 . 6 3 . 2 . 7 3 . 2 . 8 4 . 2 . 5 4 . 2 . 6 4 . 2 . 7

3 . 2 . 9

1 . 3 2 . 3 3 . 3 4 . 3 5 . 3 6 . 3 7 . 3

1 . 3 . 1 1 . 3 . 2 1 . 3 . 3 1 . 3 . 4 2 . 3 . 1 2 . 3 . 2 2 . 3 . 3 2 . 3 . 4 3 . 3 . 1 3 . 3 . 2 3 . 3 . 3 3 . 3 . 4 4 . 3 . 1 4 . 3 . 2 4 . 3 . 3 4 . 3 . 4 5 . 3 . 1 5 . 3 . 2 5 . 3 . 3 6 . 3 . 1 6 . 3 . 2 6 . 3 . 3 6 . 3 . 4 7 . 3 . 1 7 . 3 . 2 7 . 3 . 3 7 . 3 . 41 . 3 . 5 1 . 3 . 6 2 . 3 . 5 2 . 3 . 6 2 . 3 . 7 3 . 3 . 5 3 . 3 . 6 4 . 3 . 5

1 . 4 2 . 4 3 . 4 4 . 4 5 . 4

1 . 4 . 1 1 . 4 . 2 1 . 4 . 3 1 . 4 . 4 2 . 4 . 1 2 . 4 . 2 2 . 4 . 3 2 . 4 . 4 3 . 4 . 1 3 . 4 . 2 3 . 4 . 3 3 . 4 . 4 4 . 4 . 1 4 . 4 . 2 4 . 4 . 3 4 . 4 . 4 5 . 4 . 1 5 . 4 . 2 5 . 4 . 3 5 . 4 . 41 . 4 . 5 1 . 4 . 6 1 . 4 . 7 1 . 4 . 8 2 . 4 . 5 2 . 4 . 6 2 . 4 . 7 3 . 4 . 5 3 . 4 . 6 3 . 4 . 7 3 . 4 . 8 4 . 4 . 5 4 . 4 . 6 5 . 4 . 51 . 4 . 9 3 . 4 . 9 3 . 4 . 1 0 M e t r ic O K 1 . 1 . 1

M e t r ic n o t O K 1 . 1 . 1 1 . 5 2 . 5 3 . 5 4 . 5 T a s k c o m p le t e 1 . 1 . 1

1 . 5 . 1 1 . 5 . 2 1 . 5 . 3 1 . 5 . 4 2 . 5 . 1 2 . 5 . 2 2 . 5 . 3 2 . 5 . 4 3 . 5 . 1 3 . 5 . 2 3 . 5 . 3 3 . 5 . 4 4 . 5 . 1 4 . 5 . 2 4 . 5 . 3 4 . 5 . 4 T a s k o v e r d u e 1 . 1 . 11 . 5 . 5 1 . 5 . 6 1 . 5 . 7 1 . 5 . 8 2 . 5 . 5 2 . 5 . 6 2 . 5 . 7 3 . 5 . 5 3 . 5 . 6 3 . 5 . 7 6 0 d a y s 1 . 1 . 1

1 . 5 . 9 1 . 5 . 1 0 T a s k n o t d u e s o o n 1 . 1 . 1N o l o n g e r a c t i v e 1 . 1 . 1

2 . 6 3 . 6 4 . 6 N o T a s k o r m e t r i c2 . 6 . 1 2 . 6 . 2 2 . 6 . 3 2 . 6 . 4 3 . 6 . 1 3 . 6 . 2 3 . 6 . 3 3 . 6 . 4 4 . 6 . 1 4 . 6 . 2 4 . 6 . 32 . 6 . 5 2 . 6 . 6 2 . 6 . 7 2 . 6 . 8 3 . 6 . 5 3 . 6 . 6 3 . 6 . 7 3 . 6 . 8 N a v i g a t e u p

2 . 6 . 9 3 . 6 . 9 3 . 6 . 1 0 3 . 6 . 1 1 3 . 6 . 1 2 N a v i g a t e d o w n

E x t e r n a l li n k 2 . 7 3 . 7 L i n k t o g o a l s

2 . 7 . 1 2 . 7 . 2 2 . 7 . 3 2 . 7 . 4 3 . 7 . 1 3 . 7 . 2 3 . 7 . 3 3 . 7 . 42 . 7 . 5 2 . 7 . 6 2 . 7 . 7 2 . 7 . 8 3 . 7 . 5 3 . 7 . 6

2 . 8 3 . 8

2 . 8 . 1 2 . 8 . 2 2 . 8 . 3 2 . 8 . 4 3 . 8 . 1 3 . 8 . 2 3 . 8 . 32 . 8 . 5

T o d e v e l o p a n d d e p l o y a l a r g e s c a l e s c i e n c e G r i di n t h e U K f o r t h e u s e o f t h e P a r t i c l e P h y s i c s c o m m u n i t y

P r e s e n t a t i o n D e p lo y m e n t

5 6 74

1 - A p r - 0 3S t a t u s D a t e

I n t . S t a n d a r d s

O p e n S o u r c e

W o r l d w i d e I n t e g r a t i o n

U K I n t e g r a t i o n

M o n i t o r i n g

D e v e lo p in gE n g a g e m e n t

P a r t i c i p a t i o n

O t h e r

D a t a C h a l l e n g e s

R o l l o u t

T e s t b e d

W P 7

A T L A S / L H C b

C M S

B a B a r

C D F / D O

U K Q C D

W P 1

W P 2

W P 3

L C G C r e a t i o n

A p p l i c a t io n s

C E R N D a t a G r i d A p p l i c a t i o n s I n f r a s t r u c t u r e

D u e w it h in

A T L A S

G r i d P P G o a l

R e s o u r c e sI n t e r o p e r a b i l i t y D i s s e m i n a t i o n

T i e r - 1

T i e r - A

L H C b T i e r - 2

W P 8

1 2 3

D e p lo y m e n t

W P 4

W P 5

F a b r i c

T e c h n o lo g y

W P 6

U p d a t e

C l e a r

1. 1 2. 1 3. 1 4. 1 5. 1 6. 1 7. 1

1. 2 2. 2 3. 2 4. 2 5. 2 6. 2 7. 2

1. 3 2. 3 3. 3 4. 2 5. 3 6. 3 7. 3

1. 4 2. 4 3. 4 4. 3 5. 4 6. 4 7. 4

2. 5 3. 5 4. 4 5. 5

Navigate down

External link

Link to goals

TechnologyTransfer

Infrastructure

D0

OtherApplications

NetworkSupport

Support

ExperimentPhenoGrid

Non-LHC Apps761 2 3 4

ManagementProduction Grid

EngagementMonitoring

GridPP2 Goal

ATLAS

grid in the UK for the use of the Particle Physics communityTo develop and deploy a large scale production quality

5

CDF

Grid Technology

Grid Deployment UKQCDCMS

Security

Information &

LHCb

InteroperabilityDeployment

OutreachPlanningRollout

Dissemination

Computing Fabric

LCG Development LHC Apps

Applications Data & Storage

Workload

Management

Ganga

Management

BaBar

MonitoringRunning

Grid Services

Middleware


Experiment Requirements: Experiment Requirements: UK onlyUK only

CPU

0

2000

4000

6000

8000

10000

12000

2004 2005 2006 2007

Year

kS

I20

00

ye

ar

ATLAS

CMS

LHCb

ALICE

Phenomenology

ZEUS

UKQCD

UKDMC

MINOS

MICE

LISA

D0

CDF

BaBar

ANTARES

LHC

NonLHC

Disk

0

500

1000

1500

2000

2500

2004 2005 2006 2007

Year

TB

ATLASCMSLHCbALICEPhenomenologyUKQCDUKDMCMINOSMICED0CRESSTCDFBaBarANTARES

LHC

NonLHC

Tape

0

500

1000

1500

2000

2500

3000

2004 2005 2006 2007

Year

TB

ATLASCMS

LHCb

ALICEUKDMC

MINOS

MICED0

ANTARES

LHC

NonLHC

Total Requirement:

Year 2004 2005 2006 2007

CPU [kSI2000] 2395 4066 6380 9965

Disk [TB] 369 735 1424 2285

Tape [TB] 376 752 1542 2623


Meeting the Experiments’ Meeting the Experiments’ Hardware RequirementsHardware Requirements

• Significant… Production Grid inc. Tier-2 resources needed…

Disk [TB] \ Year

2004 2005 2006 2007

Tier-1 (Planned) 317 504 781 1221

Tier-2 (Estimated) 455 … … 1627

Requirement 369 735 1424 2285

Tape [TB] \ Year

2004 2005 2006 2007

Tier-1 (Planned) 256 355 455 555

Tier-2 (Estimated) 360 … … 1048

Requirement 376 752 1542 2623

CPU [kSI2000] \ Year

2004 2005 2006 2007

Tier-1 (Planned) 972 1591 2440 4030

Tier-2 (Estimated) 6045 … … 22861

Requirement 2395 4066 6380 9965


Projected Hardware ResourcesProjected Hardware Resources

Total Resources:

2004 2007(note x2 scale change)


Application InterfacesApplication Interfaces - Service Evolution- Service Evolution

Experiment Present Grid Activities Present Posts Proposed New Grid Activities

ATLAS MC production EDG integration

2.5

MC production Metadata LCG integration Persistency and data management

CMS MC production Persistency and data management Workload management Monitoring

3

MC production Persistency and data management Workload management Monitoring

LHCb MC production Metadata Persistency

2

MC production Metadata LCG integration Persistency and data management

GANGA (ATLAS+LHCb) Common Grid user interface

2

Common Grid user interface LCG integration Monitoring

BaBar Job submission Replica catalogue Persistency

2.5

Job submission Replica catalogue Persistency MC production

DØ MC production SAM development

2 MC production SAM deployment

CDF MC production SAM development

2 MC production SAM deployment

UKQCD Data access Job submission

1 Data federation Data binding

WP8 Applications integration 1 Applications integration UKDMC MC production

Analysis ZEUS MC production PhenoGrid Metadata MICE MC production

Data handling ANTARES Data handling CALICE MC production LC-ABD Metadata Total 18

• Applications– 18 FTEs: ongoing programme

of work can continue– Difficult to involve experiment

activity not already engaged within GridPP

• Project would need to build on cross-experiment collaboration – GridPP1 already has experience

– GANGA: ATLAS & LHCb– SAM: CDF & D0– Persistency: CMS &

BaBar• Encourage new joint

developments across experiments


ConclusionsConclusions

• Management under control via the Project Map and Project Plan

• GridPP Status is defined in terms of high level tasks and metrics: under control

• Major component is LCG– We contribute significantly to

LCG and our success depends critically on LCG

• Deployment – high and low level perspectives merge via accounting

• Resource centre and experiment accounting are both important

• Comprehensive accounting is a priority, built up from existing systems

• Today’s operations in the UK are built around a small team

• Future operations planning expands this team significantly: Production Manager being appointed

• Middleware deployment focussing on Information Service performance issues

• Existing IS team will be reinforced in UK within EGEE

• Security (deployment and policy) is emphasised

• GridPP2 planning status: formal feedback in November

Date post:	28-Mar-2015
Category:	Documents
Upload:	jocelyn-mcdougall
View:	213 times
Download:	0 times

Tony Doyle [email protected] Overview of UK Development and Deployment Programme, LCG PEB...

Documents