Date post: | 30-Dec-2015 |
Category: |
Documents |
Upload: | andrea-ferguson |
View: | 214 times |
Download: | 0 times |
Tony [email protected]
GridPP – Making the Grid Work for the Science, ATSE e-Science Visit, Edinburgh, 20 April 2004
Tony Doyle - University of Glasgow
ContentsContents
• Context1. General
(yesterday)
2. Process (today)
3. Operations (tomorrow)
• Start where Steve left off yesterday..
• End up where Andrew begins tomorrow..
– How does the Grid Work?– Performance Indicators– Why was the “failure rate”
~20%?
– Software Process– External dependencies– Managing a distributed
project..– Is GridPP a Grid?
• What is the Grid anyway? (from PP perspective)
– Demo..
Tony Doyle - University of Glasgow
How Does theHow Does theGrid Work?Grid Work?
1. Authenticationgrid-proxy-init
2. Job submissioneedg-job-submit
3. Monitoring and controleedg-job-statusedg-job-canceledg-job-get-output
4. Data publication and replication
globus-url-copy, RLS
5. Resource scheduling – use of Mass Storage Systems
JDL, sandboxes, storage elements
0. Web User Interface… or CLI
Tony Doyle - University of Glasgow
Job SubmissionJob Submission(behind the scenes)(behind the scenes)
UIJDL
Logging &Logging &Book-keepingBook-keeping
ResourceResourceBrokerBroker
Job SubmissionJob SubmissionServiceService
StorageStorageElementElement
ComputeComputeElementElement
Information Information ServiceService
Job Status
ReplicaReplicaCatalogueCatalogue
DataSets info
Author.&Authen.
Job S
ub
mit
Even
t
Job
Qu
ery
Job
Stat
us
Input “sandbox”
Input “sandbox” + Broker InfoGlobus RSL
Output “sandbox”
Output “sandbox”
Job Status
Pu
blis
h
grid
-pro
xy-in
it
Exp
and
ed J
DL
SE & CE info
Tony Doyle - University of Glasgow
How do I Authorize?How do I Authorize?
o=testbed,dc=eu-datagrid, dc=org
CN=Steven Hawking
ou=People
CN=Tony Doyle
mkgridmap
grid-mapfile
VOVODirectoryDirectory
““AuthorizatioAuthorizationn
Directory”Directory”
CN=Homer Simpson
o=xyz,dc=eu-datagrid, dc=org
CN=Steven Hawking
CN=Tony Doyle
Authentication
Certificate
Authentication
Certificate
Authentication
Certificate
ou=People ou=Testbed1
ou=???
local users ban list
Tony Doyle - University of Glasgow
UK Certificate Authority andUK Certificate Authority andVirtual Organisation membership Virtual Organisation membership
1. UK e-Science Certificate Authority now used in application testbed
2. PP “users” engaged from many institutes
3. UK participating in 6 ex 9 EDG Virtual Organisations
0 20 40 60 80 100 120
BaBar
Eobs
Iteam
LHCb
Alice
BioMe
CMS
Atlas
WP6
Members
UK Members
1.
2. 3.
Tony Doyle - University of Glasgow
Performance indicatorsPerformance indicators(as measured by end users)(as measured by end users)
Number of jobs successfully completed
Total Number of jobs submittedcrudeE
Efficiency (Successful Jobs / Jobs submitted)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
dé
c-0
2
jan
v-0
3
févr
-03
ma
rs-0
3
avr
-03
ma
i-0
3
juin
-03
juil-
03
ao
ût-
03
sep
t-0
3
oct
-03
no
v-0
3
dé
c-0
3
jan
v-0
4
févr
-04
Su
cc
es
sfu
l Jo
bs
/ J
ob
s s
ub
mit
ed
CMS EDGv1.4 Altlas EDGv1.4 LHCb EDGv1.4 LCG1 (EDG v2.0) EDG appl. TB v2.x
Conclusion: prototype performance, but with quality assurance mechanisms built-in
Tony Doyle - University of Glasgow
Why was the Why was the “failure rate” “failure rate”
~20%?~20%?• Component Testing e.g. RB Stress Tests (LCG) • RB never crashed• ran without problems at load for several days in a row 20 streams
with 100 jobs each ( typical error rate ~ 2 % still present)• RB stress test in a job storm of 50 streams, 20 jobs each :
– 50% of the streams ran out of connections between UI and RB. (configuration parameter – but machine constraints)
– Remaining 50% streams finished normal (2% error rate)– Time between job-submit and return of the command (acceptance by
the RB) is 3.5 seconds (independent of number of streams) • PROBLEMS ARE END-TO-END:
e.g. Site advertisement communicated via class ads to all sites (inc. e.g. CNAF) results in RB sending application jobs (e.g. AliEn for ALICE) to “black hole” – these are recorded as “failures”
(application corrects for these via re-submission)• OTHER “PROBLEM” IS INCORPORATION OF ADDED
FUNCTIONALITY– ~Resolved by adherence to software process coupled to testbed
structure… improved significantly within LCG (leading to EGEE)
III. Grid Middleware
I. Experiment Layer
II. Application Middleware
IV. Facilities and Fabrics
Tony Doyle - University of Glasgow
DataGrid Release MilestonesDataGrid Release Milestones
EU Review (2.1.13)
Evaluations (2.0.12)
• Features (2.1.13) [0.5Mloc]– Reasonable stability, reliability
– VOMS incorporated
– Bug fixes for all services.
• Features (2.0.12)– R-GMA replaced MDS
– Refactored workload mgt.
– Interactive, MPI, chkpt. jobs
– Replica Location Service
– Web Service SE
• Stabilisation time on application testbed typically a few months
Tony Doyle - University of Glasgow
Software Process InfrastructureSoftware Process Infrastructure
– Adopt the same set of tools, standards and procedures
– Adopt commonly used open-source or commercial software when easily available
– Avoid “do it yourself solutions”
– Avoid commercial software, since it may give licensing problems
Common services and infrastructure
Tools, templates, training
General QA, tests, integration, release
Similar ways of working (process)
LCG Application Area
POOL, SEAL, PI, SIMU
LCG grid software applications
(LHC experiments, projects, etc)
SPI Infrastructure
Tony Doyle - University of Glasgow
SPI Services OverviewSPI Services Overview
Provide General Services needed by each project– CVS repository, Web Site, Software Library– Mailing Lists, Bug Reports, Task Management, Collaborative Facilities
Provide solutions specific to the Software Development phases– Tools, Templates, Policies, Support, Documentations, Examples
Coding Analysis and DesignDevelopment
Release
SpecificationsTesting
Build systemsDeployment and
InstallationDocumentation
Quality Assurance
Software Development
General Services
CVS service
Collaborative Facilities Task Management Mailing Lists
Web Portal External Software
Tony Doyle - University of Glasgow
External SoftwareExternal Software
• We install software needed by Particle Physics projects
• Open Source and Public Domain software (libraries and tools) like:– Compilers (icc, ecc)– HEP made packages – Scientific libraries (GSL)– General tools (python)– Test tools (cppunit,
qmtest)– Database software (mysql,
mysql++)– Documentation generators
(lxr, doxygen)– XML parsers (XercesC)
• There are currently 50 different packages, plus others under evaluation. For more than 300 installations
• The LCG projects propose what to install in agreement with LHC needs
• The platforms are decided by the Architect Forum– Linux RedHat 7.3 with the
compilers • gcc 3.2 (rh73_gcc32) • icc 7.1 (rh73_icc71) • ecc 7.1 (rh73_ecc71)
– Windows • Visual Studio .NET 7.1:
(win32_vc7).
Tony Doyle - University of Glasgow
Ta
gg
ed
re
lea
se s
ele
cte
d f
or
cert
ifica
tion
Ce
rtifi
ed
re
lea
se s
ele
cte
d f
or
de
plo
yme
nt
Ta
gg
ed
pa
cka
ge
Problem reports
add unittested code to
repository
Run nightly build
& auto. testsGrid certification
Fix problemsApplication Certification
BuildSystem
CertificationTestbed ~40CPU
ApplicationTestbed ~1000CPU
Certified publicrelease
for use by apps.
24x7
Build system
Test Group
WPs
Unit Test Build Certification Production
Users
DevelopmentTestbed ~15CPU
Individual WP tests
IntegrationTeam
Integration
Overall release
tests
Releases candidate
Tagged Releases
Releases candidate
Certified Releases
Apps. Representatives
How Is the process applied? How Is the process applied? Middleware Validation: From Testbed to ProductionMiddleware Validation: From Testbed to Production
Process to:Test frameworks
Test supportTest policiesTest documentationTest platforms/compilers
Tony Doyle - University of Glasgow
The UK TestbedThe UK Testbed
Tony Doyle - University of Glasgow
e.g. ScotGrid: Glasgow, e.g. ScotGrid: Glasgow, Edinburgh and DurhamEdinburgh and Durham
CE SE
EDG 1.4
ScotGRID
59xWN
• Glasgow farm: WNs on a private network with outbound NAT in place
• 100,000 jobs completed (900,000 CPU hours)
• 34 dual blade servers and 5TB FastT500 being integrated now (next door)
• Shared resources (LHC, CDF and Bioinformatics)
EDG 2.1
Data Management Testbed
CE SE MON
• Edinburgh: 24TB FastT700 and 8-way server: data storage focus
• Durham: 40 node farm
• All being integrated into LCG-2
CDF
LHC BIO
Tony Doyle - University of Glasgow
1. 1 2. 1 3. 1 4. 1 5. 1 6. 1 7. 1
1. 2 2. 2 3. 2 4. 2 5. 2 6. 2 7. 2
1. 3 2. 3 3. 3 4. 3 5. 3 6. 3 7. 3
1. 4 2. 4 3. 4 4. 4 5. 4
1. 5 2. 5 3. 5 4. 5
Navigate down
External link 2. 6 3. 6 4. 6 Link to goals
2. 7 3. 7
2. 8 3. 8
System
UK Grid Rollout
Data Challenges
Dissemination
Tier-1 Centre
Tier-A Centre
Applications
CERN DataGrid Applications
LCG Creation WP1
WP2
ResourcesInfrastructure
Presentation Deployment of
related areasGANGA/GaudiATLAS/LHCb
resources
Participation in
of GridPP
Monitoring of
UK e-Science
Computing Fabric
Grid Technology
Tier-2 Centres
CMSMonte Carlo
WP3
WP4
LHCb
Standards
Open SourceImplementation
Grid Deployment
resourcesEngagementof UK groups
Attract newParticle Physics
UK Testbed
resources
WP6
WP7
WP5
SAM Framework
UKQCD
BaBar
Integration
Integration
Applications
QCD Application
Other
Data Analysis
CDF/D0
GridPP Goal
ATLAS
in the UK for the use of the Particle Physics communityTo develop and deploy a large scale science Grid
5Interoperability
International
WP8
6 71 2 3 4
Worldwide
1 . 1 2 . 1 3 . 1 4 . 1 5 . 1 6 . 1 7 . 1
1 . 1 . 1 1 . 1 . 2 1 . 1 . 3 1 . 1 . 4 2 . 1 . 1 2 . 1 . 2 2 . 1 . 3 2 . 1 . 4 3 . 1 . 1 3 . 1 . 2 3 . 1 . 3 3 . 1 . 4 4 . 1 . 1 4 . 1 . 2 4 . 1 . 3 4 . 1 . 4 5 . 1 . 1 5 . 1 . 2 5 . 1 . 3 6 . 1 . 1 6 . 1 . 2 6 . 1 . 3 6 . 1 . 4 7 . 1 . 1 7 . 1 . 2 7 . 1 . 3 7 . 1 . 41 . 1 . 5 2 . 1 . 5 2 . 1 . 6 2 . 1 . 7 2 . 1 . 8 3 . 1 . 5 3 . 1 . 6 3 . 1 . 7 3 . 1 . 8 4 . 1 . 5 4 . 1 . 6 4 . 1 . 7 4 . 1 . 8 6 . 1 . 5
2 . 1 . 9 3 . 1 . 9 3 . 1 . 1 0 4 . 1 . 9
1 . 2 2 . 2 3 . 2 4 . 2 5 . 2 6 . 2 7 . 2 1 . 2 . 1 1 . 2 . 2 1 . 2 . 3 1 . 2 . 4 2 . 2 . 1 2 . 2 . 2 2 . 2 . 3 2 . 2 . 4 3 . 2 . 1 3 . 2 . 2 3 . 2 . 3 3 . 2 . 4 4 . 2 . 1 4 . 2 . 2 4 . 2 . 3 4 . 2 . 4 5 . 2 . 1 5 . 2 . 2 5 . 2 . 3 6 . 2 . 1 6 . 2 . 2 6 . 2 . 3 7 . 2 . 1 7 . 2 . 2 7 . 2 . 31 . 2 . 5 1 . 2 . 6 1 . 2 . 7 1 . 2 . 8 2 . 2 . 5 2 . 2 . 6 2 . 2 . 7 3 . 2 . 5 3 . 2 . 6 3 . 2 . 7 3 . 2 . 8 4 . 2 . 5 4 . 2 . 6 4 . 2 . 71 . 2 . 9 1 . 2 . 1 0 3 . 2 . 9
1 . 3 2 . 3 3 . 3 4 . 3 5 . 3 6 . 3 7 . 3
1 . 3 . 1 1 . 3 . 2 1 . 3 . 3 1 . 3 . 4 2 . 3 . 1 2 . 3 . 2 2 . 3 . 3 2 . 3 . 4 3 . 3 . 1 3 . 3 . 2 3 . 3 . 3 3 . 3 . 4 4 . 3 . 1 4 . 3 . 2 4 . 3 . 3 4 . 3 . 4 5 . 3 . 1 5 . 3 . 2 5 . 3 . 3 6 . 3 . 1 6 . 3 . 2 6 . 3 . 3 6 . 3 . 4 7 . 3 . 1 7 . 3 . 2 7 . 3 . 3 7 . 3 . 41 . 3 . 5 1 . 3 . 6 1 . 3 . 7 1 . 3 . 8 2 . 3 . 5 2 . 3 . 6 2 . 3 . 7 3 . 3 . 5 3 . 3 . 6 4 . 3 . 51 . 3 . 9 1 . 3 . 1 0 1 . 3 . 1 1
1 . 4 2 . 4 3 . 4 4 . 4 5 . 4 1 . 4 . 1 1 . 4 . 2 1 . 4 . 3 1 . 4 . 4 2 . 4 . 1 2 . 4 . 2 2 . 4 . 3 2 . 4 . 4 3 . 4 . 1 3 . 4 . 2 3 . 4 . 3 3 . 4 . 4 4 . 4 . 1 4 . 4 . 2 4 . 4 . 3 4 . 4 . 4 5 . 4 . 1 5 . 4 . 2 5 . 4 . 3 5 . 4 . 41 . 4 . 5 1 . 4 . 6 1 . 4 . 7 1 . 4 . 8 2 . 4 . 5 2 . 4 . 6 2 . 4 . 7 3 . 4 . 5 3 . 4 . 6 3 . 4 . 7 3 . 4 . 8 4 . 4 . 5 4 . 4 . 6 5 . 4 . 51 . 4 . 9 3 . 4 . 9 3 . 4 . 1 0 M e t r i c O K 1 . 1 . 1
M e t r i c n o t O K 1 . 1 . 1 1 . 5 2 . 5 3 . 5 4 . 5 T a s k c o m p l e t e 1 . 1 . 1
1 . 5 . 1 1 . 5 . 2 1 . 5 . 3 1 . 5 . 4 2 . 5 . 1 2 . 5 . 2 2 . 5 . 3 2 . 5 . 4 3 . 5 . 1 3 . 5 . 2 3 . 5 . 3 3 . 5 . 4 4 . 5 . 1 4 . 5 . 2 4 . 5 . 3 4 . 5 . 4 T a s k o v e r d u e 1 . 1 . 11 . 5 . 5 1 . 5 . 6 1 . 5 . 7 1 . 5 . 8 2 . 5 . 5 2 . 5 . 6 2 . 5 . 7 3 . 5 . 5 3 . 5 . 6 3 . 5 . 7 6 0 d a y s 1 . 1 . 11 . 5 . 9 1 . 5 . 1 0 T a s k n o t d u e s o o n 1 . 1 . 1
N o t A c t i v e 1 . 1 . 1 2 . 6 3 . 6 4 . 6 N o T a s k o r m e t r i c
2 . 6 . 1 2 . 6 . 2 2 . 6 . 3 2 . 6 . 4 3 . 6 . 1 3 . 6 . 2 3 . 6 . 3 3 . 6 . 4 4 . 6 . 1 4 . 6 . 2 4 . 6 . 32 . 6 . 5 2 . 6 . 6 2 . 6 . 7 2 . 6 . 8 3 . 6 . 5 3 . 6 . 6 3 . 6 . 7 3 . 6 . 8 N a v i g a t e u p 2 . 6 . 9 3 . 6 . 9 3 . 6 . 1 0 3 . 6 . 1 1 3 . 6 . 1 2 N a v i g a t e d o w n
E x t e r n a l l i n k 2 . 7 3 . 7 L i n k t o g o a l s
2 . 7 . 1 2 . 7 . 2 2 . 7 . 3 2 . 7 . 4 3 . 7 . 1 3 . 7 . 2 3 . 7 . 3 3 . 7 . 42 . 7 . 5 2 . 7 . 6 2 . 7 . 7 2 . 7 . 8 3 . 7 . 5 3 . 7 . 6
2 . 8 3 . 8 2 . 8 . 1 2 . 8 . 2 2 . 8 . 3 2 . 8 . 4 3 . 8 . 1 3 . 8 . 2 3 . 8 . 32 . 8 . 5
W P 8
1 2 3
D e p l o y m e n t
W P 4
W P 5
F a b r i c
T e c h n o l o g y
W P 6
D u e w i t h i n
A T L A S
G r i d P P G o a l
R e s o u r c e sI n t e r o p e r a b i l i t y D i s s e m i n a t i o n
T i e r - 1
T i e r - A
L H C b T i e r - 2
C E R N D a t a G r id A p p l i c a t io n s I n f r a s t r u c t u r e
W P 1
W P 2
W P 3
L C G C r e a t i o n
A p p l i c a t i o n s
W P 7
A T L A S / L H C b
C M S
B a B a r
C D F / D O
U K Q C D
O t h e r
D a t a C h a l l e n g e s
R o l l o u t
T e s t b e d
1 - J a n - 0 4S t a t u s D a t e
I n t . S t a n d a r d s
O p e n S o u r c e
W o r l d w i d e I n t e g r a t i o n
U K I n t e g r a t i o n
M o n i t o r i n g
D e v e l o p i n gE n g a g e m e n t
P a r t i c i p a t i o n
T o d e v e l o p a n d d e p l o y a l a r g e s c a l e s c i e n c e G r i di n t h e U K f o r t h e u s e o f t h e P a r t i c l e P h y s i c s c o m m u n i t y
P r e s e n t a t i o n D e p l o y m e n t
5 6 74
U p d a t e
C l e a r
1 . 1 2 . 1 3 . 1 4 . 1 5 . 1 6 . 1 7 . 1
1 . 1 . 1 1 . 1 . 2 1 . 1 . 3 1 . 1 . 4 2 . 1 . 1 2 . 1 . 2 2 . 1 . 3 2 . 1 . 4 3 . 1 . 1 3 . 1 . 2 3 . 1 . 3 3 . 1 . 4 4 . 1 . 1 4 . 1 . 2 4 . 1 . 3 4 . 1 . 4 5 . 1 . 1 5 . 1 . 2 5 . 1 . 3 6 . 1 . 1 6 . 1 . 2 6 . 1 . 3 6 . 1 . 4 7 . 1 . 1 7 . 1 . 2 7 . 1 . 3 7 . 1 . 41 . 1 . 5 2 . 1 . 5 2 . 1 . 6 2 . 1 . 7 2 . 1 . 8 3 . 1 . 5 3 . 1 . 6 3 . 1 . 7 3 . 1 . 8 4 . 1 . 5 4 . 1 . 6 4 . 1 . 7 4 . 1 . 8 6 . 1 . 5
2 . 1 . 9 3 . 1 . 9 3 . 1 . 1 0 4 . 1 . 9
1 . 2 2 . 2 3 . 2 4 . 2 5 . 2 6 . 2 7 . 2 1 . 2 . 1 1 . 2 . 2 1 . 2 . 3 1 . 2 . 4 2 . 2 . 1 2 . 2 . 2 2 . 2 . 3 2 . 2 . 4 3 . 2 . 1 3 . 2 . 2 3 . 2 . 3 3 . 2 . 4 4 . 2 . 1 4 . 2 . 2 4 . 2 . 3 4 . 2 . 4 5 . 2 . 1 5 . 2 . 2 5 . 2 . 3 6 . 2 . 1 6 . 2 . 2 6 . 2 . 3 7 . 2 . 1 7 . 2 . 2 7 . 2 . 31 . 2 . 5 1 . 2 . 6 1 . 2 . 7 1 . 2 . 8 2 . 2 . 5 2 . 2 . 6 2 . 2 . 7 3 . 2 . 5 3 . 2 . 6 3 . 2 . 7 3 . 2 . 8 4 . 2 . 5 4 . 2 . 6 4 . 2 . 71 . 2 . 9 1 . 2 . 1 0 3 . 2 . 9
1 . 3 2 . 3 3 . 3 4 . 3 5 . 3 6 . 3 7 . 3
1 . 3 . 1 1 . 3 . 2 1 . 3 . 3 1 . 3 . 4 2 . 3 . 1 2 . 3 . 2 2 . 3 . 3 2 . 3 . 4 3 . 3 . 1 3 . 3 . 2 3 . 3 . 3 3 . 3 . 4 4 . 3 . 1 4 . 3 . 2 4 . 3 . 3 4 . 3 . 4 5 . 3 . 1 5 . 3 . 2 5 . 3 . 3 6 . 3 . 1 6 . 3 . 2 6 . 3 . 3 6 . 3 . 4 7 . 3 . 1 7 . 3 . 2 7 . 3 . 3 7 . 3 . 41 . 3 . 5 1 . 3 . 6 1 . 3 . 7 1 . 3 . 8 2 . 3 . 5 2 . 3 . 6 2 . 3 . 7 3 . 3 . 5 3 . 3 . 6 4 . 3 . 51 . 3 . 9 1 . 3 . 1 0 1 . 3 . 1 1
1 . 4 2 . 4 3 . 4 4 . 4 5 . 4 1 . 4 . 1 1 . 4 . 2 1 . 4 . 3 1 . 4 . 4 2 . 4 . 1 2 . 4 . 2 2 . 4 . 3 2 . 4 . 4 3 . 4 . 1 3 . 4 . 2 3 . 4 . 3 3 . 4 . 4 4 . 4 . 1 4 . 4 . 2 4 . 4 . 3 4 . 4 . 4 5 . 4 . 1 5 . 4 . 2 5 . 4 . 3 5 . 4 . 41 . 4 . 5 1 . 4 . 6 1 . 4 . 7 1 . 4 . 8 2 . 4 . 5 2 . 4 . 6 2 . 4 . 7 3 . 4 . 5 3 . 4 . 6 3 . 4 . 7 3 . 4 . 8 4 . 4 . 5 4 . 4 . 6 5 . 4 . 51 . 4 . 9 3 . 4 . 9 3 . 4 . 1 0 M e t r i c O K 1 . 1 . 1
M e t r i c n o t O K 1 . 1 . 1 1 . 5 2 . 5 3 . 5 4 . 5 T a s k c o m p le t e 1 . 1 . 1
1 . 5 . 1 1 . 5 . 2 1 . 5 . 3 1 . 5 . 4 2 . 5 . 1 2 . 5 . 2 2 . 5 . 3 2 . 5 . 4 3 . 5 . 1 3 . 5 . 2 3 . 5 . 3 3 . 5 . 4 4 . 5 . 1 4 . 5 . 2 4 . 5 . 3 4 . 5 . 4 T a s k o v e r d u e 1 . 1 . 11 . 5 . 5 1 . 5 . 6 1 . 5 . 7 1 . 5 . 8 2 . 5 . 5 2 . 5 . 6 2 . 5 . 7 3 . 5 . 5 3 . 5 . 6 3 . 5 . 7 6 0 d a y s 1 . 1 . 11 . 5 . 9 1 . 5 . 1 0 T a s k n o t d u e s o o n 1 . 1 . 1
N o t A c t i v e 1 . 1 . 1 2 . 6 3 . 6 4 . 6 N o T a s k o r m e t r i c
2 . 6 . 1 2 . 6 . 2 2 . 6 . 3 2 . 6 . 4 3 . 6 . 1 3 . 6 . 2 3 . 6 . 3 3 . 6 . 4 4 . 6 . 1 4 . 6 . 2 4 . 6 . 32 . 6 . 5 2 . 6 . 6 2 . 6 . 7 2 . 6 . 8 3 . 6 . 5 3 . 6 . 6 3 . 6 . 7 3 . 6 . 8 N a v ig a t e u p 2 . 6 . 9 3 . 6 . 9 3 . 6 . 1 0 3 . 6 . 1 1 3 . 6 . 1 2 N a v ig a t e d o w n
E x t e r n a l l i n k 2 . 7 3 . 7 L in k t o g o a l s
2 . 7 . 1 2 . 7 . 2 2 . 7 . 3 2 . 7 . 4 3 . 7 . 1 3 . 7 . 2 3 . 7 . 3 3 . 7 . 42 . 7 . 5 2 . 7 . 6 2 . 7 . 7 2 . 7 . 8 3 . 7 . 5 3 . 7 . 6
2 . 8 3 . 8 2 . 8 . 1 2 . 8 . 2 2 . 8 . 3 2 . 8 . 4 3 . 8 . 1 3 . 8 . 2 3 . 8 . 32 . 8 . 5
W P 8
1 2 3
D e p l o y m e n t
W P 4
W P 5
F a b r i c
T e c h n o l o g y
W P 6
D u e w i t h i n
A T L A S
G r i d P P G o a l
R e s o u r c e sI n t e r o p e r a b i l i t y D i s s e m i n a t i o n
T i e r - 1
T i e r - A
L H C b T i e r - 2
C E R N D a t a G r id A p p l i c a t io n s I n f r a s t r u c t u r e
W P 1
W P 2
W P 3
L C G C r e a t i o n
A p p l i c a t i o n s
W P 7
A T L A S / L H C b
C M S
B a B a r
C D F / D O
U K Q C D
O t h e r
D a t a C h a l l e n g e s
R o l l o u t
T e s t b e d
1 - J a n - 0 4S t a t u s D a t e
I n t . S t a n d a r d s
O p e n S o u r c e
W o r l d w i d e I n t e g r a t i o n
U K I n t e g r a t i o n
M o n i t o r i n g
D e v e l o p i n gE n g a g e m e n t
P a r t i c i p a t i o n
T o d e v e l o p a n d d e p l o y a l a r g e s c a l e s c i e n c e G r i di n t h e U K f o r t h e u s e o f t h e P a r t i c l e P h y s i c s c o m m u n i t y
P r e s e n t a t i o n D e p l o y m e n t
5 6 74
U p d a t e
C l e a r
Managing a Distributed Project: Managing a Distributed Project: GridPP1 Project Status?GridPP1 Project Status?
Metric OK
Metric not OK
Tasks Complete
Tasks Overdue
Tasks due in next 60 days
Tasks Deleted
ChangeForms Enacted
Tasks not due
43 1 145 2 1 3 17 42 76% of the 190 GridPP1 tasks have been successfully completed
Tony Doyle - University of Glasgow
What is “The Grid”What is “The Grid” Is GridPP a Grid?Is GridPP a Grid?Anyway?Anyway?
1. Coordinates resources that are not subject to centralized control
2. … using standard, open, general-purpose protocols and interfaces
3. … to deliver nontrivial qualities of service
1. YES. This is why development and maintenance of a UK-EU-US testbed is important
2. YES... Globus/CondorG/EDG meet this requirement. Common experiment application layers are also important here.
3. NO(T YET)… Experiments define whether this is true - currently only ~100,000 jobs submitted via the testbed c.f. internal component tests of up 10,000 jobs per day. Next step: LCG-2 deployment outcome… this year
http://www-fp.mcs.anl.gov/~foster/Articles/WhatIsTheGrid.pdf
Tony Doyle - University of Glasgow
What is The Grid Anyway?
From Particle Physics PerspectiveThe Grid is:
not hype, but surrounded by it
a working prototype running on testbed(s)…
about seamless discovery of PC resources around the world
using evolving standards for interoperation
the basis for particle physics computing in the 21st Century
not (yet) as transparent as end-users want it to be
Tony Doyle - University of Glasgow
The Grid: DemonstrationsThe Grid: Demonstrationshttp://www.gridpp.ac.uk/demos/http://www.gridpp.ac.uk/demos/
Demos used to establish that e.g. the two LHC multi-purpose detector collaborations
• can run jobs on an International Grid
• Use common Grid infrastructure with secure Grid access
• But doesn’t mean that the Grid works in production mode
• (yet)
• This is however
significant