Date post: | 26-Dec-2015 |
Category: |
Documents |
Upload: | derek-foster |
View: | 212 times |
Download: | 0 times |
CANS Meeting (December 1, 2004)
Paul Avery 1
Paul AveryUniversity of [email protected]
UltraLight
U.S. Grid Projects andOpen Science Grid
Chinese American NetworkingSymposium
Florida International UniversityDecember 1, 2004
CANS Meeting (December 1, 2004)
Paul Avery 2
U.S. “Trillium” Grid Consortium Trillium = PPDG + GriPhyN + iVDGL
Particle Physics Data Grid: $12M (DOE) (1999 – 2004+)GriPhyN: $12M (NSF) (2000 – 2005) iVDGL: $14M (NSF) (2001 – 2006)
Basic composition (~150 people)PPDG: 4 universities, 6 labsGriPhyN: 12 universities, SDSC, 3 labs iVDGL: 18 universities, SDSC, 4 labs, foreign partnersExpts: BaBar, D0, STAR, Jlab, CMS, ATLAS, LIGO,
SDSS/NVO
Complementarity of projectsGriPhyN: CS research, Virtual Data Toolkit (VDT)
developmentPPDG: “End to end” Grid services, monitoring, analysis iVDGL: Grid laboratory deployment using VDTExperiments provide frontier challengesUnified entity when collaborating internationally
CANS Meeting (December 1, 2004)
Paul Avery 3
Goal: Peta-scale Virtual-Data Grids
for Global Science
Virtual Data Tools
Request Planning &Scheduling Tools
Request Execution & Management Tools
Transforms
Distributed resources(code, storage, CPUs,networks)
ResourceManagement
Services
Security andPolicy
Services
Other GridServices
Interactive User Tools
Production TeamSingle Researcher Workgroups
Raw datasource
PetaOps Petabytes Performance
CANS Meeting (December 1, 2004)
Paul Avery 4
Trillium Science Drivers Experiments at Large Hadron
Collider100s of Petabytes 2007 - ?
High Energy & Nuclear Physics expts~1 Petabyte (1000 TB) 1997 –
present
LIGO (gravity wave search)100s of Terabytes 2002 –
present
Sloan Digital Sky Survey10s of Terabytes 2001 –
present
Data
gro
wth
Com
mu
nit
y g
row
th
2007
2005
2003
2001
2009
Future Grid resources Massive CPU (PetaOps) Large distributed datasets (>100PB) Global communities (1000s)
CANS Meeting (December 1, 2004)
Paul Avery 5
Sloan Digital Sky Survey (SDSS)Using Virtual Data in GriPhyN
Galaxy clustersize distribution
Sloan Data
CANS Meeting (December 1, 2004)
Paul Avery 6
The LIGO Scientific Collaboration (LSC)and the LIGO Grid
LIGO Grid: 6 US sites
* LHO, LLO: observatory sites* LSC - LIGO Scientific Collaboration - iVDGL supported
iVDGL has enabled LSC to establish a persistent production grid
Cardiff
AEI/Golm •
+ 3 EU sites (Cardiff/UK, AEI/Germany)
Birmingham•
CANS Meeting (December 1, 2004)
Paul Avery 7
Search for Origin of Mass & Supersymmetry (2007 – ?)
TOTEM
LHCb
ALICE
27 km Tunnel in Switzerland & France
CMS
ATLAS
Large Hadron Collider (LHC) @ CERN
CANS Meeting (December 1, 2004)
Paul Avery 8
CMS Experiment
LHC Global Data Grid
Online System
CERN Computer Center
USAKorea RussiaUK
Maryland
0.1 - 1.5 GB/s
>10 Gb/s
10-40 Gb/s
2.5-10 Gb/s
Tier 0
Tier 1
Tier 3
Tier 2
Physics caches
PCs
Iowa
UCSDCaltechU Florida
5000 physicists, 60 countries
10s of Petabytes/yr by 2008 1000 Petabytes in < 10 yrs?
FIU
Tier 4
CANS Meeting (December 1, 2004)
Paul Avery 9
LCG: LHC Computing Grid Global Grid infrastructure for LHC experiments
Matched to decades long research program of LHC
Large scale resourcesHundreds of resource sites throughout the worldCommon resources, tools, middleware and environments
Operated and supported 24x7 globallyA robust, stable, predictable, supportable infrastructure
CANS Meeting (December 1, 2004)
Paul Avery 10
Network Bandwidth Needs (Gb/s)
CANS Meeting (December 1, 2004)
Paul Avery 11
Analysis by Globally Distributed Teams
Non-hierarchical: Chaotic analyses + productions Superimpose significant random data flows
CANS Meeting (December 1, 2004)
Paul Avery 12
Trillium Program of Work Common experiments, leadership, participants CS research
Workflow, scheduling, virtual data
Common Grid toolkits and packagingVirtual Data Toolkit (VDT) + Pacman packaging
Common Grid infrastructure: Grid3National Grid for testing, development and production
Advanced networkingUltranet, UltraLight, etc.
Integrated education and outreach effort+ collaboration with outside projects
Unified entity in working with international projectsLCG, EGEE, Asia, South America
CANS Meeting (December 1, 2004)
Paul Avery 13
VDT Growth Over 2.5 Years
VDT 1.1.3,1.1.4 & 1.1.5 pre-SC 2002
VDT 1.0Globus 2.0bCondor 6.3.1
VDT 1.1.7Switch to Globus 2.2
VDT 1.1.11Grid3
VDT 1.1.8First real use by LCG
VDT 1.1.14May 10
CANS Meeting (December 1, 2004)
Paul Avery 14
UltraLight: 10 Gb/s Network
10 Gb/s+ network• Caltech, UF, FIU, UM, MIT• SLAC, FNAL• Int’l partners• Level(3), Cisco, NLR
Funded by ITR2004
CANS Meeting (December 1, 2004)
Paul Avery 15
Grid3: An Operational National Grid30 sites, 3500 CPUs: Universities + 4 national
labsPart of LHC GridRunning since October 2003Applications in HEP, LIGO, SDSS, Genomics, CS
http://www.ivdgl.org/grid3
CANS Meeting (December 1, 2004)
Paul Avery 16
Grid2003 Applications High energy physics
US-ATLAS analysis (DIAL),US-ATLAS GEANT3 simulation (GCE)US-CMS GEANT4 simulation (MOP)BTeV simulation
Gravity wavesLIGO: blind search for continuous sources
Digital astronomySDSS: cluster finding (maxBcg)
BioinformaticsBio-molecular analysis (SnB)Genome analysis (GADU/Gnare)
CS Demonstrators Job Exerciser, GridFTP, NetLogger-grid2003
CANS Meeting (December 1, 2004)
Paul Avery 17
Grid3 Shared Use Over 6 months
cms dc04
atlasdc2
Sep 10
Usa
ge:
CP
Us
CANS Meeting (December 1, 2004)
Paul Avery 18
Open Science Grid Build on Grid3 experience
Persistent, production-quality Grid, national + international scope
Continue U.S. leading role in international scienceGrid infrastructure for large-scale collaborative scientific
research
Create large computing infrastructureCombine resources at DOE labs and universities to
effectively become a single national computing infrastructure for science
Grid3 OSG-0 OSG-1 OSG-2 …
Maintain interoperability with LCG (LHC Grid) Provide opportunities for educators and students
Participate in building and exploiting this grid infrastructure
Develop and train scientific and technical workforce
http://www.opensciencegrid.org
CANS Meeting (December 1, 2004)
Paul Avery 19
CANS Meeting (December 1, 2004)
Paul Avery 20
Education and Outreach
CANS Meeting (December 1, 2004)
Paul Avery 21
NEWS: Bulletin: ONE TWOWELCOME BULLETIN General InformationRegistrationTravel Information Hotel RegistrationParticipant List How to Get UERJ/Hotel Computer AccountsUseful Phone Numbers ProgramContact us: Secretariat Chairmen
Grids and the Digital DivideRio de Janeiro, Feb. 16-20, 2004
Background World Summit on Information Society HEP Standing Committee on Inter-
regional Connectivity (SCIC)
Themes Global collaborations, Grids and
addressing the Digital Divide
Next meeting: May 2005 (Korea)
http://www.uerj.br/lishep2004
CANS Meeting (December 1, 2004)
Paul Avery 22
iVDGL, GriPhyN Education / Outreach
Basics $200K/yr Led by UT
Brownsville Workshops, portals Partnerships with
CHEPREO, QuarkNet, …
CANS Meeting (December 1, 2004)
Paul Avery 23
June 21-25 Grid Summer School First of its kind in the U.S. (South Padre Island,
Texas)36 students, diverse origins and types (M, F, MSIs, etc)
Marks new direction for TrilliumFirst attempt to systematically train people in Grid
technologiesFirst attempt to gather relevant materials in one placeToday: Students in CS and PhysicsLater: Students, postdocs, junior & senior scientists
Reaching a wider audiencePut lectures, exercises, video, on the webMore tutorials, perhaps 3-4/yearDedicated resources for remote tutorialsCreate “Grid book”, e.g. Georgia Tech
New funding opportunitiesNSF: new training & education programs
CHEPREO: Center for High Energy Physics Research and Educational OutreachFlorida International University
Physics Learning Center CMS Research iVDGL Grid Activities AMPATH network (S.
America)
Funded September 2003
$4M initially (3 years) 4 NSF Directorates!
CANS Meeting (December 1, 2004)
Paul Avery 25
Grid Project ReferencesGriPhyN
www.griphyn.orgiVDGL
www.ivdgl.orgPPDG
www.ppdg.netGrid3
www.ivdgl.org/grid3Open Science Grid
www.opensciencegrid.orgCHEPREO
www.chepreo.orgUltraLight
ultralight.cacr.caltech.eduGlobus
www.globus.org
LCG www.cern.ch/lcg
EU DataGrid www.eu-datagrid.org
EGEE www.eu-egee.org
CANS Meeting (December 1, 2004)
Paul Avery 26
Trillium Grid Tools: Virtual Data Toolkit
Sources(CVS)
Patching
GPT srcbundles
NMI
Build & TestCondor pool
(37 computers)
…
Build
Test
Package
VDT
Build
Contributors (VDS, etc.)
Build
Pacman cache
RPMs
Binaries
Binaries
Binaries Test
Use NMI processes later