Date post: | 30-Dec-2015 |
Category: |
Documents |
Upload: | roy-reeves |
View: | 216 times |
Download: | 0 times |
Grids at NIKHEF 2004.07.14 1
Grid Computing @ NIKHEF
David GroepNIKHEF PDP2004.07.14
• The (data) problem to solve
• beyond meta-computing: the Grid
• realizing the Grid at NIKHEF
• towards a national infrastructure
Grids at NIKHEF 2004.07.14 2
Place event info on 3D map
Trace trajectories through hits
Assign type to each track
Find particles you want
Needle in a haystack!
This is “relatively easy” case
A Glimpse of the Problem in HEP
Grids at NIKHEF 2004.07.14 3
The HEP reality
Grids at NIKHEF 2004.07.14 4
HEP Data Rates
level 1 - special hardware
40 MHz (40 TB/sec)level 2 - embedded processorslevel 3 - PCs
75 KHz (75 GB/sec)5 KHz (5 GB/sec)100 Hz(100 MB/sec)data recording &
offline analysis
• Reconstruct & analyze 1 event takes about 90 s
• Maybe only a few out of a million are interesting. But we have to check them all!
• Analysis program needs lots of calibration; determined from inspecting results of first pass.
Each event will be analyzed several times!
• Raw data rate ~ 5PByte/yr/expt.
• total volume: ~20 Pbyte/yr
• per major centre: ~2 PByte/yr
The ATLAS experiment
Grids at NIKHEF 2004.07.14 5
Data handling and computation
interactivephysicsanalysis
batchphysicsanalysis
batchphysicsanalysis
detector
event summary data
rawdata
eventreprocessing
eventreprocessing
eventsimulation
eventsimulation
analysis objects(extracted by physics topic)
event filter(selection &
reconstruction)
event filter(selection &
reconstruction)
processeddata
Grids at NIKHEF 2004.07.14 6
HEP is not unique in generating data
• LOFAR: 200 MHz,12 bits,25k antennas: 60Tbit/s
• Envisat GOME: ~ 5TByte/year
• Materials analysis (mass spectroscopy, &c):~ 2GByte/10min
• fMRI, PET/MEG, …
LHC data volume necessitates ‘provenance’ and meta-data information/data ratio even higher in other disciplines
both data and information ownership distributedaccess right for valuable data, add privacy for medical data
Grids at NIKHEF 2004.07.14 7
Beyond meta-computing: the Grid
How can the Grid help? via resource accessibility and via sharing
A grid integrates resources that are
– not owned or administered by one single organisation– speak a common, open protocol … that is generic– working as a coordinated, transparent system
And …– can be used by many people from multiple organisations – that work together in one Virtual Organisation
Grids at NIKHEF 2004.07.14 8
Virtual Organisations
A set of individuals or organisations, not under single hierarchical control, temporarily joining forces to solve a particular problem at hand, bringing to the collaboration a subset of their resources, sharing those at their discretion and each under their own conditions.
• A VO is a temporary alliance of stakeholders– Users– Service providers– Information Providers
Grids at NIKHEF 2004.07.14 9
Common and open protocols
Applications
Grid Services GRAM
Grid Security Infrastructure (GSI)
Grid FabricFARMS Supers Desktops TCP/IP Apparatus
Application ToolkitsDUROC MPICH-G2Condor-G
GridFTPInformation
VLAM-G
Replica
DBs
Grids at NIKHEF 2004.07.14 10
Standard protocols
• New Grid protocols based on popular Web Services Web Services Resource Framework (WSRF)
• Grid adds concept of ‘stateful resources’, likegrid-jobs, data elements & data bases, …
• Ensure adequate and flexible standards todayvia the Global Grid Forum
• Future developments taken up by industry
Grids at NIKHEF 2004.07.14 11
Access in a coordinated way
• Transparently crossing of domain boundariessatisfying constraints of– site autonomy
– authenticity, integrity, confidentiality
• single sign-on to all services• ways to address services collectively• APIs at the application level• every desktop, laptop, disk is part of the Grid
Grids at NIKHEF 2004.07.14 12
Realization: projects at NIKHEF
• Virtual Lab for e-Science (BSIK)– 2004-2008
• Enabling Grids for e-Science in Europe (FP6)– 2004-2005/2007
• GigaPort NG Network (BSIK)– 2004-2008
• NL-Grid Infrastructure (NCF)– 2002-…
• EU DataGrid (FP5, finished)– 2001-2003
Grids at NIKHEF 2004.07.14 13
Research threads
1. end-to-end operation for data-intensive sciences (DISc): – data acquisition – ATLAS Level-3
– wide-area transport, on-line and near-line storage – LGC SC
– data cataloguing and meta-data – D0 SAM
– common API and application layer for DISc – EGEE App+VL-E
2. design scalable and generic Grids– grid software scalability research, security
3. deployment and certification– large-scale clusters, storage, networking
Grids at NIKHEF 2004.07.14 14
End to End – the LCG Service Challenge
• 10 Pbyte per year exported from CERN (ready in 2006)• Targets for end 2004 –
1. SRM-SRM (disk) on 10 Gbps links between CERN, NIKHEF/SARA, Triumf, FZK, FNAL 500 Mb/sec sustained for days
2. Reliable data transfer service3. Mass storage system <-> mass storage system
1. SRM v.1 at all sites2. disk-disk, disk-tape, tape-tape
4. Permanent service in operation• sustained load (mixed user and generated workload)• > 10 sites • key target is reliability• load level targets to be set
slide: Alan Silverman, CERN
Grids at NIKHEF 2004.07.14 15
Networking and security
• 2x10Gbit/s Amsterdam-Chicago• 1x10Gbit/s Amsterdam-CERN
– ATLAS 3rd level trigger (distributed DACQ)
– protocol tuning and optimization
– Monitoring and micro-metering
– LCG service challenge: sustained high-throughput
• collaboration with Cees de Laat (UvA AIR) + SURFnet• ideal laboratory for our security thread (many domains)
Grids at NIKHEF 2004.07.14 16
Building the Grid
• The Grid is not a magic source of power!– Need to invest in storage, CPUs, networks
– LHC needs per major centre (assume 10 per expt.): ~ 3 PByte/yr, ~40 Gbit/s WAN, ~15 000 P4-class 2GHz
– … more for a national multi-disciplinary facility
– Collaborative build-up of expertise:NIKHEF, SARA, NCF, UvA, VU, KNMI, ASTRON, AMOLF, ASCI, …
– Resources: NIKHEF resources + NCF’s NL-Grid initiative + …
Grids at NIKHEF 2004.07.14 17
Resources today (the larger ones)
• 1.2 PByte near-line StorageTek
• 36 node IA32 cluster ‘matrix’
• 468 CPU IA64 + 1024 CPU MIPS
• multi-Gbit links to 100TByte cache
• 7 TByte cache
• 140 nodes IA32
• 1Gbit link SURFnet
• multiple links with SARA
only resources with either GridFTP or Grid job management
Grids at NIKHEF 2004.07.14 18
A Facility for e-Science
• Many (science) application with large data volumes:– Life Sciences: micro-arrays (Utrecht, SILS Amsterdam)– Medical imaging: functional MRI (AMC), MEG (VU)– ‘omics’ and molecular characterization: sequencing (Erasmus),
mass spectroscopy (AMOLF), electron microscopy (Delft, Utrecht)
today such groups are not yet equipped to deal with their >1TByte data sets, our DISc experience can help
• Common need for multi-Pbyte storage• ubiquitous networks for data exchange• sufficient compute power, accessible from anywhere
Grids at NIKHEF 2004.07.14 19
Common needs and solutions?
• VL-E Proof of Concept environment for e-Science
• grid services address the common needs (storage, computing, indexing)
• application can rely on a stable infrastructure
• valuable experience as input to industry (mainly industrial research)
• can increasingly leverage emerging industry toolsthe Grid will be a household term like the Web
by pushing on the PByte leading edge, TByte-sized storage will be an e-Science commodity
Grids at NIKHEF 2004.07.14 20
NIKHEF PDP Team
in no particular order:• End-to-end applications: Templon, Bos, Grijpink, Klous• Security: Groep, Steenbakkers, Koeroo, Venekamp• Facilities: Salomoni, Heubers, Damen, Kuipers,
v.d. Akker, Harapan• Scaling and certification: Groep, Starink
embedded in both the physics and the computing groups