Post on 25-Feb-2016
description
transcript
Exa-Scale Data Preservation in HEP
Jamie.Shiers@cern.ch APA/C-DAC Conference
February 2014
International Collaboration for Data Preservation and Long Term Analysis in High Energy Physics
2
Background
• Whilst this talk concerns data from High Energy Physics (HEP) experiments at CERN and elsewhere, many points are generic
• The scale: 100PB today, reaching ~5EB by 2030– “Trusted” repositories of this size– and with a lifetime of at
least decades – are a sine qua non of our work
• I will also talk about costs, business cases, problems and opportunities…
BEFORE!
4
200-400 MB/sec
Data flow to permanent storage: 4-6 GB/sec
1.25 GB/sec
1-2 GB/sec
1-2 GB/sec
CERN-JRC meeting Bob Jones
The LHC Computing Grid, February 2010 5
Tier 0 – Tier 1 – Tier 2Tier-0 (CERN):•Data recording•Initial data reconstruction
•Data distribution
Tier-1 (11 centres):•Permanent storage•Re-processing•Analysis
Tier-2 (~130 centres):• Simulation• End-user analysis
Tier-2 centres in India:•Kolkata (ALICE)•Mumbai (CMS)
Frédéric Hemmer
Managing 100 PBytes of data
27 January 2014 CERN-JRC meeting Bob Jones 6
LHC Schedule
CERN-JRC meeting Bob Jones 7
First run LS1 Second run LS2 Third run LS3 HL-LHC
2009 2013 2014 2015 2016 2017 201820112010 2011 2019 2023 2024 2030?20212020 2022 …
LHC startup
900 GeV
7 TeVL=6x1033 cm-2s-2
Bunch spacing = 50 ns
Phase-0 Upgrade(design energy,
nominal luminosity)
14 TeVL=1x1034 cm-2s-2
Bunch spacing = 25 ns
Phase-1 Upgrade(design energy,
design luminosity)
14 TeVL=2x1034 cm-2s-2
Bunch spacing = 25 ns
Phase-2 Upgrade(High Luminosity)
14 TeVL=1x1035 cm-2s-2
Spacing = 12.5 ns
8
ATLAS Higgs Candidates
AFTER!
10
CERN has ~100 PB archive
11LS1 Status Report – 116th LHCCFrédérick Bordry 4th December 2013
LHCb b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o
Injectorso o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o
t
LHCo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o
Injectorso o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o
LHCb b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o
Injectorsb b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o
2015 2016 2017 2018 2019Q4 Q1 Q2
2020 2021Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q3 Q4
2022 2023 2024 2025 2026 2027 2028
Q1 Q2 Q3 Q4 Q1 Q2Q3 Q4 Q1 Q2 Q3 Q4Q1 Q2 Q3
Q1 Q2 Q3 Q4Q1 Q2 Q3 Q4 Q1 Q2 Q1 Q2 Q3 Q4
2029 2030 2031 2032 2033 2034
Q3 Q4 Q1 Q2 Q3 Q4Q1 Q2 Q3 Q4 Q1 Q2Q3 Q4
Q2 Q3 Q4 Q1 Q2 Q32035
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q4Q2 Q3 Q4 Q1 Q2 Q3Q4 Q1 Q2 Q3 Q4 Q1
Only EYETS (19 weeks) (no Linac4 connection during Run2) LS2 starting in 2018 (July) 18 months + 3months BC (Beam Commissioning)LS3 LHC: starting in 2023 => 30 months + 3 BC
injectors: in 2024 => 13 months + 3 BC
But its still early days for the LHC!
Run 2 Run 3
Run 4
LS 2
LS 3
LS 4 LS 5Run 5
LHC schedule approved by CERN management and LHC experiments spokespersons and technical coordinatorsMonday 2nd December 2013
ECFA European Commit tee fo r Future Acce le ra tors
HL-LHC Workshop 12
High Luminosity LHC (HL-LHC)Update of the European Strategy for Particle Physics adopted 30 May 2013 in a special session of CERN Council at Brussels.Statement c:
October 1, 2013
c) The discovery of the Higgs boson is the start of a major programme of work to measure this particle’s properties with the highest possible precision for testing the validity of the Standard Model and to search for further new physics at the energy frontier. The LHC is in a unique position to pursue this programme. Europe’s top priority should be the exploitation of the full potential of the LHC, including the high-luminosity upgrade of the machine and detectors with a view to collecting ten times more data than in the initial design, by around 2030. This upgrade programme will also provide further exciting opportunities for the study of flavour physics and the quark-gluon plasma.
Predrag Buncic, October 3, 2013 ECFA Workshop Aix-Les-Bains - 13
Data: Outlook for HL-LHC
• Very rough estimate of a new RAW data per year of running using a simple extrapolation of current data volume scaled by the output rates. • To be added: derived data (ESD, AOD), simulation, user data…
PB
Run 1 Run 2 Run 3 Run 40.0
50.0
100.0
150.0
200.0
250.0
300.0
350.0
400.0
450.0
CMSATLASALICELHCb
14
Volume: 100PB + ~50PB/year (+400PB/year from 2020)
15
2. Digital library tools (Invenio) & services (CDS, INSPIRE, ZENODO) + related tools (HepData, RIVET, …)
3. Sustainable software, coupled with advanced virtualization techniques, “snap-shotting” and validation frameworks
4. Proven bit preservation at the 100PB scale, together with a sustainable funding model with an outlook to 2040/50
5. Open Data (“Open everything”)
1. DPHEP Portal
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
16
Case B) increasing archive growth
Start with 10PB, then +50PB/year, then +50% every 3y (or +15% / year)
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
17
Case B) increasing archive growth
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
18
Total cost: ~59.9M$(~2M$ / year)
Case B) increasing archive growth
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
19
Case B) increasing archive growth
20
Summary
1. DPHEP portal: build in collaboration with other disciplines, including RDA IG and the APA…
2. Digital libraries: continue existing collaborations3. Sustainable “bit preservation” – certified
repositories as part of EINFRA-1-20144. “Knowledge capture & preservation”: BIG
CHALLENGE not addressed in multi-disciplinary way: next funding round?
5. Open “Big Data”: a Big Opportunity (for RDA?)
21
Portal Example # 1
22
Portal Platform – Zenodo?
David South | Data Preservation and Long Term Analysis in HEP | CHEP 2012, May 21-25 2012 | Page 23
Documentation projects with INSPIRE
> The ingestion of other documents is under discussion, including theses, preliminary results, conference talks and proceedings, paper drafts, ... More experiments working with INSPIRE, including CDF, D0 as well as BaBar
> Internal notes from all HERA experiments now available on INSPIRE Experiments no longer need to provide dedicated hardware for such things Password protected now, simple to make publicly available in the future
LEP Cost would be “now” …
Completely different, of course …Direct resource cost is already
compatible with zero for LEP experiments Total ALEPH DATA + MC (analysis format) =
30 TB ALEPH: Shift50 = 320 CernUnit. One of
today’s pizza box largely exceeds this CDF data: O(10 PB), bought today for
<400kEur CDF CPU ~ 1MSi2k = 4 kHS06 = 40kEur
Here the main problem is knowledge /support, clearly Can you trust a “NP peak” 10 years later,
when experts are gone? ALEPH reproducibility test (M.Maggi, by
NO mean a DP solution) ~0.5 FTE for 3 months
Zero!!=0, but
decreasing fast
25
26
Open Data?
27
Costs and Scale• There are 4 (main) collaborations + detectors at the LHC: the largest has
3000 members
• The annual cost of WLCG (infrastructure, operations, services) is ~EUR100M
• The CERN database services costs around 2MCHF per year for Materials (licenses, maintenance, hardware) and 2MCHF for personnel
• The central grid Experiment Integration Support team varied between 4-10 people, plus significant effort at sites and within experiments
• The DPHEP Full Costs of Curation workshop concluded that a team of ~4 people, with access to experts, could “make significant progress” (be careful with this number!)
28
Conclusions• Long-term data preservation is a journey, not a destination• As such, it is best not to venture out alone
• A clear understanding of costs & benefits is necessary to secure funding
• We are eager to share our knowledge and experience (exa-scale “bit preservation”)
• We have learned a lot through collaboration through the APA – and keen to learn more in the future