Polish Contribution to the Worldwide LHC Computing Grid
WLCGM. Witek
On behalf of the team of Polish distributed Tier-2
Outline• Introduction• History and status of Polish Tier-2 Federation• Examples of physics results from LHC experiments
CERN - European Organisation for Nuclear Reseach
LHC – Large Hadron Collider
CMS
Alice
LHCb
Atlas
2
Polish WLCG Tier-2 team• Polish WLCG Tier-2 team
– A. Binczewski3, M. Bluj5, A. Cyz2, M. Dwużnik1, M. Filocha4, L. Flis1, R. Gokieli4,5, J.Iwaszkiewicz4, M. Kowalski2, P. Lasoń1, R. Lichwała3, M. Łopuszyński4, M. Magryś1, P. Malecki2, N. Meyer3, K. Nawrocki4,5, A. Olszewski2, A. Oziębło1, A. Padee4,5, H. Pałka2, M. Pospieszny3, M.Radecki1, R. Rowicki4, D. Stojda6,4, M. Stolarek4, T. Szepieniec1, T. Szymocha1,2, M.Turała1,2, K.Wawrzyniak4,5, W.Wiślicki4,5, M.Witek2, P.Wolniewicz3
• Institutions– 1 AGH University of Science and technology, ACC Cyfronet AGH, Krakow– 2 Institute of Nuclear Physics PAN, IFJ PAN, Krakow– 3 Poznan Supercomputing and Networking Center, PSNC, Poznan– 4 Interdisciplinary Centre for Mathematical and Computational Modelling, ICM, Warsaw– 5 Soltan Institute for Nuclear Studies, IPJ, Warsaw– 6Copernicus Science Centre, CSC, Warsaw, Poland
More details will be available in the article of PL-Grid Book
3
Eksperiment requirements
• Example of on-line conditions– ~108 electronic channels to read at 40
MHz– 1 PB/sec of raw info from the detector– ~few TB/sec of zero suppressed data
( channels with signal only)
• Multi-level trigger system– Reduction factor 106-107
– Output rate > 100 MB/sec
• 107 sec/year ~few PB of raw data per experiment
Level 1 Level 2
40 MHz (1000 TB/sec)
Level 3
75 KHz (75 GB/sec)5 KHz (5 GB/sec)100 Hz (100
MB/sec)Data Recording &Offline Analysis
WLCG
4
(Pre)History
• MONARC (Models of Networked Analysis at Regional Centres for LHC Experiments)– Simulate data processing (a kind of Monte Carlo model):
• a complex set of networked computing systems (CERN, Tier-1 and Tier-2 regional centres)
• the analysis process, composed of a dynamic workload of reconstruction and analysis• the behaviour of key elements of the system, such as distributed database servers and
the networks
Summary: CERN-LCB-2000-001
One of the problems @ 2000:data sharing difficult due to the prohibitively high cost of networking
Even more pronounced in Poland at that time !!!
5
From 2000 to 2011• 2000: DataGrid 2000 – HEP oriented EU project• 2000: CERN-Poland meeting (initiated by M. Turala & P. Malecki)• 2002: CrossGrid first Polish EU project with HEP task.• 2002: start of LCG (LHC Computing Grid)• 2003: First LCG testbed of 14 sites (Cyfronet participation)• 2004-2010: EGEE, EGEE II, EGEE III• 2003-2008: Participation in Data Challenges• 2005-2010: BalticGrid, BalticGrid II• 2007: Poland signs MoU for WLCG• 2009: start of PL-Grid project (WLCG infrastructure)• 2009-2011: LHC start. Smooth running of Polish WLCG sites.
6
Polish participation in networking and grid projects
Polish part of WLCG emerged out of many different projects. Some of them not directly related to HEP. Nevertheless they provided infrastructure and common tools.
PIONIER, CrossGrid, EGEE and PL-Grid were essential for the development of Polish WLCG infrastructure.
7
GTS 1,6 Gb/s
GDAŃSK
POZNAŃ
ZIELONA GÓRA
KATOWICE
KRAKÓW
LUBLIN
WARSZAWA
BYDGOSZCZ TORUŃ
CZĘSTOCHOWA
BIAŁYSTOK
OLSZTYN
RZESZÓW
Bielsko-Biała
GÉANT 10+10 Gb/s
KOSZALIN
SZCZECIN
WROCŁAW
ŁÓDŹ
KIELCEPUŁAWYOPOLE
RADOM
BASNET 34 Mb/s
CESNET, SANET
GÉANT/TELIA 2x2,5 Gb/s
DFN 10 Gb/s Gorzów
MAN
10 Gb/s(1 lambda)
2 x 10 Gb/s
1 Gb/s
CBDF 10 Gb/s
PIONIER’S FIBERS
Polish infrastructure in WLCG
Tier1 FZKKarlsruhe
Tier2 PCSSPoznań Tier2 ICM
Warszawa
Tier2 ACKCyfronet Kraków
HEP VLAN1 Gb/s
HEP VLAN1 Gb/s
8
Pionier – Polish Optical Internet• Nationwide broadband optical network represents a base for research and development
– Europe's first national academic network that uses its own dark fiber optics and DWDM 10GE transmission
– Connects 21 Academic Network Centers of Metropolitan Area Networks (MAN) and 5 of the HPC (High Performance Computing) centers.
• Cross-border links with: Germany (DFN), Czech Republic (CESNET), Slovakia (SANET), Ukraine (UARNET), Byelorus, Lithuania
Started in 2000, 5917,5 km of fibres cables @ 2010
Data sharing problem due to networking limits indicated by MONARC solved !
9
Towards physics results - that is what WLCG was built for
Every experiment (ALICE, ATLAS, CMS, LHCb) has its own computing model. All based on multi-tier structure
Successful LHC restart in 2009Excellent performance of the LHC and the experiments
3.5*1014 collisions delivered in 2011
Polish distributed Tier-2Kraków-Poznań-Warszawa
The RAW data reside in Tier-0 and Tier-1. The final physics result is an effect of many processing steps. Some steps have to be repeated when better detector calibration and alignment appear.
The role of Tier-2 sites:Monte Carlo productionAnalysis centre (larger sites)Limited reprocessing activity
10
LHCb computing model – DIRAC role
11
Polish WLCG resources
ACK Cyfronet AGH – Tier2
Polish sites provide about 2% of the total computing power in proportion to the engagement of Polish scientists in the LHC experiments.Most of the resources comes from PL-Grid infrastructure
Polish Tier-2 Federation 2010 2011 2012
CPU (HS06) 10540 13050 15800
Disk (TB) 599 810 1020 Tier-3 at IFJ PANWLCG pledges
Reliability
Availability
AtlasLHCb
Warsaw ICM – Tier2
CMSLHCb
Poznan PSNC – Tier2
AliceAtlas
July 2011
12
Poland in LCG accounting
A week with 9 % contribution of Cyfronet to the LHCb reprocessing of 2011 data.Congratulations for the site performance from the LHCb computing team.
LHCb3.9% delivered by PolandCMS
2% delivered by Poland
Month 2011
May June July Aug Sept
Usage as % of pledge
165% 266% 299% 173% 229%
Performance of Polish Tier-2 (From WLCG Accounting)
Increase over 100% mainly due to Cyfronet additional resources
Highlight from LHCb
Atlas1.6 %
Delivered by Poland
Tier-1/2
At present 69 Tier-2 federations in 33 countries. The size of Polish Tier-2 is in top 20.
13
The Standard Model – Particles and Forces
Bulding blocks of our Universe
14
ATLAS and CMS – direct discovery potential• Find the last undiscovered element of the Standard Model
– The Higgs bozon
• Direct discovery of New Physics – Supersymmetric particles or any object/phenomena beyond Standard Model we even did
not think about yet ATLAS
CMS
The Higgs boson search
15
LHCb – LHC beauty experiment• Hunting the signs of new physics by precision measurements
Bs0→μ+μ-
Bs0 (bs) meson contains two
quarks which are not present in the world around us: b beauty and s strange
Extremely rare decay3 out of 109 decays
Standard Model Supersymmery
New Physics
2 decays observed – 1.4 expected
No sign of New Physics yet2 events selected out of recorded 1010 WLCG 2 events selected out of 3.5*1014 initial collisions
New phenomena can significantly increase the decay rate
16
Summary• Polish physicists and computer scientists created distributed WLCG
Tier2, which availability and efficiency is high; the delivered resources are in the range 2-3 % of the total.
• The performance of the WLCG matches high requirements of the data processing of LHC experiments.
• Lots of highest quality physics results are being shown recently. Much more are on the way, hopefully showing New Physics.
AcknowledgmentsWe would like to thank all people for their contribution to the successful creation and operation of Polish Tier-2 Federation. In particular the management and staff of the three computing centres, ACC Cyfronet AGH in Krakow, ICM UW in Warsaw and PSNC in Poznan for their constructive cooperation. A supportive approach of the directorates of IFJ PAN Krakow and IPJ Warsaw is also appreciated. We are also grateful to many people from CERN, the FZK Karlsruhe and other WLCG collaborating institutions for their friendly advice and support.