G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
The scientific computing resources of ENEA [The Italian National Agency for New Technologies,Energy and Sustainable Economic Development] are integrated in ENEAGRID infrastructure, a production quality, service oriented system for high performance and high throughput computing. The main computing resources are the CRESCO clusters (Linux x86_64), the main site is Portici (NA), the location of the original CRESCO project (2008, Computational RESearch center for COmplex systems), funded by the Italian Minister of Research and the Education,in the framework of PON 2000-2006. This presentation describes the infrastructure and illustrates its evolution as funded by the new projects (PON 2007-2013 framework) where ENEA-UTICT, the ICT Unit of ENEA, is one of the partners.
Overview
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
Outline
• ENEAGRID– The architecture & the computing resources–Users and applications–User interface and Virtual Labs–CRESCO clusters
• New projects PON 2007-2013– IT@CHA, TEDAT, LAMRECOR
• New infrastructures–CRESCO3–CRESCO4
• Conclusions
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
The infrastructure is based on “mature” multiplatform software components assuring reliability and easy administration. Web interfaces have been developed/customized for a friendly user environment:– Kerberos 5 authentication– File systems: • AFS/OpenAFS geographic file system (HOME)• GPFS: parallel file system (also WAN)
– Resource manager: LSF Multicluster– User Web graphical interface:
• NX/FARO• Jobrama: job monitoring & Accounting
– System monitoring: Zabbix–Web management of users/projects: WARC
ENEAGRID architecture
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
ENEAGRID offers to the users computing resources based on Linux x86_64 ( ~5800 cores), AIX SP5 (~256 cpu), special systems (e.g. GPUs), virtualized hosts and distributed storage. The resources are located in 6 sites, connect by GARR network.
ENEAGRID: computing resources
ENE
ENE
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
ENEAGRID: CRESCO & Portici Site
ENEA Research Center PorticiBuildings designed by Vittorio Gregotti,1982/86
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
ENEAGRID users: – serial or small scale parallel jobs: ~200 users– large scale HPC parallel jobs: ~50 users
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
New projects : PON 2007-2013
• PON/1 Ricerca Industriale DDR MIUR 01/Ric 18/1/2010– IT@CHA - (application of new technologies to the conservation and
valorization of cultural and artistic heritage)– LAMRECOR - (Advanced logistics for persons and goods)– DIRECTFOOD - Gestione integrata filiere alimentari e canali innovativi
produttore -consumatore
• PON/2 Distretti/Laboratori Pubblico Privati DDR MIUR 713/Ric 29/10/2010
– VIS4FACTORY - Sistemi Informativi Visuali per i processi di fabbrica nel settore dei trasporti
– DATABENC - Distretto ad ALta Tecnologia per i beni culturali nella regione Campania
• PON/3 Infrastrutture DDR MIUR 254/Ric 18/5/2011– TEDAT - Centro di eccellenza per le Tecnologie e la diagnostica avanzata nel
settore dei trasporti (Advanced technologies for transport,new materials for aerospace, automotive,..)
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
Portici: CRESCO4 cluster
In the framework of TEDAT project the procurament of the new CRESCO4 cluster is currently under way. An European tender has been set up, evaluated and the contract has been awarded (E4 Company).
– 304 computing nodes Intel E2670, 2.6 GHz, 4864 cores– Supermicro Fat-Twin chassis (8 nodes in 4U)– 101 TFlops Peak– 5 Racks for computing nodes, 1 network rack – Storage: DDN S2A9900 600 TB– Infiniband QDR QLogic/Intel 12800-180 (432 ports)– New computer room – Conditioning system taking advantage of free cooling technogy– Delivered in Portici by July 2013– Final test completed October 2013
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
Portici: CRESCO4 - IB network
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
Computer rooms: CRESCO & CRESCO4
CRESCO computer room
new CRESCO4 computer
room
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
CRESCO computer room (1)
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
CRESCO computer room (2)
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
CRESCO4 computer room (1)
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
CRESCO4 computer room (2)
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
CRESCO4 computer room (3)
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
CRESCO4 computer room (4)
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
CRESCO4 computer room (5)
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
ENE
Conclusions
A significant update of the ENEAGRID HPC infrastructure is currently under way, with the new CRESCO3 and CRESCO4 clusters, funded by several PON 2007-2013 projects
The architecture of ENEAGRID provides the framework for an easy introduction of the new computing resources both from the point of view of the user and the administrator.
By the end of the year the integrated peak computing power of ENEAGRID will increase of a factor 5 up to ~150 Tflops, the most powerfull system being CRESCO4 with ~100 Tflops.
G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013
