Adaptive Hybrid Computing and Scalable Many-Core Performance Supercomputer users procure their machines to satisfy specific and demanding require- ments. But they need their systems to grow and evolve to maximize machine lifetime and return on investment. To solve these challenges today and into the future, the Cray ® XC™ supercomputer series network and compute technology has been designed to easily accom- modate new processor introductions, upgrades and enhancements. Users can augment their systems in place to upgrade to higher performance processors, or add coprocessor and accelerator components to build even higher performance Cray XC series supercom- puter configurations. Cray XC Series Compute Blade The Cray XC series Intel ® Xeon Phi™ processor compute blade, previously codenamed “Knights Landing” or “KNL,” implements four single-socket compute nodes and is networked to to ther compute elements via the Cray-developed Aries™ interconnect. Compute blades stack up to 16 per chassis and each compute cabinet can be populated with up to three chassis. Cray XC series supercomputers can be configured up to hundreds of cabinets and upgraded to exceed 100 petaflops per system. Cray ® XC40™ Supercomputer Intel ® Xeon Phi™ Processor Compute Blade Cray’s XC series supercomputer architecture has been specifically designed from the ground up to be adaptive. A holistic approach optimizes the entire system to deliver sustained real-world performance and extreme scalability across the collective integration of all hardware, networking and software. One key differentiator with this adaptive supercomputing platform is the flexible method of implementing best-in-class processing elements via processor daughter cards (PDCs) on XC series compute blades. Processor Daughter Cards (PDCs) The Cray XC series system mates processor engine technology to the main compute blades via two configurable daughter cards. The flexible PCI Express 3.0 standard accommodates scalar processors, coprocessors and accelerators to create hybrid systems that can evolve over time. For example, PDCs can be swapped out or reconfigured while keeping the original com- pute base blades in place, quickly leveraging the best possible performance technologies. Intel ® Xeon Phi TM Processors The Cray XC series expands on the founding legacy of productive hybrid supercomputing, leveraging the best-in-class performance of integrated multi-core scalar and many-core accelerator technologies. Intel ® Xeon Phi™ processors, like the codenamed “Knights Landing,” are based on Intel’s Many Integrated Core (MIC) architecture, offering an alternative performance/power con- figuration to Intel Xeon processor products. Fields of smaller and lower-power cores can provide parallelism and power efficiencies on many demanding HPC applications.
Transcript
Adaptive Hybrid Computing and Scalable Many-Core PerformanceSupercomputer users procure their machines to satisfy specific and demanding require-ments. But they need their systems to grow and evolve to maximize machine lifetime and return on investment. To solve these challenges today and into the future, the Cray® XC™ supercomputer series network and compute technology has been designed to easily accom-modate new processor introductions, upgrades and enhancements. Users can augment their systems in place to upgrade to higher performance processors, or add coprocessor and accelerator components to build even higher performance Cray XC series supercom-puter configurations.
Cray XC Series Compute BladeThe Cray XC series Intel® Xeon Phi™ processor compute blade, previously codenamed “Knights Landing” or “KNL,” implements four single-socket compute nodes and is networked to to ther compute elements via the Cray-developed Aries™ interconnect. Compute blades stack up to 16 per chassis and each compute cabinet can be populated with up to three chassis. Cray XC series supercomputers can be configured up to hundreds of cabinets and upgraded to exceed 100 petaflops per system.
Cray’s XC series supercomputer architecture has been specifically designed from the ground up to be adaptive. A holistic approach optimizes the entire system to deliver sustained real-world performance and extreme scalability across the collective integration of all hardware, networking and software.
One key differentiator with this adaptive supercomputing platform is the flexible method of implementing best-in-class processing elements via processor daughter cards (PDCs) on XC series compute blades.
Processor Daughter Cards (PDCs)The Cray XC series system mates processor engine technology to the main compute blades via two configurable daughter cards. The flexible PCI Express 3.0 standard accommodates scalar processors, coprocessors and accelerators to create hybrid systems that can evolve over time. For example, PDCs can be swapped out or reconfigured while keeping the original com-pute base blades in place, quickly leveraging the best possible performance technologies.
Intel® Xeon PhiTM Processors The Cray XC series expands on the founding legacy of productive hybrid supercomputing, leveraging the best-in-class performance of integrated multi-core scalar and many-core accelerator technologies.
Intel® Xeon Phi™ processors, like the codenamed “Knights Landing,” are based on Intel’s Many Integrated Core (MIC) architecture, offering an alternative performance/power con-figuration to Intel Xeon processor products. Fields of smaller and lower-power cores can provide parallelism and power efficiencies on many demanding HPC applications.
Leveraging the Benefits of Parallelism The Intel® Xeon PhiTM “Knights Landing” processor family is comprised of self-booting host processors which can be embedded up to 64–68 cores in Cray XC compute blade configurations. This new many-core device supports wider vector units and more threads per core to deliver in excess of 3 TF per device. One of the keys to scaling parallelism is localizing and optimizing data movement.
Exploiting Advances in Memory Technology Each of the new XC series compute nodes implements an Intel Xeon Phi processor socket with pipeline to local DDR memory, as well as PCIe Gen 3 x16 access to the high-performance Cray-developed Aries interconnect. This processor family implements innovative new onboard high-bandwidth DRAM memory (HBM configurations up to 16 GB), which is tightly coupled with the host compute die. Users can see a significant performance boost from identifying high-bandwidth data and placing that data in the on-chip memory for their HPC applications.
The high-performance memory can be configured on the flexible XC series compute nodes at boot time (job launch) to be used as local cache or as directly-accessible fast memory. This device is Xeon processor binary compatible, and whether programmers write all their own code or users load pre-existing ISVs, this productivity capability makes it easy to support different use modes.
Learn more about Intel Xeon Phi products at http://www.intel.com/content/www/us/en/processors/xeon/xeon-phi-detail.html.
Cray XC™ Series Specifications for Intel® Xeon Phi™ Processor Compute BladeProcessor Intel® Xeon Phi™ processor family, up to 192 per cabinet, 16 GB High Bandwidth Memory (HBM)
Memory96-192 GB per node
Memory bandwidth: DDR up to 115 GB/s per node, HBM up to 490 GB/s per node
Compute CabinetUp to 192 sockets per cabinet, upgradeable with processors of varying core counts
Peak performance: up to 586 TF per KNL system cabinet
Interconnect
1 Aries routing and communications ASIC per 4 compute nodes
48 switch ports per Aries chip (500 GB/s switching capacity per chip)
Dragonfly interconnect: low latency, high bandwidth topology
System Administration
Cray System Management Workstation (SMW)
Single-system view for system administration
System software rollback capability
Reliability Features (Hardware)
Integrated Cray Hardware Supervisory System (HSS)
Independent, out-of-band management network
Full ECC protection of all packet traffic in the Aries network
Redundant power supplies; redundant voltage regulator modules
Redundant paths to all system RAID
Hot swap blowers, power supplies and compute blades
Integrated pressure and temperature sensors
Reliability Features (Software)
HSS system monitors operation of all operating system kernels
Lustre® file system object storage target failover; Lustre metadata server failover
Software failover for critical system services including system database, systemlogger, and batch subsystems
NodeKARE (Node Knowledge and Reconfiguration)
Operating SystemCray Linux Environment (includes SUSE Linux SLES11, HSS and SMW software)
Extreme Scalability Mode (ESM) and Cluster Compatibility Mode (CCM)
Compilers, Libraries & Tools
Cray Compiler Environment, Intel Compiler, PGI Compiler, GNU compiler
Support for the ISO Fortran standard (2008) including parallel programming using coarrays, C/C++ and UPC
MPI 3.0, Cray SHMEM, other standard MPI libraries using CCM. Cray Apprentice and CrayPAT™ performance tools. Intel Parallel Studio Development Suite (option)
Job Management
PBS Professional job management system
Moab Adaptive Computing Suite job management system
SLURM – Simple Linux Unified Resource Manager
External I/O Interface Infiniband, 40 and 10 Gigabit Ethernet, Fibre Channel (FC) and Ethernet
Disk Storage Full line of FC, SAS and IB based disk arrays with support for FC and SATA disk drives, ClusterStor™ data storage
Parallel File System Lustre, Data Virtualization Service (DVS) allows support for NFS, external Lustre and other file systems
Power
90 kW per compute cabinet, maximum configuration
Support for 480 VAC and 400 VAC computer rooms
6 kW per blower cabinet, 20 AMP at 480 VAC or 16 AMP at 400 VAC (three-phase, ground)
Cooling Water cooled with forced transverse air flow: 6,900 cfm intake
Dimensions (Cabinets)H 80.25” x W 35.56” x D 62.00” (compute cabinet)
H 80.25” x W 18.00” x D 42.00” (blower cabinet)
Weight (Operational)3,450 lbs. per compute cabinet—liquid cooled, 243 lbs./square foot floor loading750 lbs. per blower cabinet
Regulatory Compliance
EMC: FCC Part 15 Subpart B, CE Mark, CISPR 22 & 24, ICES-003, C-tick, VCCI
