Quadrics, Inc. June 2006 – Confidential slide 1
QsNetIII and QsTenG
Networks for High PerformanceComputing
12th June 2006
Quadrics, Inc. June 2006 – Confidential slide 2
AGENDA
Company introduction
QsTenG Roadmap
QsNetIII Roadmap
Quadrics, Inc. June 2006 – Confidential slide 3
COMPANY
Quadrics, Inc. June 2006 – Confidential slide 4
Summary
Established in 1996, with locations in the US, UK, Italy and China
Leading supplier and developer of high performance networking products and resource management software
Subsidiary of Alenia Aeronautica and wholly owned by Finmeccanica. One of the largest industrial group in Europe in the defense, energy and security industries.
Developing proprietary HPC network and industry standard network products (10 GigE first)
Strategic market expansion: investments in USA (Quadrics, Inc.), Asia Pacific and Europe.
Currently 50 employees (70 % engineers)
Quadrics, Inc. June 2006 – Confidential slide 5
HistoryQuadrics R&D Team out of Meiko
1996 Incorporation as UK Limited Company, part of the Finmeccanica Group
(SIFI.MI Italian Stock Exchange)
1998 Manufacture in US through Solectron
1999 Compaq OEM agreement
2000 Compaq Third Party Manufacturing Agreement (Royalty)
2001 Shipment Direct from US to US customers
2005 Incorporation of Quadrics, Inc.
Focus on High Performance Interconnects and Resource Cluster Management with
a mission to be the market leader in such solutions
Quadrics, Inc. June 2006 – Confidential slide 6
Org Chart
G ia co m o F a v illaC F O
J im C o rkeM a nu fa c tu ring
M o ra y M cL a renH a rd w a re
D u n ca n R o w e thS o ftw a re
M o ra y M cL a renR & D
R a im o n d o F o rlinH P - S a le s N o rth A m e rica
A n ton io V oxM a rke tin g & S a les
C ris to fo ro R o m a n e lliC E O
Quadrics, Inc. June 2006 – Confidential slide 7
2004 2005 2006 2007
96 PORTS SWITCH
STANDARD10 Gbit/s ETHERNET
Strategy - Roadmap
QsNetII
ELAN 4 & ELITE 4
PROPRIETARYQsNet
ELAN 5ELITE 5
QsNetIII
288 PORTSSWITCH
Quadrics, Inc. June 2006 – Confidential slide 8
QsTenG
Quadrics, Inc. June 2006 – Confidential slide 9
QsTenG - Overview
10 Gbit/s Ethernet layer 2 switch family
Data centre and clustering market
QS-TG108 switch: up to 96 ports, copper CX4
Low latency - 0.5s to 1.5 s port to port
Compact switch chassis (8U)
Can be used with standard 10 Gbps Ethernet NICs
Quadrics, Inc. June 2006 – Confidential slide 10
QS-TG108 Switch - Architecture 12 ports microchip building block
Two stage fat-tree topology
Scalable 8-96 ports
450ns per stage latency
Source routed
Building blocks
Backplane links
Computing nodes
CX4 copper cables
Quadrics, Inc. June 2006 – Confidential slide 11
QS-TG108 - Physical Design
8U chassis
1U PSU subsystem
Front to back airflow
All connections on rear of switch
Quadrics, Inc. June 2006 – Confidential slide 12
QS–TG108 - Rear View
1 to 12 line cards
(part ID QM601)
1 to 2 control cards
(part ID QM602)
Quadrics, Inc. June 2006 – Confidential slide 13
QM601 - 8 Ports Line Card
8 CX4 ports to front panel
4 CX4 ports to backplane
Control processor
Quadrics, Inc. June 2006 – Confidential slide 14
QM602 - Control Card
Master control processor
External control and management 1 Gbps Ethernet connection
12 CX4 backplane connections
Quadrics, Inc. June 2006 – Confidential slide 15
QS–TG108 - RAS Features
Redundant control architecture
1 from 2 (2 from 3) redundant power supply
Dual sequential fans
Quadrics, Inc. June 2006 – Confidential slide 16
QS-TG108 – Control Structure
Embedded Linux control processor for each switch card
Switch configuration Stats gathering
Control card – QM602
Single point of control SNMP switch agent Fail-over to controller on alternate QM602
Dual 1 Gbps Ethernet control interfaces
Quadrics, Inc. June 2006 – Confidential slide 17
QsTenG – Firmware
Embedded Linux control software
SNMP Device management support
Power on self test
Hardware diagnostics
Performance monitoring
Quadrics, Inc. June 2006 – Confidential slide 18
QsTenG – Software
MPI-2 common to QsNet and QsTenG
Adapter specific drivers
Quadrics Resource Management System (RMS)
24 7 production cluster management
Quadrics, Inc. June 2006 – Confidential slide 19
QsTenG – CX4 Adapters
QsTenG family of switches will operate with all standard 10 Gbps Ethernet adapters
Intel – validated Chelsio – validated Myricom – validated Neterion – validation about to start Neteffect – validation about to start
Quadrics, Inc. June 2006 – Confidential slide 20
Managed Layer 2 Protocol Support – Features
LVL7 FASTPATH® Software Certified protocol support for routing and management including
VLAN - Network grouping/segmentation- IEEE 802.3ac - VLAN Tagging- IEEE 802.1v - Protocol-based VLAN - IEEE 802.1Q - Virtual LAN with port based VLANs
Spanning Trees - Broadcast storm/loop avoidance- IEEE 802.1S – Multiple Spanning Tree- IEEE 802.1W - Rapid Spanning Tree- IEEE 802.1D - Spanning Tree
Flow Control- IEEE 802.3x - Flow control
Priorities- IEEE 802.1p — Ethernet Priority with User Provisioning &
Mapping
Quadrics, Inc. June 2006 – Confidential slide 21
Managed Layer 2 Protocol Support – Features Cont…
GARP - Attribute register/de-register IGMP - Snooping - Multicast membership Link aggregation - Channel bonding
- IEEE 802.3ad — Link Aggregation
Port mirroring - Copy for analysis Jumbo Ethernet Frames Power management for individual slots Fan control Redundancy (Cold Failover) Hot plug …
Quadrics, Inc. June 2006 – Confidential slide 22
Federation Support – Features
Federation of multiple Switches
Quadrics Internal Software development
Federation Support
Support for 2000 ports
Fat Tree Topology
Quadrics, Inc. June 2006 – Confidential slide 23
QsTenG – Summary
A new generation of 10 Gbps Ethernet switch for new classes of bandwidth hungry applications.
Takes Quadrics scalable switch design into the standards based interconnect market.
Quadrics, Inc. June 2006 – Confidential slide 24
QsNetIII
Quadrics, Inc. June 2006 – Confidential slide 25
QsNetIII – Overview
New host adapter – based on Elan5 microchip
PCI Express 16x, DDR
New custom fabric – based on Elite5 microchip
High radix switches DDR CX4 links
Common mechanics and control structure
128 node building block (8U)
Quadrics, Inc. June 2006 – Confidential slide 26
Elan5 – Overview (product ID QM700)
2 QsNetIII link
Multiple Thread processors
Local memory (1Mbytes) Buffer space, routes, page
tables Memory Management
4 pages sizes 4M-32G TLB (64 16 pages)
Host Interface PCI-Express 16x DDR support
Fabric
Bridge
x8
PCIe 16 Lanes
PLL
External Code Interface
Clocks
16K x 8 x 8 banks = 1MB ECC RAM
Host I/F
PCIe
TLB Cmd Launch
Ext i/f
X2
CX4/ QsNetIII
Link
SERDES
SAC/SA SIP SOP
UAC/UA UOP UIP
Slave Access Ports (slave only)
Unit Access Ports (master/slave)
Addr Translator
Buff Mngr Host Cache
Free List
Local Memory
X2
CX4/ QsNetIII
Link
ThreadProc I$ 16K
DMEM 8K
ThreadProc I$ 16K
DMEM 8K
ThreadProc I$ 16K
DMEM 8K
ThreadProc I$ 16K
DMEM 8K
ThreadProc I$ 16K
DMEM 8K
ThreadProc I$ 16K
DMEM 8K
ThreadProc I$ 16K
DMEM 8K
Quadrics, Inc. June 2006 – Confidential slide 27
Elan5 – Thread Processor
Two thread instructions per cycle 500-600 MHz clock 128 registers 16K instruction cache 8K data buffers
- Assembling network packets- Checksum and CRC generators on write port
DMA read, write, set event, wait event
Quadrics, Inc. June 2006 – Confidential slide 28
Elite5 - Overview
16 x4, 32 x2 switch component
Or multiple 8,4,2,1 switches
Physical layer DDR XAUI
4 x 6.25Gbit/s 2.5Gbytes/s peak in each direction
Broadcast & barrier support
32 x 32 x 2 core
Quadrics, Inc. June 2006 – Confidential slide 29
Elite 5 – Switch Packaging, cards
16 ports link card (Part ID QM701)
Building block for: 256 ports stand alone switch 128 ports to computing nodes, 128 ports
to top level switches chassis
Top switch card (Part ID QM702)
Building block for: 256 ports stand alone switch Federated networks
Top switch card (Part ID QM703) Federated networks
Quadrics, Inc. June 2006 – Confidential slide 30
QsNetIII – 128 Ports chassis
Building block for federated networks
8 1024-way systems 16 2048-way systems x32 4096-way systems x64 8192-way systems
128 ports to nodes, 128 backplane links to top level switching cards
2 stage fat tree
9U high
Quadrics, Inc. June 2006 – Confidential slide 31
QsNetIII – 256 Ports chassis
High Density stand alone switch
800 Gbyte/sec bi-section bandwidth
2 stage fat tree
14U high
Can be used to provide multiple 32 and 64 top switches for very large systems
Quadrics, Inc. June 2006 – Confidential slide 32
4096 Way System – 1U Nodes
128 node scalable units
Each scalable unit has a node level switch
Upgradable bisectional bandwidths:
1/8; ¼; ½; full
Quadrics, Inc. June 2006 – Confidential slide 33
QsNetIII – 4096 Port Network Topology
Quadrics, Inc. June 2006 – Confidential slide 34
4096 1U SYSTEM - FLOOR PLAN
System size: 10m 20m (33' 66') with an aisle spacing of 5'. Maximum cable length is approximately 13m (43')
Quadrics, Inc. June 2006 – Confidential slide 35
Possible Lustre configuration with 1/8 bisectional bandwidth
48 Lustre storage servers (2 meta dataservers, 2 management nodes and a small number of login nodes).
Divided equally between the first and last node switches.
Should be configured in pairs (with one server from each pair attached to each node switch).
This configuration will maximise the Lustre I/O bandwidth with sparse connectivity between the node switches.
If all 4096 ports are used the filesystem will share uplink bandwidth with the nodes attached to its node switch.
Quadrics, Inc. June 2006 – Confidential slide 36
Quadrics Products Compared
QsNetI I QsTenG QsNetI I I
ProtocolQuadrics
proprietary10GB Ethernet
Quadrics proprietary
Physical layer 10 bits @ 1.333GHz
CX4 4 bits @ 3.125Gb/s
DDR CX4 4 bits @ 6.25Gb/s
Unidirectional rate 920 Mbytes/s 1000 Mbytes/s 2000 Mbytes/s
Bidirectional rate 900 Mbytes/s 1000 Mbytes/s 4000 Mbytes/s
Switch arity 8 12 16/32
Switch hop latency 20ns 450ns 40ns
NIC Elan4 Commodity or
Elan5-10G RDMA Elan5 - QsNetIII
Bus Interface PCI-XPCI-X or PCI
Express16x PCI Express
MPI Latency 1.3uS ~10uS ~1.3uS