Post on 02-Jul-2015
description
transcript
INFINIBAND“Bandwidth Out Of The Box”
Presented By-
Shekhar Kumar
074, CS-’B’
Guided By- Latha S.
Nair
Contents
Introduction
I/O Architectures- Fabric vs Bus
Technical Overview
InfiniBand Layers
InfiniBand Architecture
Advantages & Drawbacks
Markets
Conclusion
References
Introduction
A switch-based serial I/O interconnect architecture
Operates at a base speed of 2.5 Gb/s or 10 Gb/s in each direction (per port)
Provides both QoS (Quality of Service) and RAS (Reliability, Availability and Serviceability)
Superset of the Virtual Interface Architecture
Used in high performance computing and data centres
I/O Architectures- Fabric vs Bus
Feature Fabric Bus
Topology Switched Shared Bus
Pin Count Low High
Number Of End Points Many Few
Max Signal Length KMs Inches
Reliability Yes No
Scalable Yes No
Fault Tolerant Yes No
I/O Architectures Contd…
Shared Bus ArchitectureCPU
System
Controller
I/O Controller
I/O Controller
I/O Controller
I/O Controller
I/O Controller
PCI to PCI
Bridge
PCI to PCI
Bridge
System
Memory
I/O Architectures Contd…
Switched Fabric Architecture
Technical Overview
Switch-based point-to-point interconnect architecture
Each individual link is based on a four-wire 2.5Gb/s bidirectional connection.
Architecture defines a layered hardware protocol (Physical, Link, Network, Transport Layers) as well as a software layer
Each link can support multiple transport services for reliability
InfiniBand Feature Set
Layered Protocol- Physical, Link, Network, Transport, Upper Layers
Packet based communication Quality Of Service Three link speeds1. 1X- 2.5 Gb/s, 4 Wire2. 4X- 10 Gb/s, 16 Wire3. 12X- 30 Gb/s, 48 Wire PCB, Copper and Fiber cable
interconnect Remote DMA support Multicast and Unicast support
InfiniBand Layers
InfiniBand Layers Contd…
1) Physical Layer
Defines both electrical and mechanical
characteristics for the system
Includes cables and receptables for
fibre and copper media, backplane
connectors
Defines three link speeds, 1X,4X,12X
Each individual link is a 4-wire
differential connection that provides a
full duplex connection at 2.5 Gb/s
InfiniBand Layers Contd…
2) Link Layer
Encompasses packet layout, point-to-point operations, and switching within a local subnet
Packets- Two types: Management and Data packets
1. Management Packets- Used for link configuration and maintenance
2. Data Packets- Carry up to 4k bytes of a transaction payload
Switching- Devices within a subnet have a 16-bit Local ID assigned by the subnet manager. This LID is used for addressing.
InfiniBand Layers Contd…
Link Layer
QoS- supported through virtual lanes(VL)
VLs are separate logical communication links which share a single physical link
Data Integrity
Two CRCs per packet, Variant CRC and Invariant CRC
16-bit VCRC includes all fields in the packet and is recalculated at each hop
32-bit ICRC covers only the fields that do not change from hop to hop
InfiniBand Layers Contd…
Network Layer Handles routing of packets from one
subnet to another(within a subnet a network layer is not required)
Packets contain a global route header(GRH)
GRH contains the 128-bit IPv6 addressTransport Layer Responsible for in-order packet
delivery, channel multiplexing and transport services
Also handles transaction data segmentation when sending, and reassembly when receiving
InfiniBand Architecture
InfiniBand Architecture
Contd… Defines multiple devices for system
communication: channel adapter, switch, router, and a subnet manager
At least one channel adapter is required for each end node
Subnet Manager sets up and maintains link
Channel adapters and switches contains Subnet Management Agent(SMA) for handling communication with the subnet manager
Channel Adapters
Connects InfiniBand to other devices
Two types: Host Channel adapter(HCA) and Target Channel Adapter(TCA)
HCA- Provides interface to a host device and supports all software verbs defined by InfiniBand
TCA- Provides connection to an I/O device from InfiniBand with a subset of HCA features necessary for specific operations of each device
Switches and Routers
Switches and Routers
Contd… Switch
Contains more than one InfiniBand port and forwards packets from one of its port to another based on the LID
Can be configured to forward either unicast packets or multicast packets
Can consume or generate management packets
Router
Forwards packets from one subnet to another without consuming or generating packets
Reads the Global Route Header to forward the packet based on IPv6 address
Advantages
Superior performance
Low-latency
High-efficiency
Fabric consolidation and low energy
usage
Reliable, stable connections
Data Integrity
Highly interoperable environment
Drawbacks
Complex in design
Few platforms supports it as yet
Bleeding edge, for now, so users will
need to perform extensive testing
Markets
Application Clustering
Inter-Processor Communication(IPC)
Storage Area Networks
CONCLUSION
Greater processor performance
Supports RAS
Works both in-the-box and enables
Bandwidth Out of the Box
Supports scalability
REFERENCES
1. InfiniBand Trade Association www.infinibandta.org
2. InfiniBand Network Architecture, MindShare, Inc.
By Tom Shanley
3. InfiniBand Architecture, Development and Deployment: A strategic guide to server I/O solutions
By William T. Futral
4. Qos in InfiniBand Subnetworks:
Alfaro,F.J.; Sanchez, J.L.; Duato, J.; Department de Informatica, Univ. de Castilla-La Mancha, Albacete, Spain