InfiniBand TM Architecture Release 1.2.1 Introduction November 2007 VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE InfiniBand SM Trade Association Page 62 Proprietary and Confidential 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 CHAPTER 1: INTRODUCTION This is Volume 1 of the InfiniBand Architecture specification. It is the first in a series of documents that describe the architecture. 1.1 ACKNOWLEDGMENTS The following persons were instrumental in the development of this volume of the InfiniBand Architecture specification: Steering Committee Members Co-chairs: Tom Bradicich Tom Macdonald Members: Jacqueline Balfour Ken Jansen John Pescatore Kevin Deierling Michael Krause Jim Pinkerton Balint Fleischer Todd Matters Martin Whittaker Dr. Alfred Hartmann Chet Mehta Bob Zak David Heisey Ed Miller Technical Working Group Members Co-chairs: Dwight Barron Paul Grun Jeff Hilland Irv Robinson David Wooten Members: Dr. Alan Benner Dr. Alfred Hartmann Dr. Gregory F. Pfister Mark Bradley Michael Krause Greg Still Wolfgang Christl Bill Lynn Ken Ward Diego Crupnicoff Ed Miller Working Group Co-Chairs Link Working Group (LWG): Daniel Cassiday Michael Krause Software Working Group (SWG): Ed Miller Renato J. Recio Jim Pinkerton Management Working Group (MgtWG): David W. Abmayr Brian Forbes Jeff Hilland Dr. Pankaj Mehra Dr. Gregory F. Pfister Hal Rosenstock William H. Swortwood Dr. Mazin Yousif Application Working Group (AWG): Dwight Barron William Futral Greg Pellegrino
Transcript
InfiniBandTM Architecture Release 1.2.1 Introduction November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 62 Proprietary and Confidential
This is Volume 1 of the InfiniBand Architecture specification. It is the first in a series of documents that describe the architecture.
1.1 ACKNOWLEDGMENTS
The following persons were instrumental in the development of this volume of the InfiniBand Architecture specification:
Steering Committee MembersCo-chairs:Tom Bradicich Tom MacdonaldMembers:Jacqueline Balfour Ken Jansen John PescatoreKevin Deierling Michael Krause Jim PinkertonBalint Fleischer Todd Matters Martin WhittakerDr. Alfred Hartmann Chet Mehta Bob ZakDavid Heisey Ed Miller
Technical Working Group MembersCo-chairs:Dwight Barron Paul Grun Jeff HillandIrv Robinson David WootenMembers:Dr. Alan Benner Dr. Alfred Hartmann Dr. Gregory F. PfisterMark Bradley Michael Krause Greg Still Wolfgang Christl Bill Lynn Ken WardDiego Crupnicoff Ed Miller
Working Group Co-ChairsLink Working Group (LWG):Daniel Cassiday Michael KrauseSoftware Working Group (SWG):Ed Miller Renato J. Recio Jim PinkertonManagement Working Group (MgtWG):David W. Abmayr Brian Forbes Jeff HillandDr. Pankaj Mehra Dr. Gregory F. Pfister Hal RosenstockWilliam H. Swortwood Dr. Mazin YousifApplication Working Group (AWG):Dwight Barron William Futral Greg Pellegrino
InfiniBandTM Architecture Release 1.2.1 Introduction November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 63 Proprietary and Confidential
The InfiniBand Architecture Specification describes a first order intercon-nect technology for interconnecting processor nodes and I/O nodes to form a system area network. The architecture is independent of the host operating system (OS) and processor platform.
Figure 1 IBA System Area Network
1.2.1 THE PROBLEM
Existing interconnect technologies have failed to keep pace with computer evolution and the increased burden imposed on data servers, application processing, and enterprise computing created by the popular success of the internet. High-end computing concepts such as clustering, fail-safe, and 24x7 availability demand greater capacity to move data between pro-cessing nodes as well as between a processor node and I/O devices. These trends require higher bandwidth and lower latencies, they are
Router
Drives
TCA
Controller
StorageSubsystem
I/OChassis
SCSI
Ethernet Graphics
TC
AIO
Mod
ule
TC
AIO
Mod
ule
Fibre Channelhub & FCdevices
TC
AIO
Mod
ule
TC
AIO
Mod
ule
Switch
TC
AIO
Mod
ule
I/OChassis
TC
AIO
Mod
ule
TC
AIO
Mod
ule
TC
AIO
Mod
ule
TC
AIO
Mod
ule
TC
AIO
Mod
ule
Switch
Processor NodeCPU CPU
Mem
CPU
HCA
Switch
Processor NodeCPU CPU
MemHCA
CPU
HCA
Other IB SubnetsWANsLANsProcessor Nodes
ProcessorSCSI
SCSI
SCSI
SCSI
SCSI
Mem
HCA = InfiniBand Channel Adapter in processor nodeTCA = InfiniBand Channel Adapter in I/O node
Processor NodeCPU CPU
MemHCA
CPU
HCA
SwitchSwitchRAID Subsystem
TCASwitch
Consoles
Switch
Fabric
Video
InfiniBandTM Architecture Release 1.2.1 Introduction November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 65 Proprietary and Confidential
pushing more functionality down to the I/O device, and they are de-manding greater protection, higher isolation, deterministic behavior, and a higher quality of service than currently available.
1.2.2 FEATURES
InfiniBand Architecture (IBA) is designed around a point-to-point, switched I/O fabric, whereby end node devices (which can range from very inexpensive I/O devices like single chip SCSI or ethernet adapters to very complex host computers) are interconnected by cascaded switch de-vices. The physical properties of the IBA interconnect support two pre-dominant environments, with bandwidth, distance and cost optimizations appropriate for these environments:
• Module-to-module, as typified by computer systems that support I/O module add-in slots
• Chassis-to-chassis, as typified by interconnecting computers, ex-ternal storage systems, and external LAN/WAN access devices (such as switches, hubs, and routers) in a data-center environ-ment.
The IBA switched fabric provides a reliable transport mechanism where messages are enqueued for delivery between end nodes. In general, message content and meaning is not specified by InfiniBand Architecture, but rather is left to the designers of end node devices and the processes that are hosted on end node devices. IBA defines hardware transport pro-tocols sufficient to support both reliable messaging (send/receive) and memory manipulation semantics (e.g. remote DMA) without software in-tervention in the data movement path. IBA defines protection and error detection mechanisms that permit IBA transactions to originate and termi-nate from either privileged kernel mode (to support legacy I/O and com-munication needs) or user space (to support emerging interprocess communication demands).
The IBA Specification also addresses the need for a rich manageability in-frastructure to support interoperability between multiple generations of IBA components from many vendors over time. This infrastructure pro-vides ease of use and consistent behavior for high volume, cost sensitive deployment environments. IBA also specifies interfaces for industry stan-dard management that interoperate with enterprise class management tools for configuration, asset management, error reporting, performance metric collection, and topology management necessary for data center deployment of IBA.
1.2.3 BENEFITS
For all of the revolutionary aspects of IBA, the architecture has been care-fully designed to minimize disruption of prevailing market paradigms and business practices. By simultaneously supporting board and chassis in-
InfiniBandTM Architecture Release 1.2.1 Introduction November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 66 Proprietary and Confidential
terconnections, it is expected that vendors are able to adopt InfiniBand Ar-chitecture technology for use in future generations of existing products, within current business practices, to best support their customers needs.
IBA can support bandwidths that are anticipated to remain an order of magnitude greater than prevailing I/O media (SCSI, Fibre Channel, Ethernet). This ensures its role as the common interconnect for attaching I/O media using these technologies. Reinforcing this point is IBA’s native use of IPv6 headers, which supports extremely efficient junctions between IBA fabrics and traditional internet and intranet infrastructures.
IBA supports implementations as simple as a single computer system, and can be expanded to include: replication of components for increased system reliability, cascaded switched fabric components, additional I/O units for scalable I/O capacity and performance, additional host node computing elements for scalable computing, or any combinations thereof. InfiniBand Architecture is a revolutionary architecture that enables com-puter systems to keep up with the ever increasing customer requirement for increased scalability, increased bandwidth, decreased CPU utilization, high availability, high isolation, and support for Internet technology.
Being designed as a first order network, IBA focuses on moving data in and out of a node’s memory and is optimized for separate control and memory interfaces. This permits hardware to be closely coupled or even integrated with the node’s memory complex, removing any performance barriers. IBA is flexible enough to be implemented as a second order net-work permitting legacy and migration. Even when implemented as a second order network, IBA’s memory optimization operation permits max-imum available bandwidth utilization and increases CPU efficiency.
1.3 SCOPE
IBA supports a range of applications from being the backplane intercon-nect of a single host, to a complex system area network consisting of mul-tiple independent and clustered hosts and I/O components.
For the single host environments, as depicted in Figure 2, each IBA fabric serves as a private I/O interconnect for its host and provides connectivity
InfiniBandTM Architecture Release 1.2.1 Introduction November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 67 Proprietary and Confidential
between the host’s CPU/memory complex and a number of I/O modules. For this environment, all devices are dedicated to the host.
Figure 2 Single Host EnvironmentOn the other end of the scale is multiple host connectivity as depicted in Figure 1. Here a single fabric or even multiple fabrics interconnect nu-merous hosts and various I/O units. Some hosts might share I/O devices and others do not. Interprocess communication between hosts becomes a very significant objective. Trivial fabric management is no longer suffi-cient as network administrators desire additional features to maintain sep-aration and assure deterministic behavior.
The architecture not only specifies the mechanisms for I/O and interpro-cess communication, but it also specifies an extensive set of management mechanisms that are flexible enough to permit single host environments with out undue burden and costly fabric managers and at the same time support very complex system area networks (SAN) and feature rich fabric management.
1.4 DOCUMENT ORGANIZATION
1.4.1 SERIES OF VOLUMES
There are two volumes that comprise the InfiniBand normative specifica-tions suite:
Volume 1 - specifies the core InfiniBandTM Architecture. It provides nor-mative information required for IBA operation for switches, routers, host channel adapters for processor nodes, target channel adapters for I/O de-vices, and management.
Processor NodeCPU CPUCPU
MemHCA
I/O ChassisSwitch
SCSIEthernet
Fibre Channelhub & FCdevices
GraphicsVideo
TC
AI/
O M
odul
e
TC
AI/
O M
odul
e
TC
AI/
O M
odul
e
TC
AI/
O M
odul
e
TC
A
I/O
Mod
ule
InfiniBandTM Architecture Release 1.2.1 Introduction November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 68 Proprietary and Confidential
Volume 2 - specifies electrical & mechanical configurations. It specifies requirements for a number of different physical media and signaling rates, defines mechanical form factors, and specifies physical and chassis man-agement requirements.
1.4.2 VOLUME 1 ORGANIZATION
1.5 DOCUMENT CONVENTIONS
1.5.1 BYTE ORDERING
This specification uses Big Endian byte ordering. For fields greater than one byte in size this means that the most significant byte of each field is transmitted first as illustrated in Figure 3.
Unless specifically stated otherwise, the text of this document lists fields in the order of transmission. In most cases, multiple byte fields are aligned to start or end on a 32-bit boundary. For clarity, certain figures show fields ordered in 32 bit words. These words are in big endian format and imple-mentations targeted for little endian processing need to pay particular at-tention to byte ordering to assure correct operation since little endian processing tends to place the least significant bytes in lower byte offsets.
Byte offset previous field
+0 Most Significant Byte
+1 Second Most Significant Byte
.
.
.
ooo
+n Least Significant Byte
Figure 3 Byte Order for Multiple Byte Fields
InfiniBandTM Architecture Release 1.2.1 Introduction November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 69 Proprietary and Confidential
Figure 4 illustrates how numeric and bit significant fields should be inter-preted.
Bit fields with other than byte granularity follow the same rules - that is, the most significant bits of the field occupies the higher order bits of the lowest byte offset with least significant bits being in the lowest byte offset as illus-trated in Figure 5.
1.5.2 NUMERIC VALUES
Unless otherwise stated numerical values without qualifiers are decimal. This document uses the following qualifiers:
• 0x prefixed to a hexadecimal value (e.g., 0x15F7)• b’ prefixed to a binary value (e.g., b’0110)
An obvious exception are binary numbers used in figures and tables
In table headings a colon is used to specify a range of bits (e.g. Bits 7:0) and table values in that column are binary numbers.
A dash between two numbers represents a range (e.g. 0-3 = zero to three)
Global IDs are 128-bit values specified in the format : value:value:value:value:value:value:value:value Where each value represents a 4-digit hexadecimal number (e.g., FF02:0:0:0:0:0:0:1)
1.6 DISCLAIMER
Like any document, this specification is subject to errata for correctness, clarity, and enhancements. The InfiniBandSM Trade Association hosts a web site at http://www.InfiniBandTA.org. Please visit this site to check for errata and updates to this specification.
Active Describes an entity initiating a communication establishment request (e.g., TCP CONNECT).
Address Handle An object that contains the information necessary to transmit messages to a remote port over Unreliable Datagram service.
Address Vector A collection of address and Path information specifying a remote port and the parameters to be used when communicating with it.
AETH Ack Extended Transport Header
AM Attribute Modifier.
Asynchronous error A permanent error that cannot be reported through immediate or comple-tion error handling mechanisms at the local end. Asynchronous errors may be unaffiliated or may be affiliated with a specific Completion Queue, Endport, or Queue Pair.
Attribute The collection of management data carried in a Management Datagram.
Automatic Path Migration The process in which a Channel Adapter, on a per-Queue Pair basis, sig-nals another CA to cause Path Migration to a preset alternate Path. Auto-matic Path Migration uses a bit in a request or response packet (MigReq) to signal the other channel adapter to migrate to the predefined alternate path.
B_Key See Baseboard Management Key.
Base LID The numerically lowest Local Identifier that refers to a Port. The Path Bits of a Base LID are always zero.
Base Switch Port 0 A Switch Port 0 which is not an Endport.
Baseboard Managed Unit Any Unit which provides InfiniBand(TM) specification defined information about itself by a Baseboard method MAD operation through the Infini-BandTM link.
Baseboard Management Key
A construct that is contained in IBA management datagrams to authenti-cate that the sender is allowed to perform the requested operation.
Binding The act of associating a virtual address range in a specified Memory Region with a Memory Window.
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 72 Proprietary and Confidential
Channel The association of two queue pairs for communication.
Channel Adapter Device that terminates a link and executes transport-level functions. One of Host Channel Adapter or Target Channel Adapter.
Channel Interface The presentation of the channel to the Verbs Consumer as implemented through the combination of the Host Channel Adapter, associated firm-ware, and device driver software.
Channel, Reliable Datagram See Reliable Datagram Channel.
CI See Channel Interface.
Client The active entity in an active/passive communication establishment exchange.
CM See Communication Manager.
CME Chassis Management Entity.
Communication Manager The software, hardware, or combination of the two that supports the communication management mechanisms and protocols.
Completion Error Permanent interface or processing error reported through completion status.
Completion Event Handler A handler that is invoked when the Consumer requests completion notifi-cation and an entry is added to the completion queue associated with the handler’s identifier.
Completion Queue A queue containing one or more Completion Queue Entries. Completion Queues are internal to the Channel Interface, and are not visible to verb consumers.
Completion Queue Entry The Channel Interface-internal representation of a Work Completion.
Component Mask A field in a Management Datagram used to indicate which components of the MAD are to be considered in carrying out the operation. See Wild-carding.
Connection An association between a pair of entities (e.g., processes) over one or more Channels.
Consumer See Verbs Consumer.
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 73 Proprietary and Confidential
Data Payload The data, not including any control or header information, carried in one packet.
Data Segment A tuple in a Work Request that specifies a virtually contiguous buffer for Host Channel Adapter access. Each Data Segment consists of a Virtual Address, an associated Local Key or Remote Key, and a length.
DETH Datagram Extended Transport Header.
DGID Destination Global Identifier.
DLID Destination Local Identifier.
EEC See End to End Context.
EECN See End to End Context Number.
EE Context See End to End Context.
Endport A Port which can be a destination of LID-routed communication within the same Subnet as the sender. All Channel Adapter ports on the subnet are endports of that subnet, as is Port 0 of each Switch in the subnet. Switch ports other than Port 0 may not be endports. When port is used without qualification, it may be assumed to mean endport whenever the context indicates that it is a destination of communication.
End to End Context The endpoint of a Reliable Datagram channel.
End to End Context Number Identifies a specific End to End Context within a Channel Adapter.
End to End Flow Control A mechanism to prevent a sender from transmitting messages during pe-riods when receive buffers are not posted at the recipient.
Enhanced Switch Port 0 A Switch Port 0 which provides the functionality of a Target Channel Adapter.
External Switch Port A physical Port on a Switch. See also Switch Port 0.
Fabric The collection of Links, Switches, and Routers that connects a set of Channel Adapters.
Fast Register Physical MR A memory registration performed on an existing local L_Key, and any as-sociated R_Key, through a Post Send Work Request.
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 74 Proprietary and Confidential
General Service Interface An interface providing management services (e.g., connection, perfor-mance, diagnostics) other than subnet management.
GID See Global Identifier.
GID Index 0 The unicast GID referenced through index 0 of an endport’s GID Table, based on the endport’s invariant manufacturer-assigned EUI-64.
GID Table A table containing one or more Global Identifiers by which an endport may be referenced.
Global Identifier A 128-bit identifier used to identify an Endport or a multicast group. GIDs are valid 128-bit IPv6 addresses (per RFC 2373) with additional proper-ties / restrictions defined within IBA to facilitate efficient discovery, com-munication, and routing.
Global Route Header Routing header present in InfiniBandTM Architecture packets targeted to destinations outside the sender’s local subnet.
Globally Unique Identifier A number that uniquely identifies a device or component.
GMP General Management Packet.
GRH See Global Route Header.
GSI See General Service Interface.
GUID See Globally Unique Identifier.
HCA See Host Channel Adapter.
Host One or more Host Channel Adapters governed by a single memory/CPU complex.
Host Channel Adapter A Channel Adapter that supports the Verbs interface.
IBA InfiniBandTM Architecture.
IB-ML InfiniBandTM Management Link.
ICRC See Invariant CRC.
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 75 Proprietary and Confidential
Immediate Data Data contained in a Work Queue Element that is sent along with the pay-load to the remote Channel Adapter and placed in a Receive Work Com-pletion.
Immediate Error A permanent Interface Error reported through the verb status.
Initiator The source of requests.
Interface Error An error due to an invalid field in a Work Request.
Invalid Key See Key.
Invalidate Operation An operation that disables CI’s access to host memory through an L_Key or R_Key, but retains L_Key or R_Key translation and protection resources for use on future memory registrations.
Invariant CRC A CRC covering the fields in a packet that do not change from the source to the destination.
I/O Input/Output.
I/O Controller One of the two architectural divisions of an I/O Unit. An I/O controller (IOC) provides I/O services, while a Target Channel Adapter provides transport services.
I/O Unit An I/O unit (IOU) provides I/O service(s). An I/O unit consists of one or more I/O Controllers attached to the fabric through a single Target Chan-nel Adapter.
I/O Virtual Address An address having no direct meaning to the Host processor, intended for use only in describing a Local or Remote memory buffer to the Host Channel Adapter.
IOC See I/O Controller.
IOU See I/O Unit.
IPv6 Internet Protocol, version 6
IPv6 Address A 128-bit address constructed in accordance with IETF RFC 2460 for IPv6.
Key A construct used to limit access to one or more resources, similar to a password. The following keys are defined by the InfiniBandTM Architec-ture:
Baseboard Management Key
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 76 Proprietary and Confidential
Key Space Refers to whether the index portion of the L_Key and R_Key for a partic-ular Memory Region have the same value or not. If they do, then they are in the same Key Space. Otherwise they are in separate Key Spaces.
L_Key See Local Key.
LID See Local Identifier.
LID Mask Control A per-port value assigned by the Subnet Manager. The value of the LMC specifies the number of Path Bits in the Local Identifier.
Link A full duplex transmission path between any two network fabric ele-ments, such as Channel Adapters or Switches.
LMC See LID Mask Control.
Local Identifier An address assigned to a port by the Subnet Manager, unique within the subnet, used for directing packets within the subnet. The Source and Destination LIDs are present in the Local Route Header. A Local Identi-fier is formed by the sum of the Base LID and the value of the Path Bits.
Local Invalidate An Invalidate Operation performed through the Send Queue: on a local L_Key, R_Key, or Memory Region Handle that is associated with an MR; or an R_Key or Memory Window Handle that is associated with a Type 2 Memory Window.
Local Key An opaque object, created by a verb, referring to a Memory Registration, used with a Virtual Address to describe authorization for the HCA hard-ware to access local memory. It may also be used by the HCA hardware to identify the appropriate page tables for use in translating virtual to physical addresses.
Local Route Header Routing header present in all InfiniBandTM Architecture packets, used for routing through switches within a subnet.
Local Subnet The collection of links and Switches that connect the Channel Adapters of a particular subnet.
LRH See Local Route Header.
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 77 Proprietary and Confidential
Managed Unit A Unit which provides Vital Product Data about itself to an external entity, and is managed by that entity.
Management Datagram Refers to the contents of an Unreliable Datagram packet used for com-munication among HCAs, switches, routers, and TCAs to manage the fabric. InfiniBandTM Architecture describes the format of a number of these management commands.
Management Key A construct that is contained in IBA management datagrams to authenti-cate the sender to the receiver.
Maximum Transfer Unit The maximum Packet Payload size, which may be 256, 512, 1024, 2048, or 4096 bytes. See also MTU Capacity, Neighbor MTU, and Path Maxi-mum Transfer Unit.
MB/s Mega-bytes per second (106 bytes per second)
Memory Protection At-tributes
The access rights granted to Memory Regions.
Memory Region A virtually contiguous area of arbitrary size within a Consumer’s address space that has been registered, enabling HCA local access and optional remote access. See Memory Registration
Memory Region Handle An opaque object returned to the consumer when the consumer registers a Memory Region. The Memory Region Handle is used to specify the registered region to the memory management verbs.
Memory Registration The act of registering a host Memory Region for use by a consumer. The memory registration operation returns a Memory Region Handle. The process provides this with any reference to a virtual address within the memory region.
Memory Window An allocated resource that enables remote access after being bound to a specified area within an existing Memory Region. Each Memory Window has an associated Window Handle, set of access privileges, and current R_Key.
Message A transfer of information between two or more Channel Adapters that consists of one or more packets.
Message-Level Flow Con-trol
See End to End Flow Control.
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 78 Proprietary and Confidential
A value returned as part of an acknowledgement by the responder to the requestor, indicating the last message completed. Contrast Packet Se-quence Number.
Modifiers In a verb definition, the list of input and output objects that specify how, and on what, the verb is to be executed.
MR Memory Region
MSN See Message Sequence Number.
MTU See Maximum Transfer Unit.
MTUCap See MTU Capacity.
MTU Capacity The largest Maximum Transfer Unit that a port can support.
Multicast A facility by which a packet sent to a single address may be delivered to multiple ports.
Multicast Identifier A Local Identifier or Global Identifier for a Multicast Group.
Multicast Group A collection of Endports that receive Multicast packets sent to a single address.
MW Memory Window
Neighbor MTU The configured Maximum Transfer Unit for a Port, the value that specifies the maximum packet payload that may be sent to, or received from, the port at the other end of the Link.
NQ Notification Queue.
Out-of-band Management Management messages which traverse a transport other than the Infini-BandTM fabric.
Outstanding 1) The state of a Work Request after it has been posted on a Work Queue, but before the retrieval of the Work Completion by the con-sumer.
2) The state of a packet that has been sent onto the fabric but has not been acknowledged.
P_Key See Partition Key.
Packet The indivisible unit of IBA data transfer and routing, consisting of one or more headers, a Packet Payload, and one or two CRCs.
Packet Payload The portion of a Packet between (not including) any Transport header(s)
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 79 Proprietary and Confidential
and the CRCs at the end of each packet. The packet payload contains up to 4096 bytes.
Packet Sequence Number A value carried in the Base Transport Header that allows the detection and re-sending of lost packets.
Partition A collection of Channel Adapter ports that are allowed to communicate with one another. Ports may be members of multiple partitions simulta-neously. Ports in different partitions are unaware of each other’s pres-ence insofar as possible.
Partition Key A value carried in packets and stored in Channel Adapters that is used to determine membership in a partition.
Default Partition Key: A partition key special value providing Full mem-bership in the default partition. See Partition Membership Type.
Invalid Partition Key: A special value that indicates that the Partition Key Table entry does not contain a valid key.
Partition Key Table A table of partition keys present in each Port.
Partition Key Table Index (P_Key_ix)
An index into the partition key table.
Partition Manager The entity that manages partition keys and membership.
Partition Membership Type The high-order bit of the partition key is used to record the type of mem-bership in an Port’s partition table: 0 for Limited, 1 for Full. Limited mem-bers cannot accept information from other Limited members, but communication is allowed between every other combination of member-ship types.
Passive Describes an entity waiting to receive a communication establishment request (e.g., TCP LISTEN).
Path The collection of links, switches, and routers a message traverses from a source Channel Adapter to a destination channel adapter. Within a sub-net, a path is defined by the tuple <SLID, DLID, SL>.
Path Bits The portion of a Local Identifier that may be changed to vary the Path through the subnet to a particular Port. If the Path Bits are zero, the Local Identifier is equal to the Base LID. The number of Path Bits applicable to a particular port is specified by the Subnet Manager through the LID Mask Control value.
Path Maximum Transfer Unit
The Maximum Transfer Unit supported along a Path from source to des-tination. PMTU is described in terms of the payload size, and may be
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 80 Proprietary and Confidential
Path Migration The modification of the Path used by a connection.
PD See Protection Domain.
Peer 1) One of the agents in an active/active connection establishment ex-change.
2) A generic term for the entity at the other end of a connection.
Pinning memory A function supplied by the OS which forces the memory region to be res-ident and keeps the virtual-to-physical translations constant from the HCA point of view.
PM See Partition Manager.
PMR Physical Memory Region
PMTU See Path Maximum Transfer Unit.
Port Location on a Channel Adapter or Switch to which a link connects. There may be multiple ports on a single Channel Adapter, each with different context information that must be maintained. Switches/switch elements contain more than one port by definition.
Post To place a Work Request on a Work Queue.
Private Data A field present in Communication Management messages that is opaque at all IBA layers. Consumers may use this field to “piggy-back” additional information over the CM message exchange.
Processing Error A processing error is an error that occurs when the Host Channel Adapter is performing the unit of work described by the Work Queue Ele-ment and is unable to complete the request successfully due to an error that is returned by the transport protocol.
Protection Domain A mechanism for associating Queue Pairs, Address Handles, Memory Windows, and Memory Regions.
PSN See Packet Sequence Number.
Q_Key See Queue Key.
QoS See Quality of Service.
QP See Queue Pair.
Quality of Service Metrics that predict the behavior, reliability, speed, and latency of a given
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 81 Proprietary and Confidential
Queue Key A construct that is used to validate a remote sender’s right to access a local Receive Queue for the Unreliable Datagram and Reliable Datagram service types. If the Q_Key present in an incoming packet does not match the value stored in the receiving QP, the packet shall be dropped.
Queue Pair Consists of a Send Work Queue and a Receive Work Queue. Send and receive queues are always created as a pair and remain that way throughout their lifetime. A Queue Pair is identified by its Queue Pair Number.
Queue Pair Context The information that pertains to a particular Queue Pair, such as the cur-rent Work Queue Elements, Packet Sequence Numbers, transmission parameters, etc.
Queue Pair Handle An opaque object that refers to a specific Queue Pair. A Queue Pair Han-dle is returned by the operation that creates the QP and is supplied as an identifying parameter for other QP operations.
Queue Pair Number Identifies a specific Queue Pair within a Channel Adapter.
R_Key See Remote Key.
Raw Datagram A packet that contains an IBA Local Route Header, may contain an IBA Global Route Header, but does not contain an IBA Transport header, and is not handled by IBA transport services.
RC See Reliable Connection.
RD See Reliable Datagram.
RDC See Reliable Datagram Channel.
RDD See Reliable Datagram Domain.
RDETH Reliable Datagram Extended Transport Header.
RDMA See Remote Direct Memory Access.
Receive Queue One of the two queues associated with a Queue Pair. The receive queue contains Work Queue Elements that describe where to place incoming data.
Region Handle See Memory Region Handle.
Registered Memory A region of memory that has been through Memory Registration.
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 82 Proprietary and Confidential
Reliable Connection A Transport Service Type in which a Queue Pair is associated with only one other QP, such that messages transmitted by the send queue of one QP are reliably delivered to receive queue of the other QP. As such, each QP is said to be “connected” to the opposite QP.
Reliable Datagram A Transport Service Type in which a Queue Pair may communicate with multiple other QPs over a Reliable Datagram Channel. A message trans-mitted by an RD QP’s send queue will be reliably delivered to the receive queue of the QP specified in the associated Work Request. Despite the name, Reliable Datagram messages are not limited to a single packet.
Reliable Datagram Channel The association of two Reliable Datagram End to End Contexts. A Reli-able Datagram channel may multiplex Reliable Datagrams from many RD Queue Pairs.
Reliable Datagram Domain An association that defines which Reliable Datagram Queue Pairs may use an End to End Context.
Reliable Multi-PacketProtocol
A transaction protocol based on Management Datagrams supporting the reliable transfer of amounts of data larger than that possible in a single Management Datagram.
Remote Direct Memory Ac-cess
Method of accessing memory on a remote system without interrupting the processing of the CPU(s) on that system.
Remote Invalidate An Invalidate Operation performed on a local R_Key through an incoming Send with Invalidate Message.
Remote Key An opaque object, created by a verb, referring to a Memory Region or Memory Window, used with a Virtual Address to describe authorization for the remote device to access local memory. It may also be used by the HCA hardware to identify the appropriate page tables for use in translat-ing virtual to physical addresses.
Reserved L_Key An L_Key that can be used by a privileged Consumer to provide the HCA direct access to host physical addresses.
Retired The state of a Work Queue Element after the Host Channel Adapter completes the operation specified by the WQE, but before the Work Completion has been presented to the consumer.
RMPP See Reliable Multi-Packet Protocol.
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 83 Proprietary and Confidential
RNR Nak Receiver Not Ready. A response signifying that the receiver is not cur-rently able to accept the request, but may be able to do so in the future.
Router A device that transports packets between IBA subnets.
SA See Subnet Administration.
SAR Segmentation and Re-assembly.
Send Queue One of the two queues of a Queue Pair. The Send queue contains WQEs that describe the data to be transmitted.
Server 1) The passive entity in a connection establishment exchange.
2) An entity (e.g., a process) that provides services in response to re-quests from clients.
Service ID A value that allows a Communication Manager to associate an incoming connection request with the entity providing the service. The Service ID is similar to the TCP Port Number.
Service Level Value in the Local Route Header identifying the appropriate Virtual Lane for a packet, enabling the implementation of differentiated services. While the appropriate VL for a specific Service Level may differ over a packet’s Path, the Service Level remains constant.
Service Type See Transport Service Type.
Signaled Completion A modifier used for Work Requests submitted to the Send Queue speci-fying that a Work Completion shall be generated when the work requested completes, whether successfully or in error.
SGID Source Global Identifier.
Shared Receive Queue The Shared Receive Queue contains WQEs that can be used to receive incoming data on any QP that is associated with the Shared Receive Queue.
SLID Source Local Identifier
SL See Service Level.
SM See Subnet Manager.
SMA See Subnet Management Agent.
SMP See Subnet Management Packet.
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 84 Proprietary and Confidential
Solicited Event A facility by which a message sender may cause an event to be generated at the recipient when the message is received.
SRQ Shared Receive Queue
Subnet A set of InfiniBandTM Architecture Ports, and associated links, that have a common Subnet ID and are managed by a common Subnet Manager. Subnets may be connected to each other through routers.
Subnet Administration The architectural construct that implements the interface for querying and manipulating subnet management data.
Subnet Manager One of several entities involved in the configuration and control of the subnet.
Master Subnet Manager: The subnet manager that is authoritative, that has the reference configuration information for the subnet.
Standby Subnet Manager: A subnet manager that is currently quies-cent, and not in the role of a master SM, by agency of the master SM. Standby SMs are dormant managers.
Subnet Management Agent An entity present in all IBA Channel Adapters and Switches that pro-cesses Subnet Management Packets from Subnet Manager(s).
Subnet Management Data Vital Product Data required by the Subnet Manager.
Subnet Management Packet
The subclass of Management Datagrams used to manage the subnet. SMPs travel exclusively over Virtual Lane 15 and are addressed exclu-sively to Queue Pair Number 0.
Switch A device that routes packets from one link to another of the same Sub-net, using the Destination Local Identifier field in the Local Route Header.
Switch Management Port A virtual port by which a Switch may be managed. See Switch Port 0.
Switch Port 0 An addressable virtual port by which a Switch may be managed. May be one of Base Switch Port 0 or Endport.
TCA See Target Channel Adapter.
Target Channel Adapter A Channel Adapter typically used to support I/O devices. TCAs are not required to support the Verbs interface. See also I/O Unit.
Transport Service Type Describes the reliability, sequencing, message size, and operation types that will be used between the communicating Channel Adapters.
Transport service types that use the IBA transport:
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 85 Proprietary and Confidential
Raw Datagram service does not use the IBA transport.
Type 1 Memory Window A Memory Window that is associated with a QP through the PD and cannot be Invalidated.
Type 2 Memory Window A Memory Window that is either a Type 2A Memory Window or a Type 2B Memory Window.
Type 2A Memory Window A Memory Window that when bound is associated with a QP through the QP Number and can be Invalidated.
Type 2B Memory Window A Memory Window that when bound is associated with a QP through the QP Number and PD, and can be Invalidated.
UC See Unreliable Connection.
UD See Unreliable Datagram.
Unicast An identifier for a single port. A packet sent to a unicast address is deliv-ered to the port identified by that address.
Unit One or more sets of processes and/or functions attached to the fabric by one or more channel adapters. See Host and I/O Unit.
Unreliable Connection A Transport Service Type in which a Queue Pair is associated with only one other QP, such that messages transmitted by the send queue of one QP are, if delivered, delivered to the receive queue of the other QP. As such, each QP is said to be “connected” to the opposite QP. Messages with errors are not retried by the transport, and error handling must be provided by a higher level protocol.
Unreliable Datagram A Transport Service Type in which a Queue Pair may transmit and receive single-packet messages to/from any other QP. Ordering and delivery are not guaranteed, and delivered packets may be dropped by the receiver.
Unsignaled Completion A modifier used for Work Requests submitted to the Send Queue signify-ing that a Work Completion is to be generated only if the requested action completes in error.
Variant CRC A CRC covering all the fields of a packet, including those that may be changed by Switches.
InfiniBandTM Architecture Release 1.2.1 Glossary November 2007VOLUME 1 - GENERAL SPECIFICATIONS FINAL RELEASE
InfiniBandSM Trade Association Page 86 Proprietary and Confidential
Verbs An abstract description of the functionality of a Host Channel Adapter. An operating system may expose some or all of the verb functionality through its programming interface.
Verbs Consumer The direct user of the Verbs.
Virtual Lane A method of providing independent data streams on the same physical link.
Vital Product Data Device-specific data to support management functions.
VL See Virtual Lane.
VPD See Vital Product Data.
WC See Work Completion.
Wildcarding Setting a Component Mask bit in a Management Datagram to 0, causing that component's value to be ignored in carrying out the operation. Not all management classes define a Component Mask and Component Mask is only applicable to certain Method-Attribute combinations. An example of a management class with a Component Mask is SubnAdm.
Window Handle An opaque object that identifies a Memory Window.
Work Completion The consumer-visible representation of a Completion Queue Entry. A Work Completion may be obtained when a consumer polls a Completion Queue.
Work Queue One of Send Queue or Receive Queue.
Work Queue Element The Host Channel Adapter’s internal representation of a Work Request. The consumer does not have direct access to Work Queue Elements.
Work Queue Pair See Queue Pair.
Work Request The means by which a consumer requests the creation of a Work Queue Element.
WQ See Work Queue.
WQE Work Queue Element, commonly pronounced “wookie”.
![Sensors and Actuators A: Physicalstatic.tongtianta.site/paper_pdf/8c3a8c90-73ea-11e9-93da... · 2019. 5. 11. · method, andfrequencyleveragedmethod[2].Theinternallylever-aged method](https://static.documents.pub/doc/80x56/5fbb5c3c508f9702cb1b6e65/sensors-and-actuators-a-2019-5-11-method-andfrequencyleveragedmethod2theinternallylever-aged.jpg)
