Chapter 2, Server Motherboard & Chipset Design

8/12/2019 Chapter 2, Server Motherboard & Chipset Design

1/47

Confidential

Server Fundamentals

The performance of a server depends on the speed of data flow

between its processor and the peripheral device that are connected.

These peripheral device use a set of pathways to communicate with

the processor. This set of pathways are called as BUS. In server wemainly use PCI bus technology.


2/47

Confidential

Types of Buses

System Bus

Data Bus


3/47

Confidential

System Bus

Use to connect CPU to its MEMORY.

Also known as Local Bus

Faster than other Buses.


4/47

Confidential

Data Bus

Connects the other computer peripherals with the memory.

PCI is an example of data bus.


5/47

Confidential

Usage of PCI

In the early 1990, INTEL introduce PCI with the aim to improve I/O

operation of computers.

PCI is also used with other processors, eg. AMD


6/47

Confidential

PCI Design

Initially PCI was a 32 bit bus that operated a speed of 33 MHz

(DTR=132Mbps)

Now 64 bit PCI with a speed of 33 MHz whose DTR=264 Mbps

64 bit PCI with a speed of 66 MHz, with DTR=512 mbps


7/47Confidential

Bus Speed Setup

Synchronized--- bus speed is not depends with the memory bus

speed.

Mainly used in today's bus architecture

Asynchronized--- bus speed depends with the memory bus speed.

Normally controlled by jumpers or CMOS setup.


8/47Confidential

PCI Implementation

Data buses connect to system buses by single chip bridge device.

PCI can be implemented on servers through peer PCI bus or

hierarchical PCI bus technology.


9/47Confidential

Peer PCI Bus

In Peer PCI, several PCI buses are connected directly to a host bus. Thesehost buses provide connectivity between server and storage devices.

This enables the CPU to access each bus directly.

Peer PCI improves the performance of a server because the processor orthe bus bandwidth is evenly shared between the buses.

Also helps to separate high bandwidth and low bandwidth traffic on different

PCI buses.


10/47Confidential

Hierarchical PCI

When multiple expansion slots and connectors are needed

hierarchical PCI can be implemented.

A bridge controller is used to connect independent PCI buses so

that they can communicate with each other.


11/47Confidential

Hot plug

The hot plug facility of PCI enables to install and remove PCI device

without turning off the server.

Today most servers are equipped with the hot plug facility.

Microsoft Windows Server 2003 have native support for hot plug

operations.


12/47Confidential

Hot Plug Devices

NIC Card

SCSI Card

RAID Card


13/47Confidential

Hot Swap

Cards inserted or remove without failure when servers are running.

To support hot swap PCI hot swap adapters are required. Thesedevice provide circuit breaker functions that protect electric suppliesagainst fault conditions resulting from over current. A PCI hot swapadapter provides protection against 12V, 5V, 3.3V and -12V


14/47

Confidential

PCI- Hot Swap Levels

Basic hot swap

Full hot swap

High availability


15/47

Confidential

Basic Hot Swap VS Full Hot Swap

Requires human intervention for interaction with the host operating

system when a card is being inserted or removed.

It automates the procedure of replacing the card.


16/47

Confidential

High Availability

The high availability level focuses on having cards that can belogically and electrically added to or removed from a server.

This ensure hot spare or fail over capability for the server. The highavailability hot swap level also enables an automatic reconfigurationof software and hardware components in a running server.


17/47

Confidential

I2O

PCI is also intelligent input/output (I2O) compliant.

I2O is a standard designed by a consortium of computer companies called asI2O Special Interest Group (SIG). This standard simplifies and speed up the

I/O operations on servers.

According to I2O the drivers used for the operating system of a server must be sameas those used for the SCSI and network cards on the server.

It is recommended to use INTEL 960 chip on the motherboard of a server, to increasethe speed of its I/O subsystem.


18/47

Confidential

PCI- Bus Mastering

Bus mastering is the capability of devices other than the system chipset, on

the PCI bus to take control of the bus and perform transfer directly.

Bus mastering allows the transfer of data between devices on a PCI bus

and the main system memory even when the other devices are using CPU.

However these devices should have their own built-in processors to operate

independently of the CPU.


19/47

Confidential

MULTITASTKING

Bus Mastering facilitates multitasking by allowing data transfer

between the devices on a PCI bus and other devices in the system

without involving the CPU.

The devices that take control of the bus keep track of the

transactions. There fore the CPU utilization is low. This enables the

CPU to handle other task in the meantime.


20/47

Confidential

Bus Arbitration

Bus Mastering is achieved through the bus arbitration process. This process

determines priority levels for the request to a PCI bus so that no devices

lock each other out.

For example, when more than one processor is connected to the same

system bus, there are simultaneous requests for the bus. This request are

resolved using bus arbitration that allows devices to use the bus based on

their priority level.


21/47

Confidential

PCI-e

The Peripheral Component Interconnect Express architecture is a

serial interconnect technology designed to meet the bandwidth

requirement of advanced processors, memory subsystem and

applications.

It impose some advance features than a normal PCI.


22/47

Confidential

PCI-e

PCI-e is a point-to-point serial switch bus technology. It aims at reducing

implementation costs and providing higher bandwidth per pin and a scalable

performance.

The advantage of PCI-e over PCI is its point to point bus topology. It

enables each PCI-e device to communicate the PCI controller directly and

autonomously. This communication occurs through a PCI link.


23/47

Confidential


24/47

Confidential

PCI-e link

A PCI-e link comprise of one or more lanes. Depending on the

number of lanes used in a PCI-e link the link is called as x1 link, x2

link and so on. A link which made up for 4 lanes is known as x4 link.

PCI-e supports x1,x2,x4,x8,x12,x16 and x34 lane widths.


25/47

Confidential

PCI-e Bandwidth

Each PCI-e lane can transfer one byte at a time in both the

directions by using full duplex communication. An individual lane

can transfer data at the rate of 2 Gbps in each direction

simultaneously. The next table shows its detail.


26/47

Confidential

PCI-e bandwidth

PCI-e implementation Encoded data transferrate

Unencoded data transferrate

x1 5 gbps 4 gbps

x2 10 gbps 8 gbps

x4 20 gbps 16 gbps

x8 40 gbps 32 gbps

x16 80 gbps 64 gbps

x32 160 gbps 128 gbps


27/47

Confidential

8b/10b Clock-encoding mechanism

A PCI-e mechanism comprises dual simplex channels, where eachchannel is composed of a transmit pair and a receive pair forconcurrent transmission in both directions. Every pair is composedof dual pairs of signal that are driven by Low Voltage Differential(LVD). To attain a high speed data transfer rate, a data clock is built

into each pair by using an 8b/10b data encoding scheme.


28/47

Confidential

Point to Point Bus Topology of PCI-e

As PCI-e supports the point to point bus topology the total

bandwidth of each PCI bus is dedicated to the device at the end of

the link. This in turn enables multiple PCI-e devices to be active at

the same time without affecting each other.


29/47

Confidential

Advanced Features of PCI-e

Advanced power management

Support for real-time data traffic

Hot plug and hot swap

Data integrity and error handling


30/47

Confidential

Advanced Power Management

PCI-e maintains an active interface constantly to synchronize thetransmitter and receiver. The interface is kept active continuallytransmitting idle characters when data is not sent. As additionalpower is consumed by the receiver to decode and discard these idlecharacters, PCI-e uses two low-power link states and the active-

state power management (ASPM) protocol.


31/47

Confidential

Support for Real-Time Data Traffic

PCI-e provides native support for isochronous data transfers and different

Quality of Service (QoS) levels through virtual channels. These channels

ensure timely and reliable delivery of data packets. PCI-e supports

numerous isochronous virtual channels per lane, which may have varying

QoS levels. This feature is developed for applications that require real-time

delivery such as real-time audio and video.


32/47

Confidential

Hot Plug and Hot Swap

The hot plug and hot swap features of PCI-e are extremely

beneficial to servers and portable computers. The facility to hot plug

I/O devices reduces server downtime when peripheral cards need to

be installed or replaced.


33/47

Confidential

Data Integrity and Error Handling

PCI-e provides support for link-level data integrity for all kinds of

transactions and data-link packets. As a result, it provides end-to-end data

integrity required by high availability application running on server systems.

Additionally, fault isolation and recovery solutions can be enhanced by

using the PCI error handling and advanced error reporting and handling

capabilities of a PCI-e.


34/47

Confidential

PCI-X

PCI-X was developed jointly by IBM, HP and Compaq and was

approved by PCI Special Interest Group(PCI-SIG). This group

provides technical support, training and compliance testing. Now

PCI-X has become an open standard that is used by all computermanufactures.

C S


35/47

Confidential

PCI-X Data Speed

The PCI-X design, uses one 64 bit slot that runs at 133 MHz while

the other slots run at 66 MHz. This enables data to be exchanged at

the speed of 1.6 GBps.

So using PCI-X technology existing PCI bus can be increased from133 MBps to 1 GBps.

PCI X b k d tibilit


36/47

Confidential

PCI-X backward compatibility

PCI-X is backward compatible. It means, a PCI-X card can be

installed in a standard 32 PCI slot. However, this reduces the speed

of the bus to 33 MHz

PCI X F lt T l


37/47

Confidential

PCI-X Fault Tolerance

A PCI-X system is more fault tolerant than a Normal PCI system.

When a faulty card is installed on a computer, the PCI-X bus can re-

initialize the faulty card or take it offline before the computer fails.

PCI X i t


38/47

Confidential

PCI-X power requirements

Normal PCI----- 5 VOLT

PCI-X ----- 3.3 VOLT

C i f PCI d PCI X S t


39/47

Confidential

Comparison of PCI and PCI-X Systems

Bus Width Bus Frequency PCI/PCI-X

bandwidth

No. of slots(PCI) No. of slots(P

X)

32 bit 66 MHz 264 Mbps 4 N/A

64 bit 66 MHz 528 MBps 2 4

64 bit 100 MHz 800 MBps N/A 2

64 bit 133 MHz 1064 Mbps N/A 1

Ad t T i


40/47

Confidential

Adapter Teaming

Adapter Teaming is a process of combining two or more physical

adapters into a single logical adapter. This enables the network

administrator to assign a single IP address to the logical adapter.

A software driver is usually used to link the physical adapter to thelogical adapter.

Ad t T i b f b


41/47

Confidential

Adapter Teaming- number of members

An adapter team can have a minimum of two and a maximum of

eight adapters. The main advantage of a adapter teaming is that it

increases server uptime. This is because if a physical adapter in a

team fails, it does not affect other adapters in the team as the IPaddress is assigned to the logical adapter and not to a physical

adapter.

AFT


42/47

Confidential

AFT

Adapter teaming also enables Adapter Fault Tolerance (AFT)against broken or loose or loose cables, hubs or switch ports.

It can also provide fault tolerance to problems caused by aPeripheral Component Interconnect (PCI) slot malfunction orbreakdown in adapter hardware.

Ad t T i l d di t ib ti


43/47

Confidential

Adapter Teaming load distribution

Adapter teaming ensures that the failure of an adapter in a team

does not affect the load distribution through failover capabilities. If

any member in an adapter team fails because of problems in a

network interface card (NIC), cable, switch port, or switch, the loaddistribution is evaluated and then reassigned to the other members

of the team.

Primar Adapter


44/47

Confidential

Primary Adapter

For an effective failover, a primary adapters must be specified in an

adapter team. The primary adapter is specified based on the

priorities of the adapters.

Usually, the primary adapter is the adapter that has highest prority

Failover Restoration


45/47

Confidential

Failover Restoration

In adapter teaming, only a single active adapter transmits and

receives traffic.

The single adapter is usually the primary adapter. However, if the

primary adapter fails, a secondary adapter takes control.

After the connection of the primary adapter is restored, the control is

automatically passed back to the primary adapter.

ALB


46/47

Confidential

ALB

Adapter teaming also enables Adapter Load Balancing (ALB). ALB provides

fault tolerance by ensuring load balancing data traffic.

When load balancing takes place, all adapters in an adapter team share the

transmission load.

To enable load balancing, all the adapters in the team should be connected

to a switch and an IP Address must be assigned to the adapter team.


47/47

Thank you

Date post:	03-Jun-2018
Category:	Documents
Upload:	daron-mccoy
View:	235 times
Download:	0 times

Chapter 2, Server Motherboard & Chipset Design

Documents