The virtual switch is a guest LAN technology that bridges real hardware and virtual networking LANs, offering external LAN connectivity to the guest LAN environment. The virtual switch operates with virtual QDIO adapters (OSA-Express), and external LAN connectivity is available only through OSA-Express adapters in QDIO mode. Like the OSA-Express adapter, the virtual switch supports the transport of either IP packets or Ethernet frames.

By default, the virtual switch operates in IP mode. Each guest is identified by one or more IP addresses for the delivery of IP packets. Data is transported within IP packets, and therefore the virtual switch in IP mode supports only IP based application communications. All traffic destined for the physical portion of the LAN segment is encapsulated into an Ethernet frame with the OSA-Express's MAC as the source MAC address. On inbound, the OSA-Express strips the Ethernet frame and forwards the IP packet to the virtual switch for delivery to the guest by the destination IP address within the IP packet.

When operating in Ethernet mode, the virtual switch uses each guest's unique MAC address to forward frames. Data is transported and delivered within Ethernet frames, providing the ability to transport both IP and non-IP base application data through the fabric that the virtual switch supports. Through the ARP processing of each guest, the guest's MAC address becomes "known" (cached) by hosts residing on the physical side of the LAN segment. The generation and assignment of the locally defined MAC address is performed by z/VM under the direct management control of the LAN administrator. Each outbound or inbound frame through the OSA-Express switch trunk connection is an Ethernet frame with the guest's MAC address as the source or destination MAC address.

The virtual switch configured in Ethernet mode supports the aggregation of multiple OSA-Express adapters for external LAN connectivity. By supporting the IEEE 802.3ad Link Aggregation protocols and mechanisms the aggregation of individual physical links (adapters) makes this collection or group appear as one large link. The deployment of this type of configuration increases the virtual switch bandwidth and provides near seamless failover in the event that a port becomes unavailable. This support provides real switch-to-switch aggregation. Support is not provided for aggregating virtual NICs.

The virtual switch coupled with the OSA-Express feature provides a very powerful, flexible, and robust virtualization model. Data is transferred between guest ports of the virtual switch and between sharing partitions of the same OSA-Express feature without having to leave "the box". For installations that have security policies that require that access to the guest ports of the virtual switch be controlled, the deployment of the virtual switch port isolation facility is required. This facility actually isolates all guest port communications and also isolates the virtual switch OSA connection from all other sharing hosts/LPARs on the OSA feature or port. An external router configured as a firewall can be deployed to control access between the virtual switch guest ports themselves and between a guest port and any hosts (LPARs) sharing the same OSA port.

Page 1 of 46

Introduction

The purpose of this document is to provide a better understanding of Virtual Local Area Networks (VLANs) and their use in the Network 21 architecture. In the following sections, we will define a VLAN and describe its benefits as well as some limitations. We will explain why it is important for a LAN administrator to understand VLANs, and give some basic instructions to determine how many VLANs a department would typically need. Lastly, contact information will be provided for any additional questions you might have.

The main reasons for covering all of this are to further your understanding of the changes that will occur as part of Network 21 and to assist you in filling out the Network 21 Stage 3 Survey. Use of this information to determine a department's VLAN needs will ease in the conversion process. If VLANs are well conceived in advance, the need to readdress devices and modify VLAN configurations more than once will not become an issue. This will save everyone involved a great deal of effort and minimize the amount of changes that will be needed following the initial conversion.

Page 2 of 46

What is a VLAN?

Figure 1 - Typical Routed Network

To understand VLANs, it is first necessary to have an understanding of LANs. A Local Area Network (LAN) can generally be defined as a broadcast domain. Hubs, bridges or switches in the same physical segment or segments connect all end node devices. End nodes can communicate with each other without the need for a  router . Communications with devices on other LAN segments requires the use of a router. Figure 1 illustrates a typical LAN environment connected by routers.

In Figure 1, each LAN is separated from the other by a router. This represents the current UCDNet topology. The individual LANs and broadcast domains are represented by the areas bounded by the dotted lines and numbered 1 through 5 for future reference. Note that the router interface for each LAN is included as part of the LAN and broadcast domain.

Page 3 of 46

As networks expand, more routers are needed to separate users into broadcast and collision domains and provide connectivity to other LANs. In Figure 1, LANs 4 and 5 illustrate the use of a router to separate users in a single building into multiple broadcast domains.

One drawback to this design is that routers add latency, which essentially delays the transmission of data. This is caused by the process involved in routing data from one LAN to another. A router must use more of the data  packet  to determine destinations and route the data to the appropriate end node.

Virtual LANs (VLANs) can be viewed as a group of devices on different physical LAN segments which can communicate with each other as if they were all on the same physical LAN segment. VLANs provide a number of benefits over the network described in Figure 1, which we will discuss in the next section. In order to take advantage of the benefits of VLANs, a different network topology is needed.

Figure 2 - Typical Switched Network

Using the same end nodes as in Figure 1, the switched network in Figure 2 provides the same connectivity as Figure 1. Although the network above has some distinct speed and latency advantages over the network in Figure 1, it also has some serious drawbacks. The most notable of these for the purposes of this discussion is that all hosts (end nodes) are now in the same broadcast domain. This adds a significant amount of traffic to the network that is seen by all

Page 4 of 46

hosts on the network. As this network grows, the broadcast traffic has the potential impact of flooding the network and making it essentially unusable.

Switches using VLANs create the same division of the network into separate broadcast domains but do not have the latency problems of a router. Switches are also a more cost-effective solution. Figure 3 shows a switched network topology using VLANs.

Figure 3 - Switched Network with VLANs

Notice that the initial logical LAN topology from Figure 1 has been restored, with the major changes being the addition of Ethernet switches and the use of only one router. Notice also that the LAN identifiers appear on the single router interface. It is still necessary to use a router when moving between broadcast domains, and in this example, the router interface is a member of all of the VLANs. There are a number of ways to do this, and most are still proprietary and vendor-based.

Page 5 of 46

Figure 4 - VLAN grouping using traffic patterns

By now you are probably wondering why someone would go to all this work to end up with what appears to be the same network (at least from a logical standpoint) as the original one. Consider Figure 4, where we begin to take advantage of some of the benefits of VLANs.

In the previous examples, LANs have been grouped with physical location being the primary concern. In Figure 4, VLAN 1 has been built with traffic patterns in mind. All of the end devices in 1b, 1c, and 1d are primarily used for minicomputer access in 1a. Using VLANs, we are able to group these devices logically into a single broadcast domain. This allows us to confine broadcast traffic for this workgroup to just those devices that need to see it, and reduce traffic to the rest of the network. There is an increased connection speed due to the elimination of latency from router connections. An additional benefit of increased security could be realized if we made the decision to not allow access to the host from foreign networks, i.e., those that originate from another subnet beyond the router.

If we extend this thinking, we can now create a network that is independent of physical location and group users into logical workgroups. For instance, if a department has users in three different locations, they can now provide access to servers and printers as if they were all in the same building. Figure 5 illustrates this concept using the same end devices as in Figure 1 and logically grouped by function, traffic patterns, and workgroups.

Page 6 of 46

As in Figure 4, VLAN 1 is a group of users whose primary function is to access a database on a minicomputer. VLAN 2 is a comprised of a similar group of users that require access to local servers and the mainframe. VLAN 3 is a department with servers and user workstations on different floors and in the case of the workstations in 3b, different buildings. VLANs 4 and 5 represent different departments with workstations and servers in single buildings.

Figure 5 logically grouped VLANs

One problem remains from the picture above. In a campus environment the size of UC Davis, it is difficult to scale the model above due to physical distances and sheer numbers.

Enter  ATM  and Network 21. The solution to these problems is to install ATM in the cloud and use something called LAN Emulation (LANE) to provide backbone services to the  edge devices , or in this case, the Ethernet switches shown in Figure 5. Without going into detail, LAN Emulation over ATM provides the means to fully support existing LAN-based applications without changes. Advanced LAN Emulation software provides transparency to the underlying network's move to ATM. In addition, LANE provides the following benefits:

Higher capacity Superior allocation and management of network capacity

Easier management of the constantly changing LAN membership

Access to multiple VLANs from the same physical interface

Page 7 of 46

Ease of evolution to new applications.

Figure 6 gives us a look at VLANs in an ATM LANE environment. You'll notice that nothing has changed at the edges of the network, and a little more detail has been added at the core.

Figure 6 - VLANs with ATM backbone

We will not discuss ATM LANE in detail here. For the purpose of this discussion, the picture above shows a high level view of an ATM VLAN environment and closely mirrors the Network 21 architecture.

VLAN Benefits

As we have seen, there are several benefits to using VLANs. To summarize, VLAN architecture benefits include:

Increased performance Improved manageability

Page 8 of 46

Network tuning and simplification of software configurations

Physical topology independence

Increased security options

Increased performance

Switched networks by nature will increase performance over shared media devices in use today, primarily by reducing the size of collision domains. Grouping users into logical networks will also increase performance by limiting broadcast traffic to users performing similar functions or within individual workgroups. Additionally, less traffic will need to be routed, and the latency added by routers will be reduced.

Improved manageability

VLANs provide an easy, flexible, less costly way to modify logical groups in changing environments. VLANs make large networks more manageable by allowing centralized configuration of devices located in physically diverse locations.

Network tuning and simplification of software configurations

VLANs will allow LAN administrators to "fine tune" their networks by logically grouping users. Software configurations can be made uniform across machines with the consolidation of a department's resources into a single subnet.  IP addresses , subnet masks, and local network protocols will be more consistent across the entire VLAN. Fewer implementations of local server resources such as  BOOTP  and DHCP will be needed in this environment. These services can be more effectively deployed when they can span buildings within a VLAN.

Physical topology independence

VLANs provide independence from the physical topology of the network by allowing physically diverse workgroups to be logically connected within a single broadcast domain. If the physical infrastructure is already in place, it now becomes a simple matter to add ports in new locations to existing VLANs if a department expands or relocates. These assignments can take place in advance of the move, and it is then a simple matter to move devices with their existing configurations from one location to another. The old ports can then be "decommissioned" for future use, or reused by the department for new users on the VLAN.

Increased security options

VLANs have the ability to provide additional security not available in a shared media network environment. By nature, a switched network delivers frames only to the intended recipients, and broadcast frames only to other members of the VLAN. This allows the network administrator to segment users requiring access to sensitive information into separate VLANs from the rest of the general user community regardless of physical location. In addition, monitoring of a port with a

Page 9 of 46

traffic analyzer will only view the traffic associated with that particular port, making discreet monitoring of network traffic more difficult.

It should be noted that the enhanced security that is mentioned above is not to be considered an absolute safeguard against security infringements. What this provides is additional safeguards against "casual" but unwelcome attempts to view network traffic.

VLAN Limitations

There are a few limitations to using VLANs, some of the more notable being: Broadcast limitations Device limitations

Port constraints

Broadcast limitations

In order to handle broadcast traffic in an ATM VLAN environment it is necessary to have a special server that is an integrated part of the ATM infrastructure. This server has limitations in the number of broadcasts that may be forwarded. Some network protocols that will be running within individual VLANs, such as IPX and AppleTalk, make extensive use of broadcast traffic. This has the potential of impacting thresholds on the switches or broadcast servers and may require special consideration when determining VLAN size and configuration.

Device limitations

The number of Ethernet addresses than can be supported by each edge device is 500. This represents a distribution of about 20 devices per Network 21 port. These numbers are actual technical limitations that could be further reduced due to performance requirements of attached devices.

These limitations are above the recommended levels for high performance networking. From a pure performance standpoint, the ideal end-user device to Network 21 port ratio would be one device per port. From a practical point of view, a single Network 21 port could be shared by a number of devices that do not require a great deal of bandwidth and belong to the same VLAN. An example of this would be a desktop computer, printer, and laptop computer for an individual user.

Port Constraints

If a departmental hub or switch is connected to a Network 21 port, every port on that hub must belong to the same VLAN. Hubs do not have the capability to provide VLANs to individual ports, and VLANs cannot be extended beyond the edge device ports even if a switch capable of supporting VLANs is attached.

Preparation for VLANs

Page 10 of 46

Here are answers to some questions that you might have with regards to the implementation of Network 21 and VLANs.

How many VLANs do I need?

The Network 21 Project can accommodate 300 - 400 VLANs. In the majority of cases a department should only need one VLAN. Given that there are 250 departments included in the project, departments should try to limit their VLANs to one or two. Each LAN Administrator will need to determine appropriate logical groups for their department. It is anticipated that most departments will obtain maximum benefits by consolidating the majority (if not all) of their users into a single large VLAN. Smaller VLANs would then be used if necessary to group together power users or those requiring special handling.

What VLAN information is required by the survey?

As part of the Network 21 Stage 3 survey you will be asked to identify both the number of VLANs your department requires and the individual NAMs that comprise each VLAN. A worksheet will be provided for each of these tasks. The Department VLAN Worksheet simply asks for the number (start with one and increment accordingly), a description or the purpose, the primary department owner, and the name of any other departments on the VLAN. The Department NAM Verification worksheet lists all of the department's NAMs and their building and room number. You are asked to supply information as to which VLAN number (from the Department VLAN Worksheet) each NAM is to be connected to, and the number of devices served by that NAM. There are also check boxes to identify if any devices attached to each NAM are running AppleTalk, DECNET, or IPX. Detailed instructions and examples will be provided with the survey sheets to use for assistance in filling out these forms.

Glossary

ATM

Asynchronous Transfer Mode. International standard for cell relay in which multiple service types (such as voice, video, or data) are conveyed in fixed-length (53-byte) cells. Fixed-length cells allow cell processing to occur in hardware, thereby reducing delay. ATM is designed to take advantage of high-speed transmission media.

Bridge

A device that connects and passes packets between two network segments that use the same communications protocol. Bridges operate at the data link layer (Layer 2) of the OSI reference model. In general, a bridge will filter, forward, or flood an incoming frame based on the MAC address of that frame.

Page 11 of 46

BOOTP

Bootstrap Protocol. A protocol that is used by a network node to determine the IP address of its Ethernet interfaces, in order to effect network booting.

Broadcast Domain

The set of all devices that will receive broadcast frames originating from any device within the set. Broadcast domains can be bounded by VLANs in a stand-alone environment. In an internetworking environment, they are typically bounded by routers because routers do not forward broadcast frames.

Collision

In Ethernet, the result of two nodes that transmit simultaneously. The frames from each device impact and are damaged when they meet on the physical media.

Collision Domain

In Ethernet, the network area within which frames that have collided are propagated. Repeaters and hubs propagate collisions; LAN switches, bridges and routers do not.

CSMA/CD

Carrier Sense Multiple Access/Collision Detect. Media-access mechanism wherein devices ready to transmit data first check the channel for a carrier signal. If no carrier is sensed for a specific period, a device can transmit. A collision occurs if two devices transmit simultaneously, and the collision is detected by all colliding devices. This collision subsequently delays retransmissions from those devices for some random length of time. CSMA/CD access is used by Ethernet and IEEE 802.3.

DHCP

Dynamic Host Configuration Protocol. Provides a mechanism for allocating IP addresses dynamically so that addresses can be reused when hosts no longer needs them.

Edge Device

A physical device that is capable of forwarding packets between legacy interfaces (such as Ethernet and Token Ring) and ATM interfaces based on data-link and network layer information. An edge device does not participate in the running of any network layer routing protocol.

Ethernet

Page 12 of 46

Baseband LAN specification invented by Xerox Corporation and developed jointly by Xerox, Intel, and Digital Equipment Corporation. Ethernet networks use CSMA/CD and run over a variety of cable types at 10 Mbps. Ethernet is similar to the IEEE 802.3 series of standards.

Fast Ethernet

Any of a number of 100-Mbps Ethernet specifications, Fast Ethernet offers a speed increase ten times that of the 10BaseT Ethernet specification, while preserving such qualities as frame format, MAC mechanisms, and MTU. Such similarities allow the use of existing Ethernet applications and network management tools on Fast Ethernet networks. Fast Ethernet is based on an extension to the IEEE 802.3 specification.

Frame

The logical grouping of information sent as a data link layer unit over a transmission medium. Often refers to the header and trailer, used for synchronization and error control, which surround the user data contained in the unit.

Hub

Generally, a device that serves as the center of a star-topology shared network. Also describes a hardware or software device that contains multiple independent but connected modules of network and internetwork equipment.

IEEE

Institute of Electrical and Electronics Engineers. The IEEE is a professional organization whose activities include the development of communications and network standards. IEEE LAN standards are the predominant LAN standards today.

IP

Internet Protocol. Network layer protocol in the TCP/IP stack offering a connectionless internetwork service. IP provides features for addressing, type-of-service specification, security, and fragmentation and reassembly.

IP Address

32-bit address assigned to hosts using TCP/IP. An IP address belongs to one of five classes (A, B, C, D, or E) and is written as four octets separated by periods (dotted decimal format). Each address consists of a network number, an optional subnetwork number, and a host number. The network and subnetwork numbers together are used for routing, while the host number is used to address an individual host within the network or subnetwork. A subnet mask is used to extract network and subnetwork information from the IP address.

LAN

Page 13 of 46

Local-Area Network. High-speed, low-error data network covering a relatively small geographic area (up to a few thousand meters). LANs connect workstations, peripherals, terminals, and other devices in a single building or other geographically limited area. LAN standards specify cabling and signaling at the physical and data link layers of the OSI model. Ethernet, FDDI, and Token Ring are widely used LAN technologies.

LANE

LAN emulation. Technology that allows an ATM network to function as a LAN backbone. The ATM network must provide multicast and broadcast support, address mapping (MAC-to-ATM), SVC management, and a usable packet format. LANE also defines Ethernet and Token Ring ELANs.

Latency

Delay between the time a device requests access to a network and the time it is granted permission to transmit. It is also the delay between the time when a device receives a frame and the time that frame is forwarded out the destination port.

Node

Endpoint of a network connection or a junction common to two or more lines in a network. Nodes can be processors, controllers, or workstations. Nodes, which vary in routing and other functional capabilities, can be interconnected by links, and serve as control points in the network. Node is sometimes used generically to refer to any entity that can access a network, and is frequently used interchangeably with device.

OSI Model

Open System Interconnection reference model. Network architectural model developed by ISO and ITU-T. The model consists of seven layers, each of which specifies particular network functions such as addressing, flow control, error control, encapsulation, and reliable message transfer. The lowest layer (the physical layer) is closest to the media technology. The lower two layers are implemented in hardware and software, while the upper five layers are implemented only in software. The highest layer (the application layer) is closest to the user. The OSI reference model is used universally as a method for teaching and understanding network functionality.

Packet

A logical grouping of information that includes a header containing control information and (usually) user data, packets are most often used to refer to network layer units of data.

Router

Page 14 of 46

Network layer device that uses one or more metrics to determine the optimal path along which network traffic should be forwarded. Routers forward packets from one network to another based on network layer information. Occasionally called a gateway (although this definition of gateway is becoming increasingly outdated).

Subnet

Subnetwork. In IP networks, a network sharing a particular subnet address. Subnetworks are networks arbitrarily segmented by a network administrator in order to provide a multilevel, hierarchical routing structure while shielding the subnetwork from the addressing complexity of attached networks.

Subnet Mask

32-bit address mask used in IP to indicate the bits of an IP address that are being used for the subnet address. The subnet mask is sometimes referred to simply as mask.

Switch

A network device that filters, forwards, and floods frames based on the destination address of each frame. The switch operates at the data link layer of the OSI model.

VLAN

Virtual LAN. Group of devices on one or more LANs that are configured (using management software) so that they can communicate as if they were attached to the same wire, when in fact they are located on a number of different LAN segments. Because VLANs are based on logical instead of physical connections, they are extremely flexible.

Page 15 of 46

VIRTUAL LANs

(Based on an article from Byte - May 1995)

 

1. VIRTUAL LANs GET REAL:

To meet customers’ demands, corporations must be able to quickly bring together in-house knowledge on an ad hoc basis. Virtual LAN technology helps corporations do just that.

In essence, virtual LANs allow companies to form workgroups on the fly. This approach not only lets users share network resources but also provides dedicated bandwidth to each group. This is accomplished without flooding other parts of the corporate network with broadcast traffic that’s intended solely for other members of the same group.

Page 16 of 46

Virtual LANs are an outgrowth of switching technologies (Ethernet, Fast Ethernet and ATM) that are making their way into corporate networks. Early virtual LAN solutions came from switching hub vendors, such as Alantec, Chipcom, Lannet, and UB Networks. These solutions were followed by offerings from major internetworking vendors, such as 3Com, Bay Networks, Cabletron Systems, Cisco Systems, Standard Microsystems, and Xyplex.

All these vendors offer a way to take advantage of the port or packet-switching capabilities of their intelligent wiring hubs or switches so that users can be connected based on logical groupings rather than physical location within the company. For example, in a number of organizations, it probably makes sense to form a product development team that includes a systems engineer, a design engineer, a product marketing person, and someone responsible for the financial management of the project.

This group of folks can be scattered throughout the organization, but they need to share information relevant to the project they are working on. In the past, to create such grouping, administrators had to go into wiring closets and physically connect the users to the same LAN segment, which was fine if a group was permanent. However, in today’s business environment, it’s likely that such groups will be short-lived.

That’s where virtual LANs come in. Rather than making frequent trips into wiring closets and physically reconfiguring the network at the hub by moving cables around, administrators can define and create groups in software by issuing commands from a central management console.

Page 17 of 46

To create virtual workgroups, you assign a dedicated portion of a hub’s backplane bandwidth to each virtual LAN. Most switching hubs have high-bandwidth backplanes that support multiple, dedicated LAN segments. For example, with Lannet’s LANswitch, you can define up to 255 virtual LANs (also with this hub, you can connect users to a "global" virtual LAN).

Once you assign a person to a workgroup, the traffic generated for the other members of the group is confined to that one portion of backplane bandwidth. This reduces congestion on other portions of a network because broadcast packets generated within the group only reach the other members of that group.

Page 18 of 46

If a company wants to create logical workgroups and contain each group’s traffic to their respective group, virtual LANs are not necessary. They do, however, offer advantages over alternatives.

Consider what you can do without using virtual LANs. In traditional networks, you can segment at the network layer and use routers to control connectivity. Basically, you can assign users in an IP network to different IP subnets based on logical rather than physical groupings. In that way, two adjacent users on the same physical LAN segment may be members of different subnetworks, and traffic from one to the other must pass through a router even though they are physically "connected".

With virtual LANs, traffic generated for members of one logical group must also get routed to another group to exchange information. So in that sense, virtual LANs and a network where workgroups are synonymous with subnetworks (connected by routers) are similar. However, there are some major differences.

First, with virtual LANs, specific types of connectivity among groups can be simplified by assigning users to more than one group. For example, you may have a number of administratively defined groups (e.g., sales, marketing, and product development) where members of each group share resources, such as CAD program or shared database. You may also define "global" groups whose members have access to enterprise network resources, such as high-quality printers or an E-mail system.

Page 19 of 46

A second advantage of using virtual LANs instead of interconnected subnetworks is that groups can be formed dynamically. You can easily move people in or out of a workgroup with, essentially, the click of a mouse at a management console.

This capability is handy when forming new groups. It also saves a network administrator much time when handling changes on the network. In traditional networks, configuration changes occur on a regular basis; U.S. businesses annually spend about $1.3 billion moving, adding, and changing users on networks. Some companies experience more frequent changes than others. For example, in the banking industry, typically, 50 percent to 60 percent of the people change their location each year. With virtual LANs, the changes, as noted, are done at a management station, not by going into a wiring closet.

Companies seeking to cut the administrative costs of handling moves, adds, and changes, as well as reaping other benefits offered by using virtual LANs, will likely choose intelligent switching hubs as the core of their networks.

In the future, as corporate networks incorporate more ATM switching technology, companies will want to retain these same competitive advantages. To that end, the ATM Forum (a group of users and ATM product vendors) is developing a specification for ATM LAN emulation, which lets users create virtual LANs that run over ATM backbone networks. LAN emulation-based virtual LANs will use ATM switched circuits to carry traffic amongst members of administratively defined workgroups.

Page 20 of 46

ATM LAN emulation lets PCs on Ethernet and token-ring LANs connect with other LAN-attached PCs over an ATM network. LAN emulation is basically a way to make the ATM switching transparent to traditional LANs. In a practical sense, that means workstation adapter cards and their associated driver software does not have a change. Therefore, applications developed to run over drivers, such as NDIS (Network Driver Interface Specification) and ODI (Open Data-Link Interface), can continue to be used.

ATM LAN emulation involves two pieces of management software: LEC (LAN emulation client) and LES (LAN emulation server). The LEC acts as a proxy ATM end station for a LAN workstation and lets LAN clients connect to ATM backbones; the LES resolves LAN MAC (media access control) addresses with ATM addresses. Both pieces of software are required to build high-speed, low-latency virtual LANs based on ATM technology.

Once products that support ATM LAN emulation become available, this should provide a level of interoperability. What that means for corporations deploying virtual LANs is they will no longer be limited to a single vendor’s hardware.

 

2. NOT A PANACEA:

Virtual LANs certainly have good points. There are, however, some shortcomings that users should be aware of before implementing the technology.

The most troubling thing about virtual LAN technology is that it is proprietary. Each hub vendor offers its own management system for defining, creating, and administrating a virtual LAN environment. The virtual LAN portion of a hub vendor’s management system will not control another vendor’s switching hub. There is, however, a move toward standardization in the ATM world.

A second shortcoming of virtual LAN technology is that you still need routers to get between mixed networking environments. That’s because switching is done at the MAC layer. You would need a router when connecting an IP user to an IPX user or when connecting a user on a token-ring network to another person on an Ethernet LAN.

Another point to consider is that virtual LAN technology may be overkill in certain situations. Sometimes you can meet connectivity requirements using a simpler physical layer switching technology. For instance, a situation may call for the data on multiple LAN segments to be saved to a centrally managed backup system. Or you may have an accounting department that needs access to a high-performance server once a month when the department staffers are balancing the books.

You could handle either situation with a virtual LAN, or you could use the old method of going into the wiring closet and physically connecting the desired server to an appropriate LAN segment. In these examples, the virtual LAN approach is overkill, and the manual approach is too labor-intensive. A better approach would be to use a physical layer switch, such as Switchex

Page 21 of 46

from NHC Communications, Physical layer switches that are easier to use than traditional switching hubs operate like electronic patch panels, which you control from a management console.

Obviously, there are many ways to let a group of users share information. Virtual LANs may not be the best solution in all cases. However, in today’s rapidly changing business environment where companies must bring together in-house experts on a project-by-project basis, virtual LANs have much to offer.

Page 22 of 46

Abstract

Network Load Balancing, a clustering technology included in the Microsoft Windows 2000 Advanced Server and Datacenter Server operating systems, enhances the scalability and availability of mission-critical, TCP/IP-based services, such as Web, Terminal Services, virtual private networking, and streaming media servers. This component runs within cluster hosts as part of the Windows 2000 operating system and requires no dedicated hardware support. To scale performance, Network Load Balancing distributes IP traffic across multiple cluster hosts. It also ensures high availability by detecting host failures and automatically redistributing traffic to the surviving hosts. Network Load Balancing provides remote controllability and supports rolling upgrades from the Windows NT 4.0 operating system.

The unique and fully distributed architecture of Network Load Balancing enables it to deliver very high performance and failover protection, especially in comparison with dispatcher-based load balancers. This white paper describes the key features of this technology and explores its internal architecture and performance characteristics in detail.

On This PageIntroduction Network Load Balancing Architecture Network Load Balancing Performance Summary Glossary of Key Terms 

Introduction

Internet server programs supporting mission-critical applications such as financial transactions, database access, corporate intranets, and other key functions must run 24 hours a day, seven days a week. And networks need the ability to scale performance to handle large volumes of client requests without creating unwanted delays. For these reasons, clustering is of wide interest to the enterprise. Clustering enables a group of independent servers to be managed as a single system for higher availability, easier manageability, and greater scalability.

The Microsoft® Windows® 2000 Advanced Server and Datacenter Server operating systems include two clustering technologies designed for this purpose: Cluster service, which is intended primarily to provide failover support for critical line-of-business applications such as databases, messaging systems, and file/print services; and Network Load Balancing, which serves to balance incoming IP traffic among multi-node clusters. We will treat this latter technology in detail here.

Network Load Balancing provides scalability and high availability to enterprise-wide TCP/IP services, such as Web, Terminal Services, proxy, Virtual Private Networking (VPN), and streaming media services. Network Load Balancing brings special value to enterprises deploying TCP/IP services, such as e-commerce applications, that link clients with transaction applications and back-end databases.

Network Load Balancing servers (also called hosts) in a cluster communicate among themselves to provide key benefits, including:

Page 23 of 46

Scalability. Network Load Balancing scales the performance of a server-based program, such as a Web server, by distributing its client requests across multiple servers within the cluster. As traffic increases, additional servers can be added to the cluster, with up to 32 servers possible in any one cluster.

High availability. Network Load Balancing provides high availability by automatically detecting the failure of a server and repartitioning client traffic among the remaining servers within ten seconds, while providing users with continuous service.

Network Load Balancing distributes IP traffic to multiple copies (or instances) of a TCP/IP service, such as a Web server, each running on a host within the cluster. Network Load Balancing transparently partitions the client requests among the hosts and lets the clients access the cluster using one or more "virtual" IP addresses. From the client's point of view, the cluster appears to be a single server that answers these client requests. As enterprise traffic increases, network administrators can simply plug another server into the cluster.

For example, the clustered hosts in Figure 1 below work together to service network traffic from the Internet. Each server runs a copy of an IP-based service, such as Internet Information Services 5.0 (IIS), and Network Load Balancing distributes the networking workload among them. This speeds up normal processing so that Internet clients see faster turnaround on their requests. For added system availability, the back-end application (a database, for example) may operate on a two-node cluster running Cluster service.

Figure 1: A four-host cluster works as a single virtual server to handle network traffic. Each host runs its own copy of the server with Network Load Balancing distributing the work among the four hosts.

Advantages of Network Load BalancingNetwork Load Balancing is superior to other software solutions such as round robin DNS (RRDNS), which distributes workload among multiple servers but does not provide a mechanism for server availability. If a server within the host fails, RRDNS, unlike Network Load Balancing, will continue to send it work until a network administrator detects the failure and removes the server from the DNS address list. This results in service disruption for clients. Network Load

Page 24 of 46

Balancing also has advantages over other load balancing solutions—both hardware- and software-based—that introduce single points of failure or performance bottlenecks by using a centralized dispatcher. Because Network Load Balancing has no proprietary hardware requirements, any industry-standard compatible computer can be used. This provides significant cost savings when compared to proprietary hardware load balancing solutions.

The unique and fully distributed software architecture of Network Load Balancing enables it to deliver the industry's best load balancing performance and availability. The specific advantages of this architecture are described below in the "Network Load Balancing Architecture" section.

Installing and Managing Network Load BalancingNetwork Load Balancing is automatically installed and can be optionally enabled on the Advanced Server and Datacenter Server versions of the Windows 2000 operating system. It operates as an optional service for local area network (LAN) connections and can be enabled for one LAN connection in the system; this LAN connection is known as the cluster adapter. No hardware changes are required to install and run Network Load Balancing. Since it is compatible with almost all Ethernet and Fiber Distributed Data Interface (FDDI) network adapters, it has no specific hardware compatibility list.

IP Addresses

Once Network Load Balancing is enabled, its parameters are configured using its Properties dialog box, as described in the online help guide. The cluster is assigned a primary IP address, which represents a virtual IP address to which all cluster hosts respond. The remote control program provided as a part of Network Load Balancing uses this IP address to identify a target cluster. Each cluster host also can be assigned a dedicated IP address for network traffic unique to that particular host within the cluster. Network Load Balancing never load-balances traffic for the dedicated IP address. Instead, it load-balances incoming traffic from all IP addresses other than the dedicated IP address.

When configuring Network Load Balancing, it is important to enter the dedicated IP address, primary IP address, and other optional virtual IP addresses into the TCP/IP Properties dialog box in order to enable the host's TCP/IP stack to respond to these IP addresses. The dedicated IP address is always entered first so that outgoing connections from the cluster host are sourced with this IP address instead of a virtual IP address. Otherwise, replies to the cluster host could be inadvertently load-balanced by Network Load Balancing and delivered to another cluster host. Some services, such as the Point-to-Point Tunneling Protocol (PPTP) server, do not allow outgoing connections to be sourced from a different IP address, and thus a dedicated IP address cannot be used with them.

Host Priorities

Each cluster host is assigned a unique host priority in the range of 1 to 32, where lower numbers denote higher priorities. The host with the highest host priority (lowest numeric value) is called the default host. It handles all client traffic for the virtual IP addresses that is not specifically intended to be load-balanced. This ensures that server applications not configured for load balancing only receive client traffic on a single host. If the default host fails, the host with the next highest priority takes over as default host.

Port Rules

Page 25 of 46

Network Load Balancing uses port rules to customize load balancing for a consecutive numeric range of server ports. Port rules can select either multiple-host or single-host load-balancing policies. With multiple-host load balancing, incoming client requests are distributed among all cluster hosts, and a load percentage can be specified for each host. Load percentages allow hosts with higher capacity to receive a larger fraction of the total client load. Single-host load balancing directs all client requests to the host with highest handling priority. The handling priority essentially overrides the host priority for the port range and allows different hosts to individually handle all client traffic for specific server applications. Port rules also can be used to block undesired network access to certain IP ports.

When a port rule uses multiple-host load balancing, one of three client affinity modes is selected. When no client affinity mode is selected, Network Load Balancing load-balances client traffic from one IP address and different source ports on multiple-cluster hosts. This maximizes the granularity of load balancing and minimizes response time to clients. To assist in managing client sessions, the default single-client affinity mode load-balances all network traffic from a given client's IP address on a single-cluster host. The class C affinity mode further constrains this to load-balance all client traffic from a single class C address space. See the "Managing Application State" section below for more information on session support.

By default, Network Load Balancing is configured with a single port rule that covers all ports (0-65,535) with multiple-host load balancing and single-client affinity. This rule can be used for most applications. It is important that this rule not be modified for VPN applications and whenever IP fragmentation is expected. This ensures that fragments are efficiently handled by the cluster hosts.

Remote Control

Network Load Balancing provides a remote control program (Wlbs.exe) that allows system administrators to remotely query the status of clusters and control operations from a cluster host or from any networked computer running Windows 2000. This program can be incorporated into scripts and monitoring programs to automate cluster control. Monitoring services are widely available for most client/server applications. Remote control operations include starting and stopping either single hosts or the entire cluster. In addition, load balancing for individual port rules can be enabled or disabled on one or more hosts. New traffic can be blocked on a host while allowing ongoing TCP connections to complete prior to removing the host from the cluster. Although remote control commands are password-protected, individual cluster hosts can disable remote control operations to enhance security.

Managing Server Applications

Server applications need not be modified for load balancing. However, the system administrator starts load-balanced applications on all cluster hosts. Network Load Balancing does not directly monitor server applications, such as Web servers, for continuous and correct operation. Monitoring services are widely available for most client/server applications. Instead, Network Load Balancing provides the mechanisms needed by application monitors to control cluster operations—for example, to remove a host from the cluster if an application fails or displays erratic behavior. When an application failure is detected, the application monitor uses the Network Load Balancing remote control program to stop individual cluster hosts and/or disable load balancing for specific port ranges.

Page 26 of 46

Maintenance and Rolling Upgrades

Computers can be taken offline for preventive maintenance without disturbing cluster operations. Network Load Balancing also supports rolling upgrades to allow software or hardware upgrades without shutting down the cluster or disrupting service. Upgrades can be individually applied to each server, which immediately rejoins the cluster. Network Load Balancing hosts can run in mixed clusters with hosts running the Windows NT® Load Balancing Service (WLBS) under Windows NT 4.0. Rolling upgrades can be performed without interrupting cluster services by taking individual hosts out of the cluster, upgrading them to Windows 2000, and then placing them back in the cluster. (Note that the first port in the default port range has been changed for Windows 2000 from 1 to 0, and the port rules must always be compatible for all cluster hosts.)

How Network Load Balancing WorksNetwork Load Balancing scales the performance of a server-based program, such as a Web server, by distributing its client requests among multiple servers within the cluster. With Network Load Balancing, each incoming IP packet is received by each host, but only accepted by the intended recipient. The cluster hosts concurrently respond to different client requests, even multiple requests from the same client. For example, a Web browser may obtain the various images within a single Web page from different hosts in a load-balanced cluster. This speeds up processing and shortens the response time to clients.

Each Network Load Balancing host can specify the load percentage that it will handle, or the load can be equally distributed among all of the hosts. Using these load percentages, each Network Load Balancing server selects and handles a portion of the workload. Clients are statistically distributed among cluster hosts so that each server receives its percentage of incoming requests. This load balance dynamically changes when hosts enter or leave the cluster. In this version, the load balance does not change in response to varying server loads (such as CPU or memory usage). For applications, such as Web servers, which have numerous clients and relatively short-lived client requests, the ability of Network Load Balancing to distribute workload through statistical mapping efficiently balances loads and provides fast response to cluster changes.

Network Load Balancing cluster servers emit a heartbeat message to other hosts in the cluster, and listen for the heartbeat of other hosts. If a server in a cluster fails, the remaining hosts adjust and redistribute the workload while maintaining continuous service to their clients. Although existing connections to an offline host are lost, the Internet services nevertheless remain continuously available. In most cases (for example, with Web servers), client software automatically retries the failed connections, and the clients experience only a few seconds' delay in receiving a response.

Managing Application StateApplication state refers to data maintained by a server application on behalf of its clients. If a server application (such as a Web server) maintains state information about a client session—that is, when it maintains a client's session state—that spans multiple TCP connections, it is usually important that all TCP connections for this client be directed to the same cluster host. Shopping cart contents at an e-commerce site and Secure Sockets Layer (SSL) authentication data are examples of a client's session state. Network Load Balancing can be used to scale applications that manage session state spanning multiple connections. When its client affinity parameter

Page 27 of 46

setting is enabled, Network Load Balancing directs all TCP connections from one client IP address to the same cluster host. This allows session state to be maintained in host memory. However, should a server or network failure occur during a client session, a new logon may be required to re-authenticate the client and re-establish session state. Also, adding a new cluster host redirects some client traffic to the new host, which can affect sessions, although ongoing TCP connections are not disturbed. Client/server applications that manage client state so that it can be retrieved from any cluster host (for example, by embedding state within cookies or pushing it to a back-end database) do not need to use client affinity.

To further assist in managing session state, Network Load Balancing provides an optional client affinity setting that directs all client requests from a TCP/IP class C address range to a single cluster host. With this feature, clients that use multiple proxy servers can have their TCP connections directed to the same cluster host. The use of multiple proxy servers at the client's site causes requests from a single client to appear to originate from different systems. Assuming that all of the client's proxy servers are located within the same 254-host class C address range, Network Load Balancing ensures that the same host handles client sessions with minimum impact on load distribution among the cluster hosts. Some very large client sites may use multiple proxy servers that span class C address spaces.

In addition to session state, server applications often maintain persistent, server-based state information that is updated by client transactions, such as merchandise inventory at an e-commerce site. Network Load Balancing should not be used to directly scale applications, such as Microsoft SQL Server™ (other than for read-only database access), that independently update inter-client state because updates made on one cluster host will not be visible to other cluster hosts. To benefit from Network Load Balancing, applications must be designed to permit multiple instances to simultaneously access a shared database server that synchronizes updates. For example, Web servers with Active Server Pages should have their client updates pushed to a shared back-end database server.

Top of page 

Network Load Balancing Architecture

To maximize throughput and high availability, Network Load Balancing uses a fully distributed software architecture. An identical copy of the Network Load Balancing driver runs in parallel on each cluster host. The drivers arrange for all cluster hosts on a single subnet to concurrently detect incoming network traffic for the cluster's primary IP address (and for additional IP addresses on multihomed hosts). On each cluster host, the driver acts as a filter between the network adapter's driver and the TCP/IP stack, allowing a portion of the incoming network traffic to be received by the host. By this means incoming client requests are partitioned and load-balanced among the cluster hosts.

Network Load Balancing runs as a network driver logically situated beneath higher-level application protocols, such as HTTP and FTP. Figure 2 below shows the implementation of Network Load Balancing as an intermediate driver in the Windows 2000 network stack.

Page 28 of 46

Figure 2: Network Load Balancing runs as an intermediate driver between the TCP/IP protocol and network adapter drivers within the Windows 2000 protocol stack Note that although two network adapters are shown, only one adapter is needed to use Network Load Balancing.

This architecture maximizes throughput by using the broadcast subnet to deliver incoming network traffic to all cluster hosts and by eliminating the need to route incoming packets to individual cluster hosts. Since filtering unwanted packets is faster than routing packets (which involves receiving, examining, rewriting, and resending), Network Load Balancing delivers higher network throughput than dispatcher-based solutions. As network and server speeds grow, its throughput also grows proportionally, thus eliminating any dependency on a particular hardware routing implementation. For example, Network Load Balancing has demonstrated 250 megabits per second (Mbps) throughput on Gigabit networks.

Another key advantage to Network Load Balancing's fully distributed architecture is the enhanced availability resulting from (N-1)-way failover in a cluster with N hosts. In contrast, dispatcher-based solutions create an inherent single point of failure that must be eliminated using a redundant dispatcher that provides only 1-way failover. This offers a less robust failover solution than does a fully distributed architecture.

Network Load Balancing's architecture takes advantage of the subnet's hub and/or switch architecture to simultaneously deliver incoming network traffic to all cluster hosts. However, this approach increases the burden on switches by occupying additional port bandwidth. (Please refer to the "Network Load Balancing Performance" section of this paper for performance

Page 29 of 46

measurements of switch usage.) This is usually not a concern in most intended applications, such as Web services and streaming media, since the percentage of incoming traffic is a small fraction of total network traffic. However, if the client-side network connections to the switch are significantly faster than the server-side connections, incoming traffic can occupy a prohibitively large portion of the server-side port bandwidth. The same problem arises if multiple clusters are hosted on the same switch and measures are not taken to setup virtual LANs for individual clusters.

During packet reception, Network Load Balancing's fully pipelined implementation overlaps the delivery of incoming packets to TCP/IP and the reception of other packets by the network adapter driver. This speeds up overall processing and reduces latency because TCP/IP can process a packet while the network driver interface specification (NDIS) driver receives a subsequent packet. It also reduces the overhead required for TCP/IP and the NDIS driver to coordinate their actions, and in many cases, it eliminates an extra copy of packet data in memory. During packet sending, Network Load Balancing also enhances throughput and reduces latency and overhead by increasing the number of packets that TCP/IP can send with one NDIS call. To achieve these performance enhancements, Network Load Balancing allocates and manages a pool of packet buffers and descriptors that it uses to overlap the actions of TCP/IP and the NDIS driver.

Distribution of Cluster TrafficNetwork Load Balancing uses layer-two broadcast or multicast to simultaneously distribute incoming network traffic to all cluster hosts. In its default unicast mode of operation, Network Load Balancing reassigns the station address ("MAC" address) of the network adapter for which it is enabled (called the cluster adapter), and all cluster hosts are assigned the same MAC address. Incoming packets are thereby received by all cluster hosts and passed up to the Network Load Balancing driver for filtering. To insure uniqueness, the MAC address is derived from the cluster's primary IP address entered in the Network Load Balancing Properties dialog box. For a primary IP address of 1.2.3.4, the unicast MAC address is set to 02-BF-1-2-3-4. Network Load Balancing automatically modifies the cluster adapter's MAC address by setting a registry entry and then reloading the adapter's driver; the operating system does not have to be restarted.

If the cluster hosts are attached to a switch instead of a hub, the use of a common MAC address would create a conflict since layer-two switches expect to see unique source MAC addresses on all switch ports. To avoid this problem, Network Load Balancing uniquely modifies the source MAC address for outgoing packets; a cluster MAC address of 02-BF-1-2-3-4 is set to 02-h-1-2-3-4, where h is the host's priority within the cluster (set in the Network Load Balancing Properties dialog box). This technique prevents the switch from learning the cluster's actual MAC address, and as a result, incoming packets for the cluster are delivered to all switch ports. If the cluster hosts are connected directly to a hub instead of to a switch, Network Load Balancing's masking of the source MAC address in unicast mode can be disabled to avoid flooding upstream switches. This is accomplished by setting the Network Load Balancing registry parameter MaskSourceMAC to 0. The use of an upstream level three switch will also limit switch flooding.

Network Load Balancing's unicast mode has the side effect of disabling communication between cluster hosts using the cluster adapters. Since outgoing packets for another cluster host are sent to the same MAC address as the sender, these packets are looped back within the sender by the

Page 30 of 46

network stack and never reach the wire. This limitation can be avoided by adding a second network adapter card to each cluster host. In this configuration, Network Load Balancing is bound to the network adapter on the subnet that receives incoming client requests, and the other adapter is typically placed on a separate, local subnet for communication between cluster hosts and with back-end file and database servers. Network Load Balancing only uses the cluster adapter for its heartbeat and remote control traffic.

Note that communication between cluster hosts and hosts outside the cluster is never affected by Network Load Balancing's unicast mode. Network traffic for a host's dedicated IP address (on the cluster adapter) is received by all cluster hosts since they all use the same MAC address. Since Network Load Balancing never load balances traffic for the dedicated IP address, Network Load Balancing immediately delivers this traffic to TCP/IP on the intended host. On other cluster hosts, Network Load Balancing treats this traffic as load balanced traffic (since the target IP address does not match another host's dedicated IP address), and it may deliver it to TCP/IP, which will discard it. Note that excessive incoming network traffic for dedicated IP addresses can impose a performance penalty when Network Load Balancing operates in unicast mode due to the need for TCP/IP to discard unwanted packets.

Network Load Balancing provides a second mode for distributing incoming network traffic to all cluster hosts. Called multicast mode, this mode assigns a layer two multicast address to the cluster adapter instead of changing the adapter's station address. The multicast MAC address is set to 03-BF-1-2-3-4 for a cluster's primary IP address of 1.2.3.4. Since each cluster host retains a unique station address, this mode alleviates the need for a second network adapter for communication between cluster hosts, and it also removes any performance penalty from the use of dedicated IP addresses.

Network Load Balancing's unicast mode induces switch flooding in order to simultaneously deliver incoming network traffic to all cluster hosts. Also, when Network Load Balancing uses multicast mode, switches often flood all ports by default to deliver multicast traffic. However, Network Load Balancing's multicast mode gives the system administrator the opportunity to limit switch flooding by configuring a virtual LAN within the switch for the ports corresponding to the cluster hosts. This can be accomplished by manually programming the switch or by using the Internet Group Management Protocol (IGMP) or the GARP (Generic Attribute Registration Protocol) Multicast Registration Protocol (GMRP). The current version of Network Load Balancing does not provide automatic support for IGMP or GMRP.

Network Load Balancing implements the Address Resolution Protocol (ARP) functionality needed to ensure that the cluster's primary IP address and other virtual IP addresses resolve to the cluster's multicast MAC address. (The dedicated IP address continues to resolve to the cluster adapter's station address.) Experience has shown that Cisco routers currently do not accept an ARP response from the cluster that resolves unicast IP addresses to multicast MAC addresses. This problem can be overcome by adding a static ARP entry to the router for each virtual IP address, and the cluster's multicast MAC address can be obtained from the Network Load Balancing Properties dialog box or from the Wlbs.exe remote control program. The default unicast mode avoids this problem because the cluster's MAC address is a unicast MAC address.

Network Load Balancing does not manage any incoming IP traffic other than TCP traffic, User Datagram Protocol (UDP) traffic, and Generic Routing Encapsulation (GRE) traffic (as part of PPTP traffic) for specified ports. It does not filter IGMP, ARP (except as described above), the

Page 31 of 46

Internet Control Message Protocol (ICMP), or other IP protocols. All such traffic is passed unchanged to the TCP/IP protocol software on all of the hosts within the cluster. As a result, the cluster can generate duplicate responses from certain point-to-point TCP/IP programs (such as ping) when the cluster IP address is used. Because of the robustness of TCP/IP and its ability to deal with replicated datagrams, other protocols behave correctly in the clustered environment. These programs can use the dedicated IP address for each host to avoid this behavior.

Load Balancing AlgorithmNetwork Load Balancing employs a fully distributed filtering algorithm to map incoming clients to the cluster hosts. This algorithm was chosen to enable cluster hosts to make a load-balancing decision independently and quickly for each incoming packet. It was optimized to deliver statistically even load balance for a large client population making numerous, relatively small requests, such as those typically made to Web servers. When the client population is small and/or the client connections produce widely varying loads on the server, Network Load Balancing's load balancing algorithm is less effective. However, the simplicity and speed of its algorithm allows it to deliver very high performance, including both high throughput and low response time, in a wide range of useful client/server applications.

Network Load Balancing load-balances incoming client requests by directing a selected percentage of new requests to each cluster host; the load percentage is set in the Network Load Balancing Properties dialog box for each port range to be load-balanced. The algorithm does not respond to changes in the load on each cluster host (such as the CPU load or memory usage). However, the mapping is modified when the cluster membership changes, and load percentages are renormalized accordingly.

When inspecting an arriving packet, all hosts simultaneously perform a statistical mapping to quickly determine which host should handle the packet. The mapping uses a randomization function that calculates a host priority based on the client's IP address, port, and other state information maintained to optimize load balance. The corresponding host forwards the packet up the network stack to TCP/IP, and the other cluster hosts discard it. The mapping does not vary unless the membership of cluster hosts changes, ensuring that a given client's IP address and port will always map to the same cluster host. However, the particular cluster host to which the client's IP address and port map cannot be predetermined since the randomization function takes into account the current and past cluster's membership to minimize remappings.

The load-balancing algorithm assumes that client IP addresses and port numbers (when client affinity is not enabled) are statistically independent. This assumption can break down if a server-side firewall is used that proxies client addresses with one IP address and, at the same time, client affinity is enabled. In this case, all client requests will be handled by one cluster host and load balancing is defeated. However, if client affinity is not enabled, the distribution of client ports within the firewall usually provides good load balance.

In general, the quality of load balance is statistically determined by the number of clients making requests. This behavior is analogous to coin tosses where the two sides of the coin correspond to the number of cluster hosts (thus, in this analogy, two), and the number of tosses corresponds to the number of client requests. The load distribution improves as the number of client requests increases just as the fraction of coin tosses resulting in "heads" approaches 1/2 with an increasing number of tosses. As a rule of thumb, with client affinity set, there must be many more clients than cluster hosts to begin to observe even load balance.

Page 32 of 46

As the statistical nature of the client population fluctuates, the evenness of load balance can be observed to vary slightly over time. It is important to note that achieving precisely identical load balance on each cluster host imposes a performance penalty (throughput and response time) due to the overhead required to measure and react to load changes. This performance penalty must be weighed against the benefit of maximizing the use of cluster resources (principally CPU and memory). In any case, excess cluster resources must be maintained to absorb the client load in case of failover. Network Load Balancing takes the approach of using a very simple but powerful load-balancing algorithm that delivers the highest possible performance and availability.

Network Load Balancing's client affinity settings are implemented by modifying the statistical mapping algorithm's input data. When client affinity is selected in the Network Load Balancing Properties dialog box, the client's port information is not used as part of the mapping. Hence, all requests from the same client always map to the same host within the cluster. Note that this constraint has no timeout value (as is often the case in dispatcher-based implementations) and persists until there is a change in cluster membership. When single affinity is selected, the mapping algorithm uses the client's full IP address. However, when class C affinity is selected, the algorithm uses only the class C portion (the upper 24 bits) of the client's IP address. This ensures that all clients within the same class C address space map to the same cluster host.

In mapping clients to hosts, Network Load Balancing cannot directly track the boundaries of sessions (such as SSL sessions) since it makes its load balancing decisions when TCP connections are established and prior to the arrival of application data within the packets. Also, it cannot track the boundaries of UDP streams, since the logical session boundaries are defined by particular applications. Instead, Network Load Balancing's affinity settings are used to assist in preserving client sessions. When a cluster host fails or leaves the cluster, its client connections are always dropped. After a new cluster membership is determined by convergence (described below), clients that previously mapped to the failed host are remapped among the surviving hosts. All other client sessions are unaffected by the failure and continue to receive uninterrupted service from the cluster. In this manner, Network Load Balancing's load-balancing algorithm minimizes disruption to clients when a failure occurs.

When a new host joins the cluster, it induces convergence, and a new cluster membership is computed. When convergence completes, a minimal portion of the clients will be remapped to the new host. Network Load Balancing tracks TCP connections on each host, and, after their current TCP connection completes, the next connection from the affected clients will be handled by the new cluster host; UDP streams are immediately handled by the new cluster host. This can potentially break some client sessions that span multiple connections or comprise UDP streams. Hence, hosts should be added to the cluster at times that minimize disruption of sessions. To completely avoid this problem, session state must be managed by the server application so that it can be reconstructed or retrieved from any cluster host. For example, session state can be pushed to a back-end database server or kept in client cookies. SSL session state is automatically recreated by re-authenticating the client.

The GRE stream within the PPTP protocol is a special case of a session that is unaffected by adding a cluster host. Since the GRE stream is temporally contained within the duration of its TCP control connection, Network Load Balancing tracks this GRE stream along with its corresponding control connection. This prevents the addition of a cluster host from disrupting the PPTP tunnel.

Page 33 of 46

ConvergenceNetwork Load Balancing hosts periodically exchange multicast or broadcast heartbeat messages within the cluster. This allows them to monitor the status of the cluster. When the state of the cluster changes (such as when hosts fail, leave, or join the cluster), Network Load Balancing invokes a process known as convergence, in which the hosts exchange heartbeat messages to determine a new, consistent state of the cluster and to elect the host with the highest host priority as the new default host. When all cluster hosts have reached consensus on the correct new state of the cluster, they record the change in cluster membership upon completion of convergence in the Windows 2000 event log.

During convergence, the hosts continue to handle incoming network traffic as usual, except that traffic for a failed host does not receive service. Client requests to surviving hosts are unaffected. Convergence terminates when all cluster hosts report a consistent view of the cluster membership for several heartbeat periods. If a host attempts to join the cluster with inconsistent port rules or an overlapping host priority, completion of convergence is inhibited. This prevents an improperly configured host from handling cluster traffic.

At the completion of convergence, client traffic for a failed host is redistributed to the remaining hosts. If a host is added to the cluster, convergence allows this host to receive its share of load-balanced traffic. Expansion of the cluster does not affect ongoing cluster operations and is achieved in a manner transparent to both Internet clients and to server programs. However, it may affect client sessions because clients may be remapped to different cluster hosts between connections, as described in the previous section.

In unicast mode, each cluster host periodically broadcasts heartbeat messages, and in multicast mode, it multicasts these messages. Each heartbeat message occupies one Ethernet frame and is tagged with the cluster's primary IP address so that multiple clusters can reside on the same subnet. Network Load Balancing's heartbeat messages are assigned an ether type-value of hexadecimal 886F. The default period between sending heartbeats is one second, and this value can be adjusted with the AliveMsgPeriod registry parameter. During convergence, the exchange period is reduced by half in order to expedite completion. Even for large clusters, the bandwidth required for heartbeat messages is very low (for example, 24 Kbytes/second for a 16-way cluster).

Network Load Balancing assumes that a host is functioning properly within the cluster as long as it participates in the normal heartbeat exchange among the cluster hosts. If other hosts do not receive a heartbeat message from any member for several periods of message exchange, they initiate convergence. The number of missed heartbeat messages required to initiate convergence is set to five by default and can be adjusted using the AliveMsgToleranceregistry parameter.

A cluster host will immediately initiate convergence if it receives a heartbeat message from a new host or if it receives an inconsistent heartbeat message that indicates a problem in the load distribution. When receiving a heartbeat message from a new host, the host checks to see if the other host has been handling traffic from the same clients. This problem could arise if the cluster subnet was rejoined after having been partitioned. More likely, the new host was already converged alone in a disjoint subnet and has received no client traffic. This can occur if the switch introduces a lengthy delay in connecting the host to the subnet. If a cluster host detects this problem and the other host has received more client connections since the last convergence, it immediately stops handling client traffic in the affected port range. Since both hosts are

Page 34 of 46

exchanging heartbeats, the host that has received more connections continues to handle traffic, while the other host waits for the end of convergence to begin handling its portion of the load. This heuristic algorithm eliminates potential conflicts in load handling when a previously partitioned cluster subnet is rejoined; this event is recorded in the event log.

Remote ControlNetwork Load Balancing's remote control mechanism uses the UDP protocol and is assigned port 2504. Remote control datagrams are sent to the cluster's primary IP address. Since the Network Load Balancing driver on each cluster host handles them, these datagrams must be routed to the cluster subnet (instead of to a back-end subnet to which the cluster is attached). When remote control commands are issued from within the cluster, they are broadcast on the local subnet. This ensures that all cluster hosts receive them even if the cluster runs in unicast mode.

Top of page 

Network Load Balancing Performance

The performance impact of Network Load Balancing can be measured in four key areas:

CPU overhead on the cluster hosts, which is the CPU percentage required to analyze and filter network packets (lower is better).

Response time to clients, which increases with the non-overlapped portion of CPU overhead, called latency (lower is better).

Throughput to clients, which increases with additional client traffic that the cluster can handle prior to saturating the cluster hosts (higher is better).

Switch occupancy, which increases with additional client traffic (lower is better) and must not adversely affect port bandwidth.

In addition, Network Load Balancing's scalability determines how its performance improves as hosts are added to the cluster. Scalable performance requires that CPU overhead and latency not grow faster than the number of hosts.

CPU OverheadAll load-balancing solutions require system resources to examine incoming packets and make load-balancing decisions, and thus impose an overhead on network performance. As previously noted, dispatcher-based solutions examine, modify, and retransmit packets to particular cluster hosts. (They usually modify IP addresses to retarget packets from a virtual IP address to a particular host's IP address.) In contrast, Network Load Balancing simultaneously delivers incoming packets to all cluster hosts and applies a filtering algorithm that discards packets on all but the desired host. Filtering imposes less overhead on packet delivery than re-routing, which results in lower response time and higher overall throughput.

Network Load Balancing's filtering algorithm runs in parallel on all cluster hosts. Filtering overhead can be measured as a percentage of the CPU used on all hosts. This overhead grows in proportion to the incoming packet rate (as does the routing overhead for dispatcher-based solutions), independent of the number of cluster hosts. For example, if a two-host cluster

Page 35 of 46

experiences P percent CPU overhead per-host to sustain a given incoming packet rate, this percentage will remain the same for a 32-host cluster. If the packet rate doubles, the CPU overhead grows to 2P.

In practice, hosts are added to a Network Load Balancing cluster in proportion to the request rate as the client load increases. When this is the case, the CPU overhead grows linearly with both the packet rate and number of hosts. In the example used above, a 32-host cluster would experience a per-host CPU overhead of 16P to sustain a 16-fold increase in request-rate and throughput over a two-way cluster with CPU overhead P.

Network Load Balancing's CPU overhead has been measured for Web loads. The chart in Figure 3 below shows CPU overhead versus total throughput on a two-host cluster with 450-megahertz (MHz) CPUs. The Web load consisted of HTTP GET requests, with each request pulling a 10-kilobyte (KB) static Web page. As an example, approximately 4.1 percent of each host's CPU (out of 45.7 percent total CPU per host) is required to filter 773 GET requests per second with 64.6 megabits per second (Mbps) throughput. This overhead, which is used to analyze and filter packets, was measured by setting the load percentage to zero on one of the two hosts and measuring the residual CPU percentage while the other host handled the entire load.

Figure 3: The CPU overhead required by Network Load Balancing to filter incoming packets as a function of total throughput for Web access to 10-KB static Web pages in a two-host cluster with 450-MHz CPUs.

The chart above shows that Network Load Balancing's filtering overhead grows linearly with packet rate as expected. At peak Fast Ethernet capacity of 100 Mbps, Network Load Balancing would require about 5.8 percent of a 450-MHz CPU to perform its packet filtering. Note that for multiprocessor hosts, this CPU percentage represents only a portion of the total CPU available on the host. Extrapolating from this chart, a cluster handling 250 Mbps total throughput would use 12.9 percent of a CPU on each host for packet filtering to serve 2,993 10-KB Web-page requests per second (or more than 258 million Web hits per day).

The ratio of CPU filtering overhead to throughput varies with the size and type of client requests. For example, at a given throughput level, GET requests for 1-KB Web pages will require more filtering overhead than do GET requests for 10-KB Web pages. Since Network Load Balancing's

Page 36 of 46

filtering overhead for tracking client connections is higher than for tracking packets within a connection, client requests that generate larger replies experience lower filtering overhead.

A second type of CPU overhead due to Network Load Balancing is its packet-handling overhead during data transfers, called transfer overhead. Because Network Load Balancing is implemented as an intermediate driver, it forwards all network packets flowing through the cluster adapter. This packet handling is heavily pipelined and adds very little latency to data transfers. Its CPU overhead is proportional to the packet rate for an individual host, not the total packet rate for the cluster. Hence, it remains fixed as cluster hosts are added in proportion to increases in client load.

Measurements of transfer overhead are shown in Figure 4 below. Transfer overhead was measured by comparing the total CPU percentage for a system running Network Load Balancing to the total CPU percentage for a system with Network Load Balancing disabled and subtracting out the filtering overhead. As an example, in a four-host cluster with 80 Mbps total throughput, each host would handle 20 Mbps throughput, which would require about 2.6 percent of the CPU.

Figure 4: The CPU overhead required by Network Load Balancing to transfer packets as a function of total throughput per host for Web access to 10-KB static Web pages using 450-MHz cluster hosts.

It is useful to regard a cluster as a set of CPU "containers" that are available to handle client load. Network Load Balancing distributes incoming client load among these containers in order to distribute the CPU load as evenly as possible. It also requires a small amount of CPU in each container to accomplish the load balancing, and this amount increases in proportion to the packet rate (with filtering overhead increasing in proportion to the total packet rate and transfer overhead increasing in proportion to the per-host packet rate). The total number of hosts needed in the cluster depends on the speed of the hosts and the characteristics of the server application. CPU-intensive applications, such as Web servers with Active Server Pages may require relatively large amounts of CPU per client request compared to Web servers pulling static Web pages. To meet CPU demand, they would handle less network traffic per host and require more hosts than would applications with lower CPU demand per client request. Figure 5 below shows a cluster of "CPU containers," each occupied in part by a server application load as well as the filtering and transfer overhead loads imposed by Network Load Balancing.

Page 37 of 46

Figure 5: Hosts are added to the cluster as needed to handle the CPU demand of client requests. Network Load Balancing uses a portion of the CPU on each cluster host for filtering and transfer overheads. Excess CPU capacity is reserved to handle extra client load in the event of host failures.

Throughput and Response TimeNetwork Load Balancing scales performance by increasing throughput and minimizing response time to clients. When the capacity of a cluster host is reached, it cannot deliver additional throughput, and response time grows non-linearly as clients awaiting service encounter queuing delays. Adding another cluster host enables throughput to continue to climb and reduces queuing delays, which minimizes response time. As customer demand for throughput continues to increase, more hosts are added until the network's subnet becomes saturated. At that point, throughput can be further scaled by using multiple Network Load Balancing clusters and distributing traffic to them using Round Robin DNS. For example, this approach is used by Microsoft's Web site, Microsoft.com, which currently has approximately five six-host Network Load Balancing clusters.

In practice, cluster hosts are added until all hosts are using a moderate amount of CPU at the anticipated request rate. Excess CPU capacity is maintained in each host to handle excess load after a failover. For example, the hosts for Microsoft.com typically run at about 60 percent of capacity so that they can handle host outages without difficulty.

All load balancers introduce overheads that impact throughput scaling and response time. For Network Load Balancing to scale application performance, it must not introduce bottlenecks that would constrain throughput as hosts are added. The overall throughput should increase linearly as hosts are added to the cluster and join in serving the client load. Network Load Balancing uses a highly pipelined implementation that minimizes the increase in response time, called latency, due to filtering and transfer overheads. As a result, only a small portion of Network Load Balancing's CPU overhead contributes to latency. The complexity of operating system interactions makes latency difficult to calculate directly. Preliminary measurements made during the previously described tests for accessing static, 10–KB Web pages indicated that the latency increase was approximately one percent of total response time (to receive the first byte of a GET request).

Network Load Balancing's latency and related CPU filtering overhead affect performance by eventually limiting the maximum possible throughput that can be obtained as packet rate increases. (CPU transfer overhead limits the peak throughput of a single host in comparison to a

Page 38 of 46

host with Network Load Balancing disabled but does not affect scalability.) Since a host's application service rate usually grows in proportion to its CPU usage, Network Load Balancing's CPU overhead has the effect of constraining the host's total throughput. Assuming that the client request rate grows with the number of hosts, throughput scaling for N maximally loaded cluster hosts is limited by growth in CPU filtering overhead. If R1 represents the maximum throughput on a single host, the maximum throughput per host in an N-host cluster can be calculated as:

RN = C * (1 – (N * (RN / R1)*OF)) where R1 = C * (1 – OF)

where OF is the CPU percentage required for Network Load Balancing to filter client requests at rate R1 and C is a constant relating service rate to its associated CPU usage. This filtering overhead grows with the increased request rate in proportion to the number of hosts (but limited by the feedback from RN / R1) and reduces each host's maximum service rate as hosts are added. Note that if OF = 0, the cluster produces linear N-fold increase in throughput, as expected. For applications with increasingly large CPU requirements per client request, Network Load Balancing will have lower CPU percentage overhead to sustain a given request rate, thus improving scalability as cluster hosts are added.

Figure 6 below shows throughput scaling by plotting the formula above for maximum throughput versus the number of hosts for various filter overheads, including 0 percent overhead (which yields the ideal N-fold increase in throughput). Measurements of GET requests for 10-KB static Web pages indicate that a 450-MHz CPU saturates at an approximate request rate of 70.1 Mbps (836 requests/second) with about 4.4 percent CPU filtering overhead. As shown in the graph, throughput rolls off rapidly as hosts running at this request rate are added to the cluster. However, if the application consumes four times the CPU load per client request, Network Load Balancing's filtering overhead drops to 1.1 percent, and throughput remains within 90 percent of the ideal N-fold throughput increase for 12 cluster hosts. With the same hosts, the request rate would be lower by four-fold. To maintain the request rate shown in the graph, four-way multiprocessor hosts could be used.

Figure 6: Network Load Balancing's calculated maximum throughput and comparison to ideal as cluster hosts are added. Throughput with 4.4 percent Network Load Balancing CPU overhead corresponds to a Web load of GET requests for 10-KB static Web pages using 450-MHz cluster hosts. Throughput with 1.1 percent Network Load Balancing CPU

Page 39 of 46

overhead corresponds to a four-fold increase in application CPU usage per client request and four-way symmetric multiprocessing (SMP) cluster hosts.

This throughput model closely matches measurements recently made on a 30-host cluster used as a demonstration during the launch of Windows 2000. This cluster ran Internet Information Server serving the home page for an actual stock quotation Web site. The home page consisted of a mix of text, graphics, and calculations using Active Server Pages. The cluster hosts consisted of dual processor Dell Power Edge 4350 systems with 512 MB of memory; 500 Dell client systems were used to generate load. Throughput measurements (GET requests per second) were taken for various cluster sizes They are shown in Figure 7 below along with the calculated maximum throughput curve using 1 percent CPU filtering overhead for one host. Network Load Balancing throughput remains close to the ideal N-fold increase for 10 hosts. With more than 10 hosts, throughput stays within 85 percent of the ideal for 20 hosts and 77 percent for 30 hosts. The maximum aggregate throughput for a 30 host cluster was approximately 18,000 GET requests per second (400 Mbps), or 1.55 billion hits per day, which corresponds to about ten times the client traffic rate that the actual Web site currently experiences. This demonstration showed that Network Load Balancing provides the scaled performance needed to handle very large Web sites.

Figure 7: Network Load Balancing's measured maximum throughput and comparison to calculated throughput curves as cluster hosts are added for the Windows 2000 launch demonstration. Throughput measurements closely match calculated throughput with 1 percent NLB CPU filtering overhead.

It is important to observe that the throughput curves above show the maximum throughput that can be achieved when all cluster hosts are saturated. This situation rarely occurs in practice because excess CPU capacity is always needed to handle extra client load after the loss of a cluster host. When the cluster runs at less than maximum throughput, Network Load Balancing's latency and CPU overhead do not reduce delivered throughput, and the cluster exhibits linear throughput growth. For example, Figure 8 below shows throughput measurements for a Web load of GET requests for 1-KB static web pages. The graph follows the ideal N-fold growth in throughput as both packet rate and the number of hosts increases. The total CPU load for each cluster host was under 20 percent in these measurements.

Page 40 of 46

Figure 8: Example of Network Load Balancing's linear increase in throughput for Web load of GET requests for 1-KB static Web pages using 450-MHz hosts.

Switch OccupancyNetwork Load Balancing's filtering architecture relies on the broadcast subnet of the LAN to deliver client requests to all cluster hosts simultaneously. In small clusters, this can be achieved using a hub to interconnect cluster hosts. Each incoming client packet is automatically presented to all cluster hosts. Larger clusters use a switch to interconnect cluster hosts, and, by default, Network Load Balancing induces switch-flooding to deliver client requests to all hosts simultaneously. It is important to ensure that switch-flooding does not use an excessive amount of switch capacity, especially when the switch is shared with computers outside the cluster. (Note that computers outside the cluster do not receive packets as a result of switch-flooding.) The percentage of switch bandwidth consumed by Network Load Balancing's flooding of client requests is called its switch occupancy.

For most Network Load Balancing applications, the bandwidth used for request traffic is a small fraction of the total bandwidth needed for client/server communication. Figure 9 below shows the percentage of each 100 Mbps switch port consumed by client GET requests for 10-KB Web pages at various throughput rates. The chart confirms that client requests use less than 2 percent of switch bandwidth for this Web load.

Page 41 of 46

Figure 9: Percentage of 100 Mbps switch ports used by client GET requests for 10-KB static Web pages.

Switch-flooding can become a problem in applications with a significant percentage of network traffic directed to the cluster (such as file uploads in FTP applications). It can also become a concern when multiple clusters share a switch and their combined flooding becomes significant. Finally, if a switch is configured to use ports to connect to the backbone network that have a higher speed than those used to connect to cluster hosts, switch occupancy can become prohibitively high. For example, if the switch uses gigabit ports to connect to the backbone and 100 Mbps ports for the cluster, switch occupancy can increase by a factor of ten. In the example above, at 500 Mbps total bandwidth, switch occupancy would increase to 10 percent. In these cases, running Network Load Balancing in multicast mode and setting up a virtual LAN in the switch can limit switch-flooding.

Top of page 

Summary

Internet technology has been widely embraced, serving as the foundation for delivering enterprise-wide and frequently mission-critical applications such as Web, streaming media, and VPN servers. As an integral part of Windows 2000 Advanced Server and Datacenter Server, Network Load Balancing provides an ideal, cost-effective solution for enhancing the scalability and high availability of these applications in both Internet and intranet contexts.

Network Load Balancing lets system administrators build clusters with up to 32 hosts among which it load-balances incoming client requests. Clients are unable to distinguish the cluster from a single server, and server programs are not aware that they are running in a cluster.

Network Load Balancing gives network administrators excellent control, including the ability to remotely manage (with password protection) the cluster from any point on the network. Network administrators also have the ability to tailor clusters to specific services, with control defined on a port-by-port level. Hosts can be added to or removed from a cluster without interrupting services. In addition, they can upgrade software on cluster hosts without interrupting services to clients.

Network Load Balancing uses a fully distributed algorithm to partition client workload among the hosts. Unlike dispatcher-based solutions, this architecture delivers very high performance by reducing the overhead required to distribute client traffic. It also offers inherently high availability with (N-1)-way failover in an N-host cluster. All of this is accomplished without the expense and support burden of using special purpose hardware or software.

Network Load Balancing emits periodic heartbeat messages so that all members of a cluster can monitor the presence of other hosts. Host failures are detected within five seconds, and recovery is accomplished within ten seconds. Should a host be lost or a new one brought online, the workload is automatically and transparently redistributed among the cluster hosts.

Performance measurements have shown that Network Load Balancing's efficient software implementation imposes very low overhead on network traffic-handling and delivers excellent performance-scaling limited only by subnet bandwidth. Network Load Balancing has

Page 42 of 46

demonstrated more than 200 Mbps throughput in realistic customer scenarios handling e-commerce loads of more than 800 million requests per day.

For More InformationFor the latest information on Windows 2000 Server, please visit our Web site at http://www.microsoft.com/windows2000 and the Windows NT Server Forum in the MSN™ network of Internet services athttp://computingcentral.msn.com/topics/windowsnt.

Top of page 

Glossary of Key Terms

ARP: Address Resolution Protocol. This is a TCP/IP protocol that resolves IP addresses used by TCP/IP-based software to media access control addresses used by LAN hardware.

Availability: A measure of the fault tolerance of a computer and its programs. A highly available computer system provides continuous service without interruptions due to software or hardware failures.

Client affinity: A configuration option for the multiple-host filtering mode within a port rule that specifies whether Network Load Balancing should direct multiple requests from the same client IP address or class C address space to the same cluster host. Three affinity settings are possible: none, single client, and class C.

Client request: A service request from a client computer to a server computer or a cluster of computers. Network Load Balancing forwards each client request to a specific host within a cluster according to the system administrator's load-balancing policy.

Cluster: A set of computers that work together to provide a service. The use of a cluster enhances both the availability and scalability of the service. Network Load Balancing provides a software solution for clustering multiple computers running networked client/server applications.

Cluster adapter: The network adapter in a Network Load Balancing cluster that handles the network traffic for cluster operations (that is, the traffic for all hosts in the cluster). This adapter is assigned one or more virtual IP address and optionally a dedicated IP address.

Convergence: A process by which Network Load Balancing hosts exchange messages to determine a new, consistent state of the cluster and to elect the host with the highest host priority, known as the default host. During convergence, a new load distribution is determined for hosts that share the handling of network traffic for specific TCP or UDP ports.

CPU filtering overhead: A performance measure defined here as Network Load Balancing's CPU percentage on a cluster host required to analyze and filter network packets (lower is better).

CPU transfer overhead: A performance measure defined here as Network Load Balancing's CPU percentage on a cluster host required for it to transfer network packets through a cluster host.

Dedicated IP address: A cluster host's unique IP address used for network traffic not associated with the cluster (for example, Telnet access to a specific host within the cluster).

Page 43 of 46

Default host: The host with the highest host priority and which handles all of the network traffic for TCP and UDP ports that are not otherwise covered by port rules.

Filtering mode: A configuration setting for a port rule that determines load-balancing policy for the range of ports covered by the port rule. There are three possible filtering modes: multiple host, single host, and disabled.

Handling priority: A configuration setting in single-host filtering mode that specifies a host's priority for handling all of the cluster's network traffic for that port rule. Handling priority overrides host priority for the range of ports covered by the port rule.

Heartbeat Message: A network packet periodically broadcast by each cluster host to inform other hosts of its health and configuration. Network Load Balancing initiates convergence when it fails to receive heartbeat messages from another host or when it receives a heartbeat message from a new host.

High Availability: See availability.

Host: As used here, a computer that participates in a Network Load Balancing cluster. A host is also called a server.

Host priority: A configuration setting that specifies a cluster host's priority for handling all of the cluster's network traffic not covered by port rules. The host with the highest priority (lowest numerical value in the range of 1 to 32) is called the default host.

Latency: A performance measure defined here as the non-overlapped portion of Network Load Balancing's CPU overhead (lower is better). Latency adds to the client response time.

Load balancing: A technique for scaling performance by distributing load among multiple servers. Network Load Balancing distributes load for networked client/server applications in the form of client requests that it partitions across multiple cluster hosts.

Load weight: A configuration setting for the multiple-host filtering mode within a port rule that specifies the percentage of load-balanced network traffic that this host should handle; allowed values range from 0 (zero) to 100. The actual fraction of traffic handled by each host is computed as the local load weight divided by the sum of all load weights across the cluster.

MAC address: A link-layer network address, called a media access control address that is used to communicate with other network adapters on the same subnet. Each network adapter has an associated MAC address.

Multicast MAC address: A type of media access control address used by multiple, networked computers to concurrently receive the same incoming network packets. In multicast mode, Network Load Balancing optionally uses multicast MAC addresses to efficiently distribute incoming network traffic to cluster hosts.

Multicast mode: A configuration setting which instructs Network Load Balancing to add a multicast MAC address to the cluster adapters on all hosts in a cluster. The adapters' existing MAC addresses are not changed. See also unicast mode.

Multihomed: A computer that has multiple network cards or that has been configured with multiple IP addresses for a single network card. Network Load Balancing supports multihomed servers by allowing multiple virtual IP addresses to be assigned to the cluster adapter.

Page 44 of 46

Network adapter: A plug-in board that connects a computer to a local area network (LAN). A Network Load Balancing cluster communicates using a cluster adapter over the LAN connecting the cluster's hosts to clients.

Overhead: A performance measure defined here as the CPU percentage on a cluster host used by Network Load Balancing, consisting primarily of CPU filtering overhead and CPU transfer overhead. A portion of overhead contributes to latency.

Port rule: A configuration setting within the Network Load Balancing Properties dialog box that specifies how cluster network traffic for a range of ports is to be handled. The method by which a port's network traffic is handled is called its filtering mode.

Primary IP address: A virtual IP address used by Network Load Balancing to identify the cluster for remote control operations and within heartbeat messages. See virtual IP address.

Response time: A performance measure defined as the round-trip delay to process a client request. Response time increases with the non-overlapped portion of CPU overhead, called latency (lower is better).

RRDNS: A type of domain name service, called Round Robin DNS, which distributes clients to multiple servers in a round-robin manner. RRDNS scales performance but does not provide high availability.

Scalability: A measure of how well a computer, service, or application can grow to meet increasing performance demands. For clusters, scalability is measured by the ability to incrementally add one or more systems to an existing cluster when the overall load of the cluster exceeds its capabilities.

Session: In the context of load balancing TCP/IP traffic, a set of client requests directed to a server. These requests can be invoked with multiple, possibly concurrent, TCP connections. The server program sometimes maintains state information between requests. To preserve access to the server state, system administrators can have Network Load Balancing direct all requests within a session to the same cluster host when load balancing by setting client affinity.

State: As used here, data maintained by a server application on behalf of its clients. To scale a server application by running multiple instances with Network Load Balancing, client state must be accessible and properly coordinated by all instances of the application.

Switch: A central network device that forwards packets to specific ports rather than, as in conventional hubs, broadcasting every packet to every port. Switches can deliver higher total bandwidth than can hubs by supporting multiple, simultaneous connections.

Switch-flooding: A switch behavior induced by Network Load Balancing to simultaneously deliver all incoming client packets to all switch ports. This behavior enables Network Load balancing to deliver very high throughput, but it may cause high switch occupancy.

Switch occupancy: A performance measure defined here as the fraction of a switch port's bandwidth occupied by Network Load Balancing's incoming client requests (lower is better). Switch occupancy increases with additional client traffic and must not adversely impact port bandwidth.

TCP/IP: Transmission Control Protocol/Internet Protocol. A set of networking protocols that is used on the Internet to provide communications among interconnected networks made up of

Page 45 of 46

computers with diverse hardware architectures and various operating systems. TCP/IP includes standards for how computers communicate and conventions for connecting networks and routing traffic.

Throughput: A performance measure defined here as the number of client requests processed by a Network Load Balancing cluster per unit time (higher is better). Throughput increases with additional client traffic that the cluster can handle prior to saturating its hosts.

Unicast mode: A configuration setting which instructs Network Load Balancing to change the MAC address of the cluster adapters to the same value for all hosts in a cluster. This is the default mode of operation. See also multicast mode.

Virtual IP address: An IP address that is shared among the hosts of a Network Load Balancing cluster and used by clients to address the cluster as a whole. A Network Load Balancing cluster supports multiple virtual IP addresses, such as in a cluster of multihomed Web servers. One of the virtual IP addresses is the primary IP address, which is used to identify the cluster for remote control operations.

Page 46 of 46