New Generation of Cisco Switching

7/27/2019 New Generation of Cisco Switching

1/50

New Generation of Cisco Switchingby Howard Berkowitz

IntroductionOld and New Models: Hierarchical, SAFE, and ECNM

"Is it SAFE?"What to study -- and Not to StudyNew Paradigms and MetaphorsCisco's Switch Product PositioningFailover Requirements

CertificationZone Subscribers ShouldWhat does this mean in the context of switches?

Availability TerminologyPaging Mr. Murphy

Selecting Recovery StrategiesCost and Complexity in Selecting StrategiesRecovery Time Requirements in Selecting Strategies1:N, 1:1, and 1+1 Protection Strategies

Switch Platform Architecture: A ModelPractical Issues: What Are Ports?Management

HardwareSoftware

ControlForwarding Tables and Populating Them

ForwardingIngress Buffering and Processing

Pattern Recognition

Advances in Forwarding Tables: CAM and TCAMIntroducing Ternary Tables

TemplatesFor further details...Forwarding models

FabricShared BusShared MemoryCrossbar

Egress ProcessingQoS at the Switch

Interfacing: the GBIC (Gigabit Ethernet Interface Converter)Characterizing Switch PerformanceThroughput

BlockingOutput Blocking

Grandfather Switch: Catalyst 5x00 Platform FamilyStacking and Clustering: 3750 and 2950Midrange Flexibility: Catalyst 3550 Platform FamilyA New Interface Paradigm

Hardware Aspects of Voice PortsManagement and ControlForwarding

Catalyst 4000/4500 Platform Family

Tutorial

Page 1 of 50Certification Zone - Tutorial

5/31/2005http://www.certificationzone.com/cisco/studyguides/component.html?module=studyguides...


2/50

Management and ControlForwarding

Catalyst 6000/6500 Platform FamilyManagement and Control

Database ManagerForwarding

Switching Functions for High AvailabilityLayer 1/2 High Availability for Links and Interfaces

Layer 1 FailoverSONET and POS

Unidirectional Links: Detection Protocol (UDLD) and configuring Unidirectional EthernetLayer 2 AggregationPreventing Broadcast Storms

Other Layer 2 Security and Management EnhancementsPrivate VLANs802.1x -- Port Based AuthenticationDHCP-related Security Features

Growing Frames beyond Normal SizeSingle Spanning Tree High Availability

Layer 2 TracerouteCore/Backbone Switch FailureIndirect Root FailuresRoot WarsDistribution Switch Failure

Performance Enhancements to Individual Spanning TreesIEEE 802.1w Rapid Spanning Tree Protocol (RSTP)

Port Types in 802.1d and 802.1wPort States in 802.1d and 802.1wPortFast, BPDU Guard, and 802.1w Functional Equivalence

Root Wars and Root GuardSTP Convergence Time

Performance Enhancements to Multiple Spanning TreesMSTP: Subdividing the Spanning Tree for Faster Convergence

MSTP RegionsIST, CIST, and CST

VLAN Tagging and VLAN Trunk Protocol (VTP)VTP Pruning

VLAN-to-Spanning Tree RelationshipsPVST

ConclusionReferences

Introduction

While most of the focus of this paper is on L2 switching, there is a significant amount on thearchitecture and implementation of "L3 switching". L3 switching is really routing, but the term L3switching has tended to become associated with implementation techniques that do much of the workin specialized hardware.

Please, please don't get confused by trying to see how L3 switching is somehow different, in basicprinciples, from routing. It isn't. At worst, it's purely a marketing term; at best, it emphasizes certainimplementations. There's no accident that the Cisco 12000 is called the Gigabit Switch Router (GSR),because it makes extensive use of hardware processing. Since it's targeted at a WAN and ISP market,however, Cisco doesn't designate it a switch to avoid confusion with enterprise and server farm relays.

This particular paper has many cross-references to other CertificationZone tutorials, and for goodreason. The focus here is how a switch does something, while such things as QoS, high availability, andsecurity tutorials define why something is done.




3/50


4/50

you are studying for the CCNPSwitching or CCIE Writtenexaminations, you need toknow about platforms that arenot in the CCIE lab. The 6500switch, for example, is Cisco'sflagship product for largeenterprises and internal usewithin ISPs. It has someunique features on which youmight be tested.

For many switches, you willneed to recognize that there isa product family that includesmore than one numberedseries. For example, the 4000series switches are modular,but the 2948G switches arevery similar devices whose configurations are fixed.

Table 1. General Positioning Model for Enterprise Switches

You will find switches positioned for different functions, and for the same function within organizationsof different size. Fixed configuration platforms are most associated with the smaller enterprises, but

they also can be quite useful as aggregation platforms inside larger enterprises.

New Paradigms and Metaphors

Many of Cisco's earlier switches are the result of acquisitions, although modern switches are designedand manufactured by Cisco. As a result, there was a confusing assortment of operating systems andhuman interfaces across platforms. The "Catalyst Interface", for example, came from Cisco's acquisitionof Catalyst.

Table 2. Switch Operating Systems and their Interfaces

Real consolidation and a clear picture of future trend came with the introduction with the 3550 and itsIOS-based interface. This interface has considerable QoS capability, especially important for Cisco

Internet-Draft I coauthored, whichhopefully will soon move to RFC,"Terminology for Benchmarking BGPDevice Convergence in the ControlPlane", http://www.ietf.org/internet-drafts/draft-ietf-bmwg-conterm-05.txt,where we draw a distinction between twofunctions in the Cisco "distribution tier",

the "provider edge router" and the"inter-provider border router," asopposed to the "subscriber edge router".This distinction, while informal, capturessome of the flavor of Cisco's "campusedge". While not listed as an officialcoauthor because we weren't allowed tolist more than five coauthors, AlvaroRetana of Cisco was part of the teamthat wrote this document.

Enterprise size Wire closet Backbone

Small Fixed configurationModular

Fixed configurationModular

Midrange Fixed configurationModular

Modular

Large Modular Modular

Operating System Interface Comments

CatOS 4000, 5000, 6000

Native OS 2950, 3550, 4000 Sup 3, 6000 MSFC

Hybrid Cat OS + IOS on MSFC (5x00)

IOS Routers, MSFC




5/50

AVVID (Architecture for Voice, Video, and Integrated Data) use.

Cisco's Switch Product Positioning

Table 3. The View in 1999

Table 4. Qualifying the 1999 view for Enterprise Size

Table 5. The View in 2003

Failover Requirements

Selecting the appropriate level of availability is as much abusiness as a technical decision. In her book Planning forSurvivable Networks, Annlee Hines has written extensively

on the basis of these decisions. If you ever plan torecommend real network designs rather than simply passtests, read her book! [Hines 2002]

My WAN Survival Guide [Berkowitz 2000] discusses some of these cost-benefit trade-offs from theenterprise standpoint, and my Building Service Provider Networks [Berkowitz 2002] looks at the trade-offs from the service provider viewpoint.

Table 6. Broad Goals for High Availability [Berkowitz 2000]

Wire closet Server farm Core

2900/4000, 5000 6000 8500

Enterprise size Wire closet Backbone

Small 2900XL/2948G 50xx/55xx

Midrange 4000,5000 55xx

Large 55xx 6000/85xx

Wire closet Server farm Core

2900/4000, 5000 4000,6000 6500

CertificationZoneSubscribers Should

See my High Availability tutorial foradditional details.

AvailabilityLevel

Server Network

1 "Do nothing special"Backups Locked network equipment

2 "Increased availability: protect the data"Full or partial disk mirroring, transactionlogging

Dial/ISDN backup

3 "High availability: Protect the system"Clustered servers

Redundant routersNo single-point-of-failure local loop

4 "Disaster recovery: protect the No single-point-of-failure national




6/50

High availability involves a great many cost trade-offs, some of which are "Layer 8" business ratherthan technical considerations.

Table 7. Costs of High Availability Mechanisms

If you choose to "pay me later" and accept failures, what are some of the costs of failures when theyoccur?

Table 8. Costs of Lack of Availability

Radia Perlman's doctoral thesis [Perlman 1988] was on the"Byzantine generals problem". She demonstrated that

adding more network elements during certain kinds offailures not only does not increase availability but actuallydecreases it. The theoretical problem deals with asituation where the decision maker receives conflictinginformation from multiple sources, some of which is knownto be untrue -- but it is not known which information isuntrue. Sounds familiar from mutual redistributionproblems, hmm? It applies to most routing mechanismsand related mechanisms such as Layer 2 spanning trees.

Availability Terminology

Remember that the CCIE written exam is more concerned with protocol theory and features than

specific configuration of routers to use them. This section will give you a good deal of informationrelevant to the theory of many protocols. For more detail, see the High Availability tutorial.

We often speak of single points of failure. Multiprotocol Label Switching (MPLS) has refined thatdefinition into the shared risk group (SRG). The basic definition of an SRG is "a set of network elementsthat will be affected by the same fault".

SRGs can apply to all sorts of network resources, and a given resource can belong to more than oneSRG. A shared risk group of routers might be all of those on a common electrical power supply.

organization"Alternate server sites

backbone

Direct Indirect

Backup equipment Design

Additional lines/bandwidth Network administrator time due to additionalcomplexity; higher salaries for higher skills

Floor space, ventilation, and electrical powerfor additional resources

Performance drops due to fault tolerance overhead

Direct Indirect

Revenue loss Lost marketing opportunities

Overtime charges for repair Shareholder suits

Salaries of idle production staff Staff morale

What does this mean in thecontext of switches?

Depending on the specific switch model,you may have any or all of the features:

z Redundant processors/supervisors

z Redundant/load sharing powersupplies

z Hot-swappable line cards




7/50

Table 9. Basic Shared Risk Groups

One of the classic SRGs is the common cable or cable duct that gets cut by construction workers. Whilebuilding alternate cable runs to the telco end office historically is prohibitively expensive, new Ciscotechnology gives you some creative alternatives.

It may not be expensive, balanced against the cost of downtime, to run a wireless LAN from your mainrouter to a router in a nearby building. That alternate router would connect to the end office, at thevery least, via a different cable, and ideally would connect to an entirely different office. The bandwidthavailable to you from one wireless LAN, or a small number of parallel wireless LANs, usually will becomparable to your normal WAN uplink.

When the WAN bandwidth requirements are substantial, you still can get laser or wireless links fromnon-Cisco vendors, providing short-haul bandwidth up to OC12 (622-Mbps) rates.

Paging Mr. Murphy

Murphy's First Law states "Whatever can go wrong, will." His Second Law says "What has gone wrongwill get worse." High availability measures will never be able to deal with every possible Murphy case.

MPLS protocol designers do try to deal with most Murphy cases, and do a much more extensive job

than in other protocols.

As a result, the Framework for MPLS Recovery [RFC3469] first approaches the problem of single link(group) failures between network elements, generalizing this model to single interface and single routerfailures. The latter two are equivalent to SRG failures. My High Availability Tutorial goes into more detailthan we can fit here.

Other failure modes not considered here include congestion from broadcast storms and the like,Byzantine errors, host or host link errors, etc.

Illegal protocol packets or hardware failures clearly are error events. Inopportune events impact highavailability as well. A good example of an inopportune event is the arrival of one or more errornotifications, or explicit restart/recovery messages, while recovery or restart is in progress.

Selecting Recovery Strategies

Approaches to recovering from failures depend on whether tight resource control in the network isneeded, as, for example, where bandwidth is explicitly allocated to meet QoS actions. If overall controlof this sort is needed, there may need to be a central (or distributed) network management element,usually called a head end.

Before selecting a technology, know your tolerance for outages and your budget. This discussion

Layer Hardware Software

Infrastructure Commercial power

Physical Cable in common duct, single shared medium

Data Link Cables in common multilink bundle

Network Router Routing software session/instance

Transport TCP software

Application Single DNS server




8/50

assumes that the recovery technology does have sufficient resources to protect against at least a singlefailure without human intervention.

Outside the scope of this discussion are failures where mean time to repair (MTTR) is significantbecause it requires human intervention, possibly at unmanned sites, and possibly where spares need tobe shipped in.

You must, however, always remember why you want a particular level of survivability and build against

the defined requirements. Designers and, unfortunately, traditional telephony people often use the 50-ms cutover goal of SONET as the gold standard. This number is derived from SS7 characteristics oflarge carrier networks. VoIP is much more tolerant of drops, tolerating 140 ms to 2 s.

Cost and Complexity in Selecting Strategies

Part of the cost of any recovery strategy is the cost of resources that do not routinely carry operationaltraffic but are devoted to backup. Such resources are assumed in 1:N, 1:1, and 1+1, and possibly localrepair models. Dynamic discovery does not make this assumption. The more resources committed, themore expensive the solution.

See Table 11 for a summary of recovery strategies, which are detailed in subsequent subsections of this

discussion. Local restoration and reversion, also discussed below, can apply to any of the modes of thistable.

Another consideration is whether the recovery must consider end-to-end performance. All of thesestrategies provide restoration, but may or may not provide reversion. In restoration, the highavailability system has done its job when the failed resource is replaced by another. In reversion, thehigh availability system also needs to restore the original conditions of resources after the failure isfixed.

Another consideration is whether a backup resource needs to be found for the new working resource.Reversion implies, to some extent that the original resource backs up the new working resource, butthe risk of the original resource being down may make that inadequate.

End-to-end recovery needs to know about SRGs. It needs to know that a recovery action will eitherminimize the number of resources put into an SRG in which a failure occurred or completely avoid thatSRG. Local repair is not aware of end-to-end recovery.

Recovery Time Requirements in Selecting Strategies

Data protocols that are extremely timing-critical are becoming uncommon, such as IBM SystemNetwork Architecture (SNA) without local acknowledgement and DEC Local Area Transport (LAT).

MPLS work on availability has produced a generally useful list of timers, generalized here for IP as wellas MPLS (Table 6).

Table 10. Failure Detection Timers

Failure orDegradationType

MPLS Definition IP Routing Definition

Path Failure (PF) Recovery mechanisms have decidedthe path has totally lost connectivity.

BGP or IGP route withdrawal or loss ofkeepalives at a lower layer.

Link Failure (LF) MPLS recovery mechanisms have been Typically implementation-specific,




9/50

You may have real-time applications such as telepresence, telemetry, etc. that must have predictabledelay. Delay may also be a commercial differentiator for competitive offerings of mission-criticalbusiness applications such as automatic teller machines, credit authorization, and transaction-basedInternet commerce.

1:N, 1:1, and 1+1 Protection Strategies

In order of strength of protection (and cost), there are three basic modes for media/link protection:1:N, 1:1, and 1+1. These modes dedicate backup resources. An additional mode, dynamic discovery,assumes resources are there but does not pre-allocate them.

Do remember that to use some of these strategies, you will have to have physical topologies that makethe backup resource in physical proximity to the working resource.

Table 11. Protection types

Both 1:Nand 1:1 schemes may use the backup resource for lower-priority traffic, which can instantly

be pre-empted if the working resource fails.

1+1 protection adds application complexity, because the applications need to be able to decide whichcopy of information should to be used. In switches, you may see it in cases where Cisco NonstopForwarding supports a hot-standby processor. 1+1 is very rare in networking. You will see it in SS7telephony control networks, but it is not used extensively in enterprise networking.

Switch Platform Architecture: A Model

informed of a lower-layer total failure. although OSPF does have a specificnotification abstraction, especially fordemand circuits. Usually associated withan SNMP trap.

Fault IndicationSignal (FIS)

A signal repeatedly transmitted that afault along a path has occurred,passed along the path until it reaches

a network element capable ofinitiating recovery.

BGP or IGP withdrawal route. Generallyconsidered poor practice to announceperiodically.

Fault RecoverySignal (FRS)

Indication that a fault along a workingpath has been repaired.

BGP or IGP re-announcement of previouslywithdrawn route.

Type Description Example

1:N one backup resource for Nworking resources, N> 1 One extra link in anEtherChannel bundle

1:1 dedicates a backup resource for each workingresource.

dual ring FDDI

1+1 sends identical data on both links, so that the data isimmediately available in the event of failure

Hot standby power supplies andsupervisor engines, SS7 datalink protocol, SSCOP

Dynamicdiscovery

Relies on sufficient statistical redundancy thatrouting protocols can find a non-dedicated backuppath. This may involve determining a new multi-hoppath.

Spanning tree, L3 routing




10/50

You can look at a switch abstractly as a relay. Relays are devices with at least two interfaces, whichaccept data on one interface and send it out another. A range-extending repeater, operating at thephysical layer, is the simplest type of relay, with only one input and one output.

Ethernet hubs are still relays, although they copy the data onto an internal shared medium and fan thecontents of that medium to all other ports. You really can't get a good sense of relays until layer 2,when the platform software has to make a decision regarding which egress interface to use.

While Cisco likes to talk about frames vs. packets vs. segments vs. messages, doing so is not correctOSI terminology. OSI formalism sometimes is very pedantic, but some of its terminology can be veryprecise and unambiguous.

OSI documents speak not of specifically named units at every layer (e.g., frame at layer 2), but ofProtocol Data Units (PDU). At a specific layer, you speak of Transport PDUs or Data Link PDUs. Anotheruseful concept, especially when dealing with protocol encapsulation, is the layer above the current layeris called (N+1) while the layer below is (N-1). From the perspective of the network layer, it receives(N+1)PDUs from Transport, and sends out (N-1)PDUs to Data Link.

A relay, which is a term from the formal (yes, that's the way it's spelled), is a device (or softwarefunction) with at least two interfaces. It receives PDUs on one interface and de-encapsulates them untilit has the information on which it will make forwarding decisions. Ignoring devices such as multilayer

switches, devices such as bridges and LAN and WAN switches accept physical layer bits, build them intoData Link PDUs, and make forwarding decisions on information at Data Link.

Routers receive bits, form frames, and extract Network PDUs from the Data Link PDUs. After examiningNetwork Layer information, they internally forward Network PDUs to an outgoing interface, and thenencapsulate these into Data Link PDUs and then Physical Layer information.

To make any of these forwarding decisions, the relay must first have an association betweendestination (and possibly other) information in the PDU at which it makes decisions, and informationabout the appropriate outgoing interface. The process of learning these associations is pathdetermination. In bridges and LAN switches, path determination involves the spanning tree protocol,VLAN protocols, and source routing. In routers, path determination involves static and dynamic routing,as well as the up/down state of hardware interfaces.

Practical Issues: What Are Ports?

Ports, in general, are the physical connectors to which you can connect clients, servers, or switches to aswitch. There are virtual ports, but they are beyond the scope of this discussion.

A line card can have one or more ports.

Through manual configuration, autoconfiguration, and hardware mechanisms, a port can take on manyroles.

Table 12. Physical Port Types

Port type Attributes

Static No filtering and may be assigned to a VLAN based on physical port ID.

Dynamic Assigned to a LAN based on frame contents and the definitions in the VLAN PolicyManagement Server (VPMS)

Secure Has a MAC address filter

Trunk Runs 802.1q, 802.1v or ISL




11/50

Don't confuse the physical port types in Table 12 with the spanning tree port types in Table 35. A portcan have both a physical type and a spanning tree type.

Management

In a relay, the management function is concerned with building the forwarding "map", whether that is aspanning tree at OSI Layer 2, a routing table at Layer 3, or content switching tables at higher layers.Other functions include exception processing such as ICMP, running routing and spanning treeprotocols, etc.

Management obviously includes the automated management functions (e.g., TFTP, logging) and thehuman interface.

Hardware

Management functions usually are implemented in general-purpose processors. As performancerequirements grew more stringent, the processor often was a Reduced Instruction Set computer (RISC)design rather than a Complex Instruction Set (CISC) design.

Under some conditions, forwarding uses the same processor as is used for management.

Software

Management is primarily a software function. Clearly, this is the role of the human interface, be ittextual or Web-oriented, or be it any of the different switch operating systems.

Control

Control software runs management functions, including the human interface, as well as topologylearning with spanning tree and dynamic routing protocols.

Forwarding Tables and Populating Them

On router platforms, forwarding tables began with the routing table, that which you see with a show ip

route. This table, more formally called the Routing Information Base, is optimized for adding and

deleting routes. That optimization benefits control, but not forwarding efficiency.

In contrast, the tables used in the high-speed forwarding path are optimized for fast lookup, and are

populated from data in the RIB. While the generic computer science term for this fast-lookup datastructure is the Forwarding Information Base (FIB), Cisco uses the term cache and FIB a bit differently.The first cache example was the fast switching cache, which is a data structure in the main RAM, whichhas fewer entries than in the RIB. A fast lookup algorithm, such as hashing, is used.

First-generation fast lookup tables had to be rebuilt whenever an entry was added or deleted. Partialupdating was not practical. You could, as a result, see drops in performance whenever there was a"cache fault", or an attempt to look up a destination not present in the cache. For fast switching and itsequivalent hardware assisted variants, autonomous switching (AGS+ and early 7000) and siliconswitching (7000 with RSP), cache faults could significantly affect performance. These distributed caches

Source SPAN Source of traffic to be sent to the SPAN monitoring port

DestinationSPAN

Port associated with SPAN analysis (e.g., RMON)




12/50

were quite small, either 512 or 1024 entries. This small number of entries worked acceptably in anenterprise, which typically has a moderate number of frequently used routes, but was a severeperformance limitation in ISP routers.

Distributed switching on VIPs was a major performance advance, because the VIP FIB has a one-to-onecorrespondence with the RIB. With this correspondence, there never will be a cache fault.

ForwardingAt a general level, let's consider the forwarding modes, also called switching paths, in Cisco platforms.

Table 13. L2 switching modes

Table 14. L3 switching modes

Ingress Buffering and Processing

As long as the fabric is non-blocking, there is no need for input buffering. It is possible that buffers willbe required when doing traffic shaping at the ingress.

At the most basic, ingress processing looks up the destination address in the frame or packet header,selects the egress interface, and moves the frame or packet to the fabric. If the fabric is blocking, thepacket may go into a buffer.

In all cases where I am familiar with router or switch internals, the ingress processor prefixes the frameor packet with an internal header used by the fabric to send it to an appropriate egress interface(s).Such headers are never seen outside the platform.

Pattern Recognition

Ingress processing, in the real world, gets complicated by frequent requirements to recognize patternsin the packet or frame, patterns other than the destination. Among the most common is what wegenerically call an access control list (ACL), which checks certain fields, usually with a mask that

Switching mode Speed MIB:RIB Relationship

"Software" Slowest but most intelligent MIB and FIB are the same.

"Hardware" -- CAMfor L2

Default mode and mostcommon at layer 2

May be centralized or distributed. Uses ContentAddressable Memory requiring an exact match

"Hardware" -- TCAM

for L2 and L3

Good compromise between

speed and intelligence

May be centralized or distributed. Uses one or

more Ternary Content Addressable Memories

Switching mode Speed MIB:RIB Relationship

Process switching Slowest but most intelligent MIB and FIB are the same.

Fast switching Default mode, faster than process FIB is in RAM, and is smaller thanthe RIB.

Autonomous, silicon,optimum

Fast, hardware-assisted and platform-dependent

FIB is in special hardware, and ismuch smaller than the RIB.

Express Fastest, especially when distributed intomultiple Versatile Interface Processors

FIB is a full copy of the RIB.




13/50

indicates whether the value of a bit is to be checked, or if the pattern will accept any bit value in thatposition (i.e., wild card).

When you consider wild cards as well as a bit being one or zero, you introduce ternary logic, a stepbeyond a simple binary on-or-off decision.

Cisco now describes the individual lines in an ACL as access control entries (ACE). You can recognizepatterns, at L2 and L3, for various reasons, including security filtering, special routing (e.g., source

routing) or QoS recognition and marking.

Advances in Forwarding Tables: CAM and TCAM

One of the challenges to wire-speed forwarding is how quickly destination information can be retrievedfrom an address table. In L2 switches, this historically was the job of the Content Addressable Memory(CAM), and now the job of the Ternary Content Addressable Memory (TCAM). The TCAM has both L2and L3 fast lookup capability, as opposed to the Forwarding Information Bases in router VersatileInterface Processors (VIP) or the forwarding part of a Route Switch Processor (RSP).

Router FIBs, however, hold considerably more routes than a TCAM, a necessity for service providers.

Early switches used a CAM to look up destination MAC addresses, CAM had far fewer entries than mostrouter cache or FIB, which often was acceptable given the scope in which a switch worked.

In a CAM, you must match on every bit of a MAC address, even if some of them, such as the first 24bits of vendor ID, are not significant for the particular lookup.

Introducing Ternary Tables

TCAMs, however, can "wildcard" fields. This gives several advantages over a CAM, including longest-match selection for ACLs and CEF (i.e., in L3 forwarding), a single lookup of fixed latency, and theability to ignore fields. TCAMs are used in the 6500, 4000 and 3550 series.

There is a platform-dependent number of templates and number of entries per template type; theTCAM is partitioned into regions of templates.

In the 4000 and basic 6500, there is a single centralized forwarding table. The central forwardingengine is the limit to forwarding performance.

Switches with 100-Mbps rates and above use distributed forwarding, which allows the forwardingspeeds of multiple forwarding engines to be added. Distributed switching is present in the 3550 and inthe 6500 with DFC.

Templates

Switch Database Management for TCAMs was introduced on the 3550. Originally, there were four

templates, which would set TCAM elements to an optimal solution for:

z access

z default general-purpose

z routing

z VLAN




14/50

All the templates in assume 8 routed interfaces and 1K VLANs.

Table 15. 3550 Template Assumptions

Notice that the default template is optimized to support a large number of MAC addresses in the MACtable, and a large number of IP routes in the routing table. The trade off is fewer resources for IGMPgroups, QoS, and security related access control entries (lines in access-lists):

The routing template offers support for twice as many routes (16,000 versus 8,000), but far feweraccess control entries and QoS entries. In contrast, the VLAN template disables routing entirely, andfocuses all resources towards L2 and VLAN support.

As Chuck Larrieu put it in his 3550 Tutorial, "While it isunlikely that any CCIE Lab scenario would stress any ofthese settings, it is possible that a Candidate might beasked to 'assure that SVI support is maximized' or 'ensurethat L3 functionality is not compromised by L2considerations'." It is equally possible that a candidate for awritten exam -- CCIE or CCNP -- might be asked a similarquestion. It's likely that the template model will spread toplatforms other than the 3550.

Forwarding models

Demand-based forwarding requires that the first packet ofa flow must go through the "slow" or "software" path,which then populates a high-speed table. You will see thisin the Supervisor 1A/MSFC on the 6500.

Topology-based forwarding, on the 6500 with Supervisor 2,the 4000 with Supervisor 3, and the 3550, breaks thedependence on software lookup.

Fabric

The fabric interconnects the input and output interfaces. There are three main types of fabric:

z Shared bus

z Shared memory

z Crossbar

TCAM Access Default Routing VLAN

unicast MAC address 1024 5120 5120 8192

IGMP group 2048 1024 1024 1024

QoS Access Control Element (ACE) 1024 1024 512 1024

Security ACE 2048 1 512 0

Unicast Routes 2048 8 16384 0

Multicast Route 2048 1 1024 0

For further details...

Please refer to the 3550 Tutorial byChuck Larrieu.

"Cisco has created within the 3550platform the means of customizing andoptimizing system resource allocation

based on particular application orrequirement. For example, if a particularswitch was strictly Layer 2, or a series ofswitches had a large number ofconnected stations and a large number ofVLANs as well, then one could reallocateresources to favor VLAN, while disablingrouting and freeing up routing resources.On the other hand, if a particularinstallation required extensive QoS orsecurity configurations, an administratorcould optimize the switch to allocateresources for those activities."




15/50

A given switch will have one or more types of fabric. Indeed, on high-performance switches such as the6500, the highest-speed fabric is a separate card, not just part of the backplane.

Don't make the mistake I did, early in my career, and equate the backplane with the fabric. Thebackplane tends to be passive or nearly so. The active fabric will be on the supervisor card (orintegrated equivalent), and sometimes on a separate plug-in card. Indeed, a single platform can havemore than one fabric.

Table 16. Fabrics by Platform Type

* Cisco specifications are not always clear if the bandwidth stated is unidirectional, or adds together the two directions

[1] Depends on platform model

[2] Total bandwidth for stack

Shared Bus

Most lower-performance devices use a shared bus as the fabric. A single bus allows a connectionbetween two interfaces, with all interfaces contending for the bus. Don't fall into salesdroid traps andassume faster is always better. Shared bus is the cheapest solution, and thus appropriate forworkgroup and other small switches where cost is more important than performance.

The fabric is usually built into the backplane. Some devices, such as the 5500 switch, may have severalbusses bridged into one, and the throughput figure is the sum of the bus speeds.

z Each port must arbitrate for access

z

Broadcast and multicast are easy

z Oversubscription is normal

To make multicast and broadcast work properly,

z Flooded data decreases end-station performance

z Destination must be only those ports that need that traffic

PlatformFabric Speeds in Gbps *

Shared bus Shared Memory Crossbar

2900 8.8

2955 13.6

3550 8.8, 13.6, 24 [1]

3750 32 [2]

4000 32

4500 28, 64

5000 1.2

5500 3.6

6000 32

6500 256




16/50

z Multicast or VLAN mechanism must limit traffic to certain ports

Shared Memory

Shared memory systems keep the frame or packet in memory until the last egress interface is finishedwith it. Memory management can be simple or difficult, depending on whether or not there arerequirements for QoS and/or multicast.

QoS requires static buffer allocation in the shared memory. When you are multicasting, unless there isenough concurrent ports to the memory to service simultaneously all egress ports in the multicastgroup, the packet or frame has to stay in memory until the last egress port transmits it.

Crossbar

Crossbar designs are a full mesh, allowing concurrent communications between any pair of interfaces.Obviously, there is no contention for unicast forwarding.

Crossbars are the fastest fabric technology. There may be several cooperating crossbars within a largeswitch or router, as the ASICs involved are typically not greatly larger than 16x16.

Multicasting on crossbars can be a challenge, since the one-to-one relationship inherent to a crossbar isnot a good fit to the one-to-many of multicast involving multiple egress interfaces. Crossbar worksperfectly well in the middle of a multicast tree, where you have a single egress interface for a multicastgroup address. Shared memory fabrics may work better for multiple-egress-interface multicasting.

Egress Processing

In most switches and routers, the bulk of the processing is done at the ingress. Such functions asegress QoS, data link protocol conversion, etc., do take place in the egress card.

When the egress port connects to a server that is incapable of wire-speed operation, output bufferingmay be needed to avoid drops. In such cases, the amount of output buffering designed into the switch

involves delicate tradeoffs. Too little buffering causes data drops, but too much buffering can causeunacceptable delay.

QoS at the Switch

The discussion of QoS here is less to get into the various ways of enforcing QoS, such as shaping,policing, and queuing, and more to discuss how QoS requirements affect switch architecture.

When you do not implement a QoS marking mechanism, the DSCP fields of packets and frames aretrusted, and those fields used to sort the data units into appropriate queues. In switches, the defaultmeans of servicing queues is round-robin. Most switches support four queues, either in partitioned mainmemory or in dedicated memory

You can enable QoS marking and have the option of resetting the DSCP field to a new value, or you canset up new mappings between DSCP values and queues. See Figure 1 for the default mappings fromDSCP to queue.

Swi t ch#show qos maps dscp tx-queueDSCP- TxQueue Mappi ng Tabl e ( dscp = d1d2)d1 : d2 0 1 2 3 4 5 6 7 8 9- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -0 : 01 01 01 01 01 01 01 01 01 01




17/50

1 : 01 01 01 01 01 01 02 02 02 022 : 02 02 02 02 02 02 02 02 02 023 : 02 02 03 03 03 03 03 03 03 034 : 03 03 03 03 03 03 03 03 04 045 : 04 04 04 04 04 04 04 04 04 046 : 04 04 04 04

Figure 1. DSCP to Queue Mapping

Use the

qos map dscp dscp-values t o t x- queue queue-id

command to reset the mappings.

In switches with four queues, transmit queue 3 can be taken out of the round robin rotation anddesignated to follow strict priority queuing. This function, disabled by default, is intended for low-volume, delay-sensitive traffic such as voice and network control information. Be very conservative inassigning traffic to this queue, or you may starve the other queues.

You can find the transmit queue and priority assignment for an interface with the show run

interface command.

A specialbandwidth subcommand oftx-queue, not to be confused with interface bandwidth, can

allocate a guaranteed minimum bandwidth to each of the four queues. At present, this is only availableon non-blocking Gigabit Ethernet interfaces. For a 4000-specific example of such ports, see Table 22.

If you enable global QoS withoutbandwidth statements, each queue will get 250 Mbps. Do be aware

that the switch does not check for consistency amount the assignments, and it will let youoversubscribe (e.g., assign 250 Mbps to queues 1 and 2 and 500 Mbps to queues 3 and 4).

As long as a transmit queue is below the preconfigured share and shaping values, it is considered highpriority and served by the priority queuing discipline. Queues that do meet the share and shape values

will be serviced after the high priority queues. Only if no high priority queues exist will strict round robinbe observed.

The priority discussed here is not directly associated with the DSCP

Interfacing: the GBIC (Gigabit Ethernet Interface Converter)

Cisco standardizes the Gigabit Ethernet ports on switches, and assumes you will connect a GigabitEthernet Interface Converter (GBIC) to the ports to interface the port to the specific GE technology.There are GBICs for short- and long-wave optical GE, for long-haul systems, for coarse and densewavelength division multiplexing on optical transmission systems, for switch stacking, for GE overcopper, and a constantly growing list of optical and electrical media.

Characterizing Switch Performance

Many confusing numbers about switch and router performance.

Throughput

For the standard definition of throughput, see RFC 2544. Figure 2 shows test configurations for the




18/50

Device Under Test (DUT) with both integrated and separate load generators and receivers.

+- - - - - - - - - - - - +| |

+- - - - - - - - - - - - | t est er | | DUT | - - - - - - - - - - - - - - +

| |+- - - - - - - - - - - - +

+- - - - - - - - + +- - - - - - - - - - - - + +- - - - - - - - - - +| | | | | || sender | - - - - - - - - >| DUT | - - - - - - - - - >| recei ver || | | | | |+- - - - - - - - + +- - - - - - - - - - - - + +- - - - - - - - - - +

Figure 2. Standard Throughput Measurement topology

I find it amusing that the presentations by Cisco technical people at Networkers often give a reducedbut practical definition of throughput. For example, you'll hear the figure 256 Gbps used to state thethroughput of the fabric module on a 6500 series switch. The maximum one-way throughput, however,is 128 Gbps. The sales figure adds together the maximum speeds in each direction, doubling thethroughput.

It's less amusing if you are asked to answer a question on "speeds and feeds", and it's not clearwhether the question is looking for unidirectional or bidirectional management. There's no simplesolution here, other than to read the question carefully and see if it makes the conditions ofmeasurement clear. I'd also be more tempted to go with a salesy answer if I were taking a salescertification exam.

Somehow, we've managed to avoid widespread propagation of the idea that full-duplex Fast Ethernethas a throughput of 200 Mbps, but this "spin" of the truth still seems popular in describing thethroughput of a routing or switching platform.

Blocking

A source of much fear, uncertainty, and doubt (FUD) in switch marketing is whether a forwardingsystem is blocking or nonblocking. The usual definition of a nonblocking switching fabric is that thefabric is fast enough to transfer all traffic, without loss, while all ports are active. This definition issomewhat flawed.

A better way to speak of a nonblocking fabric is one that can keep up with a set of input ports, each of

which is outputting to a unique output port of the same or greater speed.

Sales presentations for nonblocking relays tout their advantage over blocking devices. In practice, thisis often a theoretical rather than a practical advantage. There is an underlying assumption of hownonblocking performance is measured, as shown in Figure 3. In a blocking switch, the fabric is too slowfor full noninterfering transmission. In Figure 3, input and output ports are paired, as required by RFC2544. Every input has a dedicated output.




19/50

Figure 3. Switching Fabric

Output Blocking

Output blocking is fairly common, and you must understand that it is a client or server problem, not aswitch or network problem, unless an intermediate, blocking relay is connected to the output port.Output blocking occurs when two or more ingress ports try to send simultaneously to the same egressport. Remember that the RFC 2544 throughput specification is explicit that each ingress port relays onlyto a single egress port.

In this situation, the fabric speed is irrelevant, because the problem is at the egress port (Figure 4).You can trade off delay against data loss by providing output buffering. When QoS must be controlled,

you need to think through ingress and egress parameters so unacceptable delay will never occur at anegress port.




20/50

Figure 4. Output Blocking --- don't blame the switch!

Some vendors, though not Cisco, support a technique that buffers at the ingress when the externaldestination cannot accept data fast enough, or the egress interface is busy. Input buffering, unless verycarefully designed, can lead to head of line blocking (Figure 5).

What Cisco has done is produce a GE (Gigabit Ethernet) interface for the 4000, which has 18 ports thatshare a 6 Gbps path into the fabric. These numbers were chosen because many Wintel servers can'tgenerate more than 300 Mbps of traffic. With such servers, there's still a benefit to using GE, to reducelatency in transmission, use single GE NICs rather than Fast EtherChannel, and leave room for growth

Figure 5. Head of Line Blocking




21/50

In modern switches, head-of-line blocking cannot occur, because there can be concurrent transfersbetween input and output ports. Such concurrency can be virtual, but is now generally physical,because a multiported shared memory, or a crossbar, will not get "stuck" waiting for a frame totransfer.

Even if there is a blocking fabric, modern switch design prevents head-of-line blocking, because itcreates multiple virtual queues in the input buffer, which prevent a frame from ever preventing anotherframe from reaching the fabric. With a blocking fabric and a single input buffer queue, you can have the

scenario in Figure 5.

This scenario involves ingress interfaces that have a single first-in-first-out (FIFO) buffer. Assume thattwo frames destined to output port three arrive simultaneously, one on port 1 and the other on port 2.Port 3, obviously, can only send one frame at a time.

Again remembering the input buffer is FIFO, assume another frame, destined for port 4, arrives at port1. If port 2 gained control of the fabric before port 1 could do so, port 1 can't send the 4th frame,because its path to the fabric is being blocked by the "backpressure" from the output port.

Head-of-line blocking means that the data unit "behind" the port 1 destined data unit onport 2 has to wait to be transmitted. In principle, while input port 2 waits to get a path tooutput port 1, the data unit destined to output port 2 could be transmitted in parallel. The

reality is that the fabric cannot see the input traffic in the buffer if the input buffer is afirst-in-first-out (FIFO) structure. Shared memory buffering in more modern switchestends to avoid head-of-line blocking, since all ports have access to the memory.

You encounter head-of-line blocking in daily life, when you are driving in the right lane,and come to a traffic light where you want to turn right. Your car, however, is the secondin line, and the car in front of you wants to go straight. If that car were not at the head ofthe right lane, you could turn right on red. You are, however, blocked at the head of theline.

Given an understanding that blocking may occur even in a "nonblocking" design, an any-to-any crossbar architecture may not improve performance at lower speeds. [Berkowitz1999, p. 197-199].

Grandfather Switch: Catalyst 5x00 PlatformFamily

These are obsolescent switches, but worth mentioning because so many people have experience withthem.

5000/5500 switches use the CatOS interface, except when configuring L3. Optional L3 forwardingcapability began on the NetFlow Feature Card (NFFC). This card can filter, snoop CGMP, and enforceQoS in the "fast path", but does not run the control plane of routing or switching protocols. Controlplane functions run in the Supervisor Engine.

The Route Switch Feature Card (RSFC) essentially is a full IOS 12.0 capable router, which uses a NFFCII function on the Supervisor Engine for its forwarding. The RSFC directly supports Cisco ExpressForwarding.

Stacking and Clustering: 3750 and 2950

Stacking has been in the industry for some time, and is a means of providing management for a groupof switches using only one IP address for management. Cisco extended the concept to clustering.




22/50

Clustering provides the functionality of stacking, but removes some of the limitations. Stacked switchesneeded to be in close proximity, as in a single wire closet. A cluster, however, can be defined amongswitches in different locations reachable by the same LAN. The members of the cluster are selecteddynamically rather than by the physical wiring used in a stack. Members of a cluster can be linked witha dedicated cable and GBICs as in stacks, but also with Fast Ethernet or Fast EtherChannel.

Since clustering no longer requires the physical proximity of switching, clustering is available onmixtures of up to 16 switches, including the Catalyst 3550, 2950, 3500 XL, 2900 XL, 2900 LRE XL and

1900 Series.

While clustering functionality was introduced with the 3512XL, 3524XL, and 3508G XL, only the 3508 GXL is still sold. Their replacements are the Catalyst 3550 and 2950 series. The 3508 G XL is stillsupported, primarily as a GE concentrator.

3750 series switches are only semi-modular. They have fixed ports for 10/100 Ethernet, but have somenumber of Small Form-Factor Pluggable (SFP) uplinks. Gigabit Interface Converters (GBIC) plug intothe SFP ports. You will also find SFP ports on the 3550 series.

Midrange Flexibility: Catalyst 3550 Platform

FamilyAll 3550s have L2 switching capability "out of the box". The 3550 12-, 24-, and 48-port switches haveoptional L3 switching (i.e., routing) capability. The 3550-12G and 12T come standard with the L3software image. It is very simple to upgrade the software, and migrate an L2-box to an L3-box -- allthat is required is a software license upgrade.

In life, as well as in the CCIE Lab, one should adjust to the new metaphor. This is particularly importantfor those who have concentrated their efforts around the Catalyst 5000 in their studies.

A New Interface Paradigm

You must understand the new 3550 interface metaphor and understand the relationships amongphysical ports, routed ports, port based VLANs, and switched virtual ports.

With the old set-based switches, one was concerned only with the physical ports, and the placing ofthose ports into the appropriate VLANs. This was done via the "set port" series of commands.

With the 3550, there are physical ports, and there are switched virtual interfaces (SVIs). Physical portscan be designated as L2 only or L2/L3. Without any additional configuration of the switch, all ports areLayer 2 by default. For the following discussion, these are the interfaces being referred to:

Swi t ch_2( conf i g) #interface ?Fast Et hernet Fast Et hernet I EEE 802. 3Gi gabi t Et hernet Gi gabi t Et hernet I EEE 802. 3zPor t - channel Et her net Channel of i nt er f acesVl an Cat al yst Vl ans

A "port-based" VLAN is a physical port that either has not been configured at all (in which case it is bydefault a member of VLAN 1) or which has been placed into a particular VLAN via the switchport

access vlan command. It should be apparent that port-based VLANs are Layer 2 only.

Physical ports become physical Layer 3 ports by the issuing of the no switchport interface command.

Once this has been done, the port can be given an IP address and one can enter the port into a routingdomain.




23/50

Swi t ch_2( conf i g- i f ) #ip address 10.3.3.1 255.255.255.240

% I P addr esses may not be conf i gur ed on L2 l i nks.

Swi t ch_2( conf i g- i f ) #no switchportSwi t ch_2( conf i g- i f ) #ip address 10.3.3.1 255.255.255.240Swi t ch_2( conf i g- i f ) #

Figure 6. Creating a Layer 3 Port

A switch virtual interface (SVI) is a logical interface that represents VLANs of physical switch ports tothe routed or bridged processes of the switch. Some detail will be given in the fallback bridging section.For now, let it be said that configuration and capability are similar to that of loopback interfaces. Thisreally takes the concept of VLANs just a very small step beyond the thinking on the earlier Catalystswitches. Unlike loopback interfaces, the creation of an SVI is a two-step process.

1. create the VLAN, using either the VLAN database command from the privilege exec or the VLANcommand from the global configuration mode.

2. invoke the SVI by entering the command interface vlan from the global configuration mode.

Swi t ch_2( conf i g) #vlan 307Swi t ch_2( conf i g- vl an) #name Three-oh-sevenSwi t ch_2( conf i g- vl an) #interface vlan 307Swi t ch_2( conf i g- i f ) # Z

Figure 7. Creating Switch Virtual Interfaces (SVI)

At this point, the SVI exists. Observe how the virtual interface can be displayed by a command thatwould normally be used for a physical interface. To be correct, the show interface command also can

show subinterfaces.

Swi t ch_2#show interface

Vl an307 i s up, l i ne pr ot ocol i s upHardware i s EtherSVI , addr ess i s 0009. b775. d400 ( bi a 0009. b775. d400)MTU 1500 bytes, BW 1000000 Kbi t , DLY 10 usec,

r el i abi l i t y 255/ 255, t xl oad 1/ 255, r xl oad 1/ 255Encapsul at i on ARPA, l oopback not setARP t ype: ARPA, ARP Ti meout 04: 00: 00Last i nput 00: 01: 48, out put never, out put hang neverLast cl ear i ng of " show i nt erf ace" count ers neverI nput queue: 0/ 75/ 0/ 0 ( si ze/ max/ dr ops/ f l ushes) ; Tot al out put dr ops: 0Queuei ng st r at egy: f i f oOutput queue : 0/ 40 ( si ze/ max)5 mi nut e i nput r at e 0 bi t s/ sec, 0 packet s/ sec5 mi nut e out put r ate 0 bi t s/ sec, 0 packet s/ sec

0 packet s i nput , 0 bytes, 0 no buf f er

Recei ved 0 br oadcast s, 0 r unt s, 0 gi ant s, 0 t hr ot t l es0 i nput err ors, 0 CRC, 0 f r ame, 0 over r un, 0 i gnored0 packets out put , 0 bytes, 0 underr uns0 out put er r or s, 0 i nt er f ace r eset s0 out put buf f er f ai l ur es, 0 out put buf f ers swapped out

Figure 8. Displaying an SVI

Even though the VLAN is not assigned to a physical port, and even though there is no other




24/50

configuration on the SVI, the SVI shows "up" and "up". The integration of the Layer 2 and Layer 3functionality takes place at the SVI level.

With the 3550 paradigm, you also have the capability to define an L2 port as either an access port, atrunk port, or a voice port:

Swi t ch_2( conf i g- i f ) #switchport ?access Set access mode char act er i st i cs of t he i nt er f ace

t r unk Set t r unki ng char act er i st i cs of t he i nt er f acevoi ce Voi ce appl i ance at t r i but es

Swi t ch_2( conf i g- i f ) #switchport voice vlan 77Swi t ch_2( conf i g- i f ) #switchport access vlan 78

Figure 9. SVI Functionality

The above configuration shows that voice and data VLANs can co-exist on the same port.

Management and Control

The principal human interface to the 3550 is IOS. Clustermanagement software, however, principally uses a Webinterface.

The Supervisor engine has direct access to the sharedmemory, at an internal rate faster than the interface cardslots.

Forwarding

Like the 4000 and related platforms, the 3550 has sharedmemory. All forwarding decisions take place in "Satellite"

ASICs. Control information is sent on a separate controlring to the egress interfaces, while the data part is storedin shared memory.

Table 17. 3550 Filtering Capacity

Depending on the specific 3550 model, there may be more than one TCAM.

Table 18. Number and Use of TCAMs in 3550 Models

Hardware Aspects of Voice

Ports

Many IP telephones expect -48 VDCpower to be provided to them. ManyCisco switches can send this power overan Ethernet interface, but it is not agiven that all platforms and line cardtypes will support it. Take the evenpotential need for power intoconsideration when selecting newswitches: is there any chance you willneed IP phones? If so, plan the migrationpath for the switch, which might be no

more than leaving slots for Ethernet-with-power line cards, and consideringthe additional power draw when youspecify power supplies.

Resource Limit

Access control list 512 security (256 in/256 out)

128 QoS

Access control entry (i.e., a line in an ACL) 4000 security

Model TCAMs TCAM use

3550-24 1 All interfaces on same TCAM




25/50


The original 4000 switches use a code base derived from the 5000 code set. The 4500, however, isIOS-based. 4000 switches are modular. The related 2948G is fixed configuration, as is the 4912G. Oncethe 4500 was released, the 4000 was targeted more at the wiring closet, especially with AVVID. Incontrast, the 4500 is optimized as a termination/aggregation point for metro Gigabit Ethernet.

4500 switches can use cards from both the 4000 and 6500 product lines. One of the target markets forthe 4500 is terminating Metro Ethernet between enterprises and service providers: "Gigabit to the enduser".

Table 19. 4000/4500 Platforms

For increased availability, load-sharing redundant power supplies work with all models of the 4500series. Only the 4507R supports redundant supervisors.


Originally, Catalyst 4000 Supervisors ran a derivative of the Catalyst 5000 code. Both in later 4000sand in the 4500s, the supervisor code migrated to IOS.

You can have redundant supervisors on the 4507R platform. Failover takes 30-50 seconds. Anotherhigh-availability features are the use of redundant power supplies and the ability to hot-swap line cards.

3550-48 2 Fast Ethernets 1-36 on TCAM 1All others on TCAM 2

3550-12T 3 Interfaces1-4 on TCAM 15-8 on TCAM 29-12 on TCAM 3

3550-12G 3 Interfaces1-4 on TCAM 15-8 on TCAM 29-12 on TCAM 3

Platform Line Card SlotsSupervisor

Type Slot

4003 3 III 1

4006 6 IV 1

4503 3 IV 14506 6 IV 1

4507R 7 IV 2

Feature orParameter

Supervisor

II II-Plus III IV

Platforms 4006, 4503, 4006,4503, 4006, 4503, 4006,4503,




26/50


27/50

To understand QoS filtering, you must be aware of several assumptions. First, L2 prioritization dependson the QoS value in an ISL or 802.1Q header. Second, L3 IPv4 prioritization depends either on theDifferentiated Services Code Point (DSCP) or the IP precedence value in the ToS byte. Both the DSCPand precedence value are in the Type of Service byte of the IPv4 header.

This discussion applies to the Supervisor III, and, unless specifically mentioned, to the Supervisor IV.Given that the fabric is nonblocking, there is no input queuing. Each output interface has four queues,240 packets each for Fast Ethernet and 1920 packets for non-blocking Gigabit Ethernet.

Table 22. Blocking and non-blocking port types on the Catalyst 4000 series

Switch supervisors often do not support the range of QoS measures on a router platform. For example,Weighted Random Early Discard (WRED) is not supported on switch platforms, but is available onrouters like the 7200. The 6500 switch is an exception that supports WRED.

Depending on the model, 4500 platforms will have 28 to 64 Gbps of shared memory backplane. Withthe Supervisor III or IV, the fabric is fast enough to allow all interfaces to run at wire speed, withoutfabric blocking.


There are a number of common features between the 6x00 switch series and the 7600 routers. We willconcentrate here, however, on the true 6x00 switches. Be aware that the 6x00 family tends to havemore functionality, especially in QoS, than other switches.

Table 23. 6500 Platforms

16,000 in/16,000 out QoS

Non-blocking Blocking

Supervisor III and IV uplinks 10/100/1000 T ports on the WS-X4412-2GB-TXline card

all ports on WS-X4306-GB line card

two 1000BASE-X ports on the WS-X4232-GB-RJ

line card

first two ports on the WS-X4418-GB line card All other ports

two 1000BASE-X ports on the WS-X4412-2GB-TXline card

WS-X4424-GB-RJ45 line cardWS-X4448-GB-LX line card

Platform Line Card SlotsSupervisor

Type Slot

6503 3

1, 2, 720 26506 6

6509 9




28/50

[1] Chassis, airflow, and power supply optimized for service provider environments


The Supervisor card proper contains the switch processor. Optional daughter cards include the Multi-Layer Switch Feature Card (MSFC) and Policy Feature Card (PFC). You can install both types ofdaughter card.

In the MSFC is a routing engine and EARL switching ASICs.

6500 forwarding decisions involve a pipeline of three ASICs. The first does L2 forwarding, butrecognizes traffic that must be handled at L3. The second does L3 forwarding based on informationplaced in the flow cache by the MSFC. Finally, the third ASIC does ACL processing.

In the 6000, the COIL ASIC connects forwarding cards to the control and results busses, which areseparate from the basic 32 Gbps fabric. Pinnacle ASICs connect forwarding cards to the fabric. PinnacleASICs support Weighted Round Robin and Weighted Random Early Discard QoS functions. Each Pinnaclehandles up to four GE ports.

Database Manager

On a high-end platform such as the 6500, the more traditional limiting factors such as bandwidth areless often a problem than resource contention and exhaustion. You need to understand which ACL andrelated functions are done in software, creating a centralized bottleneck.

Critical resources also can be in the distributed forwarding cards. In particular, these include masks inTCAM, the Logical Operation Units (LOUs); and the ACL-to-switch interface mapping labels.

TCAM entries, LOUs, and ACL labels are limited resources. Therefore, depending on your ACLconfiguration, you might need to be careful not to exhaust the available resources. In addition, withlarge QoS ACL and VACL configurations, you also might need to consider Non-Volatile Random AccessMemory (NVRAM) space. Remember that booting a configuration from a TFTP server is a workaroundfor configurations that won't fit into NVRAM.

Table 24. ACLs Processed in Software in Cisco Catalyst 6500 Series Switches

6509-NEB [1] 9

6513 13

Function Specific Environment Comments

ACL denied traffic Supervisor 1a with PFC -- ACLdenied packets are processedin software if interface does

not have the no ipunreachables command

configured

Supervisor 2 with PFC2 -- ACLdenied packets are leaked tothe MSFC2 if unreachables areenabled.

Packets are leaked at 10 packets persecond (pps) per VLAN (Catalyst OSsoftware with Cisco IOS Software) orone packet every two seconds perVLAN (Cisco IOS Software)




29/50

Supervisor 720 with PFC3 --ACL denied packets are leakedto the MSFC3 if unreachablesare enabled

Packets requiring ICMP unreachablesare leaked at a user-configurablerate (500 pps by default)

Traffic denied in an outputACL

(Supervisor 1a with PFC only) If traffic is denied in an output ACL,an MLS cache entry is never createdfor the flow. Therefore, subsequent

packets do not match a hardwarecache entry and are sent to theMSFC where they are denied insoftware

IPX filtering based onunsupported parameters(such as source host);

on Supervisor 720, Layer 3IPX traffic is always processedin software

ACEs requiring logging (logkeyword)

ACEs in the same ACL that donot require logging are stillprocessed in hardware;

Supervisor 2 with PFC2 andSupervisor 720 with PFC3 supportrate-limiting of packets redirected tothe MSFC for ACL logging.

TCP intercept Supervisor 1a with PFC --

Traffic permitted in a TCPintercept ACL is handled insoftware

Supervisor 2 with PFC2 andSupervisor 720 with PFC3

The TCP three-way handshake (SYN,SYN/ACK, ACK) and session close(FIN/RST) are handled in software;all remaining traffic is handled inhardware

Policy routed traffic (if matchlength, set ip

precedence, or other

unsupported parameters areused; if themls ip pbr

command is not configured

Supervisor 1a with PFC The set interface parameter issupported in software, with theexception of the set interface Null0parameter, which is handled inhardware on Supervisor 2 with PFC2

and Supervisor 720 with PFC3

Null0 parameter, which ishandled in hardware onSupervisor 2 with PFC2 andSupervisor 720 with PFC3

WCCP redirection for HTTPrequests

Supervisor 1a with PFC only)

Traffic requiring NetworkAddress Translation (NAT)

(Supervisor 1a with PFC andSupervisor 2 with PFC2);traffic requiring NATtranslation or NetFlow setup(Supervisor 720 with PFC3)

Unicast RPF check Supervisor 2 with PFC2 andSupervisor 720 with PFC3 --Traffic denied in a uRPF checkACL ACE

Supervisor 1a with PFC -- AnyuRPF check configuration

Non-IP (all Supervisors) and Supervisor 1a with PFC and




30/50

Forwarding

Depending on the model and features, the 6000 series may use one of several fabric methods. On the6000, "classic" line cards are interconnected by a Pinnacle ASICs to a 16 Gbps bus. Since the bus isbidirectional, it is marketed as 32 Mbps.

In contrast to the 6000, the 6500 has a crossbar fabric, which is mounted on a separate card. ThisSwitch Fabric Module (SFM) has a one-way 128 Gbps or 256 Gbps full-duplex capacity. Individual cardchannels are 8 Gbps; there are two channels per slot.

Maximum throughput in the 6000 is 15 Mbps, while the 6500's maximum is 30 to 210 Mbps, dependingon whether the SFM is present.

Depending on the platform, slots can have different speeds, even within the same platform. On the

6506 and 6509 switches, and the 7606 router, all with SFM or SFM2 fabrics, each slot gets 16 Gbps. Onthe 6513, slots 1-8 get 8 Gbps but slots 9-13 get 16 Gbps.

In the 6500, the Medusa ASIC interconnects the local card bus and the crossbar fabric. It also connectsfabric-enabled cards to the 32 Gbps shared memory.

Remember the 6500 supports 10 Gbps Ethernet and the 7600 supports OC-192. While these areconsidered, respectively, LAN and WAN interfaces, their physical layer is identical.

Table 25. 6500 Card Types

6500s also use TCAM tables for cEF and ACLs.

Input and output queuing take place in Pinnacle ASICs on the line card.

Table 26. 6500 Filtering Capacity (Supervisor 2)

non-IPX Supervisor 2 with PFC2 only)RACLs

Broadcast traffic denied in aRACL

Card type Function

Classic Bus only.COIL and Pinnacle ASICs

Fabric enabled Bus and fabricMedusa and Pinnacle ASICs.

Fabric only Fabric only. Medusa and Pinnacle ASICs.Can have Distributed Forwarding Card.

Switch fabric module line card Contains the actual fabric

Resource Limit

Access control list 512This number combines security RACL, QoS ACL and VLANACL (VACL)




31/50

Table 27. 65x x Forwarding Capacity

Switching Functions for High Availability

Layer 1/2 High Availability for Links and Interfaces

While redundancy isn't always the solution to high availability, it will usually be the case at Layers 1 and2. You can, of course, have multiple media, each known to routing. Especially if convergence is anissue, however, the wise course may be to bundle several links together so that Layer 3 does not see alink going up and down. The major techniques for doing this are 802.3 aggregation and Multilink PPP.SONET/SDH restoration is not quite bundling, but it is self-healing in the same general sense. When alower layer cannot repair itself, that responsibility moves to a higher layer, such as dynamic routing.

Another alternative is to use dial backup to a link. Do remember dial backup is primarily Layer 1/2,where dial-on-demand is Layer 3.

Layer 1 Failover

Remember that many Layer 1 mechanisms cannot tell when a link has failed in one direction. You needLayer 2 mechanisms, ranging from link keepalives to the Unidirectional Link Detection Protocol, orLayer 3 routing updates, to detect that condition.

The first Cisco feature to provide any sort of recovery in the event of link failure was dial backup, whichoperates at Layers 1 and 2. Subsequently, dial-on-demand routing (DDR) was adapted to give a Layer3 capability for such backup. See the CertificationZone High Availability Study Guide for more detail ondial-based recovery.

SONET and POS

While the CCIE lab has no SONET equipment, you may need to answer written questions about SONET

Alternate Protection Switching (APS. SONET/SDH can carry either ATM or Packet over SONET (POS), forSONET has become a transitional technology, just as ATM/SONET was an evolutionary step beyondTDM. Due to the large and effective SONET installed base, newer technologies must support SONET.Generalized MPLS and long-haul Ethernet and optical technologies will supplant it, but need to bebackward compatible.

SONET connectivity does not mean that you have automatic backup. You must explicitly enableAutomatic Protection Switching, a SONET high availability technology. In the original version, SONETLine Terminating Equipment connects to a primary and backup SONET medium. The specific SONETterminology used is the working and protection ring. APS supports 1+1 and 1:Nmodels.

Access control entry (i.e., a line in anACL)

32,000 entries

8,000 masks

Supervisor 1 and/or classic line cards 15 Mbps

Supervisor 2 with fabric enabled line cards 30 Mbps

Supervisor 2 with SFM and 7 6816 fabric-only line cards 107 Mbps

Supervisor 2 with SFM and 7 6816 fabric-only line cards plus card-local traffic switching 170 Mbps

6513 with DFC-enabled fabric-only line cards plus card-local traffic switching 210 Mbps




32/50

In APS, only the working ring actually carries user traffic. A management protocol runs over both rings,however. The APS Protect Group Protocol detects failures and triggers ring switchover.

However, SONET has been extremely reliable, and duplicating all rings is very expensive. In the 1:Nvariant shown on the right side of the figure below, one protection ring covers four LTEs. When a failureoccurs, the protection ring is activated only between the endpoints affected by the actual failure.

SONET no longer needs to run over its own physical fiber, but can run on a wavelength of DWDM. Thisallows links in multiple protection rings to run over the same fiber, with due regard not to put both linksof the same ring over the same physical fiber, creating an SRG.

Unidirectional Links: Detection Protocol (UDLD) and configuringUnidirectional Ethernet

UDLD is a control protocol, operating at layer 2, whose function is closely tied to layer 1. Its purpose isdetecting cases where your local device can send data to its neighbor, but you cannot receive trafficfrom the neighbor. Such a failure may not trigger physical layer alarms (e.g., as part of autodetection),but it can cause all sorts of failures, such as spanning tree loops, that can cause widespread networkchaos.

UDLD works on both copper and fiber media. When it detects the half-failure, it will shut down theinterface and force the local switch to take action as if the entire link were down. Autonegotiationcomplement UDLD as a failure detection function, but can only see layer 1 functions in one direction.

Every 60 seconds (by default) or a configurable number of seconds, a switch transmits UDLD messages(packets) to neighbor devices. UDLD messages only go out on ports where the protocol is enabled., and

both ends of the link must enable it for the mechanism to work.

Unidirectional communications are not always an error. There is a unidirectional Ethernet configurationcommand that tells the platform to use only a single fiber for one-way traffic on an interface.

Layer 2 Aggregation

Layer 2 aggregation distributes frames across two or more links, normally load-sharing but alsoproviding fallback if a link fails. LAN and MAN standards in this area come from IEEE, primarily under




33/50

802.3, but also under 802.17.

Table 28. Recent IEEE 802.3 Standards

There are two schemes for aggregating 802.3 traffic: Cisco's early and proprietary EtherChannel, andthe newer IEEE 802.3ad standard. EtherChannel uses a control protocol called Port AggregationProtocol (PAgP). 802.3ad uses the Link Aggregate Control Protocol (LACP).

These methods use at least two parallel links between two routers or switches, protecting you against asingle link failure or a failure of an interface at either end of one link.

Figure 10. Basic 802.3 Aggregation Protection between Switches

You can also use 802.3 aggregation between a switch or router and a server with a suitable NIC (Figure8). Using more links gives you protection against more link failures.

Protocol Function

802.3aa Updates to 802.3u Fast Ethernet

802.3ab Gigabit Ethernet over Cat 5

802.3ac Frame extension for baby giants

802.3ad Link aggregation




34/50

Figure 11. Multiported Servers

Other benefits of 802.3 aggregation include load balancing, in which source-destination pairs of MACaddresses are assigned to specific links in the bundle. Should a link fail, the addresses are redistributedonto the working links. Again, routing will be unaware of this redistribution.

To implement 802.3 aggregation, first be sure that your interface card supports 802.3 aggregation.Check the platform-specific restrictions, such as which ports can be bundled and if they need to becontiguously numbered. See Dan Farkas' LAN Switching tutorials for configuration details and ChuckLarrieu's paper on 3550-specific features.

Any easy way to ensure that all ports have a common configuration is to create the channel first andthen configure one port in the channel.

Perhaps the most basic application of 802.3 aggregation is having a bundle between two switches. Ifone link fails, traffic flow continues without impact on STP. It should have little effect on user traffic,although there is a possibility that a frame in transit on the failing link might be lost.




35/50

Figure 12. Basic Link/Interface Protection between Access and Distribution Switches

Figure 13 shows a fairly complex configuration I implemented for a client, which protected against bothdefault router failure and distribution failure and could protect against access switch failure. To protectagainst access switch failure, a host would need to have two NICs; each connected to a different accessswitch. STP would keep one of those NICs in the blocking state. See Dan Farkas' LAN switching papersfor configuration details.

Figure 13. Link/Interface Protection to Default Router(s)

By using multiple NICs that the host knows how to bundle into an 802.3 aggregation, you can alsoprotect against failures from network element to host. Again, a frame in transit might be lost.

Multilink PPP also protects you against failures of interfaces or links in a bundle, but the technologiesinvolved are appropriate for WANs rather than LANs.

Potentially, multichassis multilink protects you against a failure of an access server in a stack of accessservers. If you simply have one hunt group phone number for the entire stack, it will be randomwhether different calls go to the same or different access servers.

Perhaps the extreme case of using multilink to avoid single points of failure is PPP over L2TP.

Resilient Packet Rings, RPR, now under development in the IEEE 802.17 working group, is intended as amore efficient replacement for SONET/SDH, allowing better use of backup facilities. MANs, and RPR ingeneral, are intended to smooth some of the disconnects between enterprise-oriented LANs and long-haul SONET/DWDM [Vijeh 2000]. While SONET/SDH are Layer 1 technologies, RPR is a Layer 2 MACthat runs over the arbitrary physical facilities, including those compatible with SONET/SDH,metropolitan Gigabit Ethernet, etc. The basic unit of data transfer on RPR is an Ethernet frame, not a

bit.

RPR's L2 technology replaces the framing and the protection mechanisms of SONET/SDH. As opposedto Ethernet, it offers protection switching at SONET speeds.

RPR accepts that some traffic can be preempted if one ring fails, an idea certainly consistent with QoSprioritization. Other information is available from the IP over RPR Working Group in the IETF's sub-IParea, and an industry forum, the RPR Alliance, is being formed. As a technique primarily used inmetropolitan and wide area carrier networks, RPR is beyond the scope of this paper.




36/50

Preventing Broadcast Storms

Broadcast storms have become less of a problem as IP implementers learn ways to avoid them andbroadcast-intensive desktop protocols disappear. They still happen, but they can be restricted byappropriate settings on switch ports to which hosts connect. Cisco broadcast suppression countsbroadcast frame during a predefined interval, and shuts down the port if the count exceeds aconfigurable threshold.

You should never suppress all broadcasts, because that may disable perfectly legitimate functions suchas ARP and DHCP. Since broadcast storms inherently involve bursts of broadcasts that reinforce oneanother, you can't look at a port and instantly decide a broadcast storm is in process. You need tocount the broadcasts over at least 1 s.

Cisco switches have broadcast suppression disabled by default. When you enable broadcastsuppression, you specify a percentage of bandwidth that can be used by broadcasts.

In addition, on Gigabit Ethernet ports, you can suppress multicast and unicast traffic rates as well.Traffic shaping and policing, however, may be a better and more general way to deal with multicastsand unicasts.

Other Layer 2 Security and Management Enhancements

Don't assume that every possible problem in L2 networks is directly associated with operations of thespanning tree. You want to be sure that no more than the essential minimum of devices actuallyparticipate in the spanning tree, just as most of the edge routers in an enterprise network simplydefault to routers higher in the hierarchy.

Don't turn off STP, but control it. STP information can be important to error management.

Remember there can be other L2 threats that have nothing to do with STP computation, such asbroadcast storms and L2 denial of service attacks.

Private VLANsMentioned frequently in the Cisco SAFE blueprints, private VLANs impose a NBMA topology on a singleEthernet subnet. This is especially useful in broadband provider applications, where you do not wantany user to be able to see the traffic of any other user.

Table 29. Types of Ports in Private VLANs

Once you have defined the ports, you define pairs of VLANs (e.g., primary to community) that permitcommunications between them.

Table 30. Sub-VLANs in a Private VLAN

Port type Communicates with

promiscuous all other private VLAN ports and is the port used to communicate with devices such asrouters, LocalDirector, backup servers, and administrative workstations.

isolated promiscuous ports onlyCommunity communicate among themselves and with their promiscuous ports. These ports are

isolated at Layer 2 from all other ports in other communities or isolated ports withintheir private VLAN.




37/50

The simplest private VLAN consists of one primary VLAN and one of either isolated or community types.You are allowed to have additional isolated or community types, which do not communicate with oneanother. In your configuration, you must bind the isolated and/or community VLAN(s) to the primaryVLAN and assign the isolated or community ports to the appropriate sub-VLAN.

You will find that many of the private VLAN constraints (Table 31) also apply to 802.1x constraints(Table 32).

Table 31. Private VLAN constraints

! def i ne t he pr i mary VLANset vl an vlan_num pvl an- t ype pr i mary!set vl an vlan_num pvl an- t ype {i sol ated | communi t y}

Type Traffic rules

1 primary VLAN forwards incoming traffic arriving at a promiscuous port to all otherpromiscuous, isolated, and community ports

1 isolated VLAN isolated ports to communicate to the promiscuous ports.

1 or morecommunity VLANs

used by a group of community ports to communicate among themselves andtransmit traffic to outside the group via the designated promiscuous port.

Feature Constraint

BPDU Guard Automatically enabled

VLAN membership Set to static

Access ports Redefined as host ports

VTP transparent mode

VTP mode cannot be changed to client or server. VTP does not understand private VLANs.

primary VLAN only 1 isolated VLAN and/or multiple communities can be associated with it.

isolated orcommunity

only 1 primary VPAN

VLAN numbers Private VLANs cannot be numbered 1 or 1001 through 1005.

Port restrictions Private VLAN port cannot be channeling or dynamic membership. It only can betrunking if it is a MSFC port.

ASIC consistency On the same ASIC, you cannot have one port that is a trunk or a SPANdestination, and others that are community, isolated or promiscuous. This ishardware platform specific.

Spanning treeparameters

Must be identically configured on primary and isolated/

Date post:	14-Apr-2018
Category:	Documents
Upload:	salis-alvarez
View:	222 times
Download:	0 times

New Generation of Cisco Switching

Documents