Metropolitan area networks (MANs) may be implemented on the basis of various tech-nologies. Synchronous optical networking (SONET)/synchronous digital hierarchy (SDH)[1–3] is probably the most widely deployed MAN technology today. This technology usesa dual-ring architecture to achieve sub-50 ms system restoration times after a link or nodefailure. It also uses synchronous time-division multiplexing, which is suitable for circuitswitching but inefficient for packet switching because a fixed amount of bandwidth is allo-cated to every pair of nodes, and the bandwidth allocated to a given pair of nodes cannot beused by users at other nodes. Moreover, the resiliency of SONET/SDH is achieved at thecost of wasting half of the total bandwidth. This is because SONET/SDH rings reserve halfof the total bandwidth for system protection. Resilient packet ring (RPR) [4–7] is a recentlyproposed MAN technology that also operates using a dual-ring architecture. This technol-ogy is attractive because it combines the fast resiliency of SONET/SDH rings with theefficient bandwidth usage of packet switching. Though it was first proposed several yearsago, RPR has not yet been completely standardized, partly because of the complexity of itsprotocols [4–7]. It may be noted that the prevalent MAN technologies, e.g., SONET/SDHand RPR, are operated with a dual-ring topology instead of tree or mesh topologies, be-cause the dual-ring structure has been proved to be simple and inherently resilient to anylink and node failures.
The emerging 10 Gbit Ethernet (10GbE) over optical fiber is rapidly becoming anotherchoice of solution for MANs [8–10]. Ethernet is currently the major LAN technology witha very large installed base. As a mature and time-tested technology in LANs, Ethernet hasthe advantages of simplicity, familiarity, and low cost. Today, virtually all data frames be-gin and end their trips across the Internet as Ethernet frames. One of the main advantagesof extending Ethernet technology from LANs to MANs is that this will keep data in aconsistent frame format throughout the entire transport path and will eliminate the needfor additional protocol layers and network interfaces that would otherwise be needed andwould lead to extra cost and complexity. Originally designed for small campus networks,Ethernet protocols may not fulfill the stringent requirements of MANs. Traditional Ethernettechnology cannot work over topologies that contain active loops such as rings and meshes[11–13]. The IEEE 802.1D Spanning Tree Protocol (STP) [11, 12] is therefore used in Eth-ernet switches to select a spanning tree that connects all nodes from the current topology.However, STP is not desirable in MANs for two reasons. First, STP typically takes tensof seconds to reconfigure a spanning tree after a topology change, which would not be ac-ceptable to guarantee service level agreements [13]. Second, it induces unbalanced trafficamong the links (even though traffic demands in the network are uniformly distributed),since those links near the root switch tend to get saturated quickly. IEEE 802.1w RapidSpanning Tree Protocol (RSTP) [12, 14, 15] is an evolution of the STP protocol, which en-ables faster spanning tree convergence after a topology change. However, depending on thesize of the network, the convergence time may still vary from tens of milliseconds to a fullsecond or more. Moreover, the use of RSTP cannot circumvent the problem of unbalancednetwork traffic.
There have been increasingly strong interests to bring the standard Ethernet technologyfrom LANs to MANs [8–10]. Motivated by this, we propose here a new optical resilientEthernet ring (RER) solution that is based on 10GbE over optical fiber, which would allowthe Ethernet technology to be extended into MANs without making major changes to theoriginal Ethernet frame. (A preliminary version of this paper was presented in Ref. [16].)Basic RER system design issues such as architecture, frame structure, frame forwardingmechanism, and the self-learning process are described in detail. The ring topology simpli-fies the decision-making process for frame forwarding and also enables the RER to recoverfrom link or node failure within 50 ms, as in the case of SONET/SDH rings. Three differentprotection schemes are presented, and their performance differences are examined throughsimulations. Our study shows that the proposed RER is scalable to a large network sizeby the interconnection of multiple rings with a modified transparent bridging technique,without using STP.
Considered strictly from the point of view of cable lengths, it is indeed possible tooperate a 10GbE as a MAN. However, other factors such as scalability, overall through-put, protection against failures, and reconfiguration times should also be considered forthe operation of a MAN. A typical MAN would consist of a very large number of usersand would be required to carry high volumes of traffic between them. In such a situation,scalability considerations would favor an approach such as the RER that we propose inthis paper. In this, individual rings would connect users who would be reasonably closeto each other in geographical terms and would prevent traffic between users on the samering from propagating to the rest of the network. Similar advantages could also have beenobtained by incorporating suitably intelligent bridges or switches in a network with a con-ventional Ethernet architecture using 10GbE. However, the latter would not have the ad-vantages of failure resiliency that is naturally offered by a ring architecture. Moreover,transparent bridging/switching in conventional Ethernet architecture would require a span-ning tree to be set up in the topology. After a topology change (because of any reason), itmay not be feasible to modify the spanning tree rapidly enough to satisfy the demands im-
posed by service-level-agreement guarantees. We present the first steps toward extending10GbEs to MANs and focus on the medium access control (MAC) layer for such a system.
The rest of the paper is organized as follows. In Section 2 we describe the RER ar-chitecture including frame structure, frame forwarding mechanism, and the self-learningprocess. In Section 3, three protection schemes for the RER are presented and discussed.In Section 4 we discuss the interconnection of RER rings. In Section 5 we compare differ-ent ring approaches and examine the performance differences between the three differentprotection schemes using computer simulations. The conclusions are given in Section 6.
2. RER Architecture
Our proposed RER is similar to the bidirectional line-switched SONET/SDH rings and theRPR. It consists of two counterdirectional rings, as shown in Fig. 1(a), where each ringnode consists of an Ethernet ring switch. A ring node may have one or more hosts and/orsubnetworks connected to it. The structure of an individual ring node is shown in Fig. 1(b).Each switch has at least three input ports and three output ports. These ports are groupedinto pairs, with each pair consisting of one input port and one output port. Out of thesepairs, two participate in ring connection and are referred to as Ring Pair 1 and Ring Pair2. The remaining pairs, called local pairs, are used to upload traffic onto the ring fromlocal hosts or subnetworks (a subnetwork may be a LAN or just a host), or to deliver ringtraffic to local hosts or subnetworks. A ring switch may be involved in one of three rolesin forwarding frames: (a) as an ingress switch, (b) as an egress switch, or (c) as a transitswitch. An ingress switch is the entrance point where a frame enters the RER. An egressswitch is the exit point where a frame leaves the RER. A transit switch relays frames to thenext ring switch. A ring switch is also referred to as the boundary switch of a host if this isthe ring switch where the host or the host’s subnetwork is attached directly.
To facilitate forwarding frames from ingress switches to egress switches, each ringswitch is assigned a unique Switch ID (SID) for identifying individual ring switches. TheSID considered here is an 8 bit integer, which allows a total node space of 256 on an RERnetwork. (This is reasonably big for an RER.) When an RER network starts up, a topologydiscovery process will be initiated so that every switch will get to know the other switches inthe ring. At the end of the discovery process, every switch in the ring will know the numberof other switches and their relative locations on the ring. By default, all ring switches agreeto designate the switch with the lowest MAC address to be the first ring switch, denoted asS(0). After S(0) is selected, the remaining switches are assigned SIDs sequentially alongeither clockwise or counterclockwise ring direction as S(1), S(2) , . . . ,S(M−1), where Mis the number of nodes in the RER. Assigning SIDs sequentially simplifies the decision-making process in ingress switches to determine the frame forwarding direction. For a ringcontaining M nodes, hop counts from switch S(i) to switch S( j) will be simply calculatedas ( j− i) in one direction and (M− j + i) in the opposite direction, respectively, with j >i. Following the shortest-path-first rule, the direction with the smaller hop count will bechosen as the frame forwarding direction. It should be noted that the globally unique 48 bitMAC address assigned with each ring switch may also be used to identify a ring switch, asfollowed in the RPR approach [17]. Our approach for the RER, as proposed in this paper,results in lower frame overheads and simplifies frame forwarding. However, SIDs need tobe reassigned whenever a ring switch is inserted into or removed from the RER.
When a new switch is inserted between S(n−1) and S(n), S(n−1) and S(n) will de-tect the new switch as their new neighbor by receiving its keep-alive frame. This framewould have a source MAC address different from that of the previously observed neigh-bor (see Section 3 for the details of the keep-alive frame). Detecting a new switch on thering triggers the ring-topology discovery process. Let us assume that ring switches areassigned with SIDs increasing in the clockwise direction. Consider a scenario in which
switch S(n) receives in the clockwise direction a keep-alive frame whose MAC address isdifferent from what S(n) has previously observed. Switch S(n) will then immediately trig-ger a topology discovery process by sending out a topology discovery frame, which willcirculate around the ring once and will be removed by Switch S(n) itself. This process willdetermine the number of ring switches. If the number of switches is increased by 1, thenewly inserted switch will be assigned a SID as S(n), and those switches with SIDs greaterthan S(n−1) will have their SIDs increased by 1, e.g., S(n) becomes S(n+1), S(n+1)becomes S(n+2), and so on. If the number of switches is reduced by 1, those switcheswith SIDs greater than S(n−1) will have their SIDs decreased by 1, e.g., S(n) becomesS(n−1), S(n+1) becomes S(n), and so on. Since the number of nodes in the ring is un-likely to change frequently, the operational overhead due to ring switch insertion or removalis expected to be small. It is noted that when a ring switch is removed from or inserted intothe ring, the ring topology information is to be updated, and accordingly SubforwardingTable 1 in each ring switch also has to be updated. (However, Subforwarding Table 2 doesnot have to be updated.) During this updating process, frames that have already been putin the ring may get forwarded to wrong destinations where they will be subsequently dis-carded. These discarded frames will get retransmitted with use of higher-layer protocols;the details of which are beyond the scope of this paper.
The RER is flexible in bandwidth assignment, since there is no fixed bandwidth al-located between any two nodes. The spatial reuse mechanism is adopted to conserve ringbandwidth in the RER by allowing the destination (egress) switches to strip received framesoff the ring. In the rest of this section, we will describe the details on the MAC framestructure, frame forwarding tables, self-learning process, frame forwarding process, andprevention of frames circulation.
2.A. MAC Frame Structure
Frames that are transmitted within subnetworks of the RER remain the same as the originalEthernet frame format, as shown in Fig. 2(a). To simplify and speed up forwarding framesfrom one subnetwork (or host) to another via the RER, we propose to insert a short RERheader in the original Ethernet frame when a frame enters the ring. This RER header isremoved at the egress (destination) switch before the frame is forwarded to the destinationsubnetwork (or end host) and will therefore be transparent to the end hosts. As shown inFig. 2(b), all other fields from the original Ethernet frames remain unchanged. Here theRER header is assumed to be inserted between the start frame delimiter (SFD) field and thedestination address (DA) field. Another possible choice is to place the RER header withinthe preamble, since 10GbE is operated in full-duplex mode [8–10] and the preamble isno longer necessary for synchronization. (In full-duplex mode, even if the sender has nodata to send, idle frames are being transmitted and hence the receiver always remains syn-chronized.) In this case, the frame format will remain unchanged except for the preamblefield being modified. The RER header provides several other important functions such asproviding different quality of services (QoS) to various traffic classes, prevention of framecirculation within the ring, assistance in protection switching, etc.
As shown in Fig. 2(c), the RER header contains five fields. These are destination switchID (DSID) field, source switch ID (SSID) field, utility (U) field, header error check (HEC)field, and time-to-live (TTL) field. The DSID and SSID are locally unique 8 bit integersidentifying the destination (egress) switch and the source (ingress) switch, respectively.The utility field is an 8 bit field that is further divided into a type field, a wrap field, and aservice class field, as shown in Fig. 2(d). The type field is a 3 bit field used to identify upto eight different types of frames. Table 1 shows the four frame types considered in thispaper. The wrap field is a 1bit field used to indicate whether the frame has been wrappeddue to system failure (see Section 3 for details). The service class field has 4 bits, which
(b) RER frame format Preamble SFD RER Header DA SA …
40 bits
(d) Utility field Type Wrap Service Classes
4 bits 1 bit 3 bits
(a) Original Ethernet frame format
Preamble SFD DA SA L/T Data FCS SFD: Start Frame Delimiter DA: Destination Address SA: Source Address
L/T: Length/Type FCS: Frame Check Sequence
(c) RER header
8 bits 8 bits 8 bits 8 bits 8 bits
DSID SSID U HEC TTL DSID: Destination Switch ID SSID: Source Switch ID HEC: Header Error Check
U: Utility TTL: Time-to-Live
Fig. 2 MAC frame structure Fig. 2. MAC frame structure.
indicates various traffic classes. The header error check (HEC) is an 8 bit checksum, whichis computed only over the DSID, SSID, and U fields. When a switch receives a frame fromits neighbor switch, it will recompute the HEC based on the DSID, SSID, and U fields ofthe received frame to verify whether the RER header is valid or not. The received framewill be discarded immediately if the RER header is invalid. Recomputation of the HEC isnecessary, since either the DSID, SSID, or the U field may become corrupted during thetransmission. If the HEC is not recomputed at each transit switch, the corruption of one ofthese fields cannot be detected, in which case the frame will be delivered to a wrong egressswitch. The time-to-live is an 8 bit field, which is used to prevent frames from circulatingindefinitely in the ring (see Subsection 2.E for details). The value of the TTL must bedecremented every time the frame is forwarded to the next ring switch. A frame is strippedoff the ring whenever its TTL value reaches zero. Note that the TTL field is not included inthe computation of the HEC. This expedites frame forwarding, as otherwise the HEC hasto be recomputed and rewritten at every transit switch.
Table 1. Four Frame Types Considered in This PaperValue (binary) Type Description
000 Data frame001 Keep-alive frame010 Topology discovery frame011 OAM control frame
It may be observed that the switches proposed in our system are required to do theadditional processing of the RER header. This header is necessary to support the desirablefeatures proposed by our system and represent a minor modification to existing approaches.
The additional processing overhead is minimal and should be feasible with current andfuture hardware.
2.B. Frame Forwarding Table
The transmission path of a frame, from the point where it enters the ring to reaching thedestination host, may be divided into two parts: (1) from the ingress switch to the egressswitch and (2) from the egress switch to the destination host. To simplify and expediteforwarding frames from source hosts to destination hosts, we propose an efficient frameforwarding scheme here whereby every ring switch is equipped with a frame forwardingtable, consisting of two subforwarding tables: Subforwarding Table 1 and SubforwardingTable 2. Subforwarding Table 1 is looked up for forwarding frames from ingress switchesto egress switches, and Subforwarding Table 2 is used for delivering frames from egress(boundary) switches to destination hosts. Subforwarding Table 1 records all the hosts andtheir corresponding boundary switches within an RER network, and hence each entry con-tains two items: the host MAC address and the boundary switch ID of that host. Ingressswitches use Subforwarding Table 1 to determine both the egress switch as well as theforwarding direction (clockwise or counterclockwise direction) of a received frame. Aftera frame reaches its egress switch, Subforwarding Table 2 is invoked to determine the exitport via which the destination host can be reached. Each entry of Subforwarding Table 2also contains two items: the host MAC address and the exit port number of the boundaryswitch via which the host can be reached. Subforwarding Table 2 is dedicated to individualring switches, whereas Subforwarding Table 1 is common to all the ring switches, and itssize is the sum of all the Subforwarding Table 2s of all the ring switches within the RERnetwork.
Figure 3 gives an example of how frame forwarding tables are configured. Figure 3(a)shows a simple ring network where hosts a1 and a2 are connected to ring switch A andhosts g1, g2, and g3 are connected to ring switch G. For simplicity, x_mac_addr is usedto represent the MAC address of host x. Y_SID is used to represent the switch ID of ringswitch Y. Figures 3(b) and 3(c) show the forwarding tables contained in ring switches Aand G, respectively. As an example of frame forwarding, suppose that a frame F1 of hosta1 is to be sent to host g1. Upon receiving frame F1, switch A (ingress switch) looks upits Subforwarding Table 1 and finds the egress switch of frame F1 is switch G. Switch Athen decides the direction in which the frame is to be forwarded to the egress switch G. Thetransit switches in between switches A and G forward frame F1 in the same direction as itis received. Upon receiving the frame, switch G finds that the DSID of frame F1 matchesits own SID and refers to its Subforwarding Table 2 to determine which exit port frame F1is to be delivered to.
2.C. Self-Learning Process
As in the case of Ethernet switches or bridges in the original Ethernet network [18, 19],an RER ring switch learns the MAC source addresses of end hosts and associates themwith appropriate ports on which the end-host frames arrive. Ring switches build up theirSubforwarding Tables 1 and 2 through a continuous self-learning process. When a ringswitch receives a frame from one of its subnetworks (or hosts), it learns that the sourcehost is reachable on the receiving port, and then creates an entry in Subforwarding Table 2.The entry consists of the MAC source address and the receiving port. This learning processnot only builds up Subforwarding Table 2 but also continuously updates SubforwardingTable 2 by replacing old entries with new entries. When a ring switch receives a framefrom its neighboring ring switch, it learns the hosts in other ring switches by recordingthe (DSID, DA) and (SSID, SA) in the received frame. This learning process builds upSubforwarding Table 1 and also continuously updates Subforwarding Table 1. It is noted
that the MAC address associated with an Ethernet card of a host is fixed at the time of itsmanufacture. If a host changes the point of attachment in the RER network, the contentsof the tables need to be updated periodically to reflect such changes. To accomplish this,as in the conventional Ethernet [18, 19], a timer is always associated with each entry in thetables. Whenever a frame is received from a host, the corresponding timer for the entry ofthat host is reset. If no frames are received from a host within a predefined time interval,the entry is removed. Entries in Subforwarding Tables 1 and 2 are kept in a cached mannerand will be eventually removed either if the size of forwarding tables exceeds the storagecapacity or if the predefined duration has elapsed since the last update for a particular entryin the tables.
2.D. Frame Processing and Forwarding
On receiving a frame from its subnetworks, an ingress switch inserts an RER header intothe frame before it is transmitted into the RER. The operations carried out at an ingressswitch are summarized as follows:
1. Recompute the FCS and compare it with the original FCS value in the received frame.The frame is discarded if these do not match. (This can also be carried out at the endhost instead of being performed at the ingress switch.)
2. Update its Subforwarding Table 2 by recording the SA field (source MAC address)and the port receiving the frame as part of the self-learning process.
3. Look up its Subforwarding Table 1 to determine the DSID based on the DA field.
4. Select a direction to forward the frame based on the ring topology information.
5. Set the wrap field to 0 to indicate that the frame has not been wrapped (see Subsection3.B for details).
6. Calculate HEC over the DSID, SSID and utility fields.
7. Set the TTL field to be equal to the number of ring switches in the ring.
8. Insert the RER header (including DSID, SSID, utility, HEC and TTL fields) into theframe.
When a ring switch receives a transit frame from its neighbor, it has the following threeoptions: (a) discard the frame, (b) accept the frame and strip it off the ring, or (c) forwardthe frame to the next switch. If the frame is invalid, it is discarded. If the frame is validand the ring switch itself is the destination, then the frame is accepted and delivered tothe destination host. Otherwise, a valid frame is forwarded to the next switch in the samedirection in which it was received. A ring switch carries out the following operations for atransit frame received from its neighbor:
1. Decrement TTL (The frame is discarded if its TTL is zero).
2. Recompute the HEC and compare it with the original HEC value in the receivedframe. The frame is discarded if these do not match.
3. Update its Subforwarding Table 1 by recording (DSID, DA) and (SSID, SA) as partof its self-learning process.
4. Compare the DSID field with its own switch ID. If these do not match, forward theframe to the next switch. If they match and it is a unicast frame, strip the frame offthe ring, remove its RER header, and look up Subforwarding Table 2 to determinethe exit port via which the destination host can be reached. Note that multicast orbroadcast frames will not be stripped off the ring but will continue to flow in the ringuntil they are eventually stripped by their ingress (source) switches.
It should be noted that only the HEC will be recomputed to confirm the validity of thereceived frame. The FCS will not be regenerated in ring switches in order to save process-ing time and speed up frame forwarding. The reason for keeping the TTL field outsideHEC protection is that, in such a case, the HEC field does not have to be recomputed andrewritten at transit switches.
2.E. Prevention of Frame Circulation
Without the use of the TTL field, frames would continue to circulate in the ring forever iftransmitted frames fail to get stripped off the ring properly. Unicast frames are normallystripped by the egress switch. These may also be stripped by the ingress switch if the egressswitch fails to do so. Multicast and broadcast frames are stripped only by the ingress switch.If both the ingress and egress switches drop out of the ring while a unicast frame is in flight,or if the ingress switch drops out while its multicast or broadcast frame is in flight, thenthe frame may circulate around the ring indefinitely. This problem is solved by including aTTL field in each frame. Each switch forwarding the frame decrements the TTL field anddiscards the frame whenever it reaches zero. By setting the TTL field to an initial valuecorresponding to the number of switches in the ring, we can ensure that the frame willreach each switch but will be limited to only one complete rotation around the ring.
3. Failure Detection and Recovery
Dual-ring networks have built-in natural resiliency because there are always two paths be-tween any two ring switches during normal operations. These two paths are in opposite di-rections and are generally not of the same length in terms of hop counts. Normally, framesare forwarded in the direction of the shorter path. In case of a link or node failure, affectedframes are directed to the alternative path to avoid service disruption. Failure detection andrecovery is achieved through periodic exchanges of keep-alive frames. Receiving keep-alive frames from a neighbor indicates that both the neighbor and the link to the neighborare working properly. The frequency of sending keep-alive frames can be configured on thebasis of operating requirements. A high frequency implies that the switch can detect lossof connectivity with a neighbor rapidly but at the expense of consuming more bandwidth.Since the maximum frame size is 1531 bytes (including the RER header of 5bytes), whichcorresponds to a transmission time of 1.225 microseconds (µs) at the bit rate of 10 Gbit/s,a keep-alive frame sent once every 5 to 10 milliseconds (ms) is considered to be adequate.Keep-alive frames are only sent to the neighbors. A link is considered to be down if nokeep-alive frame is received from that link for three consecutive keep-alive frame-sendingperiods. Two protection schemes, i.e., wrapping and steering, which were originally usedin the SONET/SDH rings, have been specified for the RPR [17]. These two schemes arealso applicable to the RER with minor modifications. As described in Subsections 3.Aand 3.B, these two schemes have their own advantages and disadvantages. An alternativescheme discussed in Subsection 3.C is the combination of wrapping and steering schemes,which combines the advantages of both schemes. We have carried out simulations to studythe performance differences among these three protection schemes (see Subsection 5.B fordetails).
The steering protection scheme is similar to the SONET/SDH unidirectional path switchedring (UPSR) protection. When it is applied to the RER, switches that detect a link/nodefailure are required to report this failure to other switches in the RER ring. The ring topol-ogy in every ring switch then gets updated. An example is shown below. Switch S1 sendsframes to switch S5 via the path S1-S2-S3-S4-S5 in the normal situation, as illustrated inFig. 4(a). After a link failure occurs between S4 and S5, S1 redirects the frames via pathS1-S0-S9-S8-S7-S6-S5 (6 hops), as illustrated in Fig. 4(b). Before the completion of thering topology updating process, frames destined to a switch beyond the point of failure aredropped at the edges.
31
(a) Normal situationS8
S4
S5
S1 S0 S2 S3
S9 S7 S6
S8
S4 (Edge)
S5 (Edge)
S1 S0 S2 S3
S9 S7 S6
Fiber Cut
(c) Wrap protection
Fig. 4 RER protection schemes
(b) Steering protectionS8
S4 (Edge)
S5 (Edge)
S1 S0 S2 S3
S9 S7 S6
Fiber Cut
Fig. 4. RER protection schemes.
We examine here the protection-switching time and the traffic dropped due to fiber-cutfailures. The protection-switching time is the time span from the instant the failure occursto the instant the network is recovered. It can be calculated as
tswitching = tdetect + tdiscovery + tsteer, (1)
where tdetect denotes the time to detect the failure. In RER, tdetect is set to three consecutivekeep-alive frame-sending periods. After failure is detected, switches that detect the failurereport this topology change to other switches by initiating a ring topology discovery pro-cess. tdiscovery is the time required for all nodes to complete the ring topology discoveryprocess, and tsteer denotes how fast a ring switch can switch itself to the steering mode aftercompletion of the ring topology discovery process. Normally tsteer is very small comparedto tdetect or tdiscovery and is hence negligible. Equation (1) is applicable for both dual- andsingle-fiber cut failures.
The amount of traffic dropped due to fiber cut failures can be calculated approximately
The total traffic dropped D contains two parts. The first part is λ∗tswitching, which representsthe traffic dropped at the two edge switches during the protection-switching time where λ
denotes the line rate. After the protection-switching time, there might still be some trafficdestined to destination switches beyond the point of failure that has been put in the ring-switch transit buffers and has not yet reached the point of failure. This traffic will eventuallyreach the point of failure and will be dropped there (the second part of traffic dropped).Dbuffer represents the amount of traffic dropped in this fashion. Equation (2) is applicable fortransmissions along both directions in dual-fiber cut failures. For single-fiber cut failures,Eq. (2) is only applicable for transmissions along the direction where the fiber is cut, andthere will be no drop of frames for transmissions along the other direction. As can be seenin Eq. (2), traffic loss in the steering scheme is affected by the ring size. A larger ring sizeincurs a longer tdiscovery, and hence leads to a larger amount of traffic dropped.
3.B. Wrapping Protection Scheme
The wrapping protection scheme is similar to the SONET/SDH bidirectional line switchedring (BLSR) protection. With this scheme, a link failure causes frames to be wrapped(looped back) at the two switches (referred to as edge switches in the following discussion)adjacent to the failure. Using the same network as in the previous example, S1 sends framesvia path S1-S2-S3-S4-S3-S2-S1-S0-S9-S8-S7-S6-S5 (12 hops) after a link failure betweenS4 and S5, as illustrated in Fig. 4(c). In the wrapping protection scheme, only edge switchesare involved, while the other switches may not be aware of the failure. Hence this schemeis simple and very fast. Fast recovery time ensures that very few frames are dropped andconsequently few retransmissions are subsequently required. However, as shown in the ex-ample, frames may have to pass through a much larger number of switches as comparedwith the situation where the steering protection scheme is used. Under wrapping protection,edge switches need to rewrite the TTL field to the number of switches in the ring before theframe is wrapped in order to maintain the same node space as originally supported. Edgeswitches also need to rewrite the wrap field to 1 indicating that the frame has been wrapped.Multicast or broadcast frames can be stripped by source switches if the wrap-field value is0 or by edge switches if the wrap-field value is 1. Note that the ring topology informationneeds to be updated in the steering mechanism, whereas this may not be necessary in thewrapping protection scheme. Since both protection schemes are transparent to higher-layerservices such as IP, no noticeable interruption in service is expected to occur in either case.Unlike SONET/SDH rings, the self-healing RER architecture does not require one to setaside half of the fiber span capacity for protection. Hence care needs to be taken whenfailure occurs in the RER. Traffic originally flowing in the two ring directions may add upand flow in one ring direction after failure. This aggregate traffic may possibly exceed linkbandwidth capacity and result in the dropping of frames (see Subsection 5.B.2).
We once again examine here the protection-switching time and traffic dropped due tofiber cut failures. The protection-switching time for the wrapping scheme can be simplycalculated as:
tswitching = tdetect + twrap, (3)
where tdetect again denotes the time to detect the failure, and twrap denotes how fast an edgeswitch can switch itself to the wrapping mode after detecting the failure. Normally twrap isvery small compared with tdetect and hence negligible. Equation (3) is applicable for bothdual-fiber cut and single-fiber cut failures.
The amount of traffic due to fiber cut failures can be calculated as:
D = λ∗tswitching = λ
∗ (tdetect + twrap), (4)
where λ again denotes the line rate. As in the steering scheme, there might still be sometraffic destined to destination switches beyond the point of failure that has been put in thering switch transit buffers and has not yet reached the point of failure after the switchingtime. This traffic will eventually reach the edge switches and be wrapped there (instead ofbeing dropped in the steering scheme). Equation (4) is applicable for transmissions alongboth directions in dual-fiber cut failures. For single-fiber cut failures, it is only applicablefor transmissions along the direction where the single fiber is cut.
3.C. Wrapping-Then-Steering Protection Scheme
The wrapping scheme is faster, but transmitting paths after wrapping may not be opti-mal. The steering scheme is slower, but transmitting paths after steering is optimal. Thewrapping-then-steering protection scheme (denoted as WTS) discussed here combines theadvantages of both the wrapping and steering schemes. In this scheme, the nodes that de-tect the failure first perform wrapping to ensure that the system is rapidly recovered and theamount of traffic dropped is minimized. They then inform other nodes to start a new ring-topology update. After the new ring topology is updated at each node, all ingress (source)ring switches forward frames according to the new ring topology information so that framescan be transmitted in the shortest paths, as in the steering scheme. For the WTS scheme,the amount of traffic dropped is also determined by Eq. (4). The drawback of the WTSscheme is that it makes the engineering of the network somewhat more complicated thanthe situation in which only the wrapping scheme or the steering scheme is used.
It may be noted that the protection offered by our proposed RER approach is onlyeffective for one failure per ring and cannot cope with two or more failures in a given ring.This is, however, in line with usual system expectations that a single failure (or inadvertentpower disconnection of one node) is likely to be the most likely failure event in a ring,and it is also the failure event that one typically designs for in a system. (For example, theexisting SDH/SONET rings are typically designed to handle a single failure such as a fibercable cut or a node crash).
4. Interconnection of Multiple Rings
Here we consider two approaches to interconnecting multiple RER rings. The first approachis a modified version of the transparent bridging technique [18–21] used in the original Eth-ernet. With this modified transparent bridging technique, frames with unknown destinationsare broadcast and bridge switches relay broadcast frames from one ring to another. As anexample, consider two RER rings (RER1 and RER2), which are interconnected by an RERbridge switch, where a transmission is initiated in RER1 for which the destination host re-sides in RER2. The source switch cannot locate the destination host and hence broadcaststhe frame in RER1 either in the clockwise direction or in the counterclockwise direction.When the bridge switch receives the frame, it copies it, rewrites the SSID field with itsown switch ID and the TTL field to the number of ring switches in RER2. Then the bridgeswitch checks to see if it knows the destination. If it does, the broadcast frame in RER1 isstripped and the copied frame is unicast in RER2. Otherwise, the broadcast frame in RER1continues to be transmitted in RER1, and the copied frame is broadcast in RER2. It shouldbe noted that the frame broadcast in RER2 will not circulate in RER2, since the bridgeswitch will strip off frames that are sent out by itself by identifying the SSID. Since framecirculation among the rings will not take place, the STP protocol does not need to be runon the bridge switch ports. This is important as this will considerably enhance the network
throughput, particularly in situations where many RER rings are interconnected by trans-parent bridge switches. In order to keep forwarding tables in each ring switch small, theself-learning process of MAC addresses on frames from remote rings is disabled for ringswitches, i.e., a ring switch will not perform self-learning for frames whose SSID matcheswith the switch ID of the bridge switch. The modified transparent-bridging technique maynot be scalable when many rings are interconnected. This is due to the fact that a bridgeswitch would need a large forwarding table and the capability for fast processing, as it hasto record all host MAC addresses passing through it.
The second approach is to replace bridge switches with routers, which would then per-form layer-3 routing based on destination IP addresses. The routing functions are invokedonly when frames need to go across from one RER ring to another RER ring. Routersimprove network scalability but slow down frame forwarding. It is also possible to havea combination of bridge switches and routers if there are many interconnected rings. Thedetails of these choices are beyond the scope of this paper and are not discussed furtherhere.
5. Simulations and Discussions
Using network simulator version 2 (NS2) [22], we have carried out simulation studies tocompare the performance differences among the RSTP, RPR, and RER approaches, and toinvestigate different protection schemes, i.e., the wrapping scheme, the steering scheme,and the wrapping-then-steering (WTS) scheme.
5.A. Simulation on Different Ring Approaches
Three ring networks employing the RSTP, RPR, and RER with 10 Gbit/s fiber links of5 km each have been simulated. We assume that every ring switch has two buffers, i.e., atransit buffer and a transmit buffer. Transit buffers are used to receive or deliver framesfrom or to neighboring ring nodes, whereas transmit buffers are used to load or deliverframes from or to subnetworks. Frames in transit buffers are given higher priority for trans-mission than frames in transmit buffers. In the simulation study, the transit and transmitbuffers in each ring switch were set at 256 KB and 1 MB, respectively. This was doneto keep the frame losses resulting from buffer overflow small. Frame arrivals from sub-networks follow a Poisson process, and frame sizes follow a uniform distribution from64 Bytes to 1518 Bytes (both inclusive). Frames are randomly addressed to various des-tinations with equal probability. We assume that all traffic is unicast and that no networkfailures occur during simulations. RSTP, RPR, and RER protocols are applied in the samephysical ring topology. Note that RSTP will configure the ring topology as a tree by block-ing certain ports of a ring switch so as to prevent frames from circulating in a closed loop.The root node lies in the center of the RSTP network. When frames enter the RPR networkfrom subnetworks, a RPR overhead of 16 bytes (according to the latest RPR draft [17]) willbe added to the original frames. Likewise, an RER header of 5 bytes will be added to theoriginal frames when the frames enter the RER network. In this paper, the offered trafficis defined as the total bit rate generated by all the nodes. The carried traffic is defined asthe total bit rate successfully transmitted by all the nodes. The normalized offered traffic isdefined as the average total bit rate generated by all the nodes within the simulation timedivided by the link speed (10 Gbit/s). The normalized carried traffic is defined as the aver-age total bit rate successfully transmitted by all the nodes, divided by the link speed. Theaverage delay of frames is defined as the average time required to move a frame from thetime when it enters the ring to the time it leaves the ring. The throughput is defined as theratio of the normalized carried traffic to the normalized offered traffic.
Figure 5 shows the average delay versus the normalized offered load for three different
ring networks of 10 nodes. In the absence of congestion, the most important component ofthe delay is the mean distance between end nodes. The propagation delay between adjacentring switches is 25 µs, since each link is 5 km. In the RSTP network, the average delaysexperienced by frames generated at the root node and at the edge node (the farthest fromthe root node) are very different. We denote RSTP root and RSTP edge as the measure ofaverage delay of frames sourced by the root node and the edge node of the RSTP network,respectively. The RSTP root shows the best average delay performance, and RSTP edgeshows the worst in the RSTP network. As shown in the figure, the RSTP edge exhibitshigh delay (∼ 128 µs) even under low offered load, whereas RSTP root, RPR, and RERexhibit much lower delay (∼ 70 µs). As the offered load increases, all the three networksshow increasing average delay because of queuing of frames in transit buffers. RSTP startsqueuing frames at the normalized offered traffic load of 1.2, which is much lower than RPRat 2.2 and RER at 2.4. As can be seen from Fig. 6, RER achieves the highest carried trafficamong the three networks. RER saturates at a higher normalized traffic load of 3.7, whileRSTP saturates at the much lower load of 2.7. Figure 7 shows the throughput versus thenumber of nodes for all the three networks with a normalized offered load of 2.5. As can beseen, the throughput in all the three networks decreases as the number of nodes increases.Once again RER performs the best, and RSTP performs the worst.
32
Fig. 5 Average delay versus normalized offered traffic for networks with 10 nodes
Fig. 5. Average delay versus normalized offered traffic for networks with 10 nodes.
Overall, the poor performance of the RSTP network is mainly due to the followingtwo reasons. First, the RSTP network has largest mean distance between end nodes. Thetransmitting paths in RSTP are always longer than or equal to the ones in the RPR or RERnetworks because of the open-loop structure of the RSTP network. Second, traffic becomesunbalanced in the RSTP network due to its open-loop structure, even though uniform trafficis assumed for the simulations. As a result, the root switch gets saturated quickly and thatdegrades the network performance. The performance differences between the RER andRPR are primarily due to the longer header overhead in the RPR. In practice, RPR requiresmore complicated implementation than the RER. The RER is shown to be the best solutionamong these three networks. As in the case of the RPR, the RER ensures smaller meandistance between ring switches by employing the dual-ring architecture. The RER protocolis simpler than the RPR protocol, resulting in less bandwidth consumption due to frameoverheads. In addition, the RER uses bandwidth effectively and has an efficient switchingmechanism.
As discussed in Section 3, the protection schemes that may be applied to RER are thesteering scheme, the wrapping scheme and the wrapping-then-steering scheme (denoted asWTS). The objectives of this simulation are to study the performance differences betweenthe three different protection schemes under various offered loads. The simulation settingsare the same as those described in Subsection 5.A except that fiber cut failures were sched-uled to occur. The number of nodes is set to 10 for this simulation. We assume that thetraffic is started at t = 0.15 s and that a fiber cut failure between ring switches S(4) and S(5)occurs at t = 0.2 s. In addition, the period of sending keep-alive messages is set to 5 ms.The traffic loss is defined as the total bit rate dropped by all ring nodes. We have simulatedtwo scenarios for the three different protection schemes. In the first scenario, the offeredload is light (i.e., the aggregate traffic does not exceed the link speed even if a link failureoccurs). In the second scenario, the offered load is moderate (i.e., the aggregate traffic doesnot exceed the link speed in the normal operation, but exceeds the link speed when a linkfailure takes place).
5.B.1. Scenario One—Light Offered Traffic Load
In this scenario, access traffic rate at each node was set to 1 Gbit/s. Both double-fiber andsingle-fiber cut failures were simulated. Figs. 8(a) and 8(b) show the carried traffic underthe double-fiber cut failure and the single-fiber cut failure, respectively. Traffic is startedat t = 0.15 s. The carried traffic is approximately 10 Gbit/s before the fiber cut failureoccurs (0.15 s < t < 0.2 s), which is equal to the offered traffic (10×1 Gbit/s). The carriedtraffic drops sharply when the failure occurs (t = 0.2 s). After the failure is detected andthe network recovers, the carried traffic goes back to 10 Gbit/s, which is the same levelas before the failure. The wrapping scheme and the WTS scheme both make the networkrecover from failure at around t = 0.215 s. The steering scheme recovers the network fromfailure at around t = 0.235 s. As can be seen, the wrapping scheme and the WTS schemeboth make the network recover from failure rapidly (approximately 15 ms), whereas thesteering scheme takes a much longer time (approximately 35 ms) to do so. This is becausethe protection-switching time is mainly determined by the failure-detection time for boththe wrapping and WTS schemes, whereas it is determined by both the failure-detection timeand ring-topology discovery-processing time for the steering scheme.
Figures 9(a) and 9(b) show the traffic loss under the double-fiber cut failure and thesingle-fiber cut failure, respectively. The area under each curve represents the total traf-fic loss for each of three protection schemes. Although the instantaneous traffic loss ineach protection scheme is approximately 2 Gbit/s/1 Gbit/s for the respective double-fiber/single-fiber cut failures, the protection-switching time for the steering scheme islonger in comparison with the wrapping and WTS schemes. As a result, the steering schemeincurs a larger total traffic loss in comparison with the wrapping and WTS schemes. An-other observation is that the traffic loss during the single-fiber cut failure is approximatelyhalf of the traffic loss during the double-fiber cut failure. The reason is that under a single-fiber cut failure only traffic flowing in a single direction is affected, whereas traffic flowingin both directions is affected under a double-fiber cut failure.
5.B.2. Scenario Two—Moderate Offered Traffic Load
In this scenario, access traffic rate at each node was set to 2 Gbit/s. Both double-fiberand single-fiber cut failures were simulated. Figs. 10(a) and 10(b) show the carried traf-fic under the double-fiber cut failure and the single-fiber cut failure, respectively. Com-pared to the light-offered-load scenario, the protection-switching time is approximatelythe same. One obvious difference is that in this moderate offered load, the carried traf-
Fig. 8 Comparison of carried traffic for different protection schemes with a light offered traffic load of 1Gbps at each node under (a) double-fiber cut failure and (b) single-fiber cut failure.
Fig. 8. Comparison of carried traffic for different protection schemes with a light offeredtraffic load of 1 Gbit/s at each node under (a) double-fiber cut failure and (b) single-fibercut failure.
Fig. 9 Comparison of traffic loss for different protection schemes with a light offered traffic load of 1Gbps at each node under (a) double-fiber cut failure and (b) single-fiber cut failure.
Fig. 9. Comparison of traffic loss for different protection schemes with a light offeredtraffic load of 1 Gbit/s at each node under (a) double-fiber cut failure and (b) single-fibercut failure.
fic is lower than the offered traffic after the network recovers from the failure, whilein the light offered load the carried traffic is equal to the offered traffic. As shown inthe two figures, the carried traffic after the network recovers from the failure is approxi-mately 18.3 Gbit/s/19.2 Gbit/s for the respective double-fiber/single-fiber cut failure forthe wrapping scheme, and 19.4 Gbit/s/19.8 Gbit/s for the steering and WTS schemes,which is less than the offered traffic of 20 Gbit/s. Figures 11(a) and 11(b) show the trafficloss under the double-fiber and single-fiber failures, respectively.
37
(a) Double-fiber cut failure
(b) Single-fiber cut failure
Fig. 10 Comparison of carried traffic for different protection schemes with a moderate offered traffic load of 2 Gbps at each node under (a) double-fiber cut failure (b) single-fiber cut failure
Fig. 10. Comparison of carried traffic for different protection schemes with a moderateoffered traffic load of 2 Gbit/s at each node under (a) double-fiber cut failure (b) single-fiber cut failure.
Compared to the light-load scenario, the traffic loss in the network is no longer zero foreach of the protection schemes after the network recovers from failure. The non-zero trafficloss is due to the fact that traffic originally flowing in two ring directions and taking shorterpaths (fewer link spans), would have to take longer paths (more link spans) and hence
Fig. 11 Comparison of traffic loss for different protection schemes with a moderate offered traffic load of 2Gbps at each node under (a) double-fiber cut failure and (b) single-fiber cut failure.
Fig. 11. Comparison of traffic loss for different protection schemes with a moderate offeredtraffic load of 2 Gbit/s at each node under (a) double-fiber cut failure and (b) single-fibercut failure.
would consume more link bandwidth. As a result, the aggregate traffic exceeds the linkbandwidth capacity, and that results in the dropping of frames. The traffic loss after networkrecovery is the highest for the wrapping scheme at approximately 1.7 Gbit/s/0.8 Gbit/s forthe respective double-fiber/single-fiber cut failure. This is because the affected transmit-ting paths become the longest after recovery resulting in heavy congestion in the network.The traffic loss after network recovery is lowest for the steering scheme at approximately0.6 Gbit/s/0.2 Gbit/s because the affected transmitting paths are shorter than those of thewrapping scheme after recovery, resulting in light congestion in the network. For the WTSscheme, the traffic loss after network recovery is approximately 1.7 Gbit/s/0.8 Gbit/s inthe wrapping mode and then quickly drops to 0.6 Gbit/s/0.2 Gbit/s upon switching to thesteering mode. Overall, the WTS scheme combines the advantages of both the steering andwrapping schemes to achieve a fast convergence time and low traffic loss after networkrecovery, and hence is more suitable to run in MANs environment.
6. Conclusion
We have proposed a new optical resilient Ethernet ring (RER) that is very well suited toMANs. We have described in detail the basic RER system design issues including RERarchitecture, frame format, frame forwarding and switching mechanism, and self-learningprocess. The proposed optical RER extends Ethernet’s simplicity, familiarity, and low costfeatures beyond LANs and into MANs, while simultaneously meeting the requirements thatone needs from MANs. These requirements are scalability to a large network size, efficientbandwidth utilization, and increased robustness against failures. The proposed RER canoperate in a single-ring or multiple-ring environment without using RSTP. Ring topologyspeeds up frames switching in the RER and also enables the RER to recover from a linkor node failure within 50 ms. Three different protection schemes are discussed, and theirperformance differences are studied through simulations. The proposed RER is scalable toa large network size by interconnecting multiple rings using a modified transparent bridgingtechnique. Simulation results have shown that the RER network can achieve significantlybetter throughput performance than the RSTP network. The proposed RER scheme wouldtherefore be an attractive alternative for high-capacity MANs.
The adoption of the RER approach proposed here would, however, require the incor-poration of the RER header into the Ethernet frame. This is necessary in order to providethe advantages associated with our RER approach as summarized above and discussed indetail in the paper. The extra header is a minor modification of existing approaches. Theadditional processing overhead associated with this is minimal and should not be difficultto do with current and future hardware. As in other ring-based systems, the RER approachprotects only against one failure in a ring and cannot protect against two or more failuresin a given ring. This is generally deemed to be adequate, since a single failure is expectedto be the most common failure event and is typically the failure event one designs for in asystem.
The important issues which are yet to be addressed are those connected with fairnessand quality of service (QoS). These issues need to be examined in conjunction with higher-layer protocols such as multiprotocol label switching (MPLS) and virtual LAN (VLAN),and would be beyond the scope of this paper. In this paper, we provide the basic RERdesign for extending Gigabit Ethernets to MANs and have focused on the related MAClayer design issues to show the advantages of the proposed approach.
References and Links[1] ITU-T, “Recommendation G.784: synchronous digital hierarchy (SDH) management,” June
1999, http://www.itu.int/ITU-T/.[2] H. Fujita, K. Sakai, and T. Aoyama, “SONET fiber-optic transmission systems,” Hitachi Review
44, 187–192 (1995).[3] A. P. Pillai, “Synchronous digital hierarchy (SDH),” Telecommun. 45, 61–71 (1995).[4] Resilient Packet Ring Alliance, http://www.rpralliance.org.[5] F. Davik, M. Yilmaz, S. Gjessing, and N. Uzun, “IEEE 802.17 resilient packet ring tutorial,”
IEEE Commun. Mag. 42, 112–118 (2004).[6] P. Yuan, V. Gambiroza, and E. Knightly, “The IEEE 802.17 media access protocol for high-
speed metropolitan-area resilient packet rings,” IEEE Netw. 18, 8–15 (2004).[7] S. Spadaro, J. Sole-Pareta, D. Careglio, K. Wajda, and A. Szymanski, “Positioning of the RPR
telecommunications and information exchange between systems—local and metropolitanarea networks—common specifications. Part 3: media access control (MAC) bridges,” Dec.1998.
[12] “Local and Metropolitan Area Networks—Common Specifications—Part 3: Media AccessControl (MAC) Bridges—Amendment 2: Rapid Reconfiguration,” amendment to IEEE Std802.1D, 1998 ed., June 2001.
[13] M. Munafo, F. Neri, C. C. Scarpati, and A. Vasco, “Analysis of the spanning tree and sourcerouting LAN interconnection schemes,” J. Internetworking: Res. Experience 5, 121–149 (1994).
[14] IEEE Rapid Spanning Tree Protocol, http://standards.ieee.org/getieee802/.[15] Cisco, “Understanding rapid spanning tree protocol (802.1w),” http://www.cisco.com/
warp/public/473/146.pdf.[16] Z. Lian, W. D. Zhong, S. K. Bose, and Y. Wang, “Resilient Ethernet ring for metropolitan area
networks,” in Proceedings of the International Conference on Communication Systems (ICCS),Singapore, Sept. 2004, pp. 316–320.
[17] IEEE 802.17 Resilient Packet Ring Working Group, draft v. 1.1, Oct 2003, http://www.ieee802.org/17/.
![Page 1: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/1.webp)
![Page 2: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/2.webp)
![Page 3: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/3.webp)
![Page 4: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/4.webp)
![Page 5: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/5.webp)
![Page 6: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/6.webp)
![Page 7: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/7.webp)
![Page 8: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/8.webp)
![Page 9: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/9.webp)
![Page 10: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/10.webp)
![Page 11: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/11.webp)
![Page 12: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/12.webp)
![Page 13: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/13.webp)
![Page 14: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/14.webp)
![Page 15: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/15.webp)
![Page 16: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/16.webp)
![Page 17: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/17.webp)
![Page 18: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/18.webp)
![Page 19: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/19.webp)
![Page 20: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/20.webp)
![Page 21: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/21.webp)
![Page 22: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/22.webp)
![Page 23: Optical resilient Ethernet rings for high-speed MAN networks [Invited]](https://reader039.documents.pub/reader039/viewer/2023121007/6361c3662fd302ae640962bb/html5/page/23.webp)
