EPN 4.0 Mobile Services DIG A Related Documentation A-1 Contents 4 EPN 4.0 Mobile Services Design...

EPN 4.0 Mobile Transport Design and Implementation GuideSeptember 2014

Building Architectures to Solve Business Problems

iiAbout Cisco Validated Design (CVD) Program

About Cisco Validated Design (CVD) Program

The CVD program consists of systems and solutions designed, tested, and documented to facilitate faster, more reliable,

and more predictable customer deployments. For more information visit http://www.cisco.com/go/designzone.

ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLEC-

TIVELY, "DESIGNS") IN THIS MANUAL ARE PRESENTED "AS IS," WITH ALL FAULTS. CISCO AND ITS

SUPPLIERS DISCLAIM ALL WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF

MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING

FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUP-

PLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES,

INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF

THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED

OF THE POSSIBILITY OF SUCH DAMAGES.

THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE. USERS ARE SOLELY RESPONSIBLE FOR

THEIR APPLICATION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR

OTHER PROFESSIONAL ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT

THEIR OWN TECHNICAL ADVISORS BEFORE IMPLEMENTING THE DESIGNS. RESULTS MAY VARY

DEPENDING ON FACTORS NOT TESTED BY CISCO.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of

California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved.

Copyright © 1981, Regents of the University of California.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other

countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks

mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relation-

ship between Cisco and any other company. (1110R)

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses

and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in

the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative

content is unintentional and coincidental.

EPN 4.0 Mobile Transport Design and Implementation Guide

© 2014 Cisco Systems, Inc. All rights reserved.

http://www.cisco.com/go/designzone

http://www.cisco.com/go/trademarks

Design and Implementation Guide

C O N T E N T S

C H A P T E R 1 Introduction 1-1

C H A P T E R 2 Mobile Transport Services Overview 2-1

C H A P T E R 3 Mobile Transport Services Architecture 3-1

L3 MPLS VPN Service Model for LTE 3-1

Route Scale Control for LTE L3 MPLS VPN Service Model 3-3

Multicast Service Model for LTE eMBMS 3-7

L2 MPLS VPN Service Model for 2G and 3G 3-9

Mobile Transport Capacity Monitoring 3-10

C H A P T E R 4 Functional Components 4-1

Synchronization Distribution 4-1

Timing Distribution with Ethernet NIDs 4-3

Time Asymmetry Correction between Boundary Clocks 4-3

High Availability 4-4

Quality of Service 4-4

Operations, Administration, and Maintenance 4-4

C H A P T E R 5 Mobile Transport Services Implementation 5-1

Mobile Services Implementation with MPLS Access 5-1

L3 MPLS VPN Service Model for LTE Implementation 5-1

MPLS VPN Transport for LTE S1 and X2 Interfaces 5-1

MPLS VPN Control Plane 5-8

Multicast Service Model for LTE eMBMS Implementation 5-19

L2 MPLS VPN Service Model for 2G and 3G Implementation 5-27

Mobile Services Implementation with Non-MPLS Access 5-33

L3 MPLS VPN Service Model for LTE Implementation 5-33

MPLS VPN Core Transport for LTE S1 and X2 Interfaces 5-34

Mobile Backhaul over Hub-and-Spoke Access Topologies 5-38

Mobile Backhaul over Ring Access Topologies 5-42

L2 MPLS VPN Service Model for 2G and 3G Implementation 5-45

1EPN 4.0 Mobile Services

Contents

Mobile Transport Capacity Monitoring 5-55

NetFlow Configurations on MTG-1 (MTG-9006-K1501) 5-56

Report Generation from Prime Performance Manager 5-57

C H A P T E R 6 Functional Components Implementation 6-1

Synchronization Distribution Implementation 6-1

Hybrid Model Configuration with a Third-Party Grandmaster Clock Source 6-1

Aggregation Node Configurations for SyncE and 1588v2 PTP Hybrid BC with BMCA 6-6

Cisco ASR 903 Series PAN Configuration for SyncE and 1588v2 PTP Hybrid BC with BMCA and Asymmetry Correction 6-10

Access Node Configuration for SyncE and 1588 Hybrid BC with Asymmetry Correction 6-12

Hybrid Model Configuration with a Cisco ASR 9000 Series Router as Grandmaster Clock Source 6-16

High Availability Implementation 6-23

MPLS VPN-BGP FRR Edge Protection and VRRP 6-23

Mobile Transport Gateway 1 6-23

Mobile Transport Gateway 2 6-24

Cell Site Gateway 6-26

G.8032 and VRRP for Ethernet Access 6-26

G.8032 Ethernet Ring Protection Switching 6-26

VRRP 6-27

Pseudowire Redundancy for ATM and TDM Services 6-29

TDM Services 6-29

ATM Services 6-31

C H A P T E R 7 Quality of Service Implementation 7-1

CSG QoS Configuration 7-2

Class Maps 7-2

eNodeB UNI QoS Policy Map 7-3

In MPLS Access 7-3

In Non-MPLS Access 7-4

TDM CEM UNI QoS Policy Map 7-6

PAN Configuration for ATM and TDM UNIs 7-6

MTG Configuration for Ethernet, ATM and TDM UNIs 7-6

Ethernet UNI QoS Policy Maps 7-6

ATM UNI QoS Policy Maps 7-7

TDM UNI QoS Policy Map 7-8



Contents

C H A P T E R 8 OAM Implementation 8-1

Service OAM Implementation for LTE and 3G IP UMTS RAN Transport with MPLS Access 8-2

Service OAM Implementation for ATM and TDM Circuit Emulation Pseudowires for 2G and 3G RAN Transport 8-2

Transport OAM 8-3

IP SLA Configuration 8-3

IP SLA Responder Configuration 8-3

Cell Site Gateway Initiator Configuration for IP SLA 8-3

Pre-Aggregation Node Initiator Configuration for IP SLA 8-4

Mobile Transport Gateway Initiator Configuration for IP SLA 8-4

VPN: Jitter Probes 8-4

Reaction Configuration 8-4

C H A P T E R A Related Documentation A-1



Contents




C H A P T E R 1
Introduction
The Cisco® Evolved Programmable Networks (EPN) System Release 4.0 continues to develop the design of the Unified MPLS for Mobile Transport (UMMT) and Fixed Mobile Convergence (FMC) systems as part of a multi-year ongoing development program that builds towards a flexible, programmable, and cost-optimized network infrastructure, targeted to deliver in-demand fixed and mobile network services.

As described in the Cisco EPN 4.0 System Concept Guide, the EPN System follows a layered design, with each layer building on top of the previous. This guide focuses on the design and implementation aspects of the service infrastructure layer, with specific focus on Transport Services for mobile technologies spanning across all generations, including 2G, 3G, and 4G.

The following essential features complement transport of mobile services:

• Network synchronization (physical layer and packet based)

• Hierarchical quality of service (H-QoS)

• Operations, administration, and maintenance (OAM)

• Performance management

• Fast convergence

The Cisco EPN System architecture also optimizes around advanced 4G requirements such as the following:

• Direct enhanced NodeB (eNodeB) communication through the X2 interface

• IPv4 and IPv6 Multicast for optimized video transport based on Evolved Multimedia Broadcast Multicast Services (eMBMS) architecture

• Virtualization for Radio Access Network (RAN) sharing

• Distribution of the Evolved Packet Core (EPC) gateways

1-1EPN 4.0 Mobile Transport

https://docs.cisco.com/share/page/site/nextgen-edcs/document-details?nodeRef=workspace://SpacesStore/42481df4-4faa-47d8-a34f-f551eb13e746

Chapter 1 Introduction




C H A P T E R 2
Mobile Transport Services Overview
Cisco developed the Cisco EPN System to simplify the end-to-end mobile transport/backhaul architecture. EPN achieves this by decoupling the transport and service layers of the network, thereby allowing these two distinct entities to be provisioned and managed independently. The Unified MPLS Transport layer seamlessly interconnects the access, aggregation, and core MPLS domains of the network infrastructure with hierarchical Label-Switched Paths (LSPs). Once this Unified MPLS Transport is established (a task that only needs to be undertaken once), a multitude of services can be deployed on top of it. These services can span any location in the network without restricting topological boundaries.

This guide focuses on delivery of mobile backhaul services across the Cisco EPN System, which provides a comprehensive RAN backhaul solution for transport of LTE, legacy 2G Global System for Mobile Communications (GSM), existing 3G Universal Mobile Telecommunications Service (UMTS) services and small cells (Wi-Fi and Hybrid Radio). Transport of LTE and IP-enabled 3G/Wi-Fi services is provided by a highly scaled MPLS L3VPN. Legacy GSM and ATM-based UMTS backhaul are provided by pseudowire emulation edge-to-edge (PWE3)-based transport of emulated time-division multiplexing (TDM) and ATM circuits, respectively. Enhanced Multimedia Broadcast Multicast Service (eMBMS) transport is provided by multicast-based mechanisms, minimizing packet duplication within the transport network.

Figure 2-1 provides an overview of the backhaul of legacy 2G/3G and LTE services across the Cisco EPN System with an IP/MPLS-enabled access domain. All packet-based transport mechanisms originate at the cell site gateway (CSG) in this scenario.


Chapter 2 Mobile Transport Services Overview

Figure 2-1 Mobile Services Overview-Unified MPLS Access

Figure 2-2 provides an overview of the backhaul of legacy 2G/3G and LTE services across the Cisco EPN System with a native Ethernet or TDM-based access domain. The MPLS VPWS and L3 VPN transport mechanisms originate at the pre-aggregation node (PAN) in this scenario. The Ethernet-based access domain may be based on hierarchical/multipoint microwave access or on Ethernet G.8032 access rings over fiber or microwave connections.

IP/MPLSTransport

2975

85

DWDM, Fiber Rings, Mesh TopologyDWDM, Fiber Rings, H&S, Hierarchical TopologyFiber or uWave Link, Ring

Mobile Packet Core NetworkMobile Access Network Mobile Aggregation Network

IP/MPLS Transport

MPLSVPN

MPLS VPN(v4/v6)

MPLS VPN(v4/v6)

MPLS VPN(v4/v6)

IP/MPLS Transport

Mobile Transport PEASR-9000

TDM BTS, ATM Node B

IP eNB

ATM orTDM

S1/X2 and M1/M3 require different IP endpoints and VLAN interfaces in eNB when IP/PIM is used for M3/M1

SGSN

GGSN

RNCBBC

AToM Pseudowire

v4 or v6 MPLS VPN for S1, X2IP/PIM v4/v6 for eMBMS M3/M1

S/PGW

S/PGW

Covered by theISEM System

S1-C/M3

S1-U

MME

X2-C, X2-U

OptionalNID

ME-1200ZTD NID

ASR-903, ASR-9001 ASR-9000 CSR-3CRS-3 ASR-901, ASR-920Pre-Aggregation Node Aggregation Node Core NodeCore NodeCell Site Gateway

Mobile Transport Gateway





Figure 2-2 Mobile Services Overview-TDM and Ethernet Access

Figure 2-3 provides an overview of the transport of eMBMS interfaces across the Cisco EPN System with an IP/MPLS-enabled access domain. All packet-based transport mechanisms are between the cell site gateway (CSG) and the respective service gateways in this scenario. The Unified MPLS Core and Aggregation enable mLDP-labeled multicast transport and the access network distributes eMBMS with PIM v4/v6 and IP multicast.

Figure 2-3 eMBMS Service Interfaces in Cisco EPN System Design

MPLS VPN(v4/v6)

RNCBBC

2948

53ASR-903, 9001 ASR-9000

DWDM, Fiber Rings, Mesh TopologyDWDM, Fiber Rings, H&S, Hierarchical TopologyEthernet/TDM Microwave


CRS-3CRS-3

IP/MPLS Transport

Partners: NSN, NEC, SIAE,Ceragon, Dragonwave

MPLS VPN(v4/v6)

Ethernet, G.8032

Ethernet,G.8032

MPLS VPN(v4/v6)

Aggregation Node Aggregation Node Core NodeCore NodeMicrowave Systems

IP/MPLS Transport

Mobile Transport PEASR-9000

TDM BTS, ATM Node B

SDH/SONET

ATM orTDM

SGSN

GGSN

AToM Pseudowire

v4 or v6 MPLS VPNs for S1, X2IP/PIMv4/v6 for eMBMS M3/M1

X2-C, X2-U

S/PGW LMA

S/PGW LMA

Covered by theISEM System

S1-C/M3

S1-U

MME



S1/X2 and M1/M3 require different IP endpoints and VLAN interfaces in eNB when IP/PIM is used for M3/M1

Routed BVI, over Bridge Domainwith PW and EFP

OptionalNID

ME-1200ZTD NID

Cell Site GatewayASR-901

RP

W

2975

87

CSG

MPLS VPN (v4 or v6)

MBMS-GW

M3

S1-U S11

S11S1-U

S1-CMME

X2

X2

X2

MTG-3 MTG-1VRF

Global

VRF VRF

VRRP

VRF

VRF

VRF

VRF

MTG-2

IP/MPLS Ethernetor uWave Transport



IP/MPLS Transport IP/MPLS Transport

Mobile Transport Gateway PEASR-9000

Mobile Transport Gateway (S1, M3)ASR-9000

ASR-903, ASR-9001 ASR-9000 CSR-3CRS-3 ASR-901, ASR-920Pre-Aggregation Node Aggregation Node Core NodeCore NodeCell Site Gateway







C H A P T E R 3
Mobile Transport Services Architecture
The Cisco EPN System design provides transport for both legacy and current mobile services. To accomplish this on a single network, MPLS service virtualization is employed, which provides emulated circuit services via L2VPN for 2G and 3G services and L3VPN services for IP-enabled 3G, 4G and Wi-Fi services.

All the service models are outlined in this chapter, which includes the following major topics:

• L3 MPLS VPN Service Model for LTE, page 3-1

• Multicast Service Model for LTE eMBMS, page 3-7

• L2 MPLS VPN Service Model for 2G and 3G, page 3-9

• Mobile Transport Capacity Monitoring, page 3-10

L3 MPLS VPN Service Model for LTEThe Cisco EPN System supports mobile SPs that are introducing 3G Universal Mobile Telecommunications Service (UMTS)/IP, 4G LTE-based next generation mobile access, and Wi-Fi in order to scale their mobile subscribers and optimize their network infrastructure cost for the mobile broadband growth. To this end, the system proposes a highly-scaled MPLS VPN-based service model to meet the immediate needs of LTE S1 and X2 interfaces and accelerate LTE deployment. The MPLS VPN transport design can be downgraded to support the IP Iub transport for UMTS and the CAPWAP tunnel for Wi-Fi across access points and the Wireless LAN Controller. See Figure 3-1.

Figure 3-1 LTE Backhaul Service

The Mobile RAN includes cell sites with enhanced NodeBs (eNB) that are connected:

• Directly in a point-to-point fashion to the PANs utilizing Ethernet fiber or microwave, OR

2932

96


eNode B

MPLS VPN (v4 or v6)

SGW

SGW

S1-C

S1-U

S1-U

S1-C MME

X2-C, X2-U

MTG

MTG

MTG


Chapter 3 Mobile Transport Services Architecture L3 MPLS VPN Service Model for LTE

• Through CSGs connected in G.8032-protected ring topologies over Ethernet fiber or microwave transmission, OR

• Through CSGs connected in ring topologies by using MPLS/IP packet transport over Ethernet fiber or microwave transmission.

Furthermore, eNodeBs can be connected to a CSG or a PAN directly, or through an intermediate Ethernet NID device, owned by the backhaul operator and located at the mobile base stations to enhance end-to-end service visibility. The connectivity model between the CSG/PAN and the NID involves the use of two VLANs carrying the following types of traffic:

• Mobile data and control unicast traffic, for inter-eNodeB and MPC gateway communication.

• Mobile data multicast traffic, for eMBMS-enabled mobile architecture.

The cell sites in the RAN access are collected in a MPLS/IP pre-aggregation/aggregation network that may be comprised of a physical hub-and-spoke or ring connectivity that interfaces with the MPLS/IP core network that hosts the EPC gateways.

From the E-UTRAN backhaul perspective, the most important LTE/SAE reference points are the X2 and S1 interfaces. The eNodeBs are interconnected with each other via the X2 interface, and towards the EPC via the S1 interface.

• The S1-c or S1-MME interface is the reference point for the control plane between E-UTRAN and MME. The S1-MME interface is based on the S1 Application Protocol (S1AP) and is transported over the Stream Control Transmission Protocol (SCTP). The EPC architecture supports MME pooling to enable geographic redundancy, capacity increase, and load sharing. This requires the eNodeB to connect to multiple MMEs. The L3 MPLS VPN service model defined by the Cisco EPN System allows eNodeBs in the RAN access to be connected to multiple MMEs that may be distributed across regions of the core network for geographic redundancy.

• The S1-u interface is the reference point between E-UTRAN and SGW for the per-bearer user plane tunneling and inter-eNodeB path switching during handover. The application protocol used on this interface is GPRS Tunneling Protocol (GTP) v1-U, transported over User Datagram Protocol (UDP). SGW locations affect u-plane latency, and the best practice for LTE is to place S/PGWs in regions closer to the aggregation networks that they serve so that the latency budget of the eNodeBs to which they connect is not compromised. The EPC architecture supports SGW pooling to enable load balancing, resiliency, and signaling optimization by reducing the handovers. This requires the eNodeB to connect to multiple SGWs. The L3 MPLS VPN service model allows eNodeBs in the RAN access to be connected to multiple SGWs, which include ones in the core close to the local aggregation network and SGWs that are part of the pool serving neighboring core POPs.

• The X2 interface comprised of the X1-c and X2-u reference points for control and bearer plane provides direct connectivity between eNodeBs. It is used to hand over user equipment from a source eNodeB to a target eNodeB during the inter-eNodeBs handover process. For the initial phase of LTE, the traffic passed over this interface is mostly control plane related to signaling during handover. This interface is also used to carry bearer traffic for a short period (<100ms) between the eNodeBs during handovers. The stringent latency requirements of the X2 interface requires that the mesh connectivity between CSGs introduces a minimum amount of delay that is in the order of less than 10ms. The L3 MPLS VPN service model provides shortest path connectivity between eNodeBs so as to not introduce unnecessary latency.

• During initial deployments in regions with low uptake and smaller subscriber scale, MME and SGW/PGW pooling can be used to reuse mobile gateways serving neighboring core POPs. Gradually, as capacity demands and subscriber scale increases, newer gateways can be added closer to the region. L3 MPLS VPN service model for LTE backhaul that is defined by the EPN System allows migrations to newer gateways to take place without any re-provisioning of the service model or re-architecting of the underlying transport network required.




• With the distribution of the new spectrum made available for 3G and 4G services, many new SPs have entered the mobility space. These new entrants would like to monetize the spectrum they have acquired, but lack the national infrastructure coverage owned by the incumbents. LTE E-UTRAN-sharing architecture allows different core network operators to connect to a shared radio access network. The sharing of cell site infrastructure could be based on:

– A shared eNodeB—shared backhaul model where different operators are presented on different VLANs by the eNodeB to the CSG, OR

– A different eNodeB—shared backhaul model where the foreign operator's eNodeB is connected on a different interface to the CSG.

Regardless of the shared model, the Cisco EPN System provides per-mobile SP-based L3 MPLS VPNs that are able to identify, isolate, and provide secure backhaul for different operator traffic over a single converged network.

Route Scale Control for LTE L3 MPLS VPN Service ModelThe EPN System proposes a simple and efficient L3 service model, as depicted in Figure 3-2, that addresses the LTE backhaul requirements addressed above. The L3 service model is built over a Unified MPLS Transport with a common highly-scaled MPLS VPN that covers LTE S1 interfaces from all CSGs across the network and a LTE X2 interface per RAN access region. The single MPLS VPN per operator is built across the network with VRFs on the MTGs connecting the EPC gateways (SGW, MME) in the MPC, down to the RAN access with VRFs on the CSGs connecting the eNodeBs. Prefix filtering across the VPN is done using simple multiprotocol BGP (MP-BGP) route target (RT) import and export statements on the CSGs and MTGs.

Figure 3-2 L3 MPLS VPN Service Model

Regarding Figure 3-2:

• A unique RT denoted as Global RAN RT is assigned to the LTE backhaul MPLS VPN. It is either imported or exported at various locations or the VPN, depending on the role of node implementing the VRF.

• A unique RT denoted by MSE RT is assigned to the MTGs in the MPC.

29

73

06

MTG

MTG

MTG

SGW/PGW

SGW/PGW

Core NetworkAggregation Network

LTE TransportMPLS VPNv4/v6

Aggregation Network

Export: MPC RTImport: MPC RT, Common RT

MME

VRF

VRF VRF

VRF VRF

VRF VRF

VRF VRF

VRF VRFVRF VRF

VRFVRF

Export: RAN Z RT, Common RTImport: RAN Z RT, MPC RT

Export: RAN Y RT, Common RTImport: RAN Y RT, MPC RT

Export: RAN W RT,Common RT

Import: RAN W RT,MPC RT

VRF




• Each aggregation and/or each individual RAN access region in the network is assigned a unique RT denoted as the aggregation-wide or RAN-wide AGGR RT and RAN RT, respectively. The AGGR RT identifies all cell site routes in a given aggregation domain regardless of the RAN access domain of origin, while the RAN-wide RT is RAN access domain specific.

For S1 communication, all CSGs import the MSE RT and export the Global RT. In the MPC, the MTGs import the MSE RT and the Global RT and export only the MSE RT. This allows the MTGs to have connectivity to all other gateways in the MPC, as well as to the CSGs in the RAN access regions across the entire network. The MTGs are capable of handling large scale and learn all VPNv4 prefixes in the LTE VPN.

In some cases, depending on the spread of the macro cell footprint, it might be desirable to provide X2 interfaces between CSGs located in neighboring RAN access regions.

For X2 communication, each CSG in a given access domain exports the local AGGR-wide and/or RAN RTs and imports the AGGR or RAN RT of local aggregation or neighboring access domains according to its route scale capabilities. In a network with low route scale-capable CSGs, limiting import of routes to those advertised by the neighboring RAN networks through their specific RAN RTs ensures the VRF route scale of the CSGs is kept to a minimum. VPNv4 prefixes corresponding to CSGs in other non-neighbor RAN access regions—either in the local aggregation domain, or RAN access regions in remote aggregation domain across the core—are not learnt. On the contrary, in a network with only high route scale-capable CSGs, import and export of AGG-wide RTs achieves larger X2 communication domains, thus increasing the number of VPN routes learned, but greatly reduces the operational complexity associated with the management of route filtering in different access domains.

Figure 3-3 and Figure 3-4 describe further the route filtering logic implemented in inter-AS and multi-area transport architectures and for access networks designs based on Labeled BGP or IGP/LDP redistribution.




Figure 3-3 S1 and Inter-Access X2 Connectivity, Labeled BGP Access

2975

88

CN-ASBRInline RR

CN-ASBRInline RR

AGN-ASBRInline RR

AGN-ASBRInline RR

Access-2

Access-2

Access-3

Access-3

Access-4

Access-4

X2

S1 Traffic

X2

X2

MTG

X2inter-access

MTG

RAN Access networks with lowroute-scale capable CSGs

RAN Access network with highroute-scale capable CSGs

vCN-RR

vAGN-RR

Metro-1

X2inter-access

RR

RR

VRF

VRF VRF

VRF

VRF

VRF VRF VRF VRF

Unified MPLS Transport:Advertise loopbacks in iBGPlabeled-unicast with community 10:10, 10:100, 10:102

LTE MPLS VPN Service:Export: RAN-2 RT, AGGR-1 RT, Global RTImport: RAN-1 RT, RAN-2 RT, RAN-3 RT, MSE RT


MSE BGP Community 1001:1001

LTE MPLS VPN Service:Export: RAN-4 RT, Global RTImport: AGGR-1 RT, MSE RT


LTE MPLS VPN Service:Export: RAN-3 RT, AGGR-1 RT, Global RTImport: RAN-2 RT, RAN-3 RT, RAN-4 RT, MSE RT

Inter-accessX2 Traffic




Figure 3-4 Inter-Access X2 Connectivity, IGP/LDP Redistribution

As shown in Figure 3-3 and Figure 3-4, connectivity can easily be accomplished using the BGP community-based coloring of prefixes used in the Unified MPLS Transport.

• As described in the Cisco EPN 4.0 Transport Infrastructure Design and Implementation Guide, the CSG loopbacks are colored in BGP labeled-unicast with a common BGP community that represents the Global RAN community and up to two BGP communities, one unique to the RAN access region and the other to the aggregation regionwide. This tagging can be done when the CSGs advertise their loopbacks in iBGP labeled-unicast if labeled BGP is extended to the access or at the PANs when redistributing from the RAN IGP to iBGP when IGP/ LDP is used in the RAN access using the redistribution approach.

• For access domains made of low route scale-capable CSG nodes, such as the Access-2 and Access-3 RAN networks shown in Figure 3-3 and Figure 3-4, the adjacent RAN access domain CSG loopbacks can be identified at the PAN based on the unique RAN access region BGP community and be selectively propagated into the access based on egress filtering if labeled BGP is extended to the access or be selectively redistributed into the RAN IGP if IGP/LDP is used in the RAN access using the redistribution approach. Please note that X2 interfaces are based on eNodeB proximity and therefore a given RAN access domain only requires connectivity to the ones immediately adjacent. This filtering approach allows for hierarchical-labeled BGP LSPs to be set up across neighboring access regions while preserving the low route scale in the access. At the service level, any CSG in a RAN access domain that needs to establish inter-access X2 connectivity will import its neighboring CSG access region RT in addition to its own RT in the LTE MPLS VPN.

2975

89

CN-ABRInline RR

CN-ABRInline RR

Access-2

Access-3

Access-4

X2

S1 Traffic

X2

X2X2

inter-accessX2

inter-access

Inter-accessX2 Traffic

MTG

MTG

CN-RR

Metro-1

RRSelective next-hop-self in RPL

Set next-hop-selfIf community 10:01(*)

VRF

VRF VRF

VRF

VRF

VRF VRF VRF VRF

Export: RAN-2 RT, AGGR-1 RT, Global RTImport: RAN-1 RT, RAN-2 RT, RAN-3 RT, MSE RT

Redistribute RAN IGP-4 in iBGP, markBGP Community 10:10, 10:0100, 10:0104

Redistribute BGP Community 1000:1000, 10:0100, in RAN IGP-4

Redistribute RAN IGP-3 in iBGP, mark BGP Community 10:10, 10:100, 10:0103

Redistribute BGP Community 1000:1000, 10:0102, 10:0104 in RAN IGP-3

Export: RAN-3 RT, AGGR-1 RT, Global RTImport: RAN-2 RT, RAN-3 RT, RAN-4 RT, MSE RT

Redistribute RAN IGP-2 in iBGP, markBGP Community 10:10, 10:0100, 10:0102

Redistribute BGP Community 1000:1000, 10:0101, 10:0103 in RAN IGP-2

Export: RAN-4 RT, AGGR-1 RT, Global RTImport: AGGR-1 RT, , MSE RT

Access-2Access-3

Access-4

RAN Access networks with lowroute-scale capable CSGs

RAN Access network with highroute-scale capable CSGs

MSE BGP Community 1001:1001



https://docs.cisco.com/share/page/site/nextgen-edcs/document-details?nodeRef=workspace://SpacesStore/5c4d4b33-579d-4e24-84b8-09a3eeb5ca97

Chapter 3 Mobile Transport Services Architecture Multicast Service Model for LTE eMBMS

• Alternatively, for access domains made of high route scale-capable CSG nodes, such as the Access-4 RAN network shown in Figure 3-3 and Figure 3-4, loopbacks of CSGs in the same aggregation domain can be identified at the PAN based on the aggregation network-wide BGP community and be selectively propagated into the access based on egress filtering if labeled BGP is extended to the access. Alternatively, it can be selectively redistributed into the RAN IGP if IGP/LDP is used in the RAN access using the redistribution approach. This filtering approach allows for a simplified route filtering logic to the detriment of larger routing table on the CSG nodes. At the service level, any CSG in a RAN access domain that needs to establish inter-access X2 connectivity with other CSGs in the same aggregation domain will import the aggregation network-wide RT in the LTE MPLS VPN.

The CN-ABR inline-RR applies selective NHS function using route policy in the egress direction towards its local PAN neighbor group in order to provide shortest-path connectivity for the X2 interface between CSGs across neighboring RAN access regions. The routing policy language (RPL) logic involves changing the next-hop towards the PANs for only those prefixes that do not match the local RAN access or aggregation regions based on a simple regular expression matching BGP communities. This allows for the CN-ABR to change the BGP next-hop and insert itself in the data path for all prefixes that originate in the core corresponding to the S1 interface, while keeping the next-hop set by the PANs unchanged for all prefixes from local RAN regions. With this, the inter-access X2 traffic flows across adjacent access regions along the shortest path interconnecting the two PANs without having to loop through the inline-RR CN-ABR node.

Finally, the rapid adoption of LTE and the massive increase in subscriber growth is leading to an exponential increase in cell sites that are being deployed in the network. This is introducing a crunch in the number of IP addresses that need to be assigned to the eNodeBs at the cell sites. For mobile SPs that are running out of public IPv4 addresses or those that cannot obtain additional public IPv4 addresses from the registries for eNodeB assignment, the Cisco EPN System enables carrying IPv6 traffic over a IPv4 Unified MPLS Transport infrastructure using 6VPE as defined in RFC 4659. The eNodeBs and EPC gateways can be IPv6 only or dual stack-enabled to support IPv6 for S1 and X2 interfaces while using IPv4 for network management functions, if desired. The dual stack-enabled eNodeBs and EPC gateways connect to CSGs and MTGs configured with a dual stack VRF carrying VPNv4 and VPNv6 routes for the LTE MPLS VPN service. The IPv6 reachability between the eNodeBs in the cell site and the EPC gateways in the MPC is exchanged between the CSGs and MTGs acting as MPLS VPN PEs using the BGP address family [address family identifier (AFI)=2, subsequent address family identifier (SAFI)=128].

This design can be downgraded to accommodate UMTS IP IuB and Wi-Fi CAPWAP, maintaining only the hub-and-spoke VPN filtering configuration used for the LTE service, hence using only the MSE and Global RAN RT relevant import and exports across CSGs and MTGs.

Multicast Service Model for LTE eMBMSThe Cisco EPN System architecture includes support for transport of enhanced Multimedia Broadcast Multicast Service (eMBMS). The 3rd Generation Partnership Project (3GPP) has standardized eMBMS services in the LTE releases as a mechanism for effectively delivering the same content to a number of end users, such as broadcast video or file push. Content delivered via eMBMS services uses a multicast-based transport mechanism, minimizing packet duplication within the transport network.

An overview of eMBMS service implementation is illustrated in Figure 3-5.



Chapter 3 Mobile Transport Services Architecture Multicast Service Model for LTE eMBMS

Figure 3-5 Overview of eMBMS Service Implementation in LTE

The following interfaces, which are within the scope of the Cisco EPN System design, are involved in eMBMS service delivery:

• M3 interface—A unicast interface between the MME and MCE (assumed to be integrated into the eNB for the sake of Cisco EPN), which primarily carries MBMS session management signaling.

• M1 interface—A downstream user-plane interface between the MBMS Gateway (MBMS-GW) and the eNB, which delivers content to the user endpoint. IP Multicast is used to transport the M1 interface traffic.

In the context of the Cisco EPN System design, transport of the eMBMS interfaces is conducted based on the interface type. This is illustrated in Figure 3-6:

Figure 3-6 eMBMS Service Interfaces

• The M3 interface is transported within the same L3 MPLS VPN as other unicast traffic, namely the S1 and X2 interfaces. Since both the S1 and M3 interfaces are between the eNB and the MME, it makes logical sense to carry both in the same VPN.

• The M1 interface transport is handled directly via IP over mLDP transport in core and aggregation and IP Multicast with PIM SSM or IGMP/MLDP in access. This transport is reused for the wireline IPTV as well, hence the need for a common not virtualized multicast infrastructure.

Additionally, the Cisco EPN System enables the delivery of mobile multicast traffic over IPv4 and IPv6 address families. Both address families are carried over the same IPv4-enabled LSM transport infrastructure as defined in RFC 6514. The eNodeBs, MBMS and/or packet gateways can be IPv6 only or dual stack-enabled to support IPv4 and IPv6 multicast forwarding at the user plane (M1) or a combination of IPv6 only multicast at the user plane (M1) and IPv4 at the control plane (M3) interfaces,

The multicast mechanism utilized for transporting the M1 interface traffic depends upon the location in the network:

• From the MTG attached to the MBMS-GW, through the core and aggregation domains to the AGN node, LSM is utilized to transport the M1 interface traffic, using a combination of BGP signaling and mLDP transport. This provides efficient and resilient transport of the multicast traffic within these regions.

2960

16

MME

eNB MBMS-GWM1

SmSGmb

SGi-mb

M3

BM-SCUE

2960

17

Core NetworkMobile Access Network Aggregation Network

CSG

MPLS VPN (v4 or v6)

MBMS-GWRAN IGP Protocol

MPLS Access: PMv4/v6Ethernet Access: IGMP/MLDPv2

MVPN with BGP signaling andmLDP transport

M3

S1-U S11

S11S1-U

S1-CMME

X2

X2

X2

MTG-3 MTG-1VRF

Global

VRF VRF

VRRP

VRF

VRF

VRF

VRF

MTG-2



Chapter 3 Mobile Transport Services Architecture L2 MPLS VPN Service Model for 2G and 3G

• From the AGN to the CSG, in the MPLS access networks, native IP Multicast is utilized to transport the M1 interface traffic. In the Ethernet-bridged access, the transport is over a dedicated multicast VLAN (bridge domain), shared by all eNodeBs, with IGMP snooping and MLDv2 enabled, optionally, while all eNBs will join the M1 interface. This provides efficient and resilient transport of the multicast traffic while utilizing the lowest amount of resources on these smaller nodes.

– In the case of MPLS (or Layer 3) access, multicast forwarding in the access domain is based on PIM SSM v4/v6. IPv4/v6 addresses of multicast sources are re-distributed in the ISIS IGP process at level 2 on the AGN nodes, and are leaked in ISIS at level 1 on the PAN nodes for distribution to the CSGs.

– In the case of G.8032-enabled Ethernet access, multicast forwarding is based on IGMP or MLDv2 for the IPv4 and IPv6 address families, respectively.

• From the CSG to the eNodeB, two models are available depending on the capabilities of the CSG and the type of access network:

– For MPLS access and CSG nodes capable of leaking IPv4 multicast routes from the global routing table in the mobile L3MPLS VPN a single VLAN is used to deliver all mobile interfaces (S1, X2, M1 an dM3).

– For Ethernet-bridged access and all others MPLS access scenarios, two VLANs are utilized to deliver the various interfaces to the eNB. One VLAN handles unicast interface (S1, X2, M3) delivery, while the other handles M1 multicast traffic delivery and, in case of Ethernet access, it is an extension toward the eNodeB of the multicast VLAN used between the AGN and the CSG.

L2 MPLS VPN Service Model for 2G and 3GThe Cisco EPN System architecture allows mobile service providers (MSPs) with TDM-based 2G GSM and ATM-based 3G UMTS infrastructures to remove, reduce, or cap investments in SONET/SDH and ATM transport infrastructure by using MPLS-based CEoP services.

• For the MSPs that want to reduce SONET/SDH infrastructure used for GSM, the EPN System enables PWE3-based transport of emulated TDM circuits. Structured circuit emulation is achieved with CESoPSN and unstructured emulation is achieved with SAToP. E1/T1 circuits from BTS equipment connected to the CSG or to the PAN are transported to MTG, where they are bundled into channelized Synchronous Transport Module level-1 (STM1) or Optical Carrier 3 (OC-3) interfaces for handoff to the BSC. Synchronization is derived from the BSC via TDM links, or from a Primary Reference Clock (PRC), and transported across the core, aggregation, and access domains via SyncE, or via 1588 across domains where SyncE is not supported.

• For the MSPs that want to reduce their ATM infrastructure used for ATM-based UMTS, the Cisco EPN System enables PWE3-based transport of ATM virtual circuit (VC) (AAL0 or AAL5) or virtual path (VP) (AAL0) circuits. ATM E1/T1 or inverse multiplexing over ATM (IMA) interfaces from NodeB equipment connected to the CSG or PAN are transported to the MTG, where they are bundled into STM1 ATM interfaces for handoff to the Radio Frequency Subsystem (RFSS) Network Controller (RNC). Cell packing may be used to optimize the bandwidth used for this transport. Synchronization is derived from the RNC via ATM links or from a PRC and is then transported across the core, aggregation, and access domains via SyncE or via 1588 across domains where SyncE is not supported.



Chapter 3 Mobile Transport Services Architecture Mobile Transport Capacity Monitoring

Figure 3-7 ATM/TDM Transport Services

Typical GSM (2G) deployments will consist of cell sites that don't require a full E1/T1 for support. In such cell sites, a fractional E1/T1 is used. The operator can deploy these cell sites in a daisy chain fashion (for example, down a highway) or aggregate them at the BSC location. To save in the CAPEX investment on the number of channelized STM-1/OC-3 ports required on the BSC, the operator will utilize a digital XConnect to merge multiple fractional E1/T1 links into a full E1/T1. This reduces the number of T1/E1s needed on the BSC, which results in fewer channelized STM-1/OC-3 ports being needed. Deploying CESoPSN PWs from the CSG to the RAN distribution node supports these fractional T1/E1s and the aggregation of them at the BSC site. In this type of deployment, the default behavior of CESoPSN for alarm sync needs to be changed. Typically, if a T1/E1 on the ANs goes down, the PWs will forward the alarm indication signal (AIS) alarm through the PW to the distribution node and then propagate the alarm indication signal (AIS) alarm to the BSC by taking the T1/E1 down. In this multiplexed scenario, TS alarming must be enabled on a CESoPSN PW to only propagate the AIS alarm on the affected time slots, thus not affecting the other time slots (for example, cell sites) on the same T1/E1.

The same BGP-based control plane and label distribution implemented for the L3VPN services is also used for circuit emulation services. The CSGs utilize MPLS/IP routing in this system release when deployed in a physical ring topology. TDM and ATM PWE3 can be overlaid in either deployment model.

The CSGs, PAN, AGNs, and MTGs enforce the contracted ATM CoS SLA and mark the ATM and TDM PWE3 traffic with the corresponding per-hop behavior (PHB) inside the access, aggregation, and core DiffServ domains. The MTG enables multi-router automatic protection switching (MR-APS) (or single-router automatic protection switching [SR-APS] redundancy for the BSC or RNC interface, as well as pseudowire redundancy and two-way pseudowire redundancy for transport protection.

Mobile Transport Capacity Monitoring Capacity monitoring in mobile transport networks is essential for the appropriate sizing and operation planning of cell site locations based on actual consumption. The EPN System proposes the use of NetFlow functions for collection of traffic statistics from a given cell site. See Figure 3-8.

2932

99


TDM BTS, ATM Node B

e Node B

ATM orTDM ATM RNC

BBC

CSG

AToM Pseudowire

AToM Pseudowire

MTG

MTGPAN




Figure 3-8 Mobile Transport Capacity Monitoring

By enabling NetFlow exporter capabilities on the MTG L3VPN interfaces facing the mobile packet core gateways and delivering the cumulative flow records to a Collector function implemented on Cisco Prime Performance Monitoring, it is possible to visualize the amount of traffic sent and received by a given cell site at configurable interval and over customizable periods of time.

2975

90


eNode B

MPLS VPN (v4 or v6)CSG

SGW

NetFlow Collectionand Analysis

NetFlow Exporter

S1-U

S1-U

MTG

MTG SGW







C H A P T E R 4
Functional Components
This chapter includes the following major topics:

• Synchronization Distribution, page 4-1

• High Availability, page 4-4

• Quality of Service, page 4-4

• Operations, Administration, and Maintenance, page 4-4

Synchronization DistributionEvery mobile technology deployment has synchronization requirements in order to enable aspects such as radio framing accuracy, user endpoint handover between cell towers, and interference control on cell boundaries. Some technologies only require frequency synchronization across the transport network, while others require phase and time-of-day (ToD) synchronization as well. The Cisco EPN System delivers a comprehensive model for providing network-wide synchronization of all three aspects with an accuracy that exceeds the threshold requirements of any mobile technology deployed across the system.

The primary target for the current system release is to provide frequency synchronization by using the Ethernet physical layer (SyncE) and phase and Time (ToD, Phase) synchronization by using IEEE 1588-2008 PTPv2, here simply referred as PTPv2. SyncE operates on a link-by-link basis and will provide a high quality frequency reference similar to that provided by SONET and SDH networks. SyncE is complemented by Ethernet Synchronization Message Channel (ESMC), which allows transmitting over SyncE-enabled links a quality level value as done with synchronization status message in SONET and SDH. This allows the SyncE node to select a timing signal from the best available source and help detect timing loops, which is essential for the deployment of SyncE in ring topologies.

Because not all links on the network may be SyncE-capable or support synchronization distribution at the physical layer, PTPv2 may also be used for frequency distribution. IEEE 1588 packet-based synchronization distribution is overlaid across the entire system infrastructure; third-party master and third-party IP-NodeB client equipment are considered outside the scope of the system. The mechanism is standards-based and can provide frequency and/or phase distribution, relying on unicast or multicast packet-based transport. As with any packet-based mechanism, PTP traffic is subject to loss, delay, and delay variation. However, the packet delay variation (PDV) is the main factor to control. To minimize the effects of these factors and meet the requirements for synchronization delivery utilizing PTP, EF PHB treatment across the network is required.

The Cisco EPN System also supports a combination of SyncE and PTP in a hybrid synchronization architecture, aiming to improve the stability and accuracy of the phase and frequency synchronization delivered to the client for deployments such as Time Division Duplex (TDD)-LTE eNodeBs, LTE Advanced and eMBMS. In such an architecture, the packet network infrastructure is frequency


Chapter 4 Functional Components Synchronization Distribution

synchronized by SyncE. The phase signal is delivered by PTPv2. The CSG, acting as a PTPv2 ordinary clock or as a Boundary Clock (BC), may combine the two synchronization methods, using the SyncE input as the frequency reference clock for the PTPv2 engine. The combined recovered frequency and phase can be delivered to clients via 1 pulse per second (PPS), 10MHz and Building Integrated Timing Supply (BITS) timing interfaces, SyncE and PTPv2. For access networks that don't support SyncE, the hybrid 1588 BC function may be move to the PANs.

Figure 4-1 illustrates how synchronization distribution is achieved for mobile transport services over both fiber and microwave access networks in the Cisco EPN System architecture.

Figure 4-1 Synchronization Distribution

The frequency source for the mobile backhaul network is the Primary Reference Clock (PRC), which can be based on free-running atomic clock (typically Cesium), a global navigation satellite system (GNSS) receiver that derived frequency from signals received from one or more satellite system, or a combination of both.

The time (phase and Time of Day-ToD) source for the mobile backhaul network is the Primary Reference Time Clock (PRTC), which is usually based on GNSS receiver that derived time synchronization from one or more satellite systems with traceability to the Universal Coordinated Time (UTC). A PRC provides a frequency signal of G.811/Stratum-1 quality signal (traceable to UTC frequency if coming from GNSS) to the AGNs via G.703-compliant dedicated external interfaces (aka BITS input) or 10Mhz interface. A PRTC provides time via a 1PPS signal for phase, and a serial ToD interface. DOCSIS Timing Interface (DTI) is an alternative to the frequency, 1PPS and ToD interface. A PRTC can also provide frequency as a PRC. If required by the architecture, the IEEE 1588 Primary Master Clock (PMC) will also derive synchronization from the PRC or PRTC. From this point, three models of synchronization distribution are supported:

• For mobile services that only require frequency synchronization, where all network nodes support SyncE, then frequency is carried to the NodeB via SyncE. The ESMC provides source traceability between nodes through the Quality Level (QL) value which helps selecting best signal and preventing timing loops in SyncE topologies.

2975

91ASR-903 ASR-9000


CSR-3CRS-3

IP/MPLS Transport

ASR-901, ASR-920Pre-Aggregation Node Aggregation Node Core NodeCore NodeCell Site Gateway (CSG)

IP/MPLS Transport

1588 BC HC

TP-50001588 PMC/PRTC

1588 Master Clock

External SynchronizationInterface (ToD and Pbase)

IP/MPLS orG.8032 Ethernet

Transport

No PhysicalSynchronization

SyncE

SyncE, ESMC

IP/MPLS Transport Network1588 BC and OC

1588 Phase (+ Frequency)

1588 BC HCExternal Synchronization

Interface (Frequency)

Global Navigation Satellite System (e.g., GPS, GLONASS,GALILEO) - Primary Reference Time Clock (PRTC)

OptionalNID

ME-1200ZTD NID



Chapter 4 Functional Components Synchronization Distribution

• For mobile services that require synchronization over an infrastructure which does not support SyncE, PTPv2 is then utilized for frequency synchronization distribution. The PMC generates PTPv2 streams for each PTP slave that is routed globally by the regional MTG to the CSG, which then provides sync to the eNodeB. The PMC can be a network node which receives the frequency source signal via physical layer (e.g., SyncE). Proper network engineering shall prevent excessive PDV to allow timing network to provide packet-based quality signal to the slaves.

• For mobile services that require time synchronization, PTPv2 can be used in conjunction with SyncE to provide a hybrid synchronization solution, where SyncE provides accurate and stable frequency distribution, and PTPv2 is used for allowing phase and ToD synchronization. In this Cisco EPN System release, the PTPv2 streams are routed globally from the regional MTG to the CSG, which, combined with SyncE frequency, then provides synchronization to the eNodeBs.

In general, a packet-based timing mechanism such as PTPv2 has strict packet delay variation requirements, which restricts the number and type of hops over which the recovered timing from the source is still valid. With globally routed model, strict priority queuing of the PTPv2 streams is necessary. With a good implementation of PTPv2 BC on intermediate transit nodes, it is possible to provide better guarantee over more hops from the PMC to the NodeB.

Scalability and reliability of PTPv2 in the Cisco EPN System is enhanced by enabling BC in some or all of the following: the core-facing AGN, the PAN, and the CSG. Implementing BC functionality in these nodes serves two purposes:

• Increases scaling of PTPv2 phase/frequency distribution, by replicating a single stream from the PMC to multiple destinations, thus reducing the number of PTP streams needed from the PMC.

• Improves the phase stability of PTPv2, by stabilizing the frequency of the PTP servo with SyncE or another physical frequency source as described in the hybrid synchronization architecture.

Timing Distribution with Ethernet NIDsBackhaul providers deploy Ethernet Network Interface Devices (NIDs) at the base station site to enhance end-to-end service visibility and, specific to timing distribution, to regenerate a timing signal as close as possible to the radio site.

In these scenarios, the CSG distributes frequency using SyncE to an Ethernet NID, which propagates it further toward the base station over SyncE-enabled downlinks. Time synchronization to the base station bypasses the NID and involves direct PTP session establishment between the CSG and each base station.

Time Asymmetry Correction between Boundary ClocksWhen PTP peering relationships are established between Boundary Clocks located several hops away, such as in the case of BCs on PANs and core-facing AGN nodes, a timing distribution asymmetry develops on the path which negatively affects the 1PPS signal. Such asymmetry must be compensated and the correction applied to the Slave node in the PTP peering relationship.

The cumulative time error on the path is based on the number of transport hops and platform types, as described in more detail in Asymmetry Correction: Estimating Expected Time Error, page 6-21.

In the case of redundant paths between BCs, the primary and the backup paths may introduce different time errors in the distribution of the 1PPS signal. The asymmetry correction in this case is calculated as the average between the cumulative time errors introduced by the primary and the backup path.



Chapter 4 Functional Components High Availability

High AvailabilityAs highlighted in the "Redundancy and High Availability" section of the EPN 4.0 Transport Infrastructure Design and Implementation Guide, the Cisco EPN System architecture implements high availability at the transport network level and the service level. By utilizing these various technologies throughout the network, the Cisco EPN design is capable of meeting the stringent Next-Generation Mobile Network (NGMN) requirements of 200 ms recovery times for LTE real time services.

Implementation of high availability technologies at the transport layer that are common to all services is covered in the EPN 4.0 Transport Infrastructure Design and Implementation Guide. Implementation of the high availability technologies at the service level is covered in this chapter. Synchronization resiliency implementation is covered in Synchronization Distribution Implementation, page 6-1.

• For MPLS VPN services, BGP Edge protection and BGP FRR Edge protection mechanisms are supported, and Virtual Router Redundancy Protocol (VRRP) is enabled on the MTGs for redundant connectivity to the Mobile Packet Core (MPC).

• For ATM and TDM pseudowire-based services, pseudowire redundancy is supported, and MR-APS is enabled for redundant connectivity to the base station controller (BSC) or radio network controller (RNC).

• For Ethernet access based on G.8032 rings, the ring terminates on two different SE nodes and it is closed via a routed pseudowire. VRRP is then enabled between the SEs for dual homing redundancy in the access domain.

Quality of ServiceThe Cisco EPN System uses a Differentiated Services (DiffServ) QoS model across all network layers of the transport network in order to guarantee proper treatment of all services being transported. This QoS model guarantees the service-level agreement (SLA) requirements of all residential, business, and mobile backhaul services across the transport network. QoS policy enforcement is accomplished with flat QoS policies with DiffServ queuing on all Network-to-Network Interfaces (NNIs), and with H-QoS policies with parent shaping and child queuing on the UNIs and Service Edge node interfaces.

Specific to mobile services, the QoS design aims to satisfy the SLA requirements of TDM circuits, ATM classes of service (CoS), and various LTE QoS class identifier (QCI) values that correspond to different traffic types (voice, video, etc.) with varying resource types (constant bit rate [CBR], variable bit rate real time [VBR-rt], variable bit rate nonreal time [VBR-nrt], unspecified bit rate [UBR] for ATM; and guaranteed bit rate [GBR] or non-GBR for LTE).

Operations, Administration, and MaintenanceFor mobile backhaul services, the Cisco EPN System utilizes combination of protocols to provide the required service and transport OAM and PM functionality between the CSG and the MTG. The details of the required mechanism are highlighted in Figure 4-2.






Chapter 4 Functional Components Operations, Administration, and Maintenance

Figure 4-2 OAM Implementation for Mobile RAN Service Transport

At a high level, for Service OAM, the EPN System employs:

• MPLS VPN OAM and MPLS VCCV PW OAM for services carried over MPLS VPNs or pseudowires, respectively. Specifically MPLS VPN OAM is used for IP-based Mobile Transport Services, while MPLS VCCV PW OAM applies to Non-IP Mobile technologies, such as 3G ATM UMTS and 2G TDM.

• Cisco IP SLA tools for any service configured between IP enabled end points. This includes mobile transport services between CSRs and mobile transport gateways.

For transport OAM, the EPN System employs MPLS LSP OAM to monitor the health of the unified MPLS transport. Performance monitoring is based on Cisco IP SLA tools running between service end points or between any two points in the unified MPLS domain to find performance bottlenecks.

Please refer to Chapter 8, “OAM Implementation,” for the implementation details.

2934

56

RNC/BSC/SAE GWCSG MTG

MPLS VRF OAM

Node B

IPSLA PM

MPLS LSP OAM

Service OAM

MPLS VCCV PW OAM

IPSLA PM (future PW PM)

Transport OAMEnd-to-end LSPWith Unified MPLS

3G ATM UMTS,2G TDM, Transport

LTE,3G IP UMTS,Transport

IPSLAProbe

IPSLAProbe

IPSLAProbe

IPSLAProbe

VRFVRF



Chapter 4 Functional Components Operations, Administration, and Maintenance




C H A P T E R 5
Mobile Transport Services Implementation
The Cisco EPN 4.0 System divides implementation of mobile services backhaul into two major areas:

• This includes deployments with MPLS access, either via labeled Border Gateway Protocol (BGP) or Interior Gateway Protocol (IGP)/Label Distribution Protocol (LDP) control planes, where all services originate on the CSG node. This is described in Mobile Services Implementation with MPLS Access, page 5-1.

• Deployments with non-MPLS access, such as TDM over microwave, native Ethernet (Hub & Spoke and G8032 Ring), or native IP. All services originate on the PAN in this scenario. This is described in Mobile Services Implementation with Non-MPLS Access, page 5-33.

See also Mobile Transport Capacity Monitoring, page 5-55.

Mobile Services Implementation with MPLS AccessThis includes deployments with MPLS access, either via labeled Border Gateway Protocol (BGP) or Interior Gateway Protocol (IGP)/Label Distribution Protocol (LDP) control planes, where all services originate on the CSG node.

L3 MPLS VPN Service Model for LTE Implementation

MPLS VPN Transport for LTE S1 and X2 Interfaces

This section describes the L3VPN configuration aspects on the CSGs and bridging configuration on NID in the RAN access and the Mobile Transport Gateways (MTG) in the core network required for implementing the LTE backhaul service for X2 and S1 interfaces. See Figure 5-1.


Chapter 5 Mobile Transport Services Implementation Mobile Services Implementation with MPLS Access

Figure 5-1 MPLS VPN Service Implementation for LTE Backhaul

CSG MPLS VPN Configuration

We are segregating the CSG MPLS VPN configuration into two sections, depending upon the type of device used as CSGs, as their scale capabilities differ. The implementation shown is for ASR901 and ASR920 routers.

Granular X2 Interface Setup

This section describes the L3VPN configuration aspects for CSGs that have relatively less scale capability in terms of routes in the VRF, such as ASR901. The MPLS VPN configuration on the CSGs, which is minimal, is the same on all CSGs in a given RAN region.

Enhanced NodeB User-Network Interface (UNI)

!***eNodeB UNI***interface GigabitEthernet0/0 description Connected to eNodeB service-policy output PMAP-eNB-UNI-P-E service instance 100 ethernet encapsulation untagged service-policy input PMAP-eNB-UNI-I bridge-domain 100!interface Vlan100 vrf forwarding LTE224 ip address 113.30.224.1 255.255.255.0 load-interval 30 ipv6 address 2001:113:30:224::1/64

Virtual Route Forwarding (VRF) Definition

vrf definition LTE224 rd 10:104 ! address-family ipv4 export map ADDITIVE !***Common RT. *** route-target export 10:104 !***Enables X2 communication to CSGsin same RAN region*** route-target import 10:104 !***MSE RT. Imported by every CSG in the entire network.***




route-target import 1001:1001 !***Optional import of adjacent RAN region RT.*** !***Only required to enable inter-RAN-region X2 communication.*** route-target import 10:101 exit-address-family ! address-family ipv6 export map ADDITIVE route-target export 10:104 route-target import 10:104 route-target import 1001:1001 route-target import 10:101 Route map to export common route target (RT) 10:10 in addition to Local RAN RT 10:101route-map ADDITIVE permit 10 !***Common RAN RT. Exported by every CSG in the entire network*** set extcommunity rt 10:10 additive

VPNv4 BGP Configuration

router bgp 101 neighbor pan peer-group neighbor pan remote-as 101 neighbor pan password lab neighbor pan update-source Loopback0 ! address-family vpnv4 bgp nexthop trigger delay 2 neighbor pan send-community extended ! address-family vpnv6 bgp nexthop trigger delay 2 neighbor pan send-community extended ! address-family ipv4 vrf LTE224 redistribute connected exit-address-family ! address-family ipv6 vrf LTE224 redistribute connected exit-address-family

Aggregation-Wide X2 Interface Setup

This section describes the L3VPN configuration aspects for CSGs that have relatively greater scale capability in terms of routes in the VRF, such as ASR920. Here, the CSGs will use aggregation-wide RT to import/export routes in VRF, instead of having unique RTs for different access rings across the aggregation.

Enhanced NodeB User-Network Interface (UNI)interface TenGigabitEthernet0/0/5 description Connected to eNodeB service-policy output PMAP-eNB-UNI-P-E service instance 100 ethernet encapsulation untagged service-policy input PMAP-eNB-UNI-I bridge-domain 100!interface BDI100 vrf forwarding LTE200 ip address 113.30.224.1 255.255.255.0 load-interval 30




ipv6 address 2001:113:30:224::1/64end

Virtual Route Forwarding (VRF) Definitionvrf definition LTE200 rd 10:200 ! address-family ipv4 export map ADDITIVE !***Common RT*** route-target export 10:200 !***Enables X2 Communication to CSGs with Aggregation-wide RT*** route-target import 10:200 !***Aggregation wide RT imported by all ASR920 CSGs*** route-target import 1001:1001 route-target import 1001:1001 !***Optional import of adjacent RAN region RT.*** !***Only required to enable inter-RAN-region X2 communication.*** exit-address-family ! address-family ipv6 export map ADDITIVE route-target export 10:200 route-target import 10:200 route-target import 1001:1001 exit-address-family!!Route map to export common route target (RT) 10:10route-map ADDITIVE permit 10 !***Common RAN RT. Exported by every CSG in the entire network*** set extcommunity rt 10:10 additive

VPNv4 BGP Configuration

router bgp 101 neighbor pan peer-group neighbor pan remote-as 101 neighbor pan password lab neighbor pan update-source Loopback0 ! address-family vpnv4 bgp nexthop trigger delay 2 neighbor pan send-community extended ! address-family vpnv6 bgp nexthop trigger delay 2 neighbor pan send-community extended !address-family ipv4 vrf LTE200 redistribute connected exit-address-family !address-family ipv6 vrf LTE200 redistribute connected exit-address-family

Note Both the models described above can co-exist in the same domain by manipulating the RTs for import/export. For example, ASR920 CSG can additionally export RAN RT-2 for import on ASR901 CSG, and ASR901 CSG can additionally export aggregation-wide RT for import on ASR920 CSG, making both the rings establish X2 connectivity. Please refer to Figure 5-1.




NID Configuration using NID Controller

This section describes the configuration required to bridge traffic from eNodeB to CSG via NID device. To bridge traffic, one EVC with VLAN 100 on uplink port and one EVC control Entry (ECE) are created on NID. ECE is mapped to UNI port for all traffic by using NID controller. ME 3600 is used as NID Controller.

Note The following NID-related configurations are entered from the ME3600 controller.

Controller nid 1/2ProvisionEVC!***NNI port to CSG K0206 configuration ***

addEVC evcConfiguration instance 100addEVC evcConfiguration nni_ports GigabitEthernet_2_NNI enableaddEVC evcConfiguration nni_vid 100addEVC evcConfiguration learning enableaddEVC commitexit

!***UNI Port to EnodeB configurtaion ***

ProvisionEVC addECE ece_configuration ece_id 100addECE ece_configuration control ingress_match uni_ports GigabitEthernet_3_UNI enable addECE ece_configuration control ingress_match outer_tag_match match_type anyaddECE ece_configuration control egress_outer_tag mode enabledaddECE ece_configuration control egress_outer_tag pcp_mode fixedaddECE ece_configuration control egress_outer_tag pcp_value 0addECE ece_configuration control actions class specific 0addECE ece_configuration control actions tag_pop_count 0addECE ece_configuration control actions evc_id specific 100addece commitexit

Note For multicast services, a separate VLAN is required on CSG and NID. The creation of Multicast VLAN on NID is the same as for Unicast service VLAN.

NID Configuration using Cisco Prime Provisioning

This section describes the configuration required to bridge traffic from eNodeB to CSG performed via Cisco Prime Provisioning.

Step 1 Configure EVC with VLAN 100 on Uplink port at shown in Figure 5-2.




Figure 5-2 NID Configuration - EVC

Step 2 Create ECE as shown in Figure 5-3.

Figure 5-3 NID Configuration - ECE

Note For multicast services, a separate VLAN is required on CSG and NID. The creation of Multicast VLAN on NID is as same as Unicast service VLAN.

Mobile Transport Gateway MPLS VPN Configuration

This is a one-time MPLS VPN configuration done on the MTGs. No modifications are made when additional CSGs in any RAN access or other MTGs are added to the network.

UNI for System Architecture Evolution (SAE) Gateways (GWs) interface TenGigE0/0/0/2.1100 description Connected to SAE Gateway. vrf LTE102




ipv4 address 115.1.102.3 255.255.255.0 ipv6 nd dad attempts 0 ipv6 address 2001:115:1:102::3/64 encapsulation dot1q 1100

VRF Definitionvrf LTE102 address-family ipv4 unicast !***Common CSG RT imported by MTG*** !***MSE RT imported for reachability to other MPC areas*** import route-target 10:10 1001:1001 ! !***Export MSE RT.*** !***Imported by every CSG in entire network.*** export route-target 1001:1001 ! ! address-family ipv6 unicast import route-target 10:10 1001:1001 ! export route-target 1001:1001 ! !!

MTG 1 VPNv4/v6 BGP Configuration

router bgp 1000 bgp router-id 100.111.15.1 bgp update-delay 360 ! vrf LTE102 rd 1001:1001 address-family ipv4 unicast redistribute connected ! address-family ipv6 unicast redistribute connected ! !

MTG 2 VPNv4/v6 BGP Configurationrouter bgp 1000 bgp router-id 100.111.15.2! vrf LTE102 rd 1001:1002 address-family ipv4 unicast redistribute connected ! address-family ipv6 unicast redistribute connected ! !




Note Each MTG has a unique RD for the MPLS VPN VRF to properly enable BGP FRR Edge functionality.

A more detailed explanation is given in the High Availability, page 4-4.

MPLS VPN Control Plane

Whereas the MPLS VPN endpoint configuration is the same in all designs with MPLS access, the design configurations for MPLS VPN control plane configuration are different. The control plane implementations for each design option of the Cisco EPN System are covered in this section.

MPLS VPN Control Plane for Inter-AS Design

This section describes the BGP control plane aspects for the VPNv4 and VPNv6 LTE backhaul service deployed in an Inter-Autonomous System (AS) design. These configurations are designed to build upon the transport layer BGP configurations described in the EPN 4.0 Transport Infrastructure Design and Implementation Guide.

Figure 5-4 IGP Control Plane for MPLS VPN Service (Inter-AS Design)

CSG LTE VPN Provider Edge (PE) Configurationrouter bgp 101 bgp router-id 100.111.13.30 !***PAN Inline RRs*** neighbor 100.111.14.1 peer-group pan neighbor 100.111.14.2 peer-group pan ! address-family vpnv4 neighbor pan send-community extended neighbor 100.111.14.1 activate neighbor 100.111.14.2 activate exit-address-family ! address-family vpnv6 neighbor pan send-community extended neighbor 100.111.14.1 activate neighbor 100.111.14.2 activate exit-address-family

2977

59CSG CSG

CSG

MTG

MTG MTG

vCN-RRvAGN-RR

CSG

iBGPVPNv4/v6

iBGPVPNv4/v6

iBGPVPNv4/v6

iBGPVPNv4/v6iBGP

VPNv4/v6

iBGPVPNv4/v6

PANInline RR

PANInline RR

CSG CSG

Core NetworkIS-IS L2

Aggregation NetworkIS-IS L2

Aggregation NetworkIS-IS L2

Mobile AccessNetworkIS-IS L1


RRRR

vAGN-RR

RR

Multi-hop eBGPVPNv4/v6


SGW/PGW SGW

MME

AS-B AS-CAS-A

VRF

VRF

VRF

VRFVRF

VRF

VRF

VRF

VRF

VRF

VRF

VRF

VRF






! !***RT Constrained Route Distribution*** address-family rtfilter unicast neighbor pan send-community extended neighbor 100.111.14.1 activate neighbor 100.111.14.2 activate exit-address-family !

Note Please refer to Route Scale Control for LTE L3 MPLS VPN Service Model, page 3-3 for a detailed explanation of how RT-constrained RD is used in order to constrain VPNv4 routes from remote RAN access regions.

PAN Inline RR Configuration

The BGP configuration for the inline-RR function on the PAN shown below requires the small change of activating the neighborship when a new CSG is added to the local access network.

router bgp 101 bgp router-id 100.111.14.1 !***AGN-RR*** neighbor 100.111.15.5 peer-group agn-rr !***CSG RR Client*** neighbor 100.111.13.30 peer-group csg ! address-family vpnv4 bgp nexthop trigger delay 3 neighbor csg send-community extended !***CSGs are RR Clients*** neighbor csg route-reflector-client !***AGN-RR is next level RR*** neighbor agn-rr send-community both neighbor 100.111.15.5 activate neighbor 100.111.13.30 activate exit-address-family ! address-family vpnv6 bgp nexthop trigger delay 3 neighbor csg send-community extended !***CSGs are RR Clients*** neighbor csg route-reflector-client !***AGN-RR is next level RR*** neighbor agn-rr send-community both neighbor 100.111.15.5 activate neighbor 100.111.13.30 activate exit-address-family ! !***RT Constrained Route Distribution towards CSGs and AGN-RR*** address-family rtfilter unicast neighbor csg send-community extended neighbor agn-rr send-community extended neighbor 100.111.15.5 activate neighbor 100.111.13.30 activate exit-address-family !

Virtualized Centralized Aggregation-Node Route Reflector (vAGN-RR) Configuration

The BGP configuration requires the small change of activating the neighbor ship when a new PAN is added to an aggregation network.

router bgp 101 bgp router-id 100.111.11.5




!***Peer Group for AGN-ASBRs and PANs*** neighbor intra-as peer-group neighbor intra-as remote-as 101

!***Peer Group for CN-RR in core AS*** neighbor inter-as peer-group neighbor inter-as remote-as 1000 neighbor inter-as ebgp-multihop 20

!***vCN-RR*** neighbor 100.111.15.50 peer-group inter-as !***PANs*** neighbor 100.111.14.1 peer-group intra-as neighbor 100.111.14.2 peer-group intra-as ! address-family vpnv4 bgp nexthop trigger delay 3 neighbor intra-as send-community both neighbor intra-as route-reflector-client neighbor inter-as send-community both !***Next-Hop Unchanged towards CN-RR*** neighbor inter-as next-hop-unchanged neighbor 100.111.15.50 activate neighbor 100.111.14.1 activate neighbor 100.111.14.2 activate exit-address-family ! address-family vpnv6 bgp nexthop trigger delay 3 neighbor intra-as send-community both neighbor intra-as route-reflector-client neighbor inter-as send-community both !***Next-Hop Unchanged towards CN-RR*** neighbor inter-as next-hop-unchanged neighbor 100.111.15.50 activate neighbor 100.111.14.1 activate neighbor 100.111.14.2 activate exit-address-family

Virtualized Centralized Core-Node Route Reflector (vCN-RR) Configurationrouter bgp 1000 bgp router-id 100.111.11.50! address-family vpnv4 unicast nexthop trigger-delay critical 2000 ! address-family vpnv6 unicast nexthop trigger-delay critical 2000 ! session-group intra-as remote-as 1000! session-group inter-as-rr remote-as 101! !***Neighbor Group for MTGs*** neighbor-group mtg use session-group intra-as ! !***MTGs are Route-Reflector Clients*** address-family vpnv4 unicast route-reflector-client !




address-family vpnv6 unicast route-reflector-client ! ! !***Multihop Neighbor Group for AGN-RR*** neighbor-group inter-as-rr use session-group inter-as-rr !***eBGP Multihop*** ebgp-multihop 20 address-family vpnv4 unicast route-policy pass-all in !***Filters unwanted RAN prefixes towards remote AGN domains*** route-policy BGP_Egress_Transport_Filter out next-hop-unchanged ! address-family vpnv6 unicast route-policy pass-all in !***Filters unwanted RAN prefixes towards remote AGN domains*** route-policy BGP_Egress_Transport_Filter out next-hop-unchanged ! ! !***AGN-RR*** neighbor 100.111.15.5 use neighbor-group inter-as-rr ! !***MTGs*** neighbor 100.111.15.1 use neighbor-group mtg ! neighbor 100.111.15.2 use neighbor-group mtg !!***Drops common RAN RTs towards AGN-RR***route-policy BGP_Egress_Transport_Filter if community matches-any (10:10) then drop else pass endifend-policy

Note Please refer to the "BGP Transport Control Plane" sections in the EPN 4.0 Transport Infrastructure Design and Implementation Guide for a detailed explanation of how egress filtering is done at the vCN-RR in order to constrain VPN routes from remote RAN access regions.

MTG LTE VPNv4/v6 PE Configurationrouter bgp 1000 nsr bgp router-id 100.111.15.1! session-group intra-as! neighbor-group cn-rr use session-group intra-as ! address-family vpnv4 unicast ! address-family vpnv6 unicast ! !






!***CN-RR*** neighbor 100.111.15.50 use neighbor-group cn-rr !

MPLS VPN Control Plane for Single-AS Design with MPLS Access

This section describes the BGP control plane aspects for the VPNv4 and VPNv6 LTE backhaul service deployed in a single-AS design. These configurations are designed to build upon the transport layer BGP configurations described in the EPN 4.0 Transport Infrastructure Design and Implementation Guide. See Figure 5-5.

Figure 5-5 BGP Control Plane for MPLS VPN Service (Single-AS Design)

CSG LTE VPN PE Configurationrouter bgp 1000 bgp router-id 100.111.13.22 ! neighbor pan peer-group neighbor pan remote-as 1000 !***PAN Inline-RRs*** neighbor 100.111.14.3 peer-group pan neighbor 100.111.14.4 peer-group pan ! address-family vpnv4 neighbor pan send-community extended neighbor 100.111.14.3 activate neighbor 100.111.14.4 activate exit-address-family ! address-family vpnv6 neighbor pan send-community extended neighbor 100.111.14.3 activate neighbor 100.111.14.4 activate exit-address-family ! !***RT Constrained Route Distribution*** address-family rtfilter unicast neighbor pan send-community extended neighbor 100.111.14.3 activate neighbor 100.111.14.4 activate





exit-address-family !

PAN Inline-RR Configuration

The BGP configuration for the inline-RR function on the PAN shown below requires the small change of activating the neighborship when a new CSG is added to a local access network.

router bgp 1000 bgp router-id 100.111.14.3!***Peer group for CSGs in local RAN network*** neighbor csg peer-group neighbor csg remote-as 1000 !***Peer group for CN-ABRs*** neighbor abr peer-group neighbor abr remote-as 1000 !***CN-ABRs*** neighbor 100.111.11.1 peer-group abr neighbor 100.111.11.2 peer-group abr !***CSGs*** neighbor 100.111.13.22 peer-group csg neighbor 100.111.13.23 peer-group csg neighbor 100.111.13.24 peer-group csg ! address-family ipv4 bgp nexthop trigger delay 2 exit-address-family ! address-family vpnv4 bgp nexthop trigger delay 3 neighbor csg send-community extended !***CSGs are RR clients*** neighbor csg route-reflector-client !***CN-ABR is next level RR*** neighbor abr send-community both neighbor 100.111.11.1 activate neighbor 100.111.11.2 activate neighbor 100.111.13.22 activate neighbor 100.111.13.23 activate neighbor 100.111.13.24 activate exit-address-family ! address-family vpnv6 bgp nexthop trigger delay 3 neighbor csg send-community extended !***CSGs are RR clients*** neighbor csg route-reflector-client !***CN-ABR is next level RR*** neighbor abr send-community both neighbor 100.111.11.1 activate neighbor 100.111.11.2 activate neighbor 100.111.13.22 activate neighbor 100.111.13.23 activate neighbor 100.111.13.24 activate exit-address-family ! !***RT Constrained Route Distribution towards CSGs*** address-family rtfilter unicast neighbor csg send-community extended neighbor 100.111.13.22 activate neighbor 100.111.13.23 activate neighbor 100.111.13.24 activate exit-address-family




!

Core Node-Area Border Router (CN-ABR) Inline-RR Configurationrouter bgp 1000 bgp router-id 100.111.11.1! !***session group for iBGP clients*** session-group intra-as ! !***iBGP neighbor group for CN-RR*** neighbor-group cn-rr use session-group intra-as ! address-family vpnv4 unicast ! address-family vpnv6 unicast ! ! !***iBGP neighbor group for PANs in local Agg*** neighbor-group agg use session-group intra-as ! address-family vpnv4 unicast route-reflector-client ! address-family vpnv6 unicast route-reflector-client ! ! !***CN-RR*** neighbor 100.111.15.50 use neighbor-group cn-rr ! !***PAN K1403*** neighbor 100.111.14.3 use neighbor-group agg ! !***PAN K1404*** neighbor 100.111.14.4 use neighbor-group agg !!

CN-RR Configurationrouter bgp 1000 bgp router-id 100.111.11.50! address-family vpnv4 unicast nexthop trigger-delay critical 2000 ! address-family vpnv6 unicast nexthop trigger-delay critical 2000 ! session-group intra-as remote-as 1000! !***Neighbor Group for MTGs*** neighbor-group mtg use session-group intra-as ! !***MTGs are Route-Reflector Clients*** address-family vpnv4 unicast




route-reflector-client ! address-family vpnv6 unicast route-reflector-client ! ! !***Neighbor Group for CN-ABR inline RR*** neighbor-group cn-abr use session-group intra-as ! address-family vpnv4 unicast route-reflector-client !***Egress filter to drop unwanted RAN loopbacks towards neighboring aggregation regions*** route-policy BGP_Egress_RAN_Filter out ! address-family vpnv6 unicast route-reflector-client !***Egress filter to drop unwanted RAN loopbacks towards neighboring aggregation regions*** route-policy BGP_Egress_RAN_Filter out ! ! !***CN-ABRs*** neighbor 100.111.2.1 use neighbor-group cn-abr ! neighbor 100.111.4.1 use neighbor-group cn-abr ! neighbor 100.111.10.1 use neighbor-group cn-abr ! neighbor 100.111.10.2 use neighbor-group cn-abr ! !***MTGs*** neighbor 100.111.15.1 use neighbor-group mtg ! neighbor 100.111.15.2 use neighbor-group mtg

route-policy BGP_Egress_Transport_Filter !***10:10 = RAN_Community for CSGs*** if community matches-any (10:10) then drop else pass endifend-policy

MTG LTE VPNv4/v6 PE Configurationrouter bgp 1000 nsr bgp router-id 100.111.15.1! session-group intra-as! neighbor-group cn-rr use session-group intra-as ! address-family vpnv4 unicast




! address-family vpnv6 unicast ! ! !***CN-RR*** neighbor 100.111.15.50 use neighbor-group cn-rr

MPLS VPN Control Plane for Small Network Design

This section describes the BGP control plane aspects for the VPNv4 and VPNv6 LTE backhaul service deployed in a Small Network Design with an integrated core/aggregation domain. These configurations are designed to build upon the transport layer BGP configurations described in the EPN 4.0 Transport Infrastructure Design and Implementation Guide. See Figure 5-6.

Figure 5-6 BGP Control Plane for MPLS VPN Service (Small Network Design)

CSG LTE VPNv4 PE Configurationrouter bgp 1000 bgp router-id 100.111.13.8 neighbor pan peer-group neighbor pan remote-as 1000 !***PAN Inline RRs*** neighbor 100.111.5.7 peer-group pan neighbor 100.111.5.8 peer-group pan ! address-family vpnv4 neighbor pan send-community both neighbor 100.111.5.7 activate neighbor 100.111.5.8 activate exit-address-family ! address-family vpnv6 neighbor pan send-community both neighbor 100.111.5.7 activate neighbor 100.111.5.8 activate exit-address-family ! !***RT Constrained Route Distribution*** address-family rtfilter unicast neighbor pan send-community extended neighbor 100.111.14.1 activate

CN

CN

CN

CN

RR

2959

50

CSG CSG

CSG

MTG

MTG

vCN-RR

CSG

iBGPVPNv4/v6

iBGPVPNv4/v6

iBGPVPNv4/v6

iBGPVPNv4/v6

iBGPVPNv4/v6

AGNInline RR

AGNInline RR

CSG CSG

Core + AggregationNetworkIS-IS L2



SGW/PGW

MME

VRF

VRF

VRF

VRF

VRF

VRF

VRF

VRF






neighbor 100.111.14.2 activate exit-address-family !

Note Please refer to Route Scale Control for LTE L3 MPLS VPN Service Model, page 3-3 for a detailed explanation of how RT-constrained RD is used in order to constrain VPNv4 routes from remote RAN access regions.

PAN Inline-RR Configuration

The BGP configuration for the inline-RR function on the PAN shown below requires the small change of activating the neighborship when a new CSG is added to a local access network.

router bgp 1000 bgp router-id 100.111.5.8 !***Session group for all CSGs*** session-group csg remote-as 1000 ! !***Session group for all AGNs and MTGs*** session-group intra-as remote-as 1000 ! !***CN-RR Neighbor group*** neighbor-group cn-rr use session-group intra-as ! address-family vpnv4 unicast ! address-family vpnv6 unicast ! !***CSG Neighbor group*** neighbor-group csg use session-group csg ! address-family vpnv4 unicast route-reflector-client ! address-family vpnv6 unicast route-reflector-client ! !***CN-RR*** neighbor 100.111.15.50 use neighbor-group cn-rr ! !***CSGs*** neighbor 100.111.13.8 use neighbor-group csg ! neighbor 100.111.13.9 use neighbor-group csg ! neighbor 100.111.13.10 use neighbor-group csg !!

CN-RR Configuration

!router bgp 1000




nsr bgp router-id 100.111.15.50!***session group for iBGP clients (AGNs and MTGs)*** session-group intra-as remote-as 1000 ! !***MTG neighbor group*** neighbor-group mtg use session-group intra-as ! !***MTGs are RR clients*** address-family vpnv4 unicast route-reflector-client ! address-family vpnv6 unicast route-reflector-client ! ! !***PAN neighbor group*** neighbor-group pan use session-group intra-as ! !***PANs are RR clients address-family vpnv4 unicast route-reflector-client ! address-family vpnv6 unicast route-reflector-client ! ! !***MTG-K1501*** neighbor 100.111.15.1 use neighbor-group mtg ! !***MTG-K1502*** neighbor 100.111.15.2 use neighbor-group mtg ! !***PANs*** neighbor 100.111.5.7 use neighbor-group pan ! neighbor 100.111.5.8 use neighbor-group pan ! neighbor 100.111.9.21 use neighbor-group pan ! neighbor 100.111.9.22 use neighbor-group pan ! neighbor 100.111.14.3 use neighbor-group pan ! neighbor 100.111.14.4 use neighbor-group pan !!

MTG LTE VPNv4/v6 PE Configurationrouter bgp 1000 nsr bgp router-id 100.111.15.1




! session-group intra-as

! neighbor-group cn-rr use session-group intra-as ! address-family vpnv4 unicast ! address-family vpnv6 unicast ! ! !***CN-RR*** neighbor 100.111.15.50 use neighbor-group cn-rr

Multicast Service Model for LTE eMBMS Implementation

The Cisco EPN 4.0 system design incorporates support for delivery of eMBMS IPv4 and IPv6 Multicast services. The multicast delivery architecture is mainly composed of two parts:

1. Label-Switched Multicast (LSM) with Multicast Label Distribution Protocol (MLDP) Global in-band signaling profile used in the MPLS aggregation and core networks.

2. Native Ipv4/IPv6 Multicast in the access and pre-aggregation networks.

As shown in Figure 5-7, this implementation covers two types of access domains depending on the platforms involved:

1. ASR 901 Ring to ASR 903 PAN Node—Wherein Cisco ASR 901 Series routers are used as the CSGs in the access ring, while Cisco ASR 903 Series routers are used as the PAN in the pre-aggregation ring. In this model, native multicast is implemented for both IPv6 & IPv4.

2. ME 3600 Ring to ASR 9000 PAN Nodes—Wherein Cisco ME 3600X Series switches are used as the fixed access nodes (FANs) or CSGs in the access ring, while a Cisco ASR 9001 router is used as the PAN in the pre-aggregation ring. in this access model, native multicast is implemented only for IPv4.

In both access domains, multicast traffic is delivered natively over the IPv4/v6 multicast-enabled infrastructure. The implementation of multicast delivery in these domains is discussed later in this section.

The SE node acts as the boundary between the native IP Multicast domains in the access and pre-aggregation networks, and the MPLS multicast domain in the core and aggregation networks. The MPLS Multicast domain is based on LSM. LSM is a solution that enables forwarding of IP multicast traffic over MPLS, thus allowing the MPLS infrastructure to provide a common data plane (based on label switching) for both unicast and multicast traffic.




Figure 5-7 Multicast Transport Implementation for eMBMS Services

As shown in Figure 5-7, the eMBMS service is implemented attaching a multicast sources to each of the MTGs: MTG-1501 and MTG-1502 respectively. Multicast receivers are located to both access rings,

AGN K1101 and AGN K1102 mark the end of aggregation/core to where the mLDP domain extends. Both PAN rings, ASR 9001 PAN and ASR 903 PAN, and the access rings, ME 3600 and ASR 901, run native IP multicast; multicast forwarding in the access domain is based on PIM SSM v4/v6.

Native IPv4 multicast forwarding is implemented in both the ME 3600 and in the ASR901 access rings, while native IPv6 multicast is implemented in the ASR901 access ring only.

Within the LSM domain, which terminates at the SE nodes, AGN-K1101 and AGN-K1102, MLDP relies on RFC 3107-learnt IPv4/v6 multicast source addresses to build the multicast tree in core. The AGNs then redistribute the IPv4/v6 multicast source addresses learnt via RFC 3107 into the level 2 ISISv4/6 process running in the native multicast domain.

At the PAN nodes, the multicast source prefixes now available in ISISv4/6 Level 2 are leaked into ISISv4/6 Level1 such that the nodes in the access ring are aware of the multicast source prefixes and PIM can build end-to-end multicast distribution trees.

Note Please refer to the "Global Multicast" section in the EPN 4.0 Transport Infrastructure Design and Implementation Guide for implementation of ISISv6 & PIMv6 in ASR 903 PAN ring and ASR 901 access ring.

LSM MLDP-Global Configuration

MLDP should be enabled on all MPLS nodes participating in LSM. MLDP base configuration is covered in the EPN 4.0 Transport Infrastructure Design and Implementation Guide chapters on global multicast support implementation.

With MLDP-Global in-band signaling, PIM is required only at the edge. PIM-SSM is used in the Cisco EPN System architecture. SSM is enabled by default on the multicast group range mentioned below.

By default, PIM-SSM operates in the 232.0.0.0/8 Multicast group range for IPv4 and ff3x::/32 (where x is any valid scope) in IPv6. To configure these values, use the SSM range command. The default SSM range was used in this implementation--thus, there's no need to explicitly configure SSM range.







In MLDP-Global in-band signaling, the root of the MLDP LSP can be chosen from BGP-Nexthop. Hence, the source-address of the (S,G) Join must be reachable through a BGP route. Also, in MLDP-Global in-band signaling, the multicast source for the eMBMS service should be reachable in a global routing table.

Cisco EPN 4.0 only supports MLDP-Global in-band signaling for the Intra-AS scenarios. Expanding the MLDP-Global for Inter-AS scenario will be supported in future releases of Cisco EPN System architecture.

MTG-9006-K1501 (ROOT-node/Ingress-PE)

Advertise the network prefix where the Multicast source of the IPTV and eMBMS service is located.

router bgp 1000 bgp router-id 100.111.15.1address-family ipv4 unicast network 200.15.12.0/24 route-policy MSE_IGW_Community network 200.15.1.0/24 route-policy MSE_IGW_Community address-family ipv6 unicast network 2001:100:111:15::1/128 network 2001:100:192:10::/64 allocate-label allroute-policy MSE_IGW_Community set community MSE_Community set community IGW_Community additiveend-policy!community-set MSE_Community 1001:1001end-set!community-set IGW_Community 3001:3001end-set!

Note Please refer to the "Prefix Filtering" section in the EPN 4.0 Transport Infrastructure Design and Implementation Guide for explanation of the need for using the route-policy MSE_IGW_Community.

Native IP Multicast Implementation in ASR 901 to ASR 903 Access Domain

As described in the design section, from CSG to eNodeB, two options are available depending on the capabilities of CSGs and the type of access network. In the case of ASR 901 access ring, for IPv4 multicast, single VLAN option is implemented for both unicast and multicast traffic with the help of a feature called VRF route leaking.

Static mroutes are added in the VRF for the multicast sources along with the keyword "fallback-lookup global," thus enabling the RPF resolution to happen in the global routing table while the multicast source prefixes remain available for the multicast traffic coming on the VRF enabled interface.

In the case of IPv6 multicast, the VRF route leaking feature is not supported in the current release. Hence, the two VLANs option is implemented, with a dedicated VLAN for unicast and a second VLAN for multicast delivery

As depicted in Figure 5-8, the CSG node is configured differently in the two cases. For IPv6, the physical interface connected to the enhanced NodeB has two SVIs: one for Unicast in VRF for the L3VPN mobile transport service (VLAN 111), and one for Multicast in Global for the eMBMS service (VLAN 222 in this example). For IPv4, only one SVI (VLAN 111) for both unicast and multicast traffic exists.






Figure 5-8 Multicast Transport Implementation for eMBMS Services in ASR901 to ASR 903

Access Domain

The CSG node facing the eNodeB may use SSM mapping if the eNodeB/Multicast Receiver is not SSM-aware and only supports IGMP v2. The CSG node is implemented with a Cisco ASR901. The eNodeBs are emulated using a traffic generator.

CSG Cisco ASR 901 Series Router Configuration

CSG-901-K1322: IGMPv3 and mldv2 Support

Enable PIMv4/v6 in the L3 interfaces connecting other nodes in the ring and the L3 interface facing the eNodeB. Enabling PIMv6 requires, global "ipv6 multicast-routing" and enable IPv6 on the interfaces.

!***Physical interface connected to PAN ring***interface GigabitEthernet0/11 description To PAN-K1403 G0/3/1 no ip address load-interval 30 carrier-delay msec 0 negotiation auto synchronous mode cdp enable isis tag 10 service-policy input PMAP-NNI-I service-policy output PMAP-NNI-E service instance 10 ethernet encapsulation dot1q 10 rewrite ingress tag pop 1 symmetric bridge-domain 10 !end!***L3 interface connecting PAN ring***interface Vlan10 ip address 10.13.22.0 255.255.255.254 ipv6 address 2001:10:13:22::/127 ipv6 enable ip pim sparse-mode!!***Physical interface connected to other CSG node in access ring***interface GigabitEthernet0/10




description To CSG-K1323 G0/10 no ip address load-interval 30 negotiation auto cdp enable service-policy input PMAP-NNI-I service-policy output PMAP-NNI-E service instance 10 ethernet encapsulation dot1q 10 rewrite ingress tag pop 1 symmetric bridge-domain 20 !end!***L3 interface connected to other CSG node in access ring***interface Vlan20 ip address 10.13.22.2 255.255.255.254 ipv6 address 2001:10:13:22::2/127 ipv6 enable ip pim sparse-mode!!***Physical interface connected to simulated eNodeB***interface GigabitEthernet0/5 no ip address negotiation auto !***EFP for Unicast VRF for S1/X2/M3 transport*** service instance 111 ethernet encapsulation dot1q 111 rewrite ingress tag pop 1 symmetric bridge-domain 111 ! !***EFP for global Multicast transport for M1 transport*** service instance 222 ethernet encapsulation dot1q 222 rewrite ingress tag pop 1 symmetric bridge-domain 222!!***L3 interface connecting to simulated eNodeB***interface Vlan222 ipv6 address 2001:222:13:22::1/64 ipv6 enable

interface Vlan111 !***Unicast VRF*** vrf forwarding LTE128 ip address 111.13.22.1 255.255.255.0 ip pim sparse-mode!***Enable IGMPv3 (default is IGMPv2)*** ip igmp version 3

!*** static mroutes in vrf with fallback-lookup global option ***ip mroute vrf LTE128 200.15.1.2 255.255.255.255 fallback-lookup globalip mroute vrf LTE128 200.15.1.1 255.255.255.255 fallback-lookup global

ip multicast-routing vrf LTE128 ip pim vrf LTE128 ssm default

CSG-901-K1323: IGMPv2 Support

Note IGMPv2 configuration requires static SSM mapping at the CSG access node.




Enable PIM-SSM and use the default SSM range 232.0.0.0/8 for IPv4. If you are using a non-232.0.0.0/8 Multicast group address, use the ssm range command.

ip pim ssm default

Enable SSM mapping and define a static SSM map to support IGMPv2 in the PIM-SSM network.

ip igmp ssm-map enableno ip igmp ssm-map query dnsip igmp ssm-map static SSM-map2 200.15.1.2ip igmp ssm-map static SSM-map1 200.15.12.2!ip access-list standard SSM-map1 permit 232.200.13.0 0.0.0.255ip access-list standard SSM-map2 permit 232.200.14.0 0.0.0.255

Enable PIM in the L3 interfaces connecting other nodes in the ring and the L3 interface facing the simulated eNodeB.

!***Physical interface connected to other CSG node in access ring***interface GigabitEthernet0/10 description To CSG-K1322 G0/10 no ip address load-interval 30 negotiation auto cdp enable service-policy input PMAP-NNI-I service-policy output PMAP-NNI-E service instance 10 ethernet encapsulation dot1q 10 rewrite ingress tag pop 1 symmetric bridge-domain 10 !end

!***L3 interface connected to other CSG node in access ring***

interface Vlan10 ip address 10.13.23.0 255.255.255.254 ip pim sparse-mode!!***Physical interface connected to other CSG node in access ring***interface GigabitEthernet0/11 description To CSG-K1324 g0/11 no ip address load-interval 30 negotiation auto cdp enable service-policy input PMAP-NNI-I service-policy output PMAP-NNI-E service instance 10 ethernet encapsulation dot1q 10 rewrite ingress tag pop 1 symmetric bridge-domain 20 !end!***L3 interface connected to other CSG node in access ring***interface Vlan20 ip address 10.13.22.3 255.255.255.254 ip pim sparse-mode!!***Physical interface connected to simulated eNodeB***interface GigabitEthernet0/4




no ip address negotiation auto !***EFP for Unicast VRF for S1/X2/M3 transport*** service instance 111 ethernet encapsulation dot1q 111 rewrite ingress tag pop 1 symmetric bridge-domain 111 ! !***EFP for global Multicast transport for M1 transport*** service instance 222 ethernet encapsulation dot1q 222 rewrite ingress tag pop 1 symmetric bridge-domain 222!interface Vlan222 ip address 222.13.23.1 255.255.255.0 ip pim sparse-mode

interface Vlan111 !***Unicast VRF*** vrf forwarding LTE128 ip address 111.13.23.1 255.255.255.0

Native IP Multicast Implementation in ME 3600 to ASR 9000 Access Domain

As depicted in Figure 5-9, the CSG physical interface connected to the eNodeB has two SVIs: one for Unicast, in VRF, for L3VPN Mobile Transport service (VLAN 111) and one for Multicast, in Global, for eMBMS service (VLAN 222 in this example).

Figure 5-9 Multicast Transport Implementation for eMBMS Services in ME 3600 to ASR 9000

Access Domain

The CSG node facing the eNodeB may use SSM mapping if the eNodeB/Multicast Receiver is not SSM-aware and only supports IGMPv2.

The CSG node is implemented with a Cisco ME 3600X Series switch. The eNodeB is emulated using IXIA traffic generator.

CSG Cisco ME 3600X Series Switch Configuration

FAN-ME36-K0712: IGMPv3 Support

iGMPv3

iGMPv2SSM-mapping

G0/3

G0/3

2976

74

MTG-1501

AGN-K1101PAN-K0507

PAN-K0508

PAN-K0712

PAN-K0709AGN-K1102

MTG-1502

Native MulticastPIMv4-SSM

mLDPGlobal

Native MulticastPIMv4-SSM

Aggregation/Core NetworkIS-IS L2

ASR 9001 PANIS-IS L2

Access Ring ME 3600IS-IS L1

Te0/0/0/2.662

Te0/0/0/2.662G9/15

MulticastRcvr

MulticastRcvr

SVI 111: UnicastSVI 222: Multicast

IXIA

G8/7

G0/0/1/8

MulticastSrc 1

MulticastSrc 2

S = 2001:100:192:10::1G = ff33:232:200:14::14

S = 200.15.1.2G= 232.200.14.14

S = 200.15.12.2G= 232.200.13.13

S = 2001:200:15:12::2G= ff33:232:200:13::13

IXIA

G5/8IXIAL2 SW

G2/14 IXIA

SVI 111: UnicastSVI 222: Multicast

AccessASR 901 Rings as CSGs

Pre-AggregationASR 903 as PANs

Aggregation and CoreASR 9000 and CSR as AGN

and Core Nodes




Enable PIM-SSM and use the default SSM range 232.0.0.0/8 for IPv4. If you are using a non-232.0.0.0/8 Multicast group address, use the ssm range command.

ip pim ssm default

Enable PIM in the L3 interfaces connecting other nodes in the ring and the L3 interface facing the eNodeB.

interface TenGigabitEthernet0/1 no switchport ip address 10.7.11.1 255.255.255.254 ip pim sparse-mode!interface TenGigabitEthernet0/2 no switchport ip address 10.7.12.0 255.255.255.254 ip pim sparse-mode!!***L3 interface connecting to simulated eNodeB***!***Physical interface connected to simulated eNodeB***interface GigabitEthernet0/3 switchport trunk allowed vlan none switchport mode trunk !***EFP for Unicast VRF for S1/X2/M3 transport*** service instance 111 ethernet encapsulation dot1q 111 rewrite ingress tag pop 1 symmetric bridge-domain 111 ! !***EFP for global Multicast transport for M1 transport*** service instance 222 ethernet encapsulation dot1q 222 rewrite ingress tag pop 1 symmetric bridge-domain 222!interface Vlan222 ip address 222.7.12.1 255.255.255.0 ip pim sparse-mode !***Enable IGMPv3 (default is IGMPv2)*** ip igmp version 3


FAN-ME36-K0709: IGMPv2 Support

Note IGMPv2 configuration requires static SSM mapping at the CSG access node.

Enable PIM-SSM and use the default SSM range 232.0.0.0/8 for IPv4. If you are using a non-232.0.0.0/8 Multicast group address, use the SSM range command.

ip pim ssm default

Enable SSM mapping and define a static SSM map. This supports IGMPv2 in the PIM-SSM network.

ip igmp ssm-map enableno ip igmp ssm-map query dnsip igmp ssm-map static SSM-map2 200.15.1.2ip igmp ssm-map static SSM-map1 200.15.12.2




!ip access-list standard SSM-map1 permit 232.200.13.0 0.0.0.255ip access-list standard SSM-map2 permit 232.200.14.0 0.0.0.255

Enable PIM in the L3 interfaces connecting other nodes in the ring and the L3 interface facing the simulated eNodeB.

interface TenGigabitEthernet0/1 no switchport ip address 10.7.11.1 255.255.255.254 ip pim sparse-mode!interface TenGigabitEthernet0/2 no switchport ip address 10.7.12.0 255.255.255.254 ip pim sparse-mode!!***Physical interface connected to simulated eNodeB***interface GigabitEthernet0/3 switchport trunk allowed vlan none switchport mode trunk !***EFP for Unicast VRF for S1/X2/M3 transport*** service instance 111 ethernet encapsulation dot1q 111 rewrite ingress tag pop 1 symmetric bridge-domain 111 ! !***EFP for global Multicast transport for M1 transport*** service instance 222 ethernet encapsulation dot1q 222 rewrite ingress tag pop 1 symmetric bridge-domain 222!interface Vlan222 description To IXIA g2/14 (acting as eNodeB) ip address 222.7.9.1 255.255.255.0 ip pim sparse-mode


L2 MPLS VPN Service Model for 2G and 3G Implementation

Layer 2 MPLS VPN service models provide TDM CES for 2G backhaul and ATM CES for 3G backhaul. The following services were validated as part of Cisco EPN 4.0:

• TDM backhaul from the CSG to the MTG, utilizing the structured CESoPSN mechanism.

• TDM backhaul from the CSG to the MTG, utilizing the unstructured Structure Agnostic Transport over Packet (SAToP) mechanism.




CESoPSN Virtual Private Wire Service (VPWS) from CSG to MTG

CESoPSN provides structured transport of TDM circuits down to the DS0 level across an MPLS-based backhaul architecture. The configurations for the CSG and MTGs are outlined in this section, including an illustration of basic backup pseudowire configuration on the CSG in order to enable transport to redundant MTGs. Complete high availability configurations are available in High Availability, page 4-4.

Figure 5-10 CESoPSN Service Implementation for 2G and 3G Backhaul


• Cisco ASR 901 Series motherboard with built-in 12GE, 1FE, 16T1E1 (A901-12C-FT-D) is used to create the CEM interface for the TDM pseudowire.

• Both Cisco ASR 9000 Series MTGs utilize 1-port channelized OC3/STM-1 ATM and circuit emulation SPA (SPA-1CHOC3-CE-ATM) in an SIP 700 card for the TDM interfaces.

• CESoPSN encapsulates T1/E1 structured (channelized) services. Structured mode (CESoPSN) identifies framing and sends only payload, which can be channelized T1s within DS3 and DS0s within T1. DS0s can be bundled to the same packet. This mode is based on IETF RFC 5086.

• mpls ldp discovery targeted-hello accept is required because the LDP session is tunneled via PW between the PEs, since they are not directly connected. Since targeted-hello response is not configured, both sessions will show as passive.

Cisco ASR 901 Series Cell Site Gateway Configurationcard type t1 0 0!controller T1 0/0 framing esf linecode b8zs cablelength short 133 cem-group 0 timeslots 1-24!pseudowire-class CESoPSN encapsulation mpls control-word!!interface CEM0/0 no ip address load-interval 30 cem 0 xconnect 100.111.15.1 13261501 encapsulation mpls pw-class CESoPSN backup peer 100.111.15.2 13261502 pw-class CESoPSN !!interface Loopback0 ip address 100.111.13.26 255.255.255.255 isis tag 10!router isis agg-acc passive-interface Loopback0

2934

49

MR-APS BSC

TDM

MTG

MTG

CSGBTS

TDM BTSBackup TDM Pseudowire

Primary TDM Pseudowire

PG

P




!!mpls ldp discovery targeted-hello accept!*** ISIS and BGP related configuration needed to ensure MPLS LDP binding with remote PE so as to establish AToM PW***

Cisco ASR 9000 Series Mobile Transport Gateway Configuration

The other MTG configuration is identical, except with a Loopback 0 IP address of 100.111.15.2 and a pw-id of 13261502.

hw-module subslot 0/2/1 cardtype sonet!controller SONET0/2/1/0 description To ONS15454-K1410 OC3 port 4/1 ais-shut report lais report lrdi sts 1 mode vt15-t1 delay trigger 250 !!controller T1 0/2/1/0/1/1/3 cem-group framed 0 timeslots 1-24 forward-alarm AIS forward-alarm RAI clock source line!interface CEM0/2/1/0/1/1/3:0 load-interval 30 l2transport!!interface Loopback0 description Global Loopback ipv4 address 100.111.15.1 255.255.255.255!l2vpn! pw-class CESoPSN encapsulation mpls control-word ! ! xconnect group TDM-K1326 p2p T1-CESoPSN-01 interface CEM0/2/1/0/1/1/3:0 neighbor 100.111.13.26 pw-id 13261501 pw-class CESoPSN ! ! !!router isis core! interface Loopback0 passive point-to-point address-family ipv4 unicast!!router bgp 1000 bgp router-id 100.111.15.1




address-family ipv4 unicast network 100.111.15.1/32 route-policy MTG_Community!mpls ldp router-id 100.111.15.1 discovery targeted-hello accept!!*** ISIS and BGP related configuration needed to ensure MPLS LDP binding with remote PE so as to establish AToM PW***

SAToP VPWS from CSG to MTG

SAToP provides unstructured transport of TDM circuits across an MPLS-based backhaul architecture. The configurations for the CSG and MTGs are outlined in this section, including Figure 5-11, which illustrates backup pseudowire configuration on the CSG to enable transport to redundant MTGs.

Figure 5-11 SAToP VPWS Implementation for 2G Backhaul


• Cisco ASR 901 Series motherboard with built-in 12GE, 1FE, 16T1E1 (A901-12C-FT-D) is used to create the CEM interface for the TDM pseudowire.

• Cisco ME 3600X 24CX Series utilizes on-board T1/E1 interfaces.

• Both Cisco ASR 9000 Series MTGs utilize 1-port channelized OC3/STM-1 ATM and circuit emulation SPA (SPA-1CHOC3-CE-ATM) in a SIP-700 card for the TDM interfaces.

• SAToP encapsulates T1/E1 services, disregarding any structure that may be imposed on these streams, in particular the structure imposed by the standard TDM framing. This mode is based on IETF RFC 4553.


Cisco ASR 901 Series Cell-Site Gateway Configurationcard type t1 0 0!controller T1 0/1 framing unframed linecode b8zs cablelength short 133 cem-group 0 unframed!pseudowire-class SAToP encapsulation mpls control-word!!interface CEM0/1 no ip address cem 0

2934

50

MR-APS BSC

TDM

MTG

MTG

CSGBTS



PG

P




xconnect 100.111.15.1 1326150101 encapsulation mpls pw-class SAToP backup peer 100.111.15.2 1326150201 pw-class SAToP !!interface Loopback0 ip address 100.111.13.26 255.255.255.255 isis tag 10!!router isis agg-acc passive-interface Loopback0!!*** ISIS and BGP related configuration needed to ensure MPLS LDP binding with remote PE so as to establish AToM PW***!mpls ldp discovery targeted-hello accept!*** ISIS and BGP related configuration needed to ensure MPLS LDP binding with remote PE so as to establish AToM PW***

Cisco ME 3600X 24CX Series Access Node Configurationcard type t1 0 1!controller T1 0/1 framing unframed clock source internal linecode b8zs cablelength short 110 cem-group 0 unframed!pseudowire-class SAToP encapsulation mpls control-word!!interface CEM0/1 no ip address load-interval 30 cem 0 xconnect 100.111.15.1 9131501 encapsulation mpls pw-class SAToP backup peer 100.111.15.2 9131502 pw-class SAToP !!interface Loopback0 ip address 100.111.9.13 255.255.255.255!!router isis agg-acc passive-interface Loopback0!!mpls ldp discovery targeted-hello accept!***ISIS and BGP related configuration needed to ensure MPLS LDP binding with remote PE so as to establish AToM PW ***


The other MTG configuration is identical, except with a Loopback 0 IP address of 100.111.15.2 and pw-ids ending in 1502 instead of 1501.

hw-module subslot 0/2/1 cardtype sonet!controller SONET0/2/1/0 description To ONS15454-K1410 OC3 port 4/1




ais-shut report lais report lrdi sts 1 mode vt15-t1 delay trigger 250 !!controller T1 0/2/1/0/1/5/1 cem-group unframed forward-alarm AIS forward-alarm RAI clock source internal!controller T1 0/2/1/0/1/3/3 cem-group unframed forward-alarm AIS forward-alarm RAI clock source internal!interface CEM0/2/1/0/1/5/1 load-interval 30 l2transport !!interface CEM0/2/1/0/1/3/3 load-interval 30 l2transport !!!interface Loopback0 description Global Loopback ipv4 address 100.111.15.1 255.255.255.255!l2vpn pw-class SAToP encapsulation mpls control-word ! ! xconnect group TDM-K0913 p2p T1-SAToP-01 interface CEM0/2/1/0/1/5/1 neighbor ipv4 100.111.9.13 pw-id 9131501 pw-class SAToP ! ! ! xconnect group TDM-K1326 p2p T1-SAToP-01 interface CEM0/2/1/0/1/3/3 neighbor ipv4 100.111.13.26 pw-id 1326150101 pw-class SAToP ! ! !router isis core interface Loopback0 passive point-to-point address-family ipv4 unicast ! !



Chapter 5 Mobile Transport Services Implementation Mobile Services Implementation with Non-MPLS Access

!router bgp 1000 bgp router-id 100.111.15.1 address-family ipv4 unicast network 100.111.15.1/32 route-policy MTG_Community!mpls ldp router-id 100.111.15.1 discovery targeted-hello accept!!*** ISIS and BGP related configuration needed to ensure MPLS LDP binding with remote PE so as to establish AToM PW***

Mobile Services Implementation with Non-MPLS Access This includes deployments with non-MPLS access, such as TDM over microwave, native Ethernet (hub-and-spoke and G8032 ring), or native IP. All services originate on the PAN in this scenario.

L3 MPLS VPN Service Model for LTE ImplementationThis section describes the implementation details and configurations for the core transport network required for L3 MPLS VPN service model for LTE Implementation and different access models that are validated in EPN 4.0. See Figure 5-12.

This section is organized into the following sections:

• MPLS VPN Core Transport for LTE S1 and X2 Interfaces, which gives the implementation details of the core transport network that serves all the different access models.

• Mobile Backhaul over Hub and Spoke Access Topologies, which describes direct eNodeB connectivity at the PAN and PON access models.

• Mobile Backhaul over Ring Access Topologies, which provides the implementation details for G.8032-enabled Ethernet access rings made of ASR901 CSG nodes,

Note ASR903 RSP1 and RSP2 support Mobile Services with Non-MPLS access.

Figure 5-12 MPLS VPN Service Implementation for LTE Backhaul

2977

60

Core NetworkMobile Access Network Aggregation Network

MPLS VPN

SGW

SGW

S1-C

S1-U

S1-U

S1-CMME

MTG-3

MTG-4

MTG-1PAN(PE)

PAN(PE)

VRF

VRF

VRF VRF

VRRP

VRFVRF

Export: Common RAN RT(10:10) and Local RAN

RT (10:203)Import: MPC RT

(1001:1001) and Local RAN RT (10:203)

Export: MPC RT (1001:1001)Import: Common RAN

RT (10:10)

MTG-2




MPLS VPN Core Transport for LTE S1 and X2 Interfaces

This section describes the L3VPN PE configuration on the PANs in the RAN access, the L3VPN PE configuration on the MTGs in the core network, and the CN and AGN RRs required for implementing the LTE backhaul service for X2 and S1 interfaces.

This section also describes the BGP control plane aspects of the VPN LTE backhaul service in an Inter-AS design with non-MPLS access. For brevity, only the Inter-AS configurations are shown, as the PAN configurations can be easily applied to the Single-AS and small network designs. See Figure 5-13.

Figure 5-13 BGP Control Plane for MPLS VPN Service

Mobile Transport Gateway MPLS VPN Configuration

This is a one-time MPLS VPN configuration done on the MTGs. No modifications are made when additional CSGs in any RAN access or other MTGs are added to the network.

SAE GW UNI

interface TenGigE0/0/0/2.1100 description Connected to SAE Gateway. vrf LTE102 ipv4 address 115.1.102.3 255.255.255.0 ipv6 nd dad attempts 0 ipv6 address 2001:115:1:102::3/64 encapsulation dot1q 1100!

VRF Definition

vrf LTE102 address-family ipv4 unicast !***Common CSG RT imported by MTG*** import route-target 10:10 ! !***Export MSE RT.*** !***Imported by every CSG in entire network.*** export route-target 1001:1001 ! ! address-family ipv6 unicast import route-target 10:10 !

2977

61

MTG

MTG MTG

vCN-RRvAGN-RR

iBGPVPNv4/v6

iBGPVPNv4/v6iBGP

VPNv4/v6

iBGPVPNv4/v6

CoreNetwork

AggregationNetwork

AggregationNetwork

Mobile AccessNetwork

Mobile AccessNetwork

RRRR

vAGN-RR

RR



SGW/PGW SGW

MME

VRF

VRFVRF

VRF

VRF

VRF

VRF

PAN(PE)

PAN(PE)

PAN(PE)

PAN(PE)




export route-target 1001:1001 ! !!

MTG 1 VPNv4/v6 BGP Configurationrouter bgp 1000 bgp router-id 100.111.15.1 bgp update-delay 360! vrf LTE102 rd 1001:1001 address-family ipv4 unicast redistribute connected ! address-family ipv6 unicast redistribute connected ! !

MTG 2 VPNv4/v6 BGP Configurationrouter bgp 1000 bgp router-id 100.111.15.2 ! vrf LTE102 rd 1001:1002 address-family ipv4 unicast redistribute connected ! address-family ipv6 unicast redistribute connected ! !

Note Each MTG has a unique RD for the MPLS VPN VRF to properly enable BGP FRR Edge functionality.

A more detailed explanation is given in High Availability, page 4-4.

PAN LTE VPNv4 PE Configurationrouter bgp 101 bgp router-id 100.111.14.1 !***AGN-RR*** neighbor 100.111.15.5 peer-group agn-rr ! address-family vpnv4 bgp nexthop trigger delay 3 !***AGN-RR is next level RR*** neighbor agn-rr send-community both neighbor 100.111.15.5 activate exit-address-family ! address-family vpnv6 bgp nexthop trigger delay 3 !***AGN-RR is next level RR*** neighbor agn-rr send-community both neighbor 100.111.15.5 activate exit-address-family




! !***RT Constrained Route Distribution towards CSGs and AGN-RR*** address-family rtfilter unicast neighbor agn-rr send-community extended neighbor 100.111.15.5 activate exit-address-family !

Centralized vAGN-RR Configuration

The BGP configuration requires the small change of activating the neighborship when a new PAN is added to the aggregation network.

router bgp 101 bgp router-id 100.111.11.5!***Peer Group for AGN-ASBRs and PANs*** neighbor intra-as peer-group neighbor intra-as remote-as 101!***Peer Group for CN-RR in core AS*** neighbor inter-as peer-group neighbor inter-as remote-as 1000 neighbor inter-as ebgp-multihop 20!***CN-RR*** neighbor 100.111.15.50 peer-group inter-as !***PANs*** neighbor 100.111.14.1 peer-group intra-as neighbor 100.111.14.2 peer-group intra-as ! address-family vpnv4 bgp nexthop trigger delay 3 neighbor intra-as send-community both neighbor intra-as route-reflector-client neighbor inter-as send-community both !***Next-Hop Unchanged towards CN-RR*** neighbor inter-as next-hop-unchanged neighbor 100.111.11.50 activate neighbor 100.111.14.1 activate neighbor 100.111.14.2 activate exit-address-family ! address-family vpnv6 bgp nexthop trigger delay 3 neighbor intra-as send-community both neighbor intra-as route-reflector-client neighbor inter-as send-community both !***Next-Hop Unchanged towards CN-RR*** neighbor inter-as next-hop-unchanged neighbor 100.111.15.50 activate neighbor 100.111.14.1 activate neighbor 100.111.14.2 activate exit-address-family

Centralized vCN-RR Configurationrouter bgp 1000 bgp router-id 100.111.15.50! address-family vpnv4 unicast nexthop trigger-delay critical 2000 ! address-family vpnv6 unicast nexthop trigger-delay critical 2000 ! session-group intra-as




remote-as 1000! session-group inter-as-rr remote-as 101! !***Neighbor Group for MTGs*** neighbor-group mtg use session-group intra-as ! !***MTGs are Route-Reflector Clients*** address-family vpnv4 unicast route-reflector-client ! address-family vpnv6 unicast route-reflector-client ! ! !***Multihop Neighbor Group for AGN-RR*** neighbor-group inter-as-rr use session-group inter-as-rr !***eBGP Multihop*** ebgp-multihop 20 address-family vpnv4 unicast route-policy pass-all in !***Filters unwanted RAN prefixes towards remote AGN domains*** route-policy BGP_Egress_Transport_Filter out next-hop-unchanged ! address-family vpnv6 unicast route-policy pass-all in !***Filters unwanted RAN prefixes towards remote AGN domains*** route-policy BGP_Egress_Transport_Filter out next-hop-unchanged ! ! !***AGN-RR*** neighbor 100.111.15.5 use neighbor-group inter-as-rr ! !***MTGs*** neighbor 100.111.15.1 use neighbor-group mtg !neighbor 100.111.15.2 use neighbor-group mtg!!***Drops common RAN RTs towards AGN-RR***route-policy BGP_Egress_Transport_Filter if community matches-any (10:10) then drop else pass endifend-policy

MTG LTE VPNv4/v6 PE Configurationrouter bgp 1000 nsr bgp router-id 100.111.15.1! session-group intra-as! neighbor-group cn-rr




use session-group intra-as ! address-family vpnv4 unicast ! address-family vpnv6 unicast ! ! !***CN-RR*** neighbor 100.111.15.50 use neighbor-group cn-rr !

Mobile Backhaul over Hub-and-Spoke Access Topologies

This section describes the implementation details of direct eNodeB connectivity at the PAN and PON access models which forms the EPN 4.0 mobile backhaul implementation for non-MPLS access, over hub-and-spoke access topologies.

Direct eNodeB Connectivity to PAN Node

The following section shows the configuration of PAN K1401 to which eNodeB is directly connected.

MPLS VPN PE Configuration on PAN K1401

Directly-attached eNodeB UNI

interface GigabitEthernet0/3/6 vrf forwarding LTE224 ip address 114.1.224.1 255.255.255.0 load-interval 30 negotiation auto ipv6 address 2001:114:1:224::1/64

VRF Definitionvrf definition LTE224 rd 10:104 ! address-family ipv4 export map ADDITIVE route-target export 10:104 route-target import 10:104 route-target import 1001:1001 route-target import 236:236 route-target import 235:235 exit-address-family ! address-family ipv6 export map ADDITIVE route-target export 10:104 route-target import 10:104 route-target import 1001:1001 route-target import 235:235 exit-address-family!

!***Route map to export Global RT 10:10 in addition to Local RAN RT 10:203***route-map ADDITIVE permit 10 set extcommunity rt 10:10 additive

!***VPN BGP Configuration***router bgp 101




neighbor pan peer-group neighbor pan remote-as 101 neighbor pan password lab neighbor pan update-source Loopback0 ! address-family vpnv4 bgp nexthop trigger delay 2 neighbor pan send-community extended ! address-family vpnv6 bgp nexthop trigger delay 2 neighbor pan send-community extended ! address-family ipv4 vrf LTE224 !***For Directly Connected eNodeB*** redistribute connected exit-address-family ! address-family ipv6 vrf LTE224 !***For Directly Connected eNodeB*** redistribute connected exit-address-family

Mobile Backhaul over PON Access

The Cisco EPN 4.0 System design validation includes cell site connectivity via GPON access. From the CSG perspective, the connection to the ONU is configured just like any other Ethernet connection. The transport over the PON access network between the ONU and OLT is configured as "native" (or untagged) in order to transport the untagged traffic between the CSG and aggregation nodes. This section gives an overview of the CSG and PON access configurations.

ONT/ONU-OLT Setup and Configuration

Step 1 Create BitStream service in the OLT, and then associate the service to the ONU.

Step 2 To configure services in the OLT, go to Configuration > Services.

Step 3 To associate the services to the ONU, go to Configuration > Remote Equipments > ONU, and then select the correct PON port and ONU ID of the ONU to be configured.

Step 4 Click Apply, and then scroll down to the Services portion. Add the services and configure them with the appropriate VLAN.

Step 5 For BitStream service type for mobile services, the ETH-VLAN and UNI-VLAN are kept the same.

Step 6 Under Services, choose the following options:

• Type—Bitstream (this service type is used in Business Services, where all packets pass transparently by the system and MAC learning is disabled).

• Eth Vlan—Identifies the outer tag of the service used in the uplink port (should match the dot1q tag configured in the PE).

• Uni Vlan—Identifies the VLAN delivered to or received from the ONT (usually the same VLAN tag with Eth Vlan).




Figure 5-14 Mobile PON Services

From ONU > Services, choose the following options:

• cVlan—0 (cVLAN not needed unless you are doing qinq).

• Native Vlan—Checked/Enable for untagged traffic towards the CPE.

• Uni Vlan—Default Uni Vlan to be delivered in the ONT port (usually the same VLAN tag with Eth Vlan).




Figure 5-15 Mobile PON ONU

CSG Configuration

CSG-K1317 Configuration

!***Ethernet interface connected to ONU***interface GigabitEthernet0/1 description To MOB-ONT-OLT3 setup load-interval 30 negotiation auto service instance 900 ethernet encapsulation untagged bridge-domain 900

interface Vlan900 !***Link-local subinterface with PAN connected to far end of PON link*** ip address 10.5.4.14 255.255.255.254 ip router isis agg-acc load-interval 30 mpls ip mpls ldp igp sync delay 10 bfd interval 50 min_rx 50 multiplier 3 isis circuit-type level-1 isis network point-to-point isis csnp-interval 10end!***VRF Configuration***vrf definition LTE229 rd 10:209 ! address-family ipv4 export map ADDITIVE route-target export 10:209 route-target import 1001:1001 route-target import 236:236 route-target import 235:235




exit-address-family!***VRF added under BGP Configuration***router bgp 1000 address-family ipv4 vrf LTE229 redistribute connected exit-address-family!route-map ADDITIVE permit 10 set extcommunity rt 10:10 additive

Note VPN prefixes are exported with RT 10:209. All the CSGs are configured to import RT 10:209 for X2 communication and RT 1001:1001 for S1 communication. Route-map ADDITIVE is used for appending RT 10:10 to 10:209.

Mobile Backhaul over Ring Access Topologies

Mobile Backhaul over Ring access topologies are implemented for G.8032-enabled Ethernet access rings made of ASR 901 CSG nodes. The following section gives the implementation details for the same.

MPLS VPN Transport for LTE S1 and X2 Interfaces G.8032 Ethernet Access- ASR 901

This section describes the L3VPN configuration aspects on the CSGs in the RAN access and the Mobile Transport Gateways (MTGs) in the core network required for implementing the LTE backhaul service for X2 and S1 interfaces. In this scenario, the access network is deployed as a G.8032-protected Ethernet ring, and it is dual homed to a pair of SE nodes that provide the VRF Services for the LTE backhaul. SE Node Dual Homing is achieved by a combination of VRRP, Routed PW, and G.8032 providing resiliency and load balancing in the access network.

In Figure 5-16, the SE Nodes, AGN-K0301 and AGN-K0302, implement the L3 MPLS VPN Service for the transport of LTE traffic to the MPC. A routed BVI interface acts as the service endpoint. The LTE S1 interface and a X2 interface set across different access domains are carried over the L3VPN service. A X2 interface between LTE eNodeBs on the same G.8032 access domain is bridged over the ring.

Figure 5-16 MPLS VPN Service Implementation for LTE Backhaul on Ethernet G8032 Ring Access

The Ethernet access network is implemented as a G.8032 access ring and carries a dedicated VLAN to L3 MPLS VPN-based service. A PW running between the SE nodes closes the service VLAN providing full redundancy on the ring.

2959

56ASR9KK0301

ASR 901Ring

K0308

ME3400K0901

X2

Ten

0/0

Ten

0/0

Ten 0/1

Ten 0/1

Ten 0/0

IXIA10/11

IXIA10/1

Gig

0/7

Gig

0/2

Gig

0/5 K0306

K0307

K0305 ASR9KK0302

MPLS VPN v4 or v6

VRF

VRF

BV

IS

ub I/

FB

VI

SGW

K1502

K1502

MME

MTG 3

MTG 4

VRRPVRRP Routed PW

VRFVRF

VRFVRF

SGW

Sub

I/F

S1-U

S1-C

S1-C

S1-U

Ten

0/2

/1/3

Ten

0/2

/1/3




VRRP is configured on the Routed BVI interface to ensure the eNodeBs have a common default gateway regardless of the SE node forwarding the traffic. The LTE eNodeB is emulated by a Cisco ME3400 node, K0901.

AGN K0302 Configurationinterface TenGigE0/2/1/3.302 l2transport encapsulation dot1q 302 rewrite ingress tag pop 1 symmetric!l2vpn bridge group L2VPN bridge-domain EPN30-L3VPN-302 interface TenGigE0/2/1/3.302 !!*** Routed PW configured to other SE Node 100.111.3.1*** neighbor 100.111.3.1 pw-id 302 ! routed interface BVI302 ! !!***VRF Definition***vrf LTE224 address-family ipv4 unicast import route-target!***Local RAN RT. Same RT on all CSGs in a given RAN region*** 10:104 235:235 236:236 1001:1001 ! export route-policy ADDITIVE export route-target 10:104 ! !address-family ipv6 unicast import route-target 10:104 235:235 236:236 1001:1001 ! export route-policy ADDITIVE export route-target 10:104 ! !!!***BVI Interface Configuration***interface BVI302 vrf LTE224 ipv4 address 30.2.1.2 255.255.255.0 ipv6 nd dad attempts 0 ipv6 address 2001:13:2:102::2/64!!***VRRP Configuration***router vrrpinterface BVI302 address-family ipv4 vrrp 2!***Highest Priority value to be active*** priority 253




preempt delay 600 address 30.2.1.1 bfd fast-detect peer ipv4 30.2.1.3 ! !

AGN K0301 Configurationinterface TenGigE0/2/1/3.302 l2transport encapsulation dot1q 302 rewrite ingress tag pop 1 symmetric!l2vpn bridge group L2VPN bridge-domain EPN30-L3VPN-302 interface TenGigE0/2/1/3.302 !!*** Routed PW configured to other SE Node 100.111.3.2*** neighbor 100.111.3.2 pw-id 302 ! routed interface BVI302 ! !!!***VRF Definition***vrf LTE224 address-family ipv4 unicast import route-target!***Local RAN RT. Same RT on all CSGs in a given RAN region*** 10:104 235:235 236:236 1001:1001 ! export route-policy ADDITIVE export route-target 10:104 ! !address-family ipv6 unicast import route-target 10:104 235:235 236:236 1001:1001 ! export route-policy ADDITIVE export route-target 10:104 ! !!!***BVI Interface Configuration***interface BVI302 vrf LTE224 ipv4 address 30.2.1.3 255.255.255.0 ipv6 nd dad attempts 0 ipv6 address 2001:13:2:102::3/64 !!***VRRP Configuration***router vrrp interface BVI302 address-family ipv4 vrrp 2




!***Highest Priority value to be active*** priority 252 address 30.2.1.1 bfd fast-detect peer ipv4 30.2.1.2 ! !

Access Node K0306 Configurationinterface GigabitEthernet0/5!***connection to LTE*** service instance 302 ethernet encapsulation dot1q 302 rewrite ingress tag pop 1 symmetric bridge-domain 302!interface TenGigabitEthernet0/1!*** NNI port*** service instance 302 ethernet encapsulation dot1q 302 rewrite ingress tag pop 1 symmetric bridge-domain 302interface TenGigabitEthernet0/0!*** NNI port**** service instance 302 ethernet encapsulation dot1q 302 rewrite ingress tag pop 1 symmetric bridge-domain 302

Access Node K0307 Configurationinterface TenGigabitEthernet0/1!*** NNI port*** service instance 302 ethernet encapsulation dot1q 302 rewrite ingress tag pop 1 symmetric bridge-domain 302interface TenGigabitEthernet0/0!*** NNI port**** service instance 302 ethernet encapsulation dot1q 302 rewrite ingress tag pop 1 symmetric bridge-domain 302!***Same above configuration for other Access Node K0305 and K0308***

L2 MPLS VPN Service Model for 2G and 3G Implementation

Layer 2 MPLS VPN service models provide TDM CES for 2G backhaul and ATM CES for 3G backhaul. The following services were validated as part of Cisco EPN 4.0:

• TDM backhaul from the PAN to the MTG, utilizing the structured CESoPSN mechanism.

• TDM backhaul from the PAN to the MTG, utilizing the unstructured SAToP mechanism.

• ATM backhaul from the PAN to the MTG, supporting both ATM clear-channel and inverse multiplexing over ATM (IMA) circuits.

CESoPSN VPWS from PAN to MTG

CESoPSN provides structured transport of TDM circuits down to the DS0 level across an MPLS-based backhaul architecture. The configurations for the CSG and MTGs are outlined in this section, including an illustration of basic backup pseudowire configuration on the CSG in order to enable transport to




redundant MTGs. Complete high availability configurations are available in High Availability, page 4-4.

Figure 5-17 CESoPSN Service Implementation for 2G and 3G Backhaul


• The Cisco ASR 903 Series router utilizes a 16-port T1/E1 Interface Module (A900-IMA16D) for TDM interfaces.


• CESoPSN encapsulates T1/E1 structured (channelized) services. Structured mode (CESoPSN) identifies framing and sends only payload, which can be channelized T1s within DS3 and DS0s within T1. DS0s can be bundled to the same packet. This mode is based on IETF RFC 5086.


Cisco ASR 903 Series Pre-Aggregation Node Configurationcard type t1 0 5!controller T1 0/5/1 framing esf clock source internal linecode b8zs cablelength short 110 forward-alarm ais forward-alarm rai cem-group 0 timeslots 1-24!pseudowire-class CESoPSN encapsulation mpls control-word!!interface CEM0/5/1 no ip address load-interval 30 cem 0 xconnect 100.111.15.1 1401150113 encapsulation mpls pw-class CESoPSN backup peer 100.111.15.2 1401150213 pw-class CESoPSN ! hold-queue 4096 in hold-queue 4096 out!interface Loopback0 ip address 100.111.14.1 255.255.255.255!!router isis agg-acc passive-interface Loopback0

PANBTS

2934

53

MR-APS BSC

TDM

MTG

MTG



PG

P




!!!*** ISIS and BGP related configuration needed to ensure MPLS LDP binding with remote PE so as to establish AToM PW***mpls ldp discovery targeted-hello accept



hw-module subslot 0/2/1 cardtype sonet!controller SONET0/2/1/0 description To ONS15454-K1410 OC3 port 4/1 ais-shut report lais report lrdi sts 1 mode vt15-t1 delay trigger 250 !!controller T1 0/2/1/0/1/1/3 cem-group framed 0 timeslots 1-24 forward-alarm AIS forward-alarm RAI clock source internal!interface CEM0/2/1/0/1/1/3:0 load-interval 30 l2transport !!!interface Loopback0 description Global Loopback ipv4 address 100.111.15.1 255.255.255.255!l2vpn pw-class CESoPSN encapsulation mpls control-word ! ! xconnect group TDM-K1401 p2p T1-CESoPSN-01 interface CEM0/2/1/0/1/1/3:0 neighbor ipv4 100.111.14.1 pw-id 1401150113 pw-class CESoPSN ! ! !router isis core interface Loopback0 passive point-to-point address-family ipv4 unicast ! !!router bgp 1000 bgp router-id 100.111.15.1 address-family ipv4 unicast




network 100.111.15.1/32 route-policy MTG_Community!mpls ldp router-id 100.111.15.1 discovery targeted-hello accept!!*** ISIS and BGP related configuration needed to ensure MPLS LDP binding with remote PE so as to establish AToM PW***!

SAToP VPWS from PAN to MTG

SAToP provides unstructured transport of TDM circuits across an MPLS-based backhaul architecture. The configurations for the PAN and MTGs are outlined in this section, including an illustration of backup pseudowire configuration on the PAN in order to enable transport to redundant MTGs.

Figure 5-18 SAToP VPWS Implementation for 2G Backhaul


• The Cisco ASR 903 Series router utilizes a 16-port T1/E1 Interface Module (A900-IMA16D) for TDM interfaces.


• SAToP encapsulates T1/E1 services, disregarding any structure that may be imposed on these streams, in particular the structure imposed by the standard TDM framing. This mode is based on IETF RFC 4553.


Cisco ASR 903 Series Pre-Aggregation Node Configurationcard type t1 0 5!controller T1 0/5/0 framing unframed clock source internal linecode b8zs cablelength short 110 cem-group 0 unframed!pseudowire-class SAToP encapsulation mpls control-word!!interface CEM0/5/0 no ip address load-interval 30 cem 0

PANBTS

2934

53

MR-APS BSC

TDM

MTG

MTG



PG

P




xconnect 100.111.15.1 14011501 encapsulation mpls pw-class SAToP backup peer 100.111.15.2 14011502 pw-class SAToP ! hold-queue 4096 in hold-queue 4096 out!interface Loopback0 ip address 100.111.14.1 255.255.255.255!!router isis agg-acc passive-interface Loopback0!!mpls ldp discovery targeted-hello accept



hw-module subslot 0/2/1 cardtype sonet!controller SONET0/2/1/0 description To ONS15454-K1410 OC3 port 4/1 ais-shut report lais report lrdi sts 1 mode vt15-t1 delay trigger 250 !!controller T1 0/2/1/0/1/1/2 cem-group unframed forward-alarm AIS forward-alarm RAI clock source line!interface CEM0/2/1/0/1/1/2 load-interval 30 l2transport !!!interface Loopback0 description Global Loopback ipv4 address 100.111.15.1 255.255.255.255!l2vpn pw-class SAToP encapsulation mpls control-word ! ! xconnect group TDM-K1401 p2p T1-SAToP-01 interface CEM0/2/1/0/1/1/2 neighbor 100.111.14.1 pw-id 14011501 pw-class SAToP ! ! !router isis core




interface Loopback0 passive point-to-point address-family ipv4 unicast ! !!router bgp 1000 bgp router-id 100.111.15.1 address-family ipv4 unicast network 100.111.15.1/32 route-policy MTG_Community!mpls ldp router-id 100.111.15.1 discovery targeted-hello accept

ATM Clear-Channel VPWS from PAN to MTG

AToM pseudowire circuits are utilized to provide ATM circuit transport across an MPLS-based backhaul architecture. The ATM interface is configured for ATM Adaptation Layer 0 (AAL0) to allow for transparent transport of the entire permanent virtual circuit (PVC) across the transport network. The configurations for the PAN and MTGs are outlined in this section. QoS implementation for ATM circuits is covered in Chapter 7, “Quality of Service Implementation,” and resiliency via pseudowire redundancy and MR-APS are covered in High Availability, page 4-4.

Figure 5-19 ATM VPWS Implementation for 3G Backhaul


• The Cisco ASR 903 Series router utilizes a 16-port T1/E1 Interface Module (A900-IMA16D) for ATM interfaces.

• Both Cisco ASR 9000 Series MTGs utilize 1-port OC3/STM-1 ATM SPA (SPA-1XOC3-ATM-V2) in a SIP-700 card for the TDM interfaces.

• ATM transport via pseudowire over an MPLS infrastructure is detailed in IETF RFC 4447.

• The PE side of the ATM interface uses AAL0 encapsulation, and the CE side uses AAL5 Subnetwork Access Protocol (SNAP), or AAL5SNAP, encapsulation.


CE Node Connected to PAN!card type t1 0 0!controller T1 0/3 framing esf clock source line linecode b8zs

2934

54

MR-APS ATM RNC

ATM

MTG

MTG

PANNode B

ATM NodeBBackup ATM Pseudowire

Primary ATM PseudowireP

GP




cablelength long 0db mode atm!interface ATM0/3 ip address 100.14.15.26 255.255.255.252 load-interval 30 no scrambling-payload no atm enable-ilmi-trap pvc 100/4011 protocol ip 100.14.15.25 broadcast encapsulation aal5snap !!interface Vlan201 ip address 214.14.6.17 255.255.255.252 load-interval 30 no ptp enable!interface GigabitEthernet0/2 description Traffic Generator with IP 214.14.6.18 switchport access vlan 201 switchport mode access load-interval 30!ip route 214.15.3.16 255.255.255.252 100.14.15.25!

Cisco ASR 903 Series Pre-Aggregation Node Configurationcard type t1 0 5license feature atm

controller T1 0/5/2 framing esf clock source internal linecode b8zs cablelength long 0db atm!interface ATM0/5/2 no ip address no atm enable-ilmi-trap!interface ATM0/5/2.100 point-to-point no atm enable-ilmi-trap pvc 100/4011 l2transport encapsulation aal0 xconnect 100.111.15.1 1401150115 encapsulation mpls !!interface Loopback0 ip address 100.111.14.1 255.255.255.255!!router isis agg-acc passive-interface Loopback0!!***ISIS and BGP related configuration needed to ensure MPLS LDP binding with remote PE***mpls ldp discovery targeted-hello accept!






interface ATM0/2/3/0 load-interval 30!interface ATM0/2/3/0.100 l2transport pvc 100/4011 encapsulation aal0 shape vbr-rt 20000 14000 7000 !!!interface Loopback0 description Global Loopback ipv4 address 100.111.15.1 255.255.255.255!l2vpn pw-class ATM encapsulation mpls ! ! xconnect group ATM-K1401 p2p T1-ATM-01 interface ATM0/2/3/0.100 neighbor 100.111.14.1 pw-id 1401150115 pw-class ATM ! ! !router isis core interface Loopback0 passive point-to-point address-family ipv4 unicast ! !!router bgp 1000 bgp router-id 100.111.15.1 address-family ipv4 unicast network 100.111.15.1/32 route-policy MTG_Community!mpls ldp router-id 100.111.15.1 discovery targeted-hello accept!

CE Device Connected to MTGsinterface ATM3/1/0 no ip address no atm ilmi-keepalive no atm enable-ilmi-trap!interface ATM3/1/0.100 point-to-point ip address 100.14.15.25 255.255.255.252 no atm enable-ilmi-trap pvc 100/4011 protocol ip 100.14.15.26 broadcast encapsulation aal5snap !!




interface GigabitEthernet2/3/0 description Traffic Generator with IP 214.15.3.18 ip address 214.15.3.17 255.255.255.252 load-interval 30 speed 1000 no negotiation auto cdp enable!ip route 214.14.6.16 255.255.255.252 ATM3/1/0.100!

ATM IMA VPWS from PAN to MTG

AToM PW circuits are utilized to provide ATM circuit transport across an MPLS-based backhaul architecture. The ATM interface in this example is configured for IMA. The configurations for the PAN and MTGs are outlined in this section. QoS implementation for ATM circuits is covered in Chapter 6, “Functional Components Implementation,” and resiliency via pseudowire redundancy and MR-APS are covered in High Availability, page 4-4.



• The Cisco ASR 903 Series router utilizes a 16-port T1/E1 Interface Module (A900-IMA16D) for ATM interfaces.

• Both Cisco ASR 9000 Series MTGs utilize 1-port OC3/STM-1 ATM SPA (SPA-1XOC3-ATM-V2) in a SIP-700 card for the TDM interfaces.

• ATM transport via pseudowire over an MPLS infrastructure is detailed in IETF RFC 4447.

• The PE side of the ATM interface uses AAL0 encapsulation, and the CE side uses AAL5SNAP encapsulation.


CE Node Connected to PAN!card type t1 0 0!controller T1 0/3 framing esf clock source internal linecode b8zs cablelength short 110 ima-group 0 no-scrambling-payload!!interface ATM0/IMA0 ip address 100.14.15.30 255.255.255.252 ima group-id 0

2934

54

MR-APS ATM RNC

ATM

MTG

MTG

PANNode B


Primary ATM Pseudowire

PG

P




no atm ilmi-keepalive no atm enable-ilmi-trap pvc 200/4021 protocol ip 100.14.15.29 broadcast encapsulation aal5snap !!interface Vlan201 ip address 214.7.1.1 255.255.255.0 no ptp enable!interface GigabitEthernet0/3 switchport access vlan 201 switchport mode access!ip route 215.3.1.0 255.255.255.0 100.14.15.29!Cisco ASR 903 Series Pre-Aggregation Node Configurationcard type t1 0 5license feature atm

controller T1 0/5/3 framing esf clock source internal linecode b8zs cablelength short 110 ima-group 1!interface ATM0/5/ima0 no ip address atm bandwidth dynamic no atm enable-ilmi-trap no atm ilmi-keepalive pvc 200/4021 l2transport encapsulation aal0 xconnect 100.111.15.1 1402150116 encapsulation mpls backup peer 100.111.15.2 1402150216 !interface Loopback0 ip address 100.111.14.1 255.255.255.255!!router isis agg-acc passive-interface Loopback0!!*** ISIS and BGP related configuration needed to ensure MPLS LDP binding with remote PE so as to establish AToM PW***mpls ldp discovery targeted-hello accept



interface ATM0/2/3/0 load-interval 30!interface ATM0/2/3/0.200 l2transport pvc 200/4021 encapsulation aal0 !!!interface Loopback0



Chapter 5 Mobile Transport Services Implementation Mobile Transport Capacity Monitoring

description Global Loopback ipv4 address 100.111.15.1 255.255.255.255!l2vpn pw-class ATM encapsulation mpls ! ! xconnect group ATM-K1402 p2p T1-ATM-IMA-01 interface ATM0/2/3/0.200 neighbor 100.111.14.2 pw-id 1402150116 pw-class ATM ! ! !router isis core interface Loopback0 passive point-to-point address-family ipv4 unicast ! !!router bgp 1000 bgp router-id 100.111.15.1 address-family ipv4 unicast network 100.111.15.1/32 route-policy MTG_Community!mpls ldp router-id 100.111.15.1 discovery targeted-hello accept!

CE Device Connected to MTGsinterface ATM3/1/0 no ip address no atm ilmi-keepalive no atm enable-ilmi-trap!interface ATM3/1/0.200 point-to-point ip address 100.14.15.29 255.255.255.252 no atm enable-ilmi-trap pvc 200/4021 protocol ip 100.14.15.30 broadcast encapsulation aal5snap !!interface GigabitEthernet2/1/0 ip address 215.3.1.1 255.255.255.0!ip route 214.7.1.0 255.255.255.0 100.14.15.30!

Mobile Transport Capacity Monitoring As introduced briefly in Chapter 2, “Mobile Transport Services Overview,”, mobile transport capacity monitoring is very helpful in operational planning of cell site locations based on actual traffic details at each cell site. While Cisco's Prime Provisioning can be used for provisioning the mobile L3 MPLS VPN




services, Cisco's Prime Performance Manager (PPM) can now be used for collecting the performance monitoring data, in this case, NetFlow statistics, which can be used for capacity monitoring for the mobile L3 MPLS VPN services. See Figure 5-21

This section describes the implementation of mobile transport capacity monitoring, which is done leveraging the NetFlow functionality on MTG and the NetFlow collector functionality on Cisco PPM.

Figure 5-21 Mobile Transport Capacity Monitoring Topology

Enabling the NetFlow recording and exporting capabilities on MTGs involves configuring a sampler map, an exporter map, and a monitor map. The monitor map is then applied on the Mobile Packet Core-facing interface on MTG 1 and MTG 2. The connection between the MTGs and the PPM is established through a regular line card at the MTG and not through the management interface. Once this is done, the MTG is now ready to export NetFlow statistics to PPM. Such NetFlow statistics are collected in PPM in the form of reports, which can be accessed via GUI to display the amount of traffic sent and received by a given cell site.

NetFlow capabilities are enabled on MTG1 and 2 and the flow monitor map is applied on tenGigE 0/0/0/2.1100 on both MTG1 and MTG2. This interface is connected to MPC gateway.

Both the MTGs are connected to PPM through the data center connectivity via the interface GigabitEthernet0/0/1/15.

NetFlow Configurations on MTG-1 (MTG-9006-K1501)

!*** Sampler map configuration***sampler-map fsm1 random 1 out-of 1!

!*** Exporter Map Configuration***flow exporter-map fem1 version v9 options interface-table timeout 60 options sampler-table timeout 60 template timeout 5 template data timeout 5 template options timeout 5 ! dscp 10

2977

62


eNode B

MPLS VPN (v4 or v6)CSG

SGW

NetFlow Collectionand Analysis

172.18.133.92

Ten GigE 0/0/0/2.1100

GigE 0/0/1/15

NetFlow Exporter

S1-U

S1-U

MTG

MTG SGW




!*** this is the default udp port used for NetFlowNetFlow communication between MTG and PPM *** transport udp 9991 source Loopback0 !*** Prime Performance Manager IP address *** destination 172.18.133.92!

!*** Monitor Map Configuration ***flow monitor-map IPv4-fmm record ipv4 exporter fem1 cache entries 10000 cache timeout active 30 cache timeout inactive 15!

!*** Monitor Map applied on the MPC gateway facing interface ***

interface TenGigE0/0/0/2.1100 description To A-RAN-4500-K1504 Ten1/29 vrf LTE102 ipv4 address 115.1.102.3 255.255.255.0 ipv6 nd dad attempts 0 ipv6 address 2001:115:1:102::3/64 load-interval 30 flow ipv4 monitor IPv4-fmm sampler fsm1 ingress flow ipv4 monitor IPv4-fmm sampler fsm1 egress encapsulation dot1q 1100

Note Configuration on MTG-2 is identical to that shown for MTG-1.

Report Generation from Prime Performance Manager

Prime Performance Manager (PPM) is enabled for NetFlow collection by default; therefore, no specific action needs to be taken during installation. Once NetFlow is enabled on the device, PPM is ready for NetFlow statistics collection.

The steps involved for the setup of PPM for NetFlow collection and report generation are as follows:

1. Synchronize the devices database between Prime Network and PPM for the device to be reflected in PPM.

2. Verify that the device appears in PPM with NetFlow enabled.

3. Enable NetFlow report generation for the added device.

4. Select the type of report to be generated.

Note Netflow exported device must be already present in Prime Network. If not, the device must be first added in Prime Network and then synced to Prime Performance Manager.

Synchronizing the Device Database Between Prime Network and Prime Performance Manager

In order to synchronize the devices database from Prime Network to PPM, from the Administration tab select Prime Network Integration.




Figure 5-22 Synchronizing Prime Network and PPM

Choose Strict Sync and then click Refresh.

Figure 5-23 Synchronizing Prime Network and PPM with Strict Sync

Verify that the Device Appears in PPM with NetFlow Enabled

After the database is synchronized between Prime Network and PPM, from the Network tab, select Devices to check if the device is present in the PPM database and NetFlow is enabled for the same.




Figure 5-24 PPM-Network-Devices

Select the Data Collection tab.

Figure 5-25 PPM-Network-Devices-Data Collection-NetFlowNetFlow

Enable NetFlow Report Generation for the Added Device

After the device appears in the PPM database, go to Reports in the side panel. Select Report/Group Settings and then enable the required time interval options for the report generation.




Figure 5-26 Report/Group Settings

Next, select Report/Group Policies and enable all the required NetFlow reports for the report generation.

Figure 5-27 Report/Group Policies

Select Report/Group Status to check if all the necessary NetFlow report generations are enabled.




Figure 5-28 Report/Group Status-a

Figure 5-29 Report/Group Status-b

Select the Type of Report to be Generated

Select NetFlow under Reports in the side panel, and navigate to the desired reports. The following figure shows the example for the NetFlow Conversations report under NetFlow Endpoints. The report shows the conversations between individual CSG nodes (currently identified by the CSG node interface IP address) and MTG interfaces. For example,

• IP address 113.29.224.2 corresponds to CSG node K1329 (belongs to large network).

• IP address 113.30.224.1 corresponds to CSG node K1330 (belongs to large network).

• IP address 113.22.128.2 corresponds to CSG node K1322 (belongs to small network).

• IP address 115.1.102.2 corresponds to MTG1.

The different data points in terms of volume (bytes), throughput (bytes/sec) and packets pertaining to individual conversations are shown in the report.




Figure 5-30 Report-NetFlow Endpoints-Conversations

Select NetFlow under Reports in the side panel, and navigate to the desired reports. Figure 5-31 shows the example for NetFlow Applications-Conversations report under NetFlow Applications-Conversations.

Figure 5-31 Report-NetFlow Applications-Conversations




C H A P T E R 6
Functional Components Implementation
This chapter includes the following major topics:

• Synchronization Distribution Implementation, page 6-1

• High Availability Implementation, page 6-23

Synchronization Distribution ImplementationThe Cisco EPN System architecture implements a comprehensive hybrid synchronization distribution model, utilizing a combination of both SyncE and IEEE 1588v2 PTP in order to deliver frequency, phase, and ToD distribution across the transport network. This model implements a hybrid PTP boundary clock (BC) function, which uses frequency from SyncE and phase from PTP for local clock recovery. This model regenerates clock-to-delivery frequency, phase, and ToD to downstream devices.

This model can be modified to suit certain deployment scenarios or limitations. For example, if the services delivered over the transport network require only frequency synchronization, then only SyncE need be deployed. If the end-to-end transport network contains devices that do not support SyncE, then 1588v2 may be utilized for frequency distribution in that portion of the network.

This section includes the configuration for two variations of the hybrid synchronization model:

• Hybrid Model Configuration with a Third-Party Grandmaster Clock Source.

• Hybrid Model Configuration with a Cisco ASR 9000 Series Router as Grandmaster Clock Source.

The configurations for each node are divided into SyncE and 1588v2 sections so that one or the other can be configured separately if the particular deployment scenario only requires or permits one technology to be deployed.

Hybrid Model Configuration with a Third-Party Grandmaster Clock SourceThis section shows the end-to-end configurations to implement the hybrid clocking model in the Cisco EPN System architecture by using a Symmetricom TimeProvider 5000 to provide frequency synchronization as well as the grandmaster clock source for 1588v2 PTP. See Figure 6-1


Chapter 6 Functional Components Implementation Synchronization Distribution Implementation

Figure 6-1 Hybrid Model Test Topology with Symmetricom Grandmaster


• Symmetricom TimeProvider 5000 (TP5000) has two Ethernet interface connected to AGN-ASBR-K1001 and AGN-ASBR-K1002 in order to provide PTP PRC source redundancy. The PTP PRC source redundancy is marked with green and blue (green with priority 100/105, blue with priority 110/115). Both AGN-ASBR-K1001 and AGN-ASBR-K1002 have two ports configured as BC slave ports: local ports with higher local priority and inter-link ports between them as backup slave ports.

• TP5000 has two E1 output connects to AGN-ASBR-K1001 and AGN-ASBR-K1002 RSP BITS front panel ports. For redundancy under RSP switchover or node failure, it's recommended to use two IOC modules to provide a total of four E1 outputs to connect to both RSPs of AGN-ASBR nodes.

• ASBR-AGG node with 1588v2 PTP hybrid BC derives frequency from SyncE and phase from 1588. Green PTP source has higher priority, so both AGN-ASBR nodes synchronize with the green PRC source. The recovered PTP clock is regenerated from AGN-ASBR WAN interface PTP master port and provides 1588 clock to the PANs.

• The PAN with the 1588v2 PTP hybrid BC derives frequency from SyncE and phase from 1588. Each PAN, based on its metric, will choose the closest AGN-ASBR as its primary PRC source and the other as its backup.

• Access CSG nodes with 1588v2 PTP hybrid BC (Fiber Port) and with BMCA will pick up one of the PANs as a PRC source and provide synchronization to the downstream eNodeB from the recovered clock regenerated from the BC master port.

• NID nodes takes frequency reference from SyncE output signal of CSG devices.

GPS Signal SourceTS3600

(Sam source withextension module)

ASR901NSN Ring

3.1.3

TE0/1/09.12.3

TE0/0/09.12.0

TE0/1/09.12.2

TE0/0/03.2.1

G0/713.25.0

G0/11

G0/7

G0/7

G/0/3/0

13.25.2

G0/6

G0/6

13.25.3

13.26.0

13.26.113.27.0

CSG-K1330

CSG-K1329

CSG-K1331

ASR903

ASR901NSN Ring

2960

18

TE0/0/0

TE0/0/0

TE0/0/0

9.12.1TE0/1/0

3.1.2

T0/0/0/13.1.1

T0/0/0/05.1.2

T0/0/0/05.1.3

3.1.4

3.1.5

T0/2/0/23.2.0

T0/2/0/13.2.2

T0/0/0/13.2.3

T0/0/0/15.2.1

T0/0/0/15.2.0

Gi0/0/1/1.4002C.10.2.8

Gi0/0/1/1.300200.10.1.8

T0/0/0/010.1.1

T0/0/0/010.1.0

PR

SD

US

PR

SP

RS

PRS

PRS

PRS

PRS

PRSDUS

PRSDUS

PRS

DUS

PRS

DUS

PRSDUS

PR

SD

US

PRSDUS

PRSDUS

PRSDUS

PRS

DU

S

PR

SD

US

G0/8

G0/0

G0/3/014.1.2

AGN-K0501

AGN-K0502

AGN-K0301

AGN-K0302

AGN-ASBR-K1002

AGN-ASBR-K1001

ASR9K

ASR9001Fiber Agg Ring

ASR9K

ASR9K OC MasterMaster: Ten0/0/0/1

ASR9K OC MasterMaster: Ten0/0/0/1

TS3600 extension modulerequired to provide clock

source redundancy

ASR903 Hybrid BCwith BMCA

Slave: Loopback1Master: Loopback2

Loopbacks 100.111.RR.PPLinks 10.RR.PP.Z /31(RR.PP of lower router name)L3VPN(CSN)- 1RR.PP.1.1 Native – 2RR.PP.1.1For Example: K1301 RR=13 PP=01

GPS Signal SourceTS3600

PAN-K0918

PAN-K0917

PAN-K1401

PAN-K1402

T0/0/0/2

CSG-K1406

Slave Devices(MWR2941-DC)

T0/2/0/13.1.0

T0/2/0/0

T0/2/0/0 ASR901 Hybrid BCSlave: Loopback1Master: Loopback2

Disable SyncE inputselection from lower ring

back into upper ring

Disable SyncE input selectionback into clock source




Note The ASBR-AGG in this configuration is a Cisco ASR 9000 Series router. The ASR 9000 Series router currently implements a port-based 1588v2 PTP implementation, which needs PTP messages exchanged from the same port for both slave and master. A future release will have loopback support for PTP on the ASR 9000 Series router.

Master Clock (TP5000)*** TP5000 sync to lab GPS ***

tp5000> show gps

GPS Information

GPS Mode- autoGPS Mask- 10GPS Antenna Delay- 240

GPS Latitude- N35:51:18.886GPS Longitude- W78:52:31.209GPS Height- 125.6----------------------------------------------------------|Index |No.|SNR|Health|Azimuth |Elevation||------|---------|---------|---------|---------|---------||1 |3 |34 |healthy |128|69||......|.........|.........|.........|.........|.........||2 |6 |33 |healthy |84|60||......|.........|.........|.........|.........|.........||3 |7 |33 |healthy |306|25||......|.........|.........|.........|.........|.........||4 |13 |35 |healthy |299|54||......|.........|.........|.........|.........|.........||5 |16 |33 |healthy |35|52||......|.........|.........|.........|.........|.........||6 |19 |31 |healthy |169|43||......|.........|.........|.........|.........|.........|----------------------------------------------------------tp5000>

*** TP5000 PTP Status per IO Controller / Interface ***

Table 6-1 Hybrid Model Connectivity

Role Device Interface VLAN-ID IP Address Note

Master PTP clock

TP5000 ioc1-1 (eth1) 300 200.10.1.9 ASBR-9006-K1002 GigabitEther-net0/0/1/1.300

- - ioc1-2 (eth2) 400 200.10.2.9 ASBR-9006-K1001 GigabitEther-net0/0/1/1.400

Master SyncE clock

TP5000 Port1 (E1 output)

- - ABR-K1001 BITS port

Backup SyncE clock

TP5000 Port2 (E1 output)

- - ABR-K1002 BITS port




tp5000> show ptp-status ioc-1|7 |23 |35 |healthy |222|65| Grandmaster status information:

Port Enabled:yes Clock Id:00:B0:AE:FF:FE:01:90:86 Profile:unicast Clock Class:locked to reference Clock Accuracy :within 100ns Timescale:PTP Num clients:5 Client load:1% Packet load:1%

tp5000> show ptp-status ioc-2

Grandmaster status information:

Port Enabled:yes Clock Id:00:B0:AE:FF:FE:01:90:87 Profile:unicast Clock Class:locked to reference Clock Accuracy :within 100ns Timescale:PTP Num clients:2 Client load:0% Packet load:0%

tp5000>

*** TP5000 VLAN configuration towards clients ***

tp5000> show vlan-config ioc1-1

IndexVLAN-IDPriStateAddressNetmaskGateway1 100 5 enable20.1255.255.255.25420.10.1.82 300 5 enable200.1255.255.255.254200.10.1.8

tp5000> show vlan-config ioc1-2

IndexVLAN-IDPriStateAddressNetmaskGateway1 200 5 enable20.1255.255.255.25420.10.2.82 400 5 enable200.1255.255.255.254200.10.2.8

tp5000> show ptp-config common ioc1-1

PTP TimescaleAUTOPTP StateenabledPTP Max Number Clients500PTP ProfileunicastPTP ClockId00:B0:AE:FF:FE:01:90:86PTP Priority 1100PTP Priority 2105PTP Domain0PTP DSCP46PTP DSCP StateenabledPTP Sync Limit-7PTP Announce Limit-3PTP Delay Limit-7PTP Unicast NegotiationenabledPTP Unicast Lease Duration300PTP Ditherdisabled

tp5000> show ptp-config common ioc1-2




PTP TimescaleAUTOPTP StateenabledPTP Max Number Clients500PTP ProfileunicastPTP ClockId00:B0:AE:FF:FE:01:90:87PTP Priority 1110PTP Priority 2115PTP Domain0PTP DSCP46PTP DSCP StateenabledPTP Sync Limit-7PTP Announce Limit-3PTP Delay Limit-7PTP Unicast NegotiationenabledPTP Unicast Lease Duration300PTP Ditherdisabled

tp5000>

AGN-ASBR-K1001 Connections to PTP Grandmaster Clock TP5000

This provides a BMCA source for downstream devices.

!*** Interface To TP5000 Eth IOC-1 on AGN-ASBR-K1001***!interface GigabitEthernet0/0/1/1.400 ptp profile AGN-ASBR-BC-Slave port state slave-only sync frequency 64 delay-request frequency 64 ! ipv4 address 200.10.2.8 255.255.255.254 encapsulation dot1q 400!!router isis core net 49.0100.1001.1101.0001.00 address-family ipv4 unicast ! interface GigabitEthernet0/0/1/1.400 passive point-to-point address-family ipv4 unicast metric 100 ! !!

AGN-ASBR-K1002 Connections to PTP Grandmaster Clock TP5000

This provides a BMCA source for downstream devices.

!*** Interface To TP5000 Eth IOC-1 on AGN-ASBR-K1002***!interface GigabitEthernet0/0/1/1.300 ptp profile AGN-ASBR-BC-Slave port state slave-only sync frequency 64 delay-request frequency 64 ! ipv4 address 200.10.1.8 255.255.255.254 load-interval 30




encapsulation dot1q 300 !!router isis core net 49.0100.1001.1101.0002.00 address-family ipv4 unicast ! interface GigabitEthernet0/0/1/1.300 passive point-to-point address-family ipv4 unicast metric 100 ! !!

Aggregation Node Configurations for SyncE and 1588v2 PTP Hybrid BC with BMCA

AGN-ASBR K1001 SyncE Configuration

This Cisco ASR 9000 Series router is the main SyncE source node for the network.

!*** Global Configuration ***!frequency synchronization quality itu-t option 1 log selection changes!!*** Interface Configuration ***!***BITS Clock-Interface for SyncE frequency source***!***Repeat for other RSP***clock-interface sync 0 location 0/RSP0/CPU0 port-parameters bits-input e1 non-crc-4 hdb3 ! frequency synchronization selection input priority 1 wait-to-restore 0 quality receive exact itu-t option 1 PRC !!interface TenGigE0/0/0/0 description To AGN-ASBR-K1002 T0/0/0/2 frequency synchronization selection input priority 1 !!interface TenGigE0/0/0/1 description To AGN-K0502::T0/0/0/1 frequency synchronization selection input priority 2 !!

AGN-ASBR K1001 1588 Configuration

!*** PTP profile ***ptp




clock domain 0 identity mac-address router timescale PTP ! profile AGN-ASBR-BC-Slave dscp 46 transport ipv4 port state slave-only !***Secondary Master from TP5000*** master ipv4 200.10.1.9 priority 125 ! !***Primary Master from TP5000*** master ipv4 200.10.2.9 priority 120 ! sync frequency 64 clock operation one-step delay-request frequency 64 ! profile AGN-ASBR-BC-Master dscp 46 transport ipv4 sync frequency 64 clock operation one-step announce timeout 2 delay-request frequency 64 !!!*** PTP hybrid BC slave port ***interface GigabitEthernet0/0/1/1.400 ptp profile AGN-ASBR-BC-Slave port state slave-only sync frequency 64 delay-request frequency 64 ! ipv4 address 200.10.2.8 255.255.255.254 encapsulation dot1q 400!interface TenGigE0/0/0/0 description To AGN-ASBR-K1002 T0/0/0/2 ptp profile AGN-ASBR-BC-Slave port state slave-only sync frequency 64 delay-request frequency 64 ! ipv4 address 10.10.1.0 255.255.255.254 carrier-delay up 2000 down 0 load-interval 30 frequency synchronization selection input priority 1 !!!*** PTP hybrid BC master port ***interface TenGigE0/0/0/1 description To AGN-K0502::T0/0/0/1 ptp profile AGN-ASBR-BC-Master sync frequency 64 delay-request frequency 64




! ipv4 address 10.5.2.1 255.255.255.254 carrier-delay up 2000 down 0 load-interval 30 frequency synchronization selection input priority 2 !!*** IGP to reach PAN ***router isis agg-acc is-type level-2-only net 49.0100.1001.1101.0001.00 address-family ipv4 unicast ! interface TenGigE0/0/0/1 circuit-type level-2-only bfd minimum-interval 15 bfd multiplier 3 bfd fast-detect ipv4 point-to-point link-down fast-detect address-family ipv4 unicast mpls ldp sync ! !!


This Cisco ASR 9000 Series router is configured as the backup SyncE source with SSM QL-SSU-A override enabled.

!*** Global Config ***frequency synchronization quality itu-t option 1 log selection changes

!*** Interface Config ***!***BITS Clock-Interface for SyncE frequency source***!***Repeat for other RSP***clock-interface sync 0 location 0/RSP0/CPU0 port-parameters bits-input e1 non-crc-4 hdb3 ! frequency synchronization selection input priority 2 wait-to-restore 0 quality receive exact itu-t option 1 SSU-A !!interface TenGigE0/0/0/0 description To ASBR-K1001 T0/0/0/0 frequency synchronization selection input priority 1 !!interface TenGigE0/0/0/1 description To AGN-K0302::T0/0/0/1 frequency synchronization selection input




priority 2 ! ipv4 address 10.3.2.3 255.255.255.254!

AGN-ASBR K1002 1588 Configuration

!*** PTP profile ***ptp clock domain 0 identity mac-address router timescale PTP ! profile AGN-ASBR-BC-Slave dscp 46 transport ipv4 port state slave-only !***Primary Master from TP5000*** master ipv4 200.10.1.9 priority 120 ! !***Secondary Master from TP5000*** master ipv4 200.10.2.9 priority 125 ! sync frequency 64 clock operation one-step delay-request frequency 64 ! profile AGN-ASBR-BC-Master dscp 46 transport ipv4 sync frequency 64 clock operation one-step announce timeout 2 delay-request frequency 64 !!!*** PTP hybrid BC slave port ***interface GigabitEthernet0/0/1/1.300 ptp profile AGN-ASBR-BC-Slave port state slave-only sync frequency 64 delay-request frequency 64 ! ipv4 address 200.10.1.8 255.255.255.254 load-interval 30 encapsulation dot1q 300 !

!*** PTP hybrid BC master port ***interface TenGigE0/0/0/1 description To AGN-K0302::T0/0/0/1 ptp profile AGN-ASBR-BC-Master sync frequency 64 delay-request frequency 64 !




Cisco ASR 903 Series PAN Configuration for SyncE and 1588v2 PTP Hybrid BC with BMCA and Asymmetry Correction

The Cisco ASR 903 Series interface default SyncE state is master. In order to pass SyncE SSM messages from the aggregation ring into the access ring, the network-clock input-source selection from the Cisco ASR 903 Series side must be disabled. If this is not done, interlink issues between the Cisco ASR 903 Series and ASR 901 Series devices will occur.

Note This limitation doesn't apply to links between the Cisco ASR 903 Series and ASR 9000 Series devices.

Asymmetry result observed in PTP session between Master and Slave is compensated by asymmetry correction, which has been configured by using CLI command "Asymmetry offset cli set to xxxx" under PTP global command.

PAN-K1401 SyncE Configuration

!*** Global Config ***network-clock revertivenetwork-clock synchronization automaticnetwork-clock synchronization mode QL-enablednetwork-clock input-source 1 interface TenGigabitEthernet0/1/0network-clock input-source 2 interface TenGigabitEthernet0/0/0network-clock output-source system 1 External R0 e1 cas 120ohms linecode hdb3esmc process!*** Interface Config ***interface TenGigabitEthernet0/0/0 description To PAN-903-K1402::TenGigabitEthernet0/0/0 synchronous mode!interface TenGigabitEthernet0/1/0 description To AGN-K0301::T0/0/0/2 synchronous mode!interface GigabitEthernet0/3/0 description to ASR 901 access ring synchronous mode

PAN-K1401 1588v2 Configuration

*** PTP configuration ***!interface Loopback1 description PTP BC Slave to PRC transport intf ip address 100.100.14.1 255.255.255.255!interface Loopback2 description PTP BC master to CSG transport intf ip address 100.101.14.1 255.255.255.255!ptp clock boundary domain 0 hybrid!***Asymmetry offset cli set to 2000 *** output 1pps R0 offset 2000clock-port BC_Slave_K1401 slave delay-req interval -6 transport ipv4 unicast interface Lo1 negotiation !***AGG-ASBR-K1002*** clock source 10.3.2.3 !***AGG-ASBR-K1001***




clock source 10.5.2.1 1 clock-port BC_Master_K1401 master sync interval -6 transport ipv4 unicast interface Lo2 negotiation

!router isis agg-acc net 49.0100.1001.1101.4001.00 ispf level-1-2 passive-interface Loopback1 passive-interface Loopback2!

PAN-K1402 SyncE Configuration

!*** Global Config ***network-clock revertivenetwork-clock synchronization automaticnetwork-clock synchronization mode QL-enablednetwork-clock input-source 1 interface TenGigabitEthernet0/0/0network-clock input-source 2 interface TenGigabitEthernet0/1/0network-clock output-source system 1 External R0 e1 cas 120ohms linecode hdb3 esmc process

!*** Interface Config ***interface TenGigabitEthernet0/0/0 description To PAN-903-K1401::TenGigabitEthernet0/0/0 synchronous mode

interface TenGigabitEthernet0/1/0 description to PAN-903-K0917::Ten0/1 synchronous mode

interface GigabitEthernet0/3/0 description To K1331::Gi0/11 synchronous mode

PAN-K1402 1588v2 Configuration

*** PTP configuration ***!interface Loopback1 description PTP BC Slave to PRC transport intf ip address 100.100.14.2 255.255.255.255!interface Loopback2 description PTP BC master to CSG transport intf ip address 100.101.14.2 255.255.255.255!ptp clock boundary domain 0 hybrid output 1pps R0 offset 2200!***Asymmetry offset cli set to 2200 *** clock-port BC_Slave_K1402 slave delay-req interval -6 transport ipv4 unicast interface Lo1 negotiation !***AGG-ASBR-K1001*** clock source 10.5.2.1 !***AGG-ASBR-K1002*** clock source 10.3.2.3 1 clock-port BC_Master_K1402 master sync interval -6 transport ipv4 unicast interface Lo2 negotiation!




router isis agg-acc net 49.0100.1001.1101.4002.00 ispf level-1-2 passive-interface Loopback1 passive-interface Loopback2!

Access Node Configuration for SyncE and 1588 Hybrid BC with Asymmetry Correction

This section provides configuration required to activate Hybrid boundary clock on CSG Devices ASR901 and ASR920. It also provides configuration details to activate SyncE between NID and CSG devices.

Cisco ASR 901 Series CSG Configuration

The Cisco ASR 901 Series interface default SyncE state is master. Note that the A901-12C-FT-D integrated chassis has 4 combination SFP and copper GigE ports. In some applications, like when interfacing with Nokia Siemens Network (NSN) microwave devices, which require a copper RJ-45 Ethernet connection, SyncE is not supported with SFP-GE-T or GLC-T copper SFP modules. In order to pass SyncE SSM messages from the aggregation ring into the access ring, Gigabit Ethernet port 0 to 3 and 5 to 7 should be used instead. Hybrid BC on Fiber Ring supported.

Before the Cisco ASR 901 Series router GigE port is added into the network-clock input-source selection pool, SyncE state slave must be entered under the physical interface level of copper port but not on the fiber port. The reason for this is because IEEE 802.3ab and later requires GigE interface on one side to use its internal oscillator in order to drive synchronization on the link and it requires the other side to use the input clock from the line in order to drive its transmit line. This forms the proper master-slave relationship on the point-to-point GigE link. Asymmetry result observed in PTP session between master and slave is compensated by asymmetry correction, which has been configured by using CLI command "Asymmetry offset cli set to xxxx" under PTP global command.

CSG-K1329 SyncE Configuration

!*** Global Config ***network-clock synchronization automaticnetwork-clock synchronization mode QL-enablednetwork-clock input-source 1 interface GigabitEthernet0/7network-clock input-source 2 interface GigabitEthernet0/6network-clock output-source system 1 External 0/0/0 2048k network-clock revertive esmc process

!*** Interface Config ***interface GigabitEthernet0/7 description To PAN-ABR-903-K1401::GigabitEthernet0/3/0 synchronous mode synce state slave!*** Synce slave configured, only applicable for RJ45***

interface GigabitEthernet0/6 description To CSG-901-K1330::GigabitEthernet0/6(syncE support) synchronous mode synce state slave!*** Synce slave configured, only applicable for RJ45***

CSG-K1329 1588v2 Configuration

ptp clock boundary domain 0 hybrid




!*** Asymmetry offset correction *** 1pps-out 450 4096 ustime-properties gps timeScaleTRUE currentUtcOffsetValidTRUE leap59FALSE leap61FALSE 35 clock-port BC_Slave_K1329 slave transport ipv4 unicast interface Lo1 negotiation clock source 100.101.14.1!***PAN-K1401*** clock source 100.101.14.2 1!***PAN-K1402*** clock-port BC_Master_K1329 master transport ipv4 unicast interface Lo2 negotiation

CSG-K1330 1588v2 Configuration

!*** Global Config ***network-clock synchronization automaticnetwork-clock synchronization mode QL-enablednetwork-clock input-source 1 interface GigabitEthernet0/7network-clock input-source 2 interface GigabitEthernet0/6network-clock output-source system 1 External 0/0/0 2048k network-clock revertive esmc process!*** Interface Config ***interface GigabitEthernet0/7 description To CSG-901-K1329::GigabitEthernet0/6 synchronous mode synce state slave!*** Synce slave configured, only applicable for RJ45***

interface GigabitEthernet0/6 description To CSG-901-K1331::GigabitEthernet0/6(syncE support) synchronous mode synce state slave!*** Synce slave configured, only applicable for RJ45****** PTP Configuration ***!interface Loopback1 description PTP BC Slave to PAN transport intf ip address 100.100.13.30 255.255.255.255!interface Loopback2 description PTP BC master to eNB transport intf ip address 100.101.13.30 255.255.255.255!ptp clock boundary domain 0 hybrid 1pps-out 250 4096 us!*** Asymmetry offset cli set to 250ns *** time-properties gps timeScaleTRUE currentUtcOffsetValidTRUE leap59FALSE leap61FALSE 35 clock-port BC_Slave_K1330 slave transport ipv4 unicast interface Lo1 negotiation clock source 100.101.14.1!***PAN-K1401*** clock source 100.101.14.2 1!***PAN-K1402*** clock-port BC_Master_K1330 master transport ipv4 unicast interface Lo2 negotiation!router isis agg-acc advertise passive-only passive-interface Loopback1 passive-interface Loopback2!




!interface Vlan400 description To K1407 1588 PTPv2 client ip address 114.10.7.2 255.255.255.0 load-interval 30!interface GigabitEthernet0/6 description To K1406 1588 PTPv2 client service instance 400 ethernet encapsulation untagged bridge-domain 400 !

Cisco ASR 920 Series CSG Configuration

CSG-K0206 SyncE Configuration!*** Global Config ***network-clock synchronization automaticnetwork-clock synchronization mode QL-enablednetwork-clock input-source 1 interface GigabitEthernet0/7network-clock input-source 2 interface GigabitEthernet0/6network-clock revertive esmc process

!*** Interface Config ***interface TenGigabitEthernet0/0/3 description To PAN-ABR-903-K1401::GigabitEthernet Gi0/2/1 synchronous modeservice instance 62 ethernet encapsulation untagged bridge-domain 62

interface TenGigabitEthernet0/0/2 description To CSG-920-K0207:: TenGigabitEthernet0/0/3 synchronous modeservice instance 62 ethernet encapsulation untagged bridge-domain 62

interface TenGigabitEthernet0/0/2 description To NID-ME1200-K0304::GigabitEthernet Gi1/2 synchronous modeservice instance 62 ethernet encapsulation untagged bridge-domain 62

CSG-K0206 1588v2 Configuration!*** Interface Config ***interface Loopback1 description PTP BC Slave to PAN transport intf ip address 100.100.2.8 255.255.255.255!interface Loopback2 description PTP BC master to eNB transport intf ip address 100.101.2.8 255.255.255.255

!*** Advertise loopback in isis Config ***router isis agg-acc advertise passive-only passive-interface Loopback1 passive-interface Loopback2




!*** Global Config ***

ptp clock boundary domain 0 hybrid!*** Asymmetry offset correction ***output 1pps r0 offset 100clock-port BC_Slave_K0206 slave transport ipv4 unicast interface Lo1 negotiation clock source 100.101.14.1!***PAN-K1401*** clock source 100.101.14.2 1!***PAN-K1402*** clock-port BC_Master_K0206 master transport ipv4 unicast interface Lo2 negotiation

Cisco ME1200 NID Configuration using NID Controller

This section provides configuration detail required to enable SyncE on NID device from PAN Nodes. By default, no SyncE is enabled on a NID device. Configuration of SyncE on NID starts with provisioning of default clock Parameter on Global and port level. Global level clock setting configures Auto revertive mode wait to restore is 5sec. The Port level clock setting configures port with SyncE Master and SSM is disabled. ME 3600 is used as NID controller.

Note The following NID-related configurations are entered from the ME3600 controller.

!***Apply Global clock default configuration***

controller nid 1/1SyncEsetSyncEclockDefaultConfig set_synce_clock_config_defaults_reqsetSyncEclockDefaultConfig reviewsetSyncEclockDefaultConfig commit

!***Apply Port clock default configuration***

setSyncEclockDefaultConfig set_synce_clock_config_defaults_req setSyncEclockDefaultConfig commit

!***Enable SSM and SyncE Slave on port 2 towards CSG ***

setSyncEclockConfig clock_sel_config ssm_enable_ports GigabitEthernet_2_UNI enablesetSyncEclockConfig clock_sel_config source_configs 0 port 2 setSyncEclockConfig clock_sel_config source_configs 0 aneg_mode prefer_slave

!***Enable SSM and SyncE Master on port 5 towards eNodeB ***

setSyncEclockConfig clock_sel_config ssm_enable_ports GigabitEthernet_5_UNI enablesetSyncEclockConfig clock_sel_config source_configs 1 port 5 setSyncEclockConfig clock_sel_config source_configs 1 aneg_mode prefer_master

Simulated IP-NodeB 1588v2 Client (CSG-2941-K1407 is shown in this example)

!*** PTP Configuration ***!interface Vlan400 description 1588v2 client ptp announce interval 3 ptp announce timeout 2 ptp sync interval -4 ptp slave unicast negotiation




ptp clock-source 100.101.13.30 ptp enable!interface GigabitEthernet0/0 description 1588v2 client (K1330) switchport access vlan 400!network-clock-select hold-timeout infinitenetwork-clock-select mode nonrevertnetwork-clock-select 1 PACKET-TIMINGptp mode ordinaryptp priority1 128ptp priority2 128ptp domain 0!ip route 100.101.13.0 255.255.255.0 114.10.7.2!

Hybrid Model Configuration with a Cisco ASR 9000 Series Router as Grandmaster Clock Source

This section shows the end-to-end configurations to implement the hybrid clocking model in the Cisco EPN System architecture by using a Symmetricom TimeSource 3600 (TS3600) to provide frequency, phase and ToD inputs into the Cisco ASR 9000 Series router. The ASR 9000 Series router acts as the grandmaster clock source for 1588v2 PTP.

In order for the Cisco ASR 9000 Series router to perform as a grandmaster clock, an external timing source is required to provide a stable reference for frequency, phase, and ToD to the GPS 10 MHz, 1 PPS, and ToD interfaces on the RSP440. For Cisco EPN validation purposes, a Symmetricom TS3600 was used as the reference clock. The signal types from the external reference clock need to map to the Cisco ASR 9000 Series GPS interface configuration accordingly:

• SSM quality mode needs to be ITU-T option 2 generation 2

• ToD format is required to be "cisco"

• 1PPS format need to be "rs422"

The Cisco ASR 9000 Series router will use these three input signals as a reference source for generating SyncE and 1588v2 PTP in order to distribute synchronization information to the rest of the Cisco EPN network. SyncE is enabled across aggregation, pre-aggregation, and access networks in order to provide frequency for every hybrid BC node, which uses this to recover the slave port clock, synchronize the internal 1588 servo, and regenerate the clock for downstream clock devices. 1588v2 PTP signals will be used only to synchronize phase and time-of-day information across the network.




Figure 6-2 Hybrid Model Implementation with the Cisco ASR 9000 Series Grandmaster


• The Cisco ASR 9000 Series ordinary clock grandmaster (OC GM) receives frequency, phase, and time-of-day inputs from the primary reference clock via the GPS ports on the RSP440. In order to provide signals to both AGN-ASBRs, the Symmetricom TS3600 uses expansion modules to provide the necessary number of inputs into the Cisco ASR 9000 Series routers. Even with the cost of the expansion modules, this arrangement results in an approximate 20% cost savings over the TP5000-based model.

• AGN-ASBR-K1001 and AGN-ASBR-K1002 are configured as OC GM to provide clock source redundancy to the rest of the network. Since the GPS ports on standby RSP in a redundant Cisco ASR 9000 Series router are not live until an RSP switchover occurs, the network is designed to failover to the other Cisco ASR 9000 Series router in the event of an RSP failover on the primary Cisco ASR 9000 Series router.

• The PAN with the 1588v2 PTP hybrid BC derives frequency from SyncE and phase from 1588. Each PAN, based on its metric, will choose the closest ASBR-AGG as its primary PRC source and the other as its backup.

• Access CSG nodes with 1588 BC only and without BMCA will pick up one of the PANs as a PRC source and provide clock synchronization for the downstream eNodeB from the recovered clock regenerated from the BC master port. Full hybrid BC functionality on the Cisco ASR 901 Series router will be covered in an update.

Note The ASBR-AGG in this configuration is a Cisco ASR 9000 Series router. The Cisco ASR 9000 Series router currently implements a port-based 1588v2 PTP implementation, which needs PTP messages exchanged from the same port for both slave and master. A future release will have loopback support for PTP on the Cisco ASR 9000 Series router.




AGN-ASBR Connections to TS3600

The AGN-ASBR Cisco ASR 9000 Series router receives 3 signals from the TS3600: 10Mhz, 1PPS, and ToD, and it acts as an OC GM for the rest of the network, providing SyncE and PTP. The configuration is identical on both AGN-ASBR-K1001 and -K1002.

!*** Interface To TP5000 Eth IOC-1 on AGN-ASBR-K1001***!*** Global Configuration ***frequency synchronization quality itu-t option 2 generation 2 log selection changes!!*** GPS interface Configuration ***clock-interface sync 2 location 0/RSP0/CPU0 port-parameters gps-input tod-format cisco pps-input rs422 ! frequency synchronization selection input priority 1 wait-to-restore 0 ssm disable time-of-day-priority 1 quality receive exact itu-t option 2 generation 2 PRS !!

Aggregation Node Configurations for SyncE and 1588v2 PTP Hybrid BC with BMCA


This Cisco ASR 9000 Series router is the main SyncE source node for the network.

!*** Global Configuration ***!frequency synchronization quality itu-t option 2 generation 2 log selection changes!!*** Interface Configuration ***!***GPS Clock-Interface for SyncE frequency source***!***Repeat for other RSP***clock-interface sync 2 location 0/RSP0/CPU0 port-parameters gps-input tod-format cisco pps-input rs422 ! frequency synchronization selection input priority 1 wait-to-restore 0 ssm disable time-of-day-priority 1 quality receive exact itu-t option 2 generation 2 PRS !

Table 6-2 Cisco ASR 9000 Series Router OC GM Connectivity

Role Device 10 MHz 1PPS ToD Note

GPS source TS3600 10 MHz RS422 Cisco To ABR-K1001 and ABR K1002 GPS inputs




!interface TenGigE0/0/0/1 description To AGN-K0502::T0/0/0/1 frequency synchronization selection input priority 2 !!

AGN-ASBR K1001 1588 Configuration!*** Global Configuration ***!frequency synchronization quality itu-t option 2 generation 2 log selection changes!!*** Interface Configuration ***!***GPS Clock-Interface for SyncE frequency source***!***Repeat for other RSP***clock-interface sync 2 location 0/RSP0/CPU0 port-parameters gps-input tod-format cisco pps-input rs422 ! frequency synchronization selection input priority 1 wait-to-restore 0 ssm disable time-of-day-priority 1 quality receive exact itu-t option 2 generation 2 PRS !!!*** PTP profile ***ptp clock domain 0 identity mac-address router timescale PTP time-source GPS ! profile AGN-ASBR-BC-Master dscp 46 transport ipv4 sync frequency 64 clock operation one-step announce timeout 2 delay-request frequency 64 ! frequency priority 1 time-of-day priority 1!!*** PTP hybrid BC master port ***interface TenGigE0/0/0/1 description To AGN-K0502::T0/0/0/1 ptp profile AGN-ASBR-BC-Master sync frequency 64 delay-request frequency 64 ! ipv4 address 10.5.2.1 255.255.255.254 carrier-delay up 2000 down 0 load-interval 30 frequency synchronization




selection input priority 3 !!*** IGP to reach PAN ***router isis agg-acc is-type level-2-only net 49.0100.1001.1101.0001.00 address-family ipv4 unicast ! interface TenGigE0/0/0/1 circuit-type level-2-only bfd minimum-interval 15 bfd multiplier 3 bfd fast-detect ipv4 point-to-point link-down fast-detect address-family ipv4 unicast mpls ldp sync ! !!


This Cisco ASR 9000 Series router is configured as the backup SyncE source with SSM QL-SSU-A override enabled.

!*** Global Config ***

frequency synchronization quality itu-t option 2 generation 2 log selection changes!!*** Interface Configuration ***!***GPS Clock-Interface for SyncE frequency source***!***Repeat for other RSP***clock-interface sync 2 location 0/RSP0/CPU0 port-parameters gps-input tod-format cisco pps-input rs422 ! frequency synchronization selection input priority 1 wait-to-restore 0 ssm disable time-of-day-priority 1 quality receive exact itu-t option 2 generation 2 PRS !!interface TenGigE0/0/0/1 description To AGN-K0302::T0/0/0/1 frequency synchronization selection input priority 2 ! ipv4 address 10.3.2.3 255.255.255.254!

AGN-ASBR K1002 1588 Configuration!*** Global Configuration ***!frequency synchronization




quality itu-t option 2 generation 2 log selection changes!!*** Interface Configuration ***!***GPS Clock-Interface for SyncE frequency source***!***Repeat for other RSP***clock-interface sync 2 location 0/RSP0/CPU0 port-parameters gps-input tod-format cisco pps-input rs422 ! frequency synchronization selection input priority 1 wait-to-restore 0 ssm disable time-of-day-priority 1 quality receive exact itu-t option 2 generation 2 PRS !!!*** PTP profile ***ptp clock domain 0 identity mac-address router timescale PTP time-source GPS ! profile AGN-ASBR-BC-Master dscp 46 transport ipv4 sync frequency 64 clock operation one-step announce timeout 2 delay-request frequency 64 ! frequency priority 2 time-of-day priority 2!!*** PTP hybrid BC master port ***interface TenGigE0/0/0/1 description To AGN-K0302::T0/0/0/1 ptp profile AGN-ASBR-BC-Master sync frequency 64 delay-request frequency 64 !

Cisco ASR 903 Series PAN and ASR 901 Series CSG Configuration

The configurations for all other nodes in the network are identical to the configurations shown in Hybrid Model Configuration with a Third-Party Grandmaster Clock Source, page 6-1. Please refer to that section for the configuration of these devices.

Asymmetry Correction: Estimating Expected Time Error

As described in Time Asymmetry Correction between Boundary Clocks, page 4-3, when a PTP session is established between two remote PTP Nodes, a time error asymmetry may develop on the path between them that varies depending on the type and number of intermediate nodes. The accumulated asymmetry value of each segments on the PTP Path will be used for asymmetry correction. Such correction is done




at the PTP Slave using the "1pps-out <offset>" command, where <offset> is the expected Asymmetry value. If the PTP slave has multiple paths to the master, the offset is calculated as average of the expected asymmetry value of primary and secondary paths.

Table 6-3 provides the asymmetry value that is expected between any two nodes on a link. and helps in estimating the expected total asymmetry value on a path between two PTP peers. The rows and columns represent the left-most and right-most node on the segment for which we are estimating asymmetry.

In Figure 6-3, we have shown expected asymmetry values for ASR901 with Fiber and uwave link (uwave segment provides more asymmetry value than Fiber segment).

The following example calculates the expected asymmetry correction value for Node ASR901_1, which has two redundant paths to the same TP5K PTP Master clock.

The ASR901-1 node is four hops away from the PTP Master on the primary path, and five hops away on the Secondary Path. Paths are made of different combinations of Cisco ASR9000 and ASR903 nodes. The time errors calculated for the primary and secondary path are -1675 and -2475, respectively.

To address such asymmetry, the slave clock should apply a corrective OFFSET calculated as the mean value between the time errors on the two paths.

Figure 6-3 Sample Asymmetric Value Calculation

Asymmetry value on ASR901 = ( (-2475) + (-1675)) / 2 = (-2075)ns

Table 6-3 Expected Asymmetry Values

ASR9K ASR903 ASR901 ASR901uw ASR920

ASR9K -800 -350 N/A N/A N/A

ASR903 -350 -150 -350 -800 -100

ASR901 N/A -350 -250 N/A N/A

ASR901uw N/A -800 N/A -800 N/A

ASR920 N/A -100 N/A N/A -100

2959

46

-25ns -800ns -350ns -350ns-150ns

BC Master BC BC BCSlave SlaveMaster

BC

TP5K

Total Asymmetry expected = (-25) + (-800) + (-350) + (-150) + (-350) = -1675

Primary Path

ASR9K1

-25ns -800ns -800ns -350ns-150ns

BC Master BC BC BCSlave SlaveMaster

BC

TP5K

Total Asymmetry expected = (-25) + (-800) + (-800) + (-350) + (-150) + (-350) = -2475

So OFFSET value on ASR901_1 = (Primary Path + Secondary Path)/2

Secondary Path

ASR9K1 -800ns

BC

ASR9K2

ASR9031

ASR9K2

ASR9K3

ASR9031

ASR9032

ASR9031

ASR9032

ASR9031



Chapter 6 Functional Components Implementation High Availability Implementation

High Availability ImplementationAs highlighted in the "Redundancy and High Availability" section in the EPN 4.0 Transport Infrastructure Design and Implementation Guide, the Cisco EPN System architecture implements high availability at the transport network level and the service level. By utilizing these various technologies throughout the network, the Cisco EPN design is capable of meeting the stringent Next-Generation Mobile Network (NGMN) requirements of 200 ms recovery times for LTE realtime services.

Implementation of high availability technologies at the transport layer that are common to all services is covered in the EPN 4.0 Transport Infrastructure Design and Implementation Guide. Implementation of the high availability technologies at the service level is covered in this section. Synchronization resiliency implementation is covered in Synchronization Distribution, page 4-1.

For MPLS VPN services, BGP Edge protection and BGP FRR Edge protection mechanisms are supported, and VRRP is enabled on the MTGs for redundant connectivity to the MPC. For ATM and TDM pseudowire-based services, pseudowire redundancy is supported, and MR-APS is enabled for redundant connectivity to the BSC or RNC. This section covers the implementation of the relevant protection mechanisms for each service.

MPLS VPN-BGP FRR Edge Protection and VRRPBGP FRR provides deterministic network reconvergence, even with the BGP prefix scale encountered in the Cisco EPN System design. BGP FRR edge functionality is supported at the MPLS VPN service level as well as the transport network level. The following example illustrates how to configure BGP FRR edge from the MTG to the CSG across the reference topology.

Note The VRF configuration under the BGP process uses a unique route distinguisher (RD) per MTG. This unique RD configuration in each MTG, combined with the BGP and VRRP timer adjustments in MTG 1, enables the ability for the rest of the transport infrastructure to optimize MPLS VPN protection via BGP FRR. This RD does not have to match the route target defined for the MPLS VPN VRF. The need for this unique RD will be eliminated once support for BGP additional-paths receive is implemented for BGP VPNv4 address-family configuration in IOS, thus allowing for multiple MTG information to be propagated for the MPLS VPN.

Mobile Transport Gateway 1

VRF Configuration

vrf LTE102 address-family ipv4 unicast import route-target 10:10 ! export route-target 1001:1001 ! ! address-family ipv6 unicast import route-target 10:10 ! export route-target 1001:1001







! !!interface TenGigE0/0/0/2.1100 vrf LTE102 ipv4 address 115.1.102.3 255.255.255.0 ipv6 address 2001:115:1:102::3/64 encapsulation dot1q 1100!

VRRP Configuration

router vrrp interface TenGigE0/0/0/2.1100 delay minimum 1 reload 240 address-family ipv4 vrrp 110 priority 254 timer msec 100 force address 115.1.102.1 ! ! address-family ipv6 vrrp 110 priority 254 timer msec 100 force address global 2001:115:1:102::1 address linklocal autoconfig ! !

BGP FRR Edge Configuration

router bgp 1000 address-family vpnv4 unicast additional-paths receive additional-paths send additional-paths selection route-policy add-path-to-ibgp ! ! vrf LTE102 rd 1001:1001 address-family ipv4 unicast redistribute connected ! address-family ipv6 unicast redistribute connected ! !route-policy add-path-to-ibgp set path-selection backup 1 installend-policy

Mobile Transport Gateway 2

VRF Configuration

vrf LTE102 address-family ipv4 unicast import route-target 10:10




! export route-target 1001:1001 ! ! address-family ipv6 unicast import route-target 10:10 ! export route-target 1001:1001 ! !!interface TenGigE0/0/0/2.1100 vrf LTE102 ipv4 address 115.1.102.4 255.255.255.0 ipv6 address 2001:115:1:102::4/64 encapsulation dot1q 1100!

VRRP Configuration

router vrrp interface TenGigE0/0/0/2.1100 delay minimum 1 reload 240 address-family ipv4 vrrp 110 priority 253 timer msec 100 force address 115.1.102.1 ! ! address-family ipv6 vrrp 110 priority 253 timer msec 100 force address global 2001:115:1:102::1 address linklocal autoconfig ! ! !!

BGP FRR Edge Configuration

router bgp 1000 address-family vpnv4 unicast additional-paths receive additional-paths send additional-paths selection route-policy add-path-to-ibgp ! ! vrf LTE102 rd 1001:1002 address-family ipv4 unicast redistribute connected ! address-family ipv6 unicast redistribute connected ! !route-policy add-path-to-ibgp




set path-selection backup 1 installend-policy

Core Route Reflector

router bgp 1000 address-family vpnv4 unicast additional-paths receive additional-paths send additional-paths selection route-policy add-path-to-ibgp ! address-family vpnv6 unicast !!route-policy add-path-to-ibgp set path-selection backup 1 advertise installend-policy

Cell Site Gateway

router bgp 101 address-family vpnv4 bgp additional-paths install !***Enable CEF recursion for BGP host routes*** bgp recursion host ! address-family vpnv6 !***Enable CEF recursion for BGP host routes*** bgp recursion host

G.8032 and VRRP for Ethernet Access

G.8032 Ethernet Ring Protection Switching

Ethernet Ring Protection Switching, or ERPS, is an effort at ITU-T under G.8032 Recommendation to provide sub-50ms protection and recovery switching for Ethernet traffic in a ring topology and at the same time ensuring that no loops form at the Ethernet layer. Figure 6-4 and the example following explain how to configure G.8032 across the reference topology

Figure 6-4 VRRP and G8032 HA Implementation for LTE Backhaul

2959

56

ASR9KK0301

ASR 901Ring

K0308

ME3400K0901

X2

Ten

0/0

Ten

0/0

Ten 0/1

Ten 0/1

Ten 0/0

IXIA10/11

IXIA10/1

Gig

0/7

Gig

0/2

Gig

0/5 K0306

K0307

K0305 ASR9KK0302

MPLS VPN v4 or v6

VRF

VRF

BV

IS

ub I/

FB

VI

SGW

K1502

K1502

MME

MTG 3

MTG 4

VRRPVRRP Routed PW

VRFVRF

VRFVRF

SGW

Sub

I/F

S1-U

S1-C

S1-C

S1-U

Ten

0/2

/1/3

Ten

0/2

/1/3




AGN-9006-K0302 ethernet ring g8032 profile ring_profile timer wtr 10 timer guard 100 timer hold-off 0!l2vpn ethernet ring g8032 EPNRING port0 interface TenGigE0/2/1/3 ! port1 none open-ring instance 1 profile ring_profile inclusion-list vlan-ids 99,300-350,604 aps-channel port0 interface TenGigE0/2/1/3.99 port1 none ! ! instance 2 profile ring_profile rpl port0 owner inclusion-list vlan-ids 199,351-400,651 aps-channel port0 interface TenGigE0/2/1/3.199 port1 none !

CSG-K0305/CSG-K0306/CSG-K0307/CSG-k0308ethernet ring g8032 profile ring_profile timer wtr 10 timer guard 100!ethernet ring g8032 CERING open-ring port0 interface TenGigabitEthernet0/0 port1 interface TenGigabitEthernet0/1 instance 1 profile ring_profile inclusion-list vlan-ids 64,99,300-350,604 aps-channel port0 service instance 99 port1 service instance 99 ! ! instance 2 profile ring_profile inclusion-list vlan-ids 199,351-400,651 aps-channel port0 service instance 199 port1 service instance 199 !

VRRP

VRRP is used as a gateway redundancy protocol on the AGN nodes for node failure scenarios at AGN. The routed PW is implemented for communication of VRRP messages between the BVI interfaces of AGNs. The implementation of VRRP is as shown below.




AGN-9006-K0302

!*** VRRP configuration****router vrrp interface BVI302 address-family ipv4 vrrp 2 priority 253 preempt delay 600 address 30.2.1.1 bfd fast-detect peer ipv4 30.2.1.3 ! ! address-family ipv6 vrrp 2 priority 253 preempt delay 600 bfd fast-detect peer ipv6 2001:13:2:102::3 address global 2001:13:2:102::1 address linklocal autoconfig ! ! !!***BVI Interface configuration***interface BVI302 vrf LTE224 ipv4 address 30.2.1.2 255.255.255.0 ipv6 nd dad attempts 0 ipv6 address 2001:13:2:102::2/64!

AGN-9006-K0301

!*** VRRP configuration****router vrrp interface BVI302 address-family ipv4 vrrp 2 priority 253 preempt delay 600 address 30.2.1.2 bfd fast-detect peer ipv4 30.2.1.3 ! ! address-family ipv6 vrrp 2 priority 252 preempt delay 600 bfd fast-detect peer ipv6 2001:13:2:102::3 address global 2001:13:2:102::1 address linklocal autoconfig ! ! !!***BVI Interface configuration***interface BVI302 vrf LTE224 ipv4 address 30.2.1.1 255.255.255.0 ipv6 nd dad attempts 0 ipv6 address 2001:13:2:102::1/64!




Pseudowire Redundancy for ATM and TDM ServicesHigh availability for ATM and TDM services transported via CEoP PWs is achieved through PW redundancy over the transport network, where a backup PW pointing to an alternate MTG is configured for each primary PW. Corresponding to these redundant PW is MR-APS functionality on the ATM/TDM side of the MTGs, which provides redundant ATM/TDM connectivity to the MPC equipment. Configuration of both technologies is illustrated below.

TDM Services

Figure 6-5 and the example that follows illustrate TDM services.

Figure 6-5 CESoPSN/SAToP Service Implementation for 2G and 3G Backhaul

Cisco ASR 901 Series Cell Site Gateway Configuration

Note The only difference between CESoPSN and SAToP configuration is the lack of "control-word" in the pseudowire-class for SAToP configs.

pseudowire-class CESoPSN encapsulation mpls control-word!!interface CEM0/0 cem 0 xconnect 100.111.15.1 13261501 encapsulation mpls pw-class CESoPSN backup peer 100.111.15.2 13261502 pw-class CESoPSN!interface Loopback0 ip address 100.111.13.26 255.255.255.255!

Cisco ASR 9000 Series Mobile Transport Gateway 1 Configurationaps group 1 timers 10 15 channel 0 remote 100.111.15.2 channel 1 local SONET0/2/1/0!!controller SONET0/2/1/0 description To BSC ais-shut report lais report lrdi sts 1

PANBTS

2934

53

MR-APS BSC

TDM

MTG

MTG



PG

P




mode vt15-t1 delay trigger 250 ! clock source line!controller T1 0/2/1/0/1/2/2 cem-group framed 0 timeslots 1-24 forward-alarm AIS forward-alarm RAI!interface Loopback0 description Global Loopback ipv4 address 100.111.15.1 255.255.255.255!l2vpn! pw-class CESoPSN encapsulation mpls control-word ! ! xconnect group TDM-K1326 p2p T1-CESoPSN-01 interface CEM0/2/1/0/1/2/2:0 neighbor ipv4 100.111.13.26 pw-id 13261501 pw-class CESoPSN ! ! !!

Cisco ASR 9000 Series Mobile Transport Gateway 2 Configurationaps group 1 revert 8 timers 10 15 channel 0 local SONET0/2/1/0 channel 1 remote 100.111.15.1 signaling sonet!!controller SONET0/2/1/0 description To ONS15454-K1410 OC3 port 4/1 ais-shut report lais report lrdi sts 1 mode vt15-t1 delay trigger 250 ! clock source line!controller T1 0/2/1/0/1/1/3 cem-group framed 0 timeslots 1-24 forward-alarm AIS forward-alarm RAI!interface Loopback0 description Global Loopback ipv4 address 100.111.15.2 255.255.255.255!l2vpn! pw-class CESoPSN




encapsulation mpls control-word ! ! xconnect group TDM-K1326 p2p T1-CESoPSN-01 interface CEM0/2/1/0/1/2/2:0 neighbor ipv4 100.111.13.26 pw-id 13261502 pw-class CESoPSN ! ! !!

ATM Services

Figure 6-6 and the example that follows illustrate ATM PW redundancy for a clear-channel implementation. The same configurations would be used for an IMA implementation as well, just using ATM IMA interfaces instead.


Cisco ASR 903 Series Pre-Aggregation Node Configuration

interface ATM0/5/2.100 point-to-point pvc 100/4011 l2transport encapsulation aal0 xconnect 100.111.15.1 1401150115 encapsulation mpls backup peer 100.111.15.2 1401150215 !!interface Loopback0 ip address 100.111.14.1 255.255.255.255!!

Cisco ASR 9000 Series Mobile Transport Gateway 1 Configurationaps group 2 channel 0 remote 100.111.15.2 channel 1 local SONET0/2/3/0!controller SONET0/2/3/0 path delay trigger 250!interface ATM0/2/3/0.100 l2transport pvc 100/4011 encapsulation aal0 !!interface Loopback0

2934

54

MR-APS ATM RNC

ATM

MTG

MTG

PANNode B


Primary ATM Pseudowire

PG

P




description Global Loopback ipv4 address 100.111.15.1 255.255.255.255!l2vpn pw-class ATM encapsulation mpls ! ! xconnect group ATM-K1401 p2p T1-ATM-01 interface ATM0/2/3/0.100 neighbor 100.111.14.1 pw-id 1401150115 pw-class ATM ! ! !

Cisco ASR 9000 Series Mobile Transport Gateway 2 Configurationaps group 2 revert 8 channel 0 local SONET0/2/3/0 channel 1 remote 100.111.15.1 signaling sonet!controller SONET0/2/3/0 path delay trigger 250 !!interface Loopback0 description Global Loopback ipv4 address 100.111.15.2 255.255.255.255!l2vpn pw-class ATM encapsulation mpls ! ! xconnect group ATM-K1401 p2p T1-ATM-01 interface ATM0/2/3/0.100 neighbor 100.111.14.1 pw-id 1401150215 pw-class ATM ! ! !




C H A P T E R 7
Quality of Service Implementation
This chapter, which discusses the QoS implementation for mobile services, includes the following major topic:

• CSG QoS Configuration, page 7-2

The aggregate QoS policies implemented on the NNIs of the transport network are covered in the QoS chapter of the EPN 4.0 Transport Infrastructure Design and Implementation Guide. This chapter covers the QoS policies implemented on the UNIs of the CSG and MTG, covering the service-level QoS for TDM, ATM, and L3VPN services.

Note IEEE 1588v2 PTP traffic is marked with a DSCP of 46 by the grandmaster and thus receives EF PHB treatment across the transport network.

QoS policy enforcement is accomplished with H-QoS policies with parent shaping and child queuing on the UNIs. The classification criteria used to implement the DiffServ PHBs is described in detail in the EPN 4.0 Transport Infrastructure Design and Implementation Guide and summarized in Figure 7-1.

Figure 7-1 DiffServ QoS Domain

2932

41

Traffic Class PHB

Unified MPLSTransport

Service EdgeFixed/Mobile Access

Ethernet/TDM/ATM UNI

Core, Aggregation,Access

BusinessPWHE

Res/BusEthernet

M R, B, M M, B

DSCP EXP DSCP EXP 802.1P DSCP 802.1P ATM

Network Management AF 56 7 56 7 7 56 (7) VBR-nrt

Network Control Protocols AF 48 6 48 6 6 48 (6) VBR-nrt

Residential Voice

Business Real-time

Network Sync (1588 PTP)

Mobility & Signaling traffic

Mobile Conversation/Streaming

EF 46 5 46 5 5 46 5 CBR

Residential TV and Video Distribution AF 32 4 32 4 4 NA 4 NA

Business Telepresence AF 24 3 24 3 3 NA 3 NA

Business Critical

In Contract

Out of Contract

AF16

8

2

1

16

8

2

1

2

1

16

8

2

1VBR-nrt

Residential HSI

Business Best Effort

Mobile Background

VQE Fast Channel Change, Repair

BE 0 0 0 0 0 0 0 UBR




Chapter 7 Quality of Service Implementation CSG QoS Configuration

Figure 7-2 QoS Enforcement Points

Figure 7-2 shows the following elements, which are covered in this section:

• (a) H-QoS policy map on CSG UNIs

• (b) H-QoS policy map on pre-aggregation NNI connecting the microwave access network

• (4) Flat QoS policy map on ingress for ATM and TDM UNIs

• (E) H-QoS policy map on CSG UNIs for G8032 access Network

• (F) H-QoS policy map on pre-aggregation NNI connecting the G8032 access network

Note The values of all policer rates and shaper rates in these examples are simply there to show how policers and shapers are configured. The actual values to be deployed in a production network should be modeled after the actual traffic rates that will be encountered.

CSG QoS Configuration

Class MapsIn MPLS Access, QoS classification at the UNI in the ingress direction for upstream traffic is based on IP DSCP, and the marking is done by the connected eNodeB.

!***Network management traffic***class-map match-any CMAP-NMgmt-DSCP match dscp cs7!!***Voice/Real-Time traffic***class-map match-all CMAP-RT-DSCP match dscp ef!!***Broadcast Video traffic***class-map match-any CMAP-Video-DSCP match dscp cs4

In Non-MPLS Access, QoS classification at the UNI in the ingress direction for upstream traffic is based on COS, and the marking is done by the connected eNodeB.

!***Broadcast Video traffic***class-map match-any CMAP-VIDEO-COS match cos 4!***Network management traffic***

Microwave Access

Fiber Access

2977

73

eNode B CSG Pre-Agg

G8032 Access

Aggregation ASBR Core ASBR MTG SGW/MME

400 Mbps

1 2a 3

b

3 3 3 3 3 4

F

E

1G 1G

1G

10G 10G 10/100G




class-map match-any CMAP-NMgmt-COS match cos 7!***Voice/Real-Time traffic***class-map match-any CMAP-RT-COS match cos 5

• QoS classification at the UNI in the egress direction for downstream traffic is based on QoS groups, and the QoS group mapping is done at the ingress NNI.

• QoS classification at the NNI in the egress direction is based on QoS groups, and:

– QoS group mapping for upstream traffic is done at the ingress UNI.

– QoS group mapping for traffic transiting the access ring is done at the ingress NNI.

!***Network management traffic***class-map match-any CMAP-NMgmt-GRP match qos-group 7!!***Network control traffic***class-map match-any CMAP-CTRL-GRP match qos-group 6!!***Voice/Real-Time traffic***class-map match-all CMAP-RT-GRP match qos-group 5!!***Broadcast Video traffic***class-map match-any CMAP-Video-GRP match qos-group 4

eNodeB UNI QoS Policy Map• Upstream Traffic—Flat QoS policy map with policing applied in the ingress direction.

• Downstream Traffic—H-QoS policy map with parent shaper and child queuing applied in the egress direction.

In MPLS Access

!***Interface connecting eNodeB.***interface GigabitEthernet0/2 service-policy output PMAP-eNB-UNI-P-E service-policy input PMAP-eNB-UNI-I!

policy-map PMAP-eNB-UNI-I class CMAP-RT-DSCP police cir 20000000 set qos-group 5 class CMAP-NMgmt-DSCP police 5000000 set qos-group 7 class CMAP-Video-DSCP police 100000000 set qos-group 3 class class-default police 200000000!policy-map PMAP-eNB-UNI-P-E class class-default




shape average 425000000 service-policy PMAP-eNB-UNI-C-E!policy-map PMAP-eNB-UNI-C-E class CMAP-RT-GRP priority percent 5 class CMAP-NMgmt-GRP bandwidth percent 1 class CMAP-HVideo-GRP bandwidth percent 25 class class-default

In Non-MPLS Access

!***Interface connecting eNodeB.***interface GigabitEthernet0/2 service-policy output PMAP-eNB-UNI-P-E service-policy input PMAP-eNB-UNI-I!

policy-map PMAP-eNB-UNI-I!*** Ingress policy for UNI Port *** class CMAP-RT-COS police cir 20000000 set qos-group 5 class CMAP-NMgmt-COS police 5000000 set qos-group 7 class CMAP-Video-COS police 100000000 set qos-group 3 class class-default police 200000000!policy-map PMAP-eNB-UNI-P-E!*** Egress Policy for eNB UNI Port *** class class-default shape average 425000000 service-policy PMAP-eNB-UNI-C-E!policy-map PMAP-eNB-UNI-C-E class CMAP-RT-GRP priority percent 5 class CMAP-NMgmt-GRP bandwidth percent 1 class CMAP-HVideo-GRP bandwidth percent 25 class class-default

Ingress and Egress policy configured on all NNI port which makes ring of G 8032

service instance 302 ethernet encapsulation dot1q 302!*** Vlan 302 used for EnodeB service rewrite ingress tag pop 1 symmetric bridge-domain 302!policy-map PMAP-NNI-E!*** Egress policy on NNI PORT *** class CMAP-RT-GRP priority percent 20 class CMAP-BC-GRP bandwidth percent 5




class CMAP-BC-Tele-GRP bandwidth percent 10 class CMAP-NMgmt-GRP bandwidth percent 5 class CMAP-CTRL-GRP bandwidth percent 2 class CMAP-Video-GRP bandwidth percent 20 class class-default!!policy-map PMAP-NNI-I!*** Ingress Policy on NNI Port *** class CMAP-BC-COS set qos-group 2 class CMAP-RT-COS set qos-group 5 police rate 1000000 class CMAP-BC-Tele-COS set qos-group 3!

Egress policy configured on Service Edge Node

policy-map PMAP-eNB-UNI-C-E class CMAP-RT-COS police rate 50000000 bps conform-action set cos 5 ! priority level 1 ! class CMAP-NMgmt-COS bandwidth percent 1 set cos 7 ! Class CMAP-VIDEO-COS Bandwidth percent 25 Set cos 4! class class-default ! end-policy-map!

policy-map PMAP-eNB-UNI-P-E class class-default service-policy PMAP-eNB-UNI-C-E shape average 100000000 bps ! end-policy-map

interface TenGigE0/2/1/3.302 l2transport encapsulation dot1q 302 rewrite ingress tag pop 1 symmetric service-policy output PMAP-eNB-UNI-P-E !!




TDM CEM UNI QoS Policy MapThis shows how to apply the proper EXP bits marking for a TDM CEoP service on a Cisco ASR 901 Series router. A simple service policy is applied to the CEM interface to impose the correct QoS group, which is mapped to the corresponding EXP value on egress on the NNI. No downstream QoS configuration is required.

policy-map PMAP-TDM-UNI-Iclass class-default set qos-group 5

interface CEM0/0 service-policy input PMAP-TDM-UNI-I

PAN Configuration for ATM and TDM UNIsATM CEoP UNIs were validated on the Cisco ASR 903 Series platform fulfilling the PAN role in the Cisco EPN system. This shows how to properly mark the EXP value for a particular ATM PVC on the Cisco ASR 903 Series router.

policy-map PMAP-ATM-PVC-Iclass class-default set mpls experimental imposition 5

!***ATM PVC carrying RT Traffic***interface ATM0/5/2.100 point-to-point pvc 100/4011 l2transport service-policy input PMAP-ATM-PVC-I

MTG Configuration for Ethernet, ATM and TDM UNIs

Ethernet UNI QoS Policy Maps

class-map match-all CMAP-RT-DSCP match dscp ef end-class-map!class-map match-any CMAP-NMgmt-DSCP match dscp cs7 end-class-map!class-map match-any CMAP-Video-DSCP match dscp cs3 end-class-map

policy-map PMAP-UNI-E class CMAP-RT-DSCP priority level 1 police rate 10000 kbps ! ! class CMAP-NMgmt-DSCP bandwidth 50000 kbps ! class CMAP-HVideo-DSCP bandwidth 200000 kbps !




class class-default ! end-policy-map!policy-map PMAP-UNI-I class CMAP-RT-DSCP priority level 1 police rate 10000 kbps ! set mpls experimental imposition 5 ! class CMAP-NMgmt-DSCP bandwidth 50000 kbps ! set mpls experimental imposition 7 ! class CMAP-HVideo-DSCP bandwidth 200000 kbps ! set mpls experimental imposition 3 ! class class-default ! end-policy-map!

The only ingress marking to match is the ATM CLP bit on ATM UNIs, which indicates a discard preference for marked cells within a particular ATM CoS. This can be utilized to offer a bursting capability in a particular ATM CoS.

ATM UNI QoS Policy Maps

class-map match-any CMAP-ATM-CLP0-UNI-I match atm clp 0 end-class-map!class-map match-any CMAP-ATM-CLP1-UNI-I match atm clp 1 end-class-map

Two ATM policy maps are shown. The first corresponds to an ATM VBR-rt service where cells are marked with a CLP of 1 above a certain cell rate. The second corresponds to an ATM UBR service, again where cells are marked with a CLP of 1 above a certain cell rate. The proper map is applied to an ATM PVC which corresponds to the ATM CoS carried on that PVC.

policy-map PMAP-ATM-UNI-I class CMAP-ATM-CLP0-UNI-I set mpls experimental imposition 5 ! class CMAP-ATM-CLP1-UNI-I set mpls experimental imposition 4 ! class class-default ! end-policy-map!policy-map PMAP-ATM-UNI-DATA-I class CMAP-ATM-CLP0-UNI-I set mpls experimental imposition 4 ! class CMAP-ATM-CLP1-UNI-I




set mpls experimental imposition 0 ! class class-default ! end-policy-map!interface ATM0/2/3/0 load-interval 30!interface ATM0/2/3/0.100 l2transport pvc 100/4011 service-policy input PMAP-ATM-UNI-I encapsulation aal0 shape vbr-rt 20000 14000 7000 !!interface ATM0/2/3/0.101 l2transport pvc 100/4012 service-policy input PMAP-ATM-UNI-DATA-I shape ubr 40000 !!

TDM UNI QoS Policy Map

The TDM UNI policy map simply marks all traffic on a CEM interface with an MPLS EXP of 5 to ensure that all traffic associated with the CEM interface is given EF treatment. This ensures that the emulated TDM circuit is transported with minimum packet delay variation (PDV) in order to guarantee the quality of the TDM circuit.

policy-map PMAP-TDM-UNI-I class class-default set mpls experimental imposition 5 ! end-policy-map!interface CEM0/2/1/0/1/4/4 service-policy input PMAP-TDM-UNI-I load-interval 30 l2transport !!




C H A P T E R 8
OAM Implementation
This chapter, which describes the operations, administration, and maintenance (OAM) and performance management (PM) implementation for mobile RAN service transport in the Cisco EPN System, includes the following major topics:

• Service OAM Implementation for LTE and 3G IP UMTS RAN Transport with MPLS Access, page 8-2

• Service OAM Implementation for ATM and TDM Circuit Emulation Pseudowires for 2G and 3G RAN Transport, page 8-2

• Transport OAM, page 8-3

• IP SLA Configuration, page 8-3

Figure 8-1 depicts OAM implementation for Mobile RAN service transport.

Figure 8-1 OAM Implementation for Mobile RAN Service Transport

2934

56

RNC/BSC/SAE GWCSG MTG

MPLS VRF OAM

Node B

IPSLA PM

MPLS LSP OAM

Service OAM

MPLS VCCV PW OAM

IPSLA PM (future PW PM)

Transport OAMEnd-to-end LSPWith Unified MPLS

3G ATM UMTS,2G TDM, Transport

LTE,3G IP UMTS,Transport

IPSLAProbe

IPSLAProbe

IPSLAProbe

IPSLAProbe

VRFVRF


Chapter 8 OAM Implementation Service OAM Implementation for LTE and 3G IP UMTS RAN Transport with MPLS Access

Service OAM Implementation for LTE and 3G IP UMTS RAN Transport with MPLS Access

OAM and PM functions were validated between the following pairs of devices (initiator listed first, responder listed second):

• CSG to CSG, to monitor the performance of the microwave link connecting the two CSGs in the access network ring.

• PAN to CSG, to monitor the performance of the access network.

• MTG to PAN, to monitor the performance of the aggregation and core networks.

• MTG to CSG, to monitor the performance of the core-to-access transport network.

The following OAM and PM functions are enabled on the initiator and responder in each case:

• IP ping and traceroute operations for LTE deployment with native IP/MPLS support.

• VRF-aware IP ping operations for connectivity check for LTE deployment with MPLS VPNs.

• IP SLA responder enabled on responder only for both native IP/MPLS and MPLS VPN deployments.

• IP SLA User Datagram Protocol (UDP) echo probes configured on initiator for round-trip time (RTT) measurement.

• IP SLA UDP jitter probe configured on initiator for one-way latency, packet loss, and packet delay variation measurement.

Service OAM Implementation for ATM and TDM Circuit Emulation Pseudowires for 2G and 3G RAN Transport

OAM and PM functions were validated between the following pairs of devices (initiator listed first, responder listed second):

• MTG to PAN, to monitor the performance of the aggregation and core networks for ATM and TDM PWs.

• MTG to CSG, to monitor the performance of the access, aggregation, and core networks for TDM PWs.

The following OAM and PM functions are enabled on the initiator and responder in each case:

• MPLS pseudowire ping and traceroute operations for connectivity check for ATM and TDM PWs.

• IP SLA responder enabled on responder for both ATM and TDM PW deployments. Loopback address of node is used for IP SLA measurements.

• IP SLA UDP echo probes configured on initiator for round-trip time (RTT) measurement.

• IP SLA UDP jitter probe configured on initiator for one-way latency, packet loss, and packet delay variation measurement.



Chapter 8 OAM Implementation Transport OAM

Transport OAMThe MPLS transport for mobile backhaul is based on the unified MPLS approach. The unified MPLS deployment approach allows for end-to-end LSPs to be built between the RAN access domain or PAN and the centralized MTG in the MPC:

• IP ping and traceroute operations for verifying the data plane against control plane and for isolating faults within the inter-domain LSP in the MPLS/IP network between the CSGs or PANs and the MTG.

• MPLS LSP ping and traceroute operations for verifying the data plane against control plane and for isolating faults within the intra-domain LSPs in the access, aggregation, and core domains.

• MPLS LSP ping and traceroute operations for inter-domain LSPs will be supported in a future release.

IP SLA Configuration

IP SLA Responder ConfigurationThe CSG and PAN act as IP SLA responders for different measurement scenarios. Minimal configuration is required for enabling the responder function.

ip sla responder

Cell Site Gateway Initiator Configuration for IP SLAThe CSG is configured with IP SLA probes for initiating measurement of packet loss, packet delay, and packet delay variation (for example, jitter) towards the PAN.

!***Global IP SLA Probe***ip sla 1 udp-jitter 100.111.15.1 59000 num-packets 100 request-data-size 160 tos 96 verify-data frequency 30ip sla schedule 1 life forever start-time now!!***Reaction configuration for Probe***ip sla reaction-configuration 1 react rtt threshold-value 1 1 threshold-type immediate action-type trapOnlyip sla reaction-configuration 1 react jitterDSAvg threshold-value 1 1 threshold-type immediate action-type trapOnlyip sla enable reaction-alerts

!***IP SLA Probe in VRF***ip sla 5 udp-jitter 114.1.224.1 9000 num-packets 100 request-data-size 160 tos 96 verify-data vrf LTE224 frequency 30ip sla schedule 5 life forever start-time now



Chapter 8 OAM Implementation IP SLA Configuration

Pre-Aggregation Node Initiator Configuration for IP SLAThe PAN is configured with IP SLA probes for initiating measurement of packet loss, packet delay, and packet delay variation (for example, jitter) towards the CSG. It acts as a responder only to the MTG.

ip sla 5 udp-jitter 113.30.224.1 13030 num-packets 100 request-data-size 160 tos 96 verify-data vrf LTE224 frequency 30ip sla schedule 5 life forever start-time now!ip sla logging trapsip sla enable reaction-alerts

Mobile Transport Gateway Initiator Configuration for IP SLAThe MTG is configured with IP SLA probes for initiating measurement of packet loss, packet delay, and packet delay variation (for example, jitter) towards the PAN.

Note The ToS values in IOS-XR are equal to four times the desired DSCP value. Please refer to Chapter 7, “Quality of Service Implementation,” for LTE QCI to DSCP mapping.

VPN: Jitter Probes

ipsla operation 6 type udp jitter vrf LTE102 destination address 114.1.224.1 packet count 100 !***tos 184 = DSCP 46 (EF)*** tos 184 destination port 918 frequency 30 ! ! schedule operation 6 start-time now life forever ! !***Enabled IP SLA Responder*** responder !!

Reaction Configuration

The reaction configuration defines the thresholds for the previously configured probes, and it defines the corresponding actions to be taken when those thresholds are exceeded. The following configuration shows a single example of a jitter probe reaction and an echo probe reaction.

ipsla reaction operation 6




react connection-loss action logging action trigger threshold type immediate ! react jitter-average dest-to-source action logging action trigger threshold type immediate threshold lower-limit 10 upper-limit 15 ! react jitter-average source-to-dest action logging action trigger threshold type immediate threshold lower-limit 10 upper-limit 15 ! react packet-loss dest-to-source action logging action trigger threshold type immediate threshold lower-limit 3 upper-limit 5 ! !!







A
P P E N D I X A Related Documentation
The EPN 4.0 MEF Transport Services Design and Implementation Guide is part of a set of resources that comprise the Cisco EPN System documentation suite. The resources include:

• EPN 4.0 System Concept Guide: Provides general information about Cisco's EPN 4.0 System architecture, its components, service models, and the functional considerations, with specific focus on the benefits it provides to operators.

• EPN 4.0 System Brochure: At-a-glance brochure of the Cisco Evolved Programmable Network (EPN).

• EPN 4.0 Transport Infrastructure Design and Implementation Guide: Design and Implementation guide with configurations for the transport models and cross-service functional components supported by the Cisco EPN System concept.

• EPN 4.0 Residential Services Design and Implementation Guide: Design and Implementation guide with configurations for deploying the consumer service models and the unified experience use cases supported by the Cisco EPN System concept.

• EPN 4.0 Enterprise Services Design and Implementation Guide: Design and Implementation guide with configurations for deploying the enterprise L3VPN service models over any access and the personalized use cases supported by the Cisco EPN System concept.

• EPN 4.0 MEF Transport Services Design and Implementation Guide: Design and Implementation guide with configurations for deploying the Metro Ethernet Forum service transport models and use cases supported by the Cisco EPN System concept.

Note All of the documents listed above, with the exception of the System Concept Guide and System Brochure, are considered Cisco Confidential documents. Copies of these documents may be obtained under a current Non-Disclosure Agreement with Cisco. Please contact a Cisco Sales account team representative for more information about acquiring copies of these documents.

A-1EPN 4.0 Mobile Transport

https://docs.cisco.com/share/page/site/nextgen-edcs/document-details?nodeRef=workspace://SpacesStore/8aebbef8-0b39-4a93-b85d-b6531c6d7f23

https://docs.cisco.com/share/page/site/nextgen-edcs/document-details?nodeRef=workspace://SpacesStore/8aebbef8-0b39-4a93-b85d-b6531c6d7f23

https://docs.cisco.com/share/page/site/nextgen-edcs/document-details?nodeRef=workspace://SpacesStore/42481df4-4faa-47d8-a34f-f551eb13e746

https://docs.cisco.com/share/page/site/nextgen-edcs/document-details?nodeRef=workspace://SpacesStore/704a8cb7-46c3-48ce-9a6b-ac02106efe9a


https://docs.cisco.com/share/page/site/nextgen-edcs/document-details?nodeRef=workspace://SpacesStore/35a1f12e-2099-4bf6-83f4-47cbb3e44c6d

https://docs.cisco.com/share/page/site/nextgen-edcs/document-details?nodeRef=workspace://SpacesStore/5b2c8446-94e7-432c-8dc5-850f7d0095df

https://docs.cisco.com/share/page/site/nextgen-edcs/document-details?nodeRef=workspace://SpacesStore/04c8ba12-4c5a-4726-8bb5-ac192f15d3a7

Appendix A Related Documentation

A-2EPN 4.0 Mobile Transport


Date post:	21-May-2018
Category:	Documents
Upload:	vandang
View:	216 times
Download:	1 times

EPN 4.0 Mobile Services DIG A Related Documentation A-1 Contents 4 EPN 4.0 Mobile Services Design...

Documents