Future-Net 2010

Verizon‘s Requirements for IP/MPLS-Based Carrier Ethernet Networks

Andrew G. Malis & Drew RexrodeVerizon Communications

andrew.g.malis@verizon.comcharles.a.rexrode@verizon.com

2

Introduction

• Public Ethernet services are exploding in popularity

• External Ethernet interface to the customer does not necessarily

mean ―Ethernet inside‖– The Internet Engineering Task Force (IETF) has standardized

mechanisms for providing point-to-point and multipoint public Ethernet

services over IP/MPLS-based infrastructures

• This talk discusses Verizon‘s requirements for such solutions,

including functionality, conformance to Metro Ethernet forum (MEF)

service definitions, reliability, scalability, QoS, performance

monitoring, OAM, testing, and certification

3

Why Are Ethernet Services

Popular?

• Ubiquity and low cost of Ethernet interfaces in customer equipment,

universal experience with Ethernet in LANs, and perceived simplicity

• Successful marketing of the ―Ethernet‖ brand by vendors, IEEE,

MEF, and others– Little resemblance with original DIX Ethernet specifications, from

physical layer on up (e.g., today‘s Ethernet is mostly point-to-point or

ring-based rather than CSMA-CD at the physical layer)

– Most everything has changed except for the basic frame format – and

jumbograms (large frames up to 9K bytes) change even that

• Plenty of competition and favorable pricing by service providers

4

MEF Carrier Ethernet Service

Definitions

• Three service types based on the three Ethernet Virtual Connection (EVC) types

• Two ―UNI Types‖ determine whether services are ‗private‘ or ‗virtual‘– Port-based (All to One Bundling) single EVC (transparency, but uses an entire port per service)– VLAN-based ‗N‘ EVCs per UNI (not as transparent, but multiple services per port)

• Services are defined by combination of connectivity model and ‗UNI Type‘

• Also Ethernet-based access services to Layer 3 VPNs or dedicated Internet access

Connectivity ModelPort-Based

(All to One Bundling)

VLAN-Based

(EVC identified by VLAN ID)

E-Line

(point-to-point EVC)

Ethernet Private Line

(EPL)

Ethernet Virtual Private Line

(EVPL)

E-LAN

(multipoint-to-multipoint EVC)

Ethernet Private LAN

(EP-LAN)

Ethernet Virtual Private LAN

(EVP-LAN)

E-Tree

(rooted multipoint EVC)

Ethernet Private Tree

(EP-Tree)

Ethernet Virtual Private Tree

(EVP-Tree)

5

―Enterprise-Class‖ Ethernet

Limitations

• ―Enterprise-class‖ Ethernet switching has shortcomings as a basic

for public Ethernet services– Few features for high availability in protocols or equipment

– Scaling limits on MAC addresses, VLAN IDs, and spanning tree

topology limit the size of native Ethernet networks

– Spanning tree routing may take seconds to (occasionally) minutes to re-

converge

• Early Ethernet providers found that enterprise-class Ethernet cannot

naively be deployed for reliable carrier services

6

Evolving and Scaling Ethernet

Services

• A typical ―early‖ public Ethernet service provider probably uses

Ethernet switches and Q-in-Q for customer separation

• Typical end user services are– Ethernet Private LAN (EP-LAN)

– Ethernet Virtual Private LAN (EVP-LAN)

– Ethernet Private Line (EPL)

– Ethernet Virtual Private Line (EVPL)

– Each of these services requires the use of a provider VLAN tag

• As the service becomes successful, the provider will encounter the

usual Ethernet scaling limitations– MAC address scaling

– VLAN tag scaling (4K customer limit)

– Switching capacity limits

7

Typical ―Early‖ Ethernet Service

Network

• Characterized by organic

growth driven by customer

location

• All switches are ―edge

switches‖

• May be some number of

redundant links

• 802.3ad Link Aggregation may

also be used for resiliency or

for additional BW between

switches

• Flat network with spanning

tree routing– Network diameter is limited,

often to metro scope

PE – Provider Edge Switch

PE

PE

PE

PE

PE

PE

PE

PE

PE

PE

PE

PE

PE

PE

PE

PE

PE

GigE/LAG

8

Emergence of ―Carrier Ethernet‖

• Limitations in enterprise-class Ethernet have led to the development

of ―Carrier Ethernet‖

• Meant to address unique requirements for carrier Ethernet services,

including Verizon‘s services– Scaling to support a large number of customers

– Scaling to support large numbers of switches and customer interfaces

– Support both point-to-point (E-Line) and multipoint (E-LAN and E-Tree)

services

– Support for both port-based and VLAN-based services

– Support for QoS other than best-effort to support QoS-based SLAs

– Sub-second outage restoration and routing convergence to support

availability SLAs

– Policing and shaping to support sub-rate services (e.g., 200 Mbps

service on a physical GigE interface)

9

IETF Ethernet Services Support

• Point-to-point pseudowires (PWs) to carry layer two frames,

including Ethernet, over IP/MPLS networks

• Extends the MPLS LDP protocol to signal pseudowire establishment

• IETF extended PWs to a multipoint Ethernet service, VPLS (Virtual

Private LAN Service)

• PWs and VPLS extremely popular, implemented by most every

router vendor and in wide use by service providers world-wide

• Verizon uses both point-to-point PWs and VPLS to provide customer

Ethernet services

10

IP/MPLS Forum Ethernet Services

Support

• Extended IETF-defined PWs to support non-similar endpoint

interworking– Supports point-to-point Ethernet-to-Frame Relay, Ethernet-to-ATM, and

ATM-to-Frame Relay interworking over MPLS PWs

– Very useful for multiservice convergence, and to support customers with

a variety of access methods

– Can support applications such as hub location with GigE access, and

low-speed Frame Relay spokes

– Supports interworking of IP packets via ARP Mediation, and bridged

services by interworking native Ethernet with Ethernet frames

encapsulated by FR or ATM

– Can also support VPLS endpoints with FR or ATM-attached customer

equipment

11

H-VPLS vs. VPLS+PBB

• VPLS and H-VPLS as originally defined by the IETF cannot meet

Carrier Ethernet service scaling requirements:– 10s to 100s of thousands of EVCs

– Number of E-LAN bridging instances per edge switch/LER

– Up to millions of customer MAC addresses

• For these reasons, the IETF is now defining the combination of

VPLS in the core with Provider Backbone Bridges (PBB, 802.1ah) at

the edge

12

Scalable Network Architecture –

PBB + MPLS

• Metro Network Dedicated to Ethernet Service

• Investment Protection

• Hierarchy with PBB

• Administrative Traffic Eng.

• Operations skill set / OSS Leverage

• MPLS core leveraged across multiple services (e.g., Ethernet, L3 VPNs)

• Scalable and mature control plane

• Leverage control plane to ease administration (BGP-Auto Discovery, TE)

• Less touch points for cross-metro services

• PBB (B-VID) VPLS instance (reduce PW Meshiness)

• Broadcast containment per service across core (via MMRP/BGP-AD)

• PBB MAC hiding

BEB

BEB

PB

PB

PB

PB

PB

PB

BEB

BEB

BEB

PP

BCB

BEB

/PE

N-PE

PBEB

PB

N-PE

PBEB

N-PE

PBEB

BCB

13

PBB-VPLS— MAC Scaling and

Customer-Addressing Awareness

• ―Hub‖ PE-rs get visibility of 100,000s of MACs

• High customer-addressing awareness

• MAC tables reduced: one B-MAC per

node

• No customer-addressing awareness

MTU-s

MTU-s

PE-rs

No. of MAC addresses/node

0MTU-s

1000s

100,000s

PE-rs

Customer MACs

Backbone MACs

MTU-s

MTU-s PE-rs

No. of MAC addresses/node

0MTU-s

1000s

100,000s

PE-rs

PBB-VPLS

MPLSMPLS

H-VPLS

14

PBB-VPLS Benefits — Service/Pseudowire

Scaling and Customer-Service Awareness

Customer services

Customer PWs

Backbone services

Backbone PWs

MTU-sMTU-s

PE-rsB

B

B

B

B

B

BB

B

B

MTU-s

PE-rs

0

1000s

100,000s

10,000s

No. of services-PW/node No. of services-PW/node

0

1000s

100,000s

10,000s

MTU-S MTU-S PE-rs PE-rs MTU-S MTU-S PE-rs PE-rs

VPLS + PBBH-VPLS

100s

15

OAM Specifications

• The IEEE, ITU-T, and MEF have defined Ethernet OAM (Operations,

Administration and Maintenance) specifications to allow fault

detection and correction. These include:– Link OAM: IEEE 802.3-2005, Clause 57

• Enables monitoring and troubleshooting of native Ethernet links

– Ethernet Local Management Interface (E-LMI): MEF 16

• Provides EVC status

• Enables automatic configuration of Customer Equipment (CE)

– Connectivity Fault Management (CFM): IEEE 802.1ag

• Enables monitoring and troubleshooting of VLANs within a network

• Supports multiple views (Customer, Service Provider, Operator)

– Service OAM: ITU-T Y.1731

• Extends CFM to include additional FM capabilities

• Performance Monitoring (PM)

16

Carrier Ethernet over MPLS

Testing

• Requires documentation and references– MEF, BBF, IETF, IEEE

• Automation– Definitive Parameters

17

802.1 q-in-q/ad and 802.1ah - Service

Tunneling Testing Scenario (Example)

CEs

PBBN

Q-in-Q /82.1adISIDISIDISID

ISIDISIDISID ISIDISID

ISIDISID

ISID

G.8031

Tunnel protection group

BCB1 BCB2

BEB1 BEB2 BEB3 BEB4BEB5 BEB6

VLAN 100VLAN 100 VLAN 100VLAN 100

0000.c004.0102

0000.c004.0103

0000.c004.0104

0000.c004.01050000.c004.0106

0000.c004.0107

0000.c004.0108

0000.c004.0109

18

Carrier Ethernet over MPLS

Certification

• ROI

• Time to Market

• Man Hours

• Resources

19

Conclusions

• Verizon‘s Carrier Ethernet services must meet stringent

requirements for:

– Conformance to Metro Ethernet forum (MEF) service definitions

– Scalability to support customer growth

– Reliability, resilience, OAM for troubleshooting and performance

monitoring, to support high service availability

– Standards-based certification

– Pre-deployment and post-deployment testing

20

Questions?

Thank you!

andrew.g.malis@verizon.com

charles.a.rexrode@verizon.com