Whitepaper

SONET – What Next for the Backbone The All Purpose Transport Network

C. Harper Anderton

FAETWPSWN0814 ©2014

Contents SONET, the Current Standard ....................................................................................................................... 1

Core vs. Metro vs. Access.............................................................................................................................. 2

Core Network ............................................................................................................................................ 2

Metro/Aggregation ................................................................................................................................... 2

Access ........................................................................................................................................................ 2

What is an All Purpose Transport Network (APTN) ...................................................................................... 3

Packet Based Networks............................................................................................................................. 4

Connection Oriented Ethernet (CoE) .................................................................................................... 5

MultiProtocol Label Switching (MPLS) .................................................................................................. 6

MultiProtocol Label Switching-TP (MPLS-TP) ....................................................................................... 7

Comparing SONET, CoE and MPLS ........................................................................................................ 7

The All Purpose Transport Network.............................................................................................................. 8

Integrating Multiple Technologies ........................................................................................................ 9

FAE Telecom’s All Purpose Transport Network Infrastructure ................................................................... 11

About the Author ........................................................................................................................................ 12

SONET – What Next for the Backbone

SONET, the Current Standard SONET has been a preferred backbone network technology for many years and rightly so as it has many features that address the needs of the critical network infrastructure for utilities:

• As a well-established technology o Well known how to deploy o There are many systems available to monitor it o SONET troubleshooting is well understood

• It delivers dedicated bandwidth with channelized transport • It is deterministic – latency and jitter are controlled • It is manageable with standard performance indicators • It is reliable with sub-50ms protection switching

All of which made SONET an ideal backbone to support a myriad of different application specific requirements that were continuously being introduced. Each of these different connection layers tended to be supported by different communication and connection protocols including physical interface differences, particularly where low speed serial and analog communications (e.g. C37.94, RS-232/422, FXS/FXO, etc.) were required.

SONET – What Next for the Backbone | 1

Core vs. Metro vs. Access FAE Telecom sees the network as being divided into three “segments”: Core, Metro/Aggregation and Access.

Core Network The core network is increasingly multi-service running both IT and telecom applications running over carrier-grade optical OC-192 that provides SONET and/or 10Gb Ethernet transport. Fiber (as either privately owned or a mix of private fiber and commercial Ethernet services) is the preferred medium, as even higher aggregate bandwidth rates can be achieved by upgrading existing fiber with wave division multiplexing technologies.

Metro/Aggregation The metro/aggregation network is the focal point for the aggregation and management of the many different access network protocols, with OC-48/1 GB Ethernet infrastructure providing the key interface to the high speed core network, using a Multi-Service Provisioning Platform.

Access The access network is where the significant migration issues lie. This is where the key legacy asynchronous devices are still residing today. These legacy devices offer proven reliability, employing such protocols as EIA-232/V.24, EIA-422/V.11, EIA-449/V.36, and EIA-530 to transmit low-overhead, low-speed (e.g.115.2 kbps) control data streams as well as mission critical voice communications, including encryption and E&M signaling.

2 | SONET – What Next for the Backbone

TDM multiplexers are employed today to combine asynchronous outputs for transmission across a synchronous-serial V.35, X.21, T1, or E1 composite to ensure the low latency and guaranteed delivery required by the legacy applications.

What is an All Purpose Transport Network (APTN) An All Purpose Transport Network is a combination of hardware and Management Software that allows an orderly migration and managed solution for the Next Generation Network (NGN).

A Next Generation Network addresses everything from the low speed existing data interfaces such as RS.232 to the current IP routers using high speed Gigabit Ethernet as the connection to the network core.

Developing a coherent All Purpose Transport Network infrastructure, addressing both the bandwidth requirements of new Smart Grid applications currently being deployed and the legacy TDM based applications, will be of critical importance in the network design process.

Bandwidth requirements have significantly grown, and are predicted to continue to grow as completely new sets of applications are rolled out (advanced substation automation, Smart Grid etc),

Type of Facility Bandwidth 2006-2010 Bandwidth 2011-2015 Large Generating Station 1 DS3 1 OC3 ROC 1 DS3 1 OC3 Small Plant Combustion Turbine 4 DS1 8 DS1 Field Maint. Office/Warehouse 4 DS1 8 Ds1 230KV Sub Station, DC Tie 2 DS1 4 DS1 69-138Kv Substation 1 DS1 2 DS1 Pole Top remote/Switch 1 DS0 1 DS0

Source: UTCG

SONET – What Next for the Backbone | 3

Driving utilities to look for ways to better utilize the available bandwidth and integrate with IT and corporate applications that are also running on the backbone.

Carriers have also moved their networks from SONET to packet (a far easier task as they do not the legacy applications to support – they just discontinue them!), the market for SONET equipment has decreased to the point where many manufacturers are no longer supporting it.

Packet based (Ethernet and IP) network topologies are seen as the way of the future, however there are technical issues with any new infrastructure supporting those legacy applications which need the assurance of constant time factors present in Time Division Networking, which has been the fundamental technology for critical network infrastructures over the past fifty years.

Packet Based Networks Today there are two prominent packet based technologies:

• Multi-protocol Label Switching (MPLS) o A methodology for establishing “connection-like” end-to-end paths for IP and other

protocols o An evolution of IETF L3 IP routing

• Connection-Oriented Ethernet (COE) o A methodology for delivering circuit-switched performance in a packet switched

network o A standards-based evolution of (IEEE) L2 Ethernet

Whilst both of these are packet based they take different approaches to carrying the different multiple traffic types supported in a utilities network.

From a utility network architecture perspective, the ultimate goal is having a single network infrastructure handling all required data traffic flows. A complete solution architecture built on one technology that can offer legacy service emulation in addition to new multipoint services is very appealing, especially if that one technology can meet the service requirements of both past, present, and future utility applications.

Where these different technologies sit in the hierarchy of the OSI 7 Layer Model is shown in the table below:

4 | SONET – What Next for the Backbone

This ubiquitous network is the story being told by the packet equipment vendors and on the surface it is a very compelling one.

Peeling back the layers and taking a good look under the hood throws up issues regarding the support for the legacy applications (which are not going away in the short term) and migration – how do we get from today’s SONET infrastructure to the Next Generation Network in an orderly fashion.

Connection Oriented Ethernet (CoE) CoE is a layer 2 switching technology that closely resembles SONET. As its name suggests, CoE is primarily focused at connection oriented (point to point and point to multipoint) network topologies.

This makes CoE is an ideal technology in the access and metro/aggregation portions of the network particularly where low cost, guaranteed performance and high security are required, such as substation backhaul connections. It is also ideal where a user wants the look and feel of TDM, but the lower cost and greater simplicity of the packet world, while still having the ability to configure determinism and low-latency similar to TDM:

• Establish a transport path/tunnel across the network • Allocate services and account for resources inside the tunnel

SONET – What Next for the Backbone | 5

• Create defined back up paths across the network with known latency.

MultiProtocol Label Switching (MPLS) In an MPLS network, data packets are assigned labels. Packet-forwarding decisions are made solely on the contents of this label, without the need to examine the packet itself.

6 | SONET – What Next for the Backbone

MPLS operates at a layer that is generally considered to lie between traditional definitions of layer 2 (switching) and layer 3 (routing), and thus is often referred to as a “layer 2.5” protocol. MPLS provides a “connection-like” service for IP (and other protocols) in that it has the mechanism to establish a path across the network, end-to-end, but it is still a hop-by-hop forwarding mechanism (the decision where to route the packet is made at each node the packet passes through). As a new node is added to the network then all possible routes through the network need to be recalculated and managed (not a trivial matter in a network with hundreds of nodes)

Back up routes are predetermined and therefore latency on these routes is not deterministic.

MultiProtocol Label Switching-TP (MPLS-TP) As the name implies, MPLS-TP (MPLS-Transport Protocol) is a variant of MPLS. It is a layer 2 connection-oriented packet-switched (CO-PS) application. It was developed by the IETF to address concerns that MPLS was too complex for deployment at the edge of the network. It is a dedicated MPLS implementation that removes features that are not relevant to CO-PS applications and adding mechanisms that provide support of critical transport functionality.

Comparing SONET, CoE and MPLS

(For a more detailed comparison of CoE and MPLS see FAE Telecom’s Whitepaper “The Packet Wars – CoE vs MPLS)

SONET – What Next for the Backbone | 7

The All Purpose Transport Network The drive towards a ubiquitous network infrastructure is like a rock rolling down a hillside. It has started slowly with many utilities unwilling to move away from a tried, trusted and understood technology such as SONET. Also many telecommunications planners remain wary of claims being made by the “packet” evangelists – “packet networks can do everything you need”, but it is gaining momentum as utilities are looking for ways to:

• Lower initial equipment costs • Have flexible configuration and bandwidth assignment • Make more efficient use of limited bandwidths • Provide adaptive bandwidth allocation • Have native Ethernet interfaces, reducing/eliminating the high cost of TDM interfaces in routers • Have the ability to transport legacy TDM over packet or packet over legacy SONET.

An All Purpose Transport Network (APTN) addresses the above requirements by taking a “real world” approach to addressing the desire to move to a Next Generation Network, whilst supporting legacy applications that are not being replaced overnight and enabling an orderly, controlled migration from one technology to another.

An APTN builds on the fact that there is no “one size fits all” solution. There is no one technology that can effectively deliver solutions to disparate requirements of different parts of the network. The requirements of the core network, connecting centers, control centers and “Big Data” corporate applications are very different to the requirements of a remote substation supporting critical monitoring and control applications. A single technology solution today invokes the old adage, “Jack of all trades, master of none”.

Whilst packet based infrastructures have become the de-facto solution in the core of the network, the debate is now about how far toward the edge does it make sense to push more complex infrastructures.

8 | SONET – What Next for the Backbone

The cost per transported bit increases as more complexity is added, and the more complex the technology the more issues are raised:

• They are more difficult to manage • Guaranteed delivery/timing issues –

how deterministic are they • There is a steep learning curve – are

current staff trained on new technologies?

• Capex vs Opex – new technologies can reduce one off Capex costs, but if the continuing Opex costs increase significantly because of higher support costs, is that actually a savings?

• NERC/FERC/CIP – what impact does the new technology have on compliance costs?

Integrating Multiple Technologies APTNs utilize a new generation of hardware appliances and management software that will support both the diverse and complex access network requirements of today’s critical applications and the new IP/packet based applications.

APTNs also recognize the different requirements at the different areas of the network and utilize the best architecture to address those requirements, but providing an overall coherent network.

Taking the three areas of the network described earlier in this paper:

• Core • Metro/Aggregation • Access

An APTN integrates the best technology for each area, supporting both current and future technologies alongside each other, enabling the controlled and orderly migration of the legacy applications as the technology develops to support them.

This support of multiple physical infrastructures for connecting remote locations into the network, recognizes that although fiber is today the preferred choice, there are many factors (cost, geography, timescale), that makes fiber an unjustifiable solution in many locations.

SONET – What Next for the Backbone | 9

The APTN Multi-Services Provisioning Platform (MSPP) serves as both TDM and Ethernet/IP integration at the access and metro/aggregation areas of transport. This platform provides the ideal migration path to the Next Generation Network (NGN) as it supports all traffic from all sources on a common transport backbone or uplink into either an all packet core being carried over optical paths, or into a current SONET network at the same time.

This allows mission critical legacy data applications and the special voice services that need to maintain the reliability of TDM trunks to remain in service as far into the future as needed.

At the same time they support full mapping to Gig-Ethernet transport for all those IP based applications that are ready to take advantage of the CORE network’s lower cost per bit for supporting the new applications that are moving large amounts of information on TCP/IP.

An APTN solution can be applied to all networks large and small, with FAE Telecom offering the technology and services in a one stop solution. FAE Telecom can define, design and deliver a completely new and optimized transport network that will take advantage of APTN hardware and software to significantly reduce OPEX costs.

10 | SONET – What Next for the Backbone

FAE Telecom’s All Purpose Transport Network Infrastructure

SONET – What Next for the Backbone | 11

About the Author C. Harper Anderton – President, FAE Telecom. Anderton is co-founder and President of FAE Telecom Inc. a technology company focused on the support of legacy networks and their migration to Next Generation Network infrastructures. He has over 25 years’ experience designing, engineering, selling, and marketing high technology systems for both large as well as startup companies addressing critical infrastructure networks. He worked extensively in both Europe and the United States. Previous companies include British Telecom, 3Com, Cray Networks, Ascom Timeplex, Computer Network Technology and Expand Networks.

PO Box 2842 Acton, MA 01720

info@FAETelecom.com www.FAETelecom.com

855.GO-FAETEL (855.463.2383)

12 | SONET – What Next for the Backbone