METRO-HAUL: METRO High bandwidth, 5G Application-aware optical network, with edge storage, compute and low Latency
H2020-ICT-2016-2 Metro-Haul Grant No. 761727http://metro-haul.eu
The Impact of the Optical Network on 5G –the Metro-Haul Project
Andrew Lord [email protected]
ONDM, Athens 13th May, 2019
Contributions from many in Metro-Haul including:
Nicola Calabretta and team at TuE on novel WSS
Antonio D’Errico and Filippo Cugini and teams at Ericsson and CNIT on novel WSS
Pablo Pavon and team at UPCT on 5G network modelling using Net2Plan
Alex Stavdas and team at OLC on novel optical transport metro architectures
Talk flow
2
Metro-Haul overview
The key question – what kind of optical layer will be needed for 5G?
Lots of bandwidth to handle all applications?
And what should the nodes look like?
Lots of compute and storage at local nodes to provide low latency applications?
How will we measure all of this?
KPIs
What technologies shall we use?
Focus on some of the optical solutions and discussion
Ultimately we could do anything
With enough money, power, space…. Metro-Haul techno-economic modelling using Net2Plan
How can we prove our new technology?
Metro-Haul planned demos
Conclusions
The Metro-Haul Solution
METRO TRANSPORT
AccessMetroEdge
Metro CoreEdge Photonic
Core
Metro CoreEdge
Metro Node
Metro Node
Metro Node
Metro scope
10-20 km
0.1-0.2ms
Control / Orchestration
5G Access
50-200 km
0.5-2ms
100-1000 km
1-10ms
Access Metro Edge Node (AMEN) – multiple ubiquitous access technologies, cloud enabled (storage, compute)
Metro Transport Network – metro node: pure transport
Metro Core Edge Node (MCEN) – Larger cloud capabilities
Metro Control Plane – full orchestration
5G PPP - https://5g-ppp.eu/ - Metro-Haul Golden Nuggets
4
5G-PPP EU body overseeing all the 5G projects
Golden Nuggets are available from every project to capture the essence of the innovations from each one
GN1 - High Capacity & Flexible Metro Optical Network with Edge ComputingDynamic data plane with intelligent control plane involving multiple network segments and layers, spanning multiple geographicalData Centre (DC) locations and addressing resource heterogeneity including, notably, the optical transport. Without this data & control plane architectures, network resources supporting future 5G services would require enormous over provisioning, of both optical transport capacity across metro and core networks, and edge Data-Centre resources such as compute and storage.
GN2 - Real-Time Performance Monitoring & AnalyticsTelemetry/monitoring framework which provides a global, real-time view of the E2E network performance. This new technology enables services configuration and reliable operation. It provides pro-active actions on early detection of issues. Machine-Learning within the decision engine allows this new Metro-Haul technology to continually learn and improve as real network data is collected. It includes state-of-the-art advanced planning, placement and re-optimization/re-configuration tools, enabling holistic (joint) optimization across heterogeneous resources.
GN3 - Open Multi-Layer Disaggregated NetworkSystematic and unified approach based on model driven development for the SDN control of multilayer disaggregated and open transport networks, while allowing flexibility in deployment choices, extensibility for the integration of new technologies and agility
in migration processes without vendor lock-in.
5G-PPP KPIs
5
KPIs – Key Performance Indicators
Designed to measure the performance of the E2E solution
Designed to ensure the wide range of potential 5G vertical applications will work
Metro-Haul has converted 5G-PPP KPIs into quantities pertinent to the optical layer
KPI CategoryTarget
1Optical Point-to-Point
connection set-up time≤ 1 min
2Metro-Haul E2E Point-
to-Point connection set-up time
≤ 2 min
3Set-up time of network
service slice across Metro-Haul
≤ 1 hr
4Capacity of Metro-Haul
Controller
Control of 10 – 100 nodes (AMENs/MCENs, i.e., Open
Disaggregated ROADMs)
Optical layer time to set up or reconfigure services handling 5G applications enabled by
SDN-based management framework – includes control plane latency, optical node
reconfiguration delay, network instantiation time.
Set-up time between two VNF elements as part of a service slice. Includes packet over
optical pt-pt connection.
Time to set up a network slice as a set of interconnected VNFs
Maximum number of Netconf devices that a single SDN optical controller can support.
5G-PPP KPIs (2)
6
KPI CategoryTarget
5Fault / degradation
detection timeTo be defined
6Capacity of Metro-Haul
infrastructure
100x more 5G capacity supported over the same optical fibre infrastructure
7New optical components
/ systemsTo be defined
8
CapEx ReductionTo be defined
9
Energy ConsumtionTo be defined
Time between instant fault happens (e.g. some threshold is violated) until it is detected
Number of service instances that can be supported. Combines optical connections and
AMEN capacities (throughput, storage, compute)
New components defined in Metro-Haul
Relative cost reduction compared to baseline network to support a pre-defined set of
verticals
Reduction of energy using new node technologies such as PIC, filterless and dynamic
service infrastructure (service set-up / tear down)
Metro-Haul is an optical project – what are the optical issues?
7
Optics in the metro will need to serve thousands of 5G wireless endpoints with high bit rates (10G + )
Unlike the core network (where there is only one), this is a large distribution across all regions of a country
Distances are less than the core but cost is critical
We can’t just use core transport technologies
Core coherent transmission is far too expensive.
Core optical switching (advanced WSS based ROADMs) are far too expensive.
Fully integrated one-vendor solutions lock the solution in and this is also far too expensive as well as restrictive
But we DO need optical switching. We DO need optical networking (e.g. to pick up multiple wireless base stations on a single horse-shoe
On the other hand – distances are limited so performance is not so challenging
Under consideration:
Ericsson and TuE both developing low cost integrated WSS components
Filterless solutions also considered – e.g. DuFiNet using PON technology
Whitebox mentality – requiring SDN-based architectures to allow control and monitoring of components to allow an E2E multivendor / multitechnology dynamic solution
Tx1
Tx2
Tx3CAPS 25-50 Gb/s
PM-QPSK100 Gb/s
NRZ 25-50 Gb/s
Photonic Integrated White Box
OPEN ROADM
AgentOpen Config Agent
Agent
Agent
Agent
NETCONF
AgentOpen Config Agent
• Photonic Integrated Whitebox
• Silicon photonics chips will enable huge cost reduction
• Performance doesn’t have to match LCoS-based WSS
• 200 mm wafer realization
Rx
Several modulation formats used
Dispersion tolerant CAPS3
transmitter
Photonic adaptable
dispersion
compensator
100 Gb/s Commercial Apparatus
Open Config Agent
Open Config Agent
Open Config Agent
Ericsson and CNIT Innovation
Cell size: 4 mm x 4,6 mm
InGaAsP/InP SOAs: 950 μm gate SOA, 2 mm pre-amplifier
AWG: Channel Spacing = 2,4 nm, Free Spectral Range = 8 x 2,4 nm
9
TuE Photonic integrated 8 WDM Add/drop switch
Input
PassPre-amplifier
SOA
Booster
SOA
WSS
module
DropCh1
Ch8:
Spectral characterization of the switch
10
On/Off switching ratio > 35 dB
On-chip gain estimated >11 dB
1.00E-11
1.00E-10
1.00E-09
1.00E-08
1.00E-07
1.00E-06
1.00E-05
1.00E-04
1.00E-03
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0
BER
Received power [dBm]
B2B CH 1 CH 2 CH 3 CH 4
. . . .
Access-Metro
Edge Node
#1
Access-Metro
Edge Node
#K
Upstream
Path
Downstream
Path
3dB coupler
mux/demux
Rx’s
DR
OP
AD
D
BM-Tx
DR
OP Rx’s
AD
D
BM-Tx
Metro Core Edge Node
* Uzunidis et al DuFiNet: A Flexible, High-Capacity and Cost-Effective Solution for Metropolitan Area Networks; ECOC 2018
DuFiNet - a filterless dual-bus Metro architecture using PON technology
Standard protocols are used
(NetConf, OpenFlow)
no extensions introduced
The LTs and ONUs, are
represented as legacy
OF/Netconf - L2 switches
Openflow for flow
management over the
PON
Broadband Forum’s
switch YANG model for
configuring queues
Overall, the Metro abstraction
was realised using YANG
modeling
Control Plane
Techno-economics – what is so hard about it?
12
Has to take a wide range of 5G scenarios with multiple KPI definitions
Has to take a geography (multiple operators in the project)
Deploy resources – optical, compute, storage to deliver 5G
With constraints –
Cost, Power, Space
Critical questions –
Do we simply over provision optical fat pipes? What is the cost penalty in doing this? Do we have the space / power in exchanges to allow this?
How do we handle latency? Do we simply over provision the local exchanges? What is the cost, space, power penalty for doing this?
Three main scenarios to be compared
Over provisioned everything – no Metro-Haul tech needed
Static DC optimisation – Slow CP management only
Dynamic DC optimisation – fully dynamic slice assignment changing during a day
All of this needs a highly advanced simulation tool
Pretty sure this is better
This will be even better – but by how
much and is it worth the hassle?
Methodology for techno-ecnonomic analysis
Reference Topology
Traffic Model
Excel Template
NIWNFV over IP over
WDM Net2Plan Library
Scenario Definition Software frameworkReusable modules /
multiple partners
Algorithms
Aut. reports
Benchmark tests scripts
• NIW library developed within MH (open-source)
• Based on abstract model of an IP over WDM network with IT resources in the nodes.
• Simplifies development of Net2Plan algorithms, automatic reports etc. for these networks.
• Simplifies import/export from defined Excel-based template.
• Publicly available: shipped with Net2Plan (www.net2plan.com)
Methodology for techno-economic analysis. Workflow example
Dense-urbanMost L < 20 kmSome 20 ≤ L < 50 km
SuburbanSome 20 ≤ L < 50 kmMost 50 ≤ L < 100 kmSome 100 ≤ L < 200 km
Aggregation node
Core node
Core node BB
L: length of horseshoe
UrbanMost 20 ≤ L < 50 kmSome 50 ≤ L < 100 km • Three reference topologies (Telecom Italia)
• Diameter <200 km, [52, 102, 159] nodes• Anonymized in node position
• With traffic information in agreement with Deliverable 2.3• Background traffic: P2P, regular, cacheable (video)• Traffic from verticals: VNF & traffic requisites from key
vertical services considered (D2.3).
Reference Topology &
Traffic
Capacity Planning
alg.
IP BOM
DC BOM
WDM BOM
Cost model
Energy model
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 2 4 6 8 10 12 14 16 18 20 22 24
T1 (Business)
T1 (Residential)
Time of day
Nor
mal
ized
tra
ffic Multi-hour
tests
Multi-period
tests
Methodology for techno-economic analysis. Workflow example
• JOINT capacity planning of IT & network resources• Different algs from different partners for
different problem variants / approaches• ML algorithms in resource allocation in dynamic
traffic environments• Code reuse: Algorithms can leverage on proven
routines or other algorithms
Reference Topology &
Traffic
Capacity Planning
alg.
IP BOM
DC BOM
WDM BOM
Cost model
Energy model
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 2 4 6 8 10 12 14 16 18 20 22 24
T1 (Business)
T1 (Residential)
Time of day
Nor
mal
ized
tra
ffic Multi-hour
tests
Multi-period
tests
RUN
Methodology for techno-economic analysis . Workflow example
Reference Topology &
Traffic
Capacity Planning
alg.
IP BOM
DC BOM
WDM BOM
Cost model
Energy model
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 2 4 6 8 10 12 14 16 18 20 22 24
T1 (Business)
T1 (Residential)
Time of day
Nor
mal
ized
tra
ffic Multi-hour
tests
Multi-period
tests
Automatic scripts creating Bill-of-Materials reports• IT: BOMs for different DC architectures• IP & WDM: BOMs for different capabilities of
optical equipment
Methodology for techno-economic analysis . Workflow example
Datacenter ComponentsDescription (text)
Compute Nodes
Examples: HP Proliant DL 360, Huawei 1288-v5 or
2288H-V5, Dell R740 or R940
Small
Intel Xeon Gold 6134 with 8 cores, 64 GB RAM, 600GB
HDD
Medium
Intel Xeon Gold 6140 with 16 cores, 128 GB RAM, 1.2TB
HDD
Large
Intel Xeon Platinum 8160 with 2 CPUs x 24 cores, 128 GB
RAM, 3.6 TB HDD
ExtraLarge
Intel Xeon Platinum 8160 with 4 CPUs x 24 cores, 192 GB
RAM, 3.6 TB HDD
Storage
Examples: Samsung SSD, iXSystems TrueNAS X10 2U,
HPE Nimble Storage HF40/60, Dell EMC Unity 300
SSD Small 4 TB
SSD Large 8 TB
NAS Small 20 TB
NAS Medium 60 TB
NAS Large 120 TB
NAS ExtraLarge 400 TB
Other specialized hardware
BRAS
HW Firewall eg Fortinet FortiGate 3000D
Load Balancer
eg HPE OpenCall Load Balancer, Radware Alteon. F5Big-
IP, etc
Reference Topology &
Traffic
Capacity Planning
alg.
IP BOM
DC BOM
WDM BOM
Cost model
Energy model
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 2 4 6 8 10 12 14 16 18 20 22 24
T1 (Business)
T1 (Residential)
Time of day
Nor
mal
ized
tra
ffic Multi-hour
tests
Multi-period
tests
MH Cost model & energy consumption model• A cost model is being produced, updating and
expanding efforts in previous Eus• Cost model includes energy consumption figures• Automatic reports are being developped to
produce cost & energy figures from the BOMs & networks.
Methodology for techno-economic analysis . Workflow example
Reference Topology &
Traffic
Capacity Planning
alg.
IP BOM
DC BOM
WDM BOM
Cost model
Energy model
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 2 4 6 8 10 12 14 16 18 20 22 24
T1 (Business)
T1 (Residential)
Time of day
Nor
mal
ized
tra
ffic Multi-hour
tests
Multi-period
tests
Different tests are being conducted. Scripts for bulk definition of the multiple tests can be also shared.• Multi-hour: Exploit knowledge of well known traffic profile variation along day.• Multi-period: Studies considering traffic growth. Future-proof network evolution.
Results coming soon... example question to address: how the operational benefit of having the MH Dynamic Control Plane, that on-demand allocates IP flows and VNFs, translates into CAPEX savings whan dimensioning the IT+IP+WDM metro resources?
Connection with on-demand resource allocation
19
RES
T/A
PI NFV-O
WIM Connector
OaaS(N2P Server)
DataREST/API
Oaa
S C
lien
t
OSS
RES
T/A
PI
SDN-C(Packet)SDN-C
(Packet)SDN-C
(Optical)SDN-C
(Optical)
PoP (VIM) 1 PoP (VIM) 2
NetworkInterface
NetworkInterface
VNF3VNF1 VNF2
OXC OXC
OXC
VLAN-A VLAN-B
Packet Switch Packet Switch
Parent-C OaaS Client
NIW-based routines used in techno-ec analysis can be reused in dynamic allocation algorithms in MH demos
• Optimization-as-a-Service: planning tool making on-line joint VNF & IP & WDM resource allocation decisions
• Net2Plan-OaaS module in demos, uses NIW-based algorithms for dynamic allocation.
End of project demo #1 – Crowdsourcing application
20
End of project demo #2 – video security
21
Conclusions
22
Metro-Haul nearly 2 years in
Most of the base technologies in place
Control Plane
Monitoring
Optical layer options
Disaggregation options
Techno-economic modelling with KPI constraints
Final phase demonstrations
Control Plane demonstration at EUCNC in Valencia in June
Two full project demos
BUT –
Although operators will almost certainly need more cost effective optical transport solutions in the metor
The case for disaggregation isn’t proven. Operators may go different ways depending on appetite for innovation
Local caching / compute for latency reasons will be v expensive – do those use cases really exist?
Dynamic slice handling – still VERY ambitious