LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
TURBO: Terabits/s Using Reconfigurable
Bandwidth Optics
Dan Kilper, Madeleine Glick UA
Keren Bergman Columbia University
February 18, 2016
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Adding Airlines to Google Maps
2
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
TURBO Concept
• Develop SDN control methods to establish end to end intra domain flexible optical spectrum paths• Layer 2 inter-domain connections
• Develop transparent cross-domain handover methods
• Create optical physical layer orchestrator
Long Haul
ROADMOA
OA
ROADM
OA
ROADM
ROADMROADMROADM
OTN/Eth
OPL-O
OTN/Eth OTN/EthOTN/Eth
MetroCampus A
OTN/Eth
OTN/Eth
ROADMROADM
OCS
OTN/Eth OTN/Eth
OTN/Eth
OCS
OTN/Eth
Data Center
Metro Campus B
ROADMROADM
OTN/EthOTN/Eth
Layer 2 Handover
Transparent Handover
Intra-Domain End-to-End Transparent
OPL-OOPL-O
OPL-O
OPL-OOPL-O
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Team
•University of Arizona• D. Kilper
• M. Glick
• Transparent Optical Aggregation Networks Testbed• 30 Gbaud coherent modulation
• Mesh network emulation platform
• Columbia University• K. Bergman
• Lightwave Research Lab• Inter-data center testbed
• CIAN: Center for Integrated Access Networks• NSF ERC, 18 Industry members, 8
Universities
4
Arizona TOAN Lab
Columbia LRL Data Center Testbed
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Lightpath Configuration Time
5
DICONET Project, Angelou, et. al. IEEE Comm Mag. 2012
Does not include physical device control and tuning delays from physical setup
Computation, signaling, and protocol part: ~ 1 second CORONET: 700ms
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Optical Switches, but no optical switching networks
Infinera record:
8 Tb/s provisioned in 19 minutes
•16 channels
•Over pre-determined path Amsterdam to Hamburg
“Optical burst switch developer Intune Networks in receivership” – LightwaveMag.
6www.guinessworldrecords.com
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
ON Lab, AT&T: Want White Box Programmable ROADMs
7
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Optical Power Dynamics: Unsolved
•Network problem not an amplifier problem• Present even for perfect constant gain control
•Only a problem for optical transmission with real-time wavelength switching•Power dynamics, similar to noise performance,
polarization effects, nonlinear impairments, must be addressed and lead to a trade off between cost and performance•Higher performance (less dynamics) leads to higher
cost• Better/more components• Better/more complex control algorithms
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Optical Transients: Solved Problem
• Single EDFA problem
• Fast Feedforward Control & Nonlinear Feedback
9Tian & Kinoshita, JLT 2003
No Feedforward With Fast Feedforward
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
10
Optical Power Dynamics
F. Smyth, et. al. JLT 2009
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
TURBO Goals
• Realize a white box optical transmission system control system• Only proprietary today• Allow researchers to program new algorithms and investigate new
abstractions
• Realize a programmable and transparent SDX
• Demonstrate end to end intra- and inter-domain flex grid transparent path creation using these new capabilities
• In each case these are v1.0 implementations to open the door for further development and promote opening up of transmission systems
11
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
TURBO Impact•Bring optical transmission system (ROADM) control
into research domain• Don’t expect commercial systems to ever be entirely open,
but this approach enables experimentation of where that line should be drawn
•Enables White Box ROADMs, Disaggregated ROADMs, etc.• Stimulate new area of research• Promote creation of SDN interfaces in optical elements
• Enable demonstrations (ON Lab) to include optical trans.• Encourage system vendors to open up their products to more
programmability
12
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Scheduler
Cross-Domain Interface
Policy Manager
Network State/ Abstraction
Ntwk State/ Abstraction
DataCenterSDNos
OPL Controller
MetroSDNos
WAN Network OS
Control Data Translator
DPCE
DPCE
OPL Orchestrator
ROADMOA OA
ROADMOA
ROADMROADMROADMOCS
OTN/Eth
Layer 2 Controller
Layer 2 CntrlOPL
Orchestrator
OTN/Eth OTN/Eth
Layer 2 Controller
OPL Orchestrator
OTN/Eth
Domain 2 Domain 3Domain 1
OPL-O: Optical Physical Layer Orchestrator
13
SENSE, BDESDNos, Network OS
OpenFlow, Proprietary, TL1
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Role of OPL Orchestrator
•Control diverse set of optical transmission hardware to operate network• Provision channels• Tune amplifiers• Tune channel powers• Manage faults, e.g. node loss, transient recovery• Defragmentation• Real time route selection*• Real time elastic bandwidth/flex grid
management*• Wavelength layer protection • ….
14
Future
Current propriety solution
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Worst Case Power Excursion: Remove 1 Chn
15
5 10 15 20-16
-14
-12
-10
-8
-6
-4
-2
0
ROADM Node Hops
Po
we
r E
xcu
rsio
n (
dB
)
2 4 6 8 10 1210
-1
100
101
102
103
De
lay (
s)
ROADM Node Hops
1dB
6dB
3dB
tn=10 s
tn=5 s
tn=1 s
tn=0.1 s
Peak to Peak Ripple
Consider accumulated ripple between ROADM nodes: 3-4 dBChannels flattened at each ROADM in steady state
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Delay Accumulation with Node Hops
16
5 10 15 20-16
-14
-12
-10
-8
-6
-4
-2
0
ROADM Node Hops
Po
we
r E
xcu
rsio
n (
dB
)
2 4 6 8 10 1210
-1
100
101
102
103
De
lay (
s)
ROADM Node Hops
1dB
6dB
3dB
tn=10 s
tn=5 s
tn=1 s
tn=0.1 s
te=2tn, DPref = 1 dB, 3 dB ripple
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
• Open channel and bring up/down slowly
Wavelength Switching: Step by Step
ROADM1 OA ROADM2 ROADM3OA
Decreasing power on 1535 nm Increasing power on 1560 nm
Power (dBm)
-25
-2
6-2
7-2
8-2
9-3
0-3
1-3
1
-33
-32
-31
-30
-29
-28
-27
-26
-2
5
-24
-2
3
-22
-2
1-2
0
-24
-24
-23
-22
-21
-20
-19
-19
No
λ-32
-33
-190
0.2
0.4
0.6
0
Sati
c ch
ann
el
Excu
rsio
n (
dB
)
**
* WSS adjustment to negate excursion
*
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
• Switch laser faster than EDFA response time
• Vary dwell time on each wavelength
• Full power at all times
• Reduce number of steps
Wavelength Switching: Fast Tunable Laser
10
0%
95
%
90
%
80
%
70
%
50
%
40
%
30
%
20
%
10
%
60
%
5%
1535nm dwell time %
0%
0
0.2
0.4
0.6
0
Stat
ic C
han
nel
Ex
curs
ion
(d
B)
50
%
20
%
0%
0%
5%
10
%
20
%
30
%
50
%
60
%
70
%
80
%
90
%
40
%
95
%
10
0%
50
%
80
%
10
0%
1560nm dwell time %
**
WSS adjustment
*
*
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
SDN Control Plane
19
R1 OA OA R9R2 R7R3 R4 R5 R6 R8 R10
Z. Wang JLT 2014
Parallel and independent ROADM node control
9 dB!
Scheduler
Cross-Domain Interface
Policy Manager
Network State/ Abstraction
Ntwk State/ Abstraction
OPL Controller
WAN Network OS
Control Data Translator
DPCE
DPCE
OPL Orchestrator
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Factors that Impact Power Dynamics
•Order and timing of node adjustments
•Node control algorithm
•Channel loading and configuration
•Timing of wavelength switching
•Wavelength Route Characteristics
20
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
SDN Control Plane
21
R1 OA OA R9R2 R7R3 R4 R5 R6 R8 R10
Z. Wang JLT 2014
Parallel Modified ROADM node control
<2.5 dB!
Scheduler
Cross-Domain Interface
Policy Manager
Network State/ Abstraction
Ntwk State/ Abstraction
OPL Controller
WAN Network OS
Control Data Translator
DPCE
DPCE
OPL Orchestrator
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Appropriate Abstractions for OPL?
WSS OA WSSTx/Rx
Tx/Rx
… Tx/Rx
Tx/Rx
...
• Layer 2/Ethernet switch: details are hidden, complexity goes to OPL• Problem: Only works for small networks, large margins• Problem: wavelength blocking, reach and power dynamics
• Network Models: a network of five nodes with multiple ports• Stable reconfiguration using OPL orchestrators for each domain• Colored ports: BW, latency, setup time, osnr, …
D2
D1
D4
D3
D5
N1N3
N4N5
N2
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
OPL Orchestrator
• Dynamic PCE• Rules and logic for spectrum assignment and path selection
• Abstraction Model• Representations of available spectrum and paths throughout network
• Cross-Domain Interface• What spectral information gets shared?
• How to represent path dependent spectral information between domains?
• OPL Controller• Implements flexgrid assignments across
many nodes and switches• Tuning of WSS controls
• What happens when a filter encroaches on a live signal to allow room for a new signal?
Scheduler
Cross-Domain Interface
Policy Manager
Network State/ Abstraction
Ntwk State/ Abstraction
OPL Controller
WAN Network OS
Control Data Translator
DPCE
DPCE
OPL Orchestrator
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Optical Flexigrid
• Fixed grid: assign wavelengths to rigid spectral slots or channels
• Flex grid: wavelengths can be assigned to an arbitrary spectral location and allocated different spectral widths • Channel widths can be variable
• Spatial dimentions (multi-fiber, multicore) can also be used
• Flexgrid capabilities have unique operational and control characteristics
• Spectrum may become fragmented
A B C D
unused spectrum
contiguous spectrum block
spectru
m
spectrum continuity constraint
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Multi-Hop Spectrum Continuity
A B C D
500k0x200G
unused spectrum
contiguous spectrum block
spectru
m
Filter
500k1x50G
1x filtered spectrum block
signal spectrum
• Spectrum is not all or nothing
• Wide passbands for filter cascades & spectral broadening• Varying filtering conditions can be managed along path• Need to manage crosstalk
• How to manage used BC spectrum that overlaps ABCD assignment?
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Cross-Domain Abstractions
• Performance weights for available routes
• Investigate defragmentation policies for common cross
domain spectrum
• DPCE: Flex grid performance constraints with IA-RWA
• Use AOPM at boundaries
AB
Scheduler Policy: Reserve Shortest
Domain State/ Abstraction
Domain 1 Orchestrator Inter-Domain Protocol & AbstractionCross-
Domain Interface
Domain 2 Orchestrator
Domain 1 Abstraction
Domain 2
6
9
6
3
2
2
104
9
free spectrum
Domain 1 OPL Orchestrator
SchedulerPolicy: Reserve Shortest
Domain State/ Abstraction
Cross-Domain
Interface
Domain 2 OPL Orchestrator
10, 9
2
6
BA
A B
Domain 1
Domain 2 Abstraction
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Transparent Domain Boundary
•How to achieve spectrum continuity across domains?
•Need guarantees for SLAs• Who’s to blame if something goes wrong
Long Haul
ROADMOA
OA
ROADM
OA
ROADM
ROADMROADMROADM
OTN/Eth
OPL-O
OTN/Eth OTN/EthOTN/Eth
MetroCampus A
OTN/Eth
OTN/Eth
ROADMROADM
OCS
OTN/Eth OTN/Eth
OTN/Eth
OCS
OTN/Eth
Data Center
Metro Campus B
ROADMROADM
OTN/EthOTN/Eth
Layer 2 Handover
Transparent Handover
Intra-Domain End-to-End Transparent
OPL-OOPL-O
OPL-O
OPL-OOPL-O
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Dual Layer SDX Handover
•SLA based on advanced Optical Performance Monitoring (AOPM)
•Flexgrid WSS: enforce spectrum continuity
AOPM
OTN/Eth
FWSS
FWSS
FTL
CT
OTN/Eth
AOPM
FOPC FOPC
FWSS
FTL
CT
OpaqueGateway
TransparentGateway
OPL Orchestrator
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Transparent SLAs
• Use spectrum continuity and guardbands as specification
• Use OSNR for performance guarantee• Key performance parameter for
coherent
• Silicon photonic OSNR monitor• Coherent
• Polarizaiton Independent
• Modulation Calibrated
29
Chip Scale OSNR Monitor
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Network Transmission Emulation
• Introduce multiple impairments to reproduce range of phenomena
30
PDL
VOA
POL Filter
ROADM
ROADM
ROADMROADM
ASE
High Power
Large Ripple
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Distance Emulation
31
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
TURBO DeliverablesYear 1
• Define use cases
• Optical physical layer orchestrator implemented in testbed
• Prototype of transparent SDX
Year 2
• Optical physical layer abstraction models for single domain control
• Optical physical layer control algorithms for single domain, performance with SDN
• Multi-domain transparent path establishment
Year 3
• Physical layer abstraction models for multi-domain control
• Experiments on dynamic real-time provisioning of optical connections across multiple network domains in the emulated optical networking platform
32
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Roles and Responsibilities
•UA• Path setup and control (OPL Controller)• Flexgrid Optical transmission, inter- and intra- domain• Transparent SDX• OPL abstraction
•Columbia• SDN Implementation and interface to network OS• Wavelength resource management/rwa: DPCE
33
LIGHTWAVE RESEARCH LABORATORYC O L U M B I A U N I V E R S I T Y
Looking forward 10 years
34
Space Switch
Coupler #1
...
Coupler #k
...
...
Demux #1
...
Demux #k
...
AWG#1
...
... ...
...
...drop
add
fibers
...AWG#s
...
...
E-Switch#1
...
...
E-Switch#t
...
... ...
...
Space Switch
...
Millisecond Switching
Microsecond Switching
Millisecond WAN WDM Switching
Data center/hpc server racks
Data center/hpc server racks
Millisecond Switching
TOR
TOR