Networks ∙ Services ∙ People www.geant.org
Guy Roberts
Technology Exchange, Cleveland
Alien waves, packet aggregation and router bypass
A smarter DWDM/OTN layer in your network
7th October 2015
Transport Network Architect, GÉANT
Networks ∙ Services ∙ People www.geant.org 2
Contents
• Alien waves• Alien wave background
• Economic case for alien waves
• Field trial results
• Alien wave production services
• OTN Router bypass and packet aggregation
Networks ∙ Services ∙ People www.geant.org
Alien WavesTheory and economics
Networks ∙ Services ∙ People www.geant.org
Alien wave over DWDMScaling Fiber Capacity
Optical fiber
Tx
Tx
Tx
Rx
Rx
Rx
Transmitters Receivers
Combining and separating colors
Alien Wave provider – fibre, amplifiers and muxing
Alien Wave user – Txand Rx transponders
Amplifier
Demarcation between provider and user
• DWDM uses the 1525–1565 nm C band to make use of erbium doped fibre amplifiers (EDFAs)
• Approx. 80 waves per fibre on 50GHz ITU-T gridThe AW provider is selling spectrum on their DWDM transmission system
Networks ∙ Services ∙ People www.geant.org 5
The economics traditional transmission services
• Cost structure for transmission layer:
• Fixed one-off costs – system design costs, installation fees
• Fixed annualized costs – common equipment depreciation, fibre
lease, equipment maintenance fee, power etc.
• Variable costs – transponders added on a per-service basis.
• For the first few services added, the fixed costs greatly exceeds the
variable costs.
• As new services are added the fixed costs per service falls.
• The break-even point occurs when revenues exceed the overall cost
of running the service
Networks ∙ Services ∙ People www.geant.org 6
• Alien waves allow multiple providers to share one common fibre infrastructure
• This means that there will be more services on the fibre, and the fixed cost are shared across many
services, reducing the overall cost.
• The collaboration model agreed between Surfnet and GÉANT is for the fixed costs to be shared
equally between each of the providers using the common fibre infrastructure.
• In the case of Amsterdam to Hamburg this is Surfnet, Nordunet and GÉANT. In future this may also
include PSNC.
• Common costs are reduced by 2/3s compared to GÉANT only fibre.
• Solution also retains GMPLS control plane and ability to rapidly turn up new services.
• TeleGeography: “huge demand (for alien waves) from nontraditional operators”
Economics of alien waves
Networks ∙ Services ∙ People www.geant.org 7
Economics of alien wavesActual costs for Ams-Ham
Annualized cost
over 3 years in
Euros
AW brings savings cf.
leased waves with only
3 x 100Gbps
1 x 100 2 x 100 3 x 100 4 x 100 5 x 100
GÉANT DF € 198,623 € 123,623 € 98,623 € 86,123 € 78,623
Leased Wavelengths € 65,000 € 65,000 € 65,000 € 65,000 € 65,000
Alien Waves € 98,623 € 73,623 € 65,290 € 61,123 € 58,623
€ 0
€ 50,000
€ 100,000
€ 150,000
€ 200,000
€ 250,000
Ams-Ham, Alien Waves cost per 100Gbps
Networks ∙ Services ∙ People www.geant.org 8
Source: TeleGeography
Alien waves growth driven by content providers
Most of the growth has been
in content provider networks
Networks ∙ Services ∙ People www.geant.org
Alien WaveField trial
Networks ∙ Services ∙ People www.geant.org 10
Amsterdam – Hamburg fibreEliminating redundant fibre infrastructure
• R&E community has two parallel fibre systems between Amsterdam and Hamburg.
• Yellow cable is Surfnet.
• Green cable is GÉANT.
• Duplication of infrastructure dilutes utilization.
• Objective 1: remove one fibre system and share remaining fibre
• Objective 2: retain GMPLS control plane
• Solution: alien waves
Networks ∙ Services ∙ People www.geant.org
Phase I
• Tested the technology and operational procedures.
• Objective was to understand if GÉANT can make use of SURFnet’s dark fibre to carry our DWDM trunks.
Fixed Filters
• Phase I used fixed filters to insert alien waves
• Convenient for fast trial, but does not scale well
• After successful competition of trail, the production solution will use ROADMs
• ROADMs allow waves to be remotely turned up and reduces inter-site fibres
11
Field TrialThe setup
Networks ∙ Services ∙ People www.geant.org
AOLX-500
AOLM-500
AOLX-500
OTC OTC
DANTE Amsterdam
DANTEHamburg
DTN-X
AOLX-500
AOLX-500
SURFnetAmsterdam
6500 CPL6500 CPL
SURFnetHamburg
AOLM-500
Pair of loan AOLMs in the blue spectrum
(OCG 2)
Existing DANTE fibre
Switch Matrix
ATC ATC
N x amps
N x amps
SURFnet/Cienaline system
Pair of loan ATCs with OLA and OFM-4-D
Existing AOLX on OCG 7
AOLX-500
Switch Matrix
DTN-X
Field TrialConnectivity
Surfnet system operated in parallel with GÉANT production fibre for duration of trial
Networks ∙ Services ∙ People www.geant.org 13
1.00E-07
1.00E-06
1.00E-05
1.00E-04
1.00E-03
1.00E-02
-30 -25 -20 -15 -10
pre
-FEC
BER
channel Rx power (dBm)
AOLM-500-T4-1-C5 Hamburg Pre-FEC sensitiviy
Ham 1556.56nm
Ham 1554.94nm
• Receive sensitivity measured on two wavelengths in each direction
• Better than 10 dB receive margin in both directions – i.e power into Rx can fall by 10dB before errors are seen
1.00E-07
1.00E-06
1.00E-05
1.00E-04
1.00E-03
1.00E-02
-30 -25 -20 -15 -10p
re-F
EC B
ERChannel Rx power (dBm)
AOLM-500-T4-1-C5 Amsterdam re-FEC sensitiviy
Ams 1556.56nm
Ams 1554.94nm
Field TrialTest results – excellent receive margin
Networks ∙ Services ∙ People www.geant.org 14
1.00E-07
1.00E-06
1.00E-05
1.00E-04
1.00E-03
1.00E-02
1.00E-01
1.00E+00
-7 -6 -5 -4 -3 -2 -1 0
Pre-FEC BER @ AMS vs insertion loss at input to Ciena mux @ HAM
1556.56nm 1554.94nm
1.00E-07
1.00E-06
1.00E-05
1.00E-04
1.00E-03
1.00E-02
1.00E-01
1.00E+00
-3.5 -3 -2.5 -2 -1.5 -1 -0.5 0
pre-FEC BER vs insertion loss at point of insertion of alien wave into fibre
1556.56 1554.94
• Power injection is reduced at two points:• Where the AW signal is injected into the fibre
• Where the AW signal is injected into the Ciena equipment
• In both cases the reduction in power did not result in a significant change in the pre-FEC BER.
• Demonstrates that system is stable under power level changes.
Field TrialTest results – robust to input power variation
Networks ∙ Services ∙ People www.geant.org 15
Infinera pre-FEC Q out of range alarm
Infinera Loss of Frame alarm
Alarms and notification aid debugging: four types signal deterioration can be distinguished
1. If there is a loss of fidelity of the signal, this can be detected using pre-FEC threshold crossing notification
2. If the alien wave becomes so degraded that the Q-factor drops below 10, Infinera will raise a pre-FEC Q out of range alarm
3. A cut in the SURFnet fibre results in a Loss of Frame (LOF) alarm, some signal from the optical amplifiers leaks through, so there will not be a Loss of Signal (LOS) alarm.
4. If the local patching between GÉANT and SURFnet sites is broken, then a Loss of Signal (LOS) alarm is raised.
Field TrialTest results – Infinera alarms allow debugging of alien wave
Networks ∙ Services ∙ People www.geant.org
Alien WavesNext steps
Networks ∙ Services ∙ People www.geant.org
Production Alien WaveFlexible solution uses ROADM technology
• These positive field trial results mean that GÉANT will go ahead and work with SURFnet to a solution that scales up to 5 x 100G of alien waves from Amsterdam to Hamburg.
• The production solution will use ROADMs for a more scalable solution.
• The addition of Wavelength Selective Switches (WSS) on the drop side of the SURFnet ROADMs adds support for colourless add-drop.
• This means that the multiplexed set of 10 x 50Gbps waves from the Infinera Photonic Integrated Circuit (PIC) can be injected straight into the Ciena ROADM without the need for fixed mux/demux.
• Reduces the number of optical patch cords between GÉANT and SURFnet racks.
Networks ∙ Services ∙ People www.geant.org
OTM
DTN-X
AOLX-500
Ciena6500 CPL
AOLX-500
Up to 10 waves of 50Gbps each on single fibre pair
Switch Matrix
GÉANT Amsterdam PoP SURFnet Amsterdam PoP
TrueWavereduced slopefibre
OTC
AOLX-500
AOLX interleaving
WSSWSS
OCG-7
WSS for Directional Independent
Access (DIA) line side
Production Alien WaveConnectivity
The use of a wavelength selective switch (WSS) on the access ports allows each wavelength to be individually turned up and adjusted
Networks ∙ Services ∙ People www.geant.org
OTN & PXMFlexible capacity management
Networks ∙ Services ∙ People www.geant.org
PXM in DTN-XPacket Aware OTN Core
Port Consolidation Transport Efficiency Stat-Mux
• Reduce router ports/fiber• Improve router efficiency• Aggregate packet flows
• Right-size transport• Build ODUflex as needed• Efficient optical protection
• Multiple flows combined• Traffic profiles per flow• Comprehensive QoS & TM
100GbEN x 10GbE
N x 10GbE
N x 10GbE
Packet Aware OTN Core
PXM: 200G Ethernet switch on a blade
Pt2pt ELINE or multipoint ELAN services.Built over OTN circuits
Networks ∙ Services ∙ People www.geant.org 21
• Current IP architecture mirrors the underlying optical fibre links
• As a result IP trunks chain through all routers
HU
CH DE CZ
ATIT SK
HRSI
Fibrelayer
Routinglayer
Old Eastern ringIP trunks follow fibre
100G
40G100G
100G 100G
100G
40G
20G20G
20G
Router interfaces are filled upwith transit traffic
Networks ∙ Services ∙ People www.geant.org 22
• OTN allows flexible and rapid reallocation network services
• PXM cards allow multiple services to terminate on a single DTN-X 100G port
• Allows for fine-grained allocation of GÉANT trunks
• Express links saves on high cost router interfaces
• Trunk capacity can be rapidly adjusted without the need to change routers interfaces
New eastern ring IPIP trunks follow traffic not fibre
HU
CH DE CZ
ATIT SK
HRSI
Transmission layer
Routinglayer
50G
100G 50G
50G
50G 50G
40G
50G
10G
10G10G
10G10G
FR
50G
Trunks follow traffic rather than fibre
Networks ∙ Services ∙ People www.geant.org 23
Summary
• GÉANT’s first production alien wave has been commissioned on the Amsterdam-
Hamburg section.
• Good stable operation with no bit errors. Operational model is well understood.
• The next AW system is planed to be from Hamburg to the Polish border.
• We have re-designed the IP trunks on the Eastern ring to make use of packet aggregation
cards and router bypass.
Networks ∙ Services ∙ People www.geant.org
Thank you and any questions
Networks ∙ Services ∙ People www.geant.org
24