All-Optical Networks for Grids: Dream or Reality?

All-Optical Networks for Grids:

Dream or Reality?Payam Torab

Lambda Optical Systems CorporationSeptember 28, 2005

www.lambdaopticalsystems.com

Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005

2

Grids – Tflops vs. Tbps. Emergence of grids is the result of the synergism between

communications and computing, just like cybernetic systems that came out of synergism between communications and control

Role of the network in Grids: to provide throughput– Application-aware networks, or network-aware applications?– Network providing services, or network as a services?– Throughput is the theme unifying connectivity, delay and

bandwidth Balanced growth of networking and computing results in Grids

Networking power (Tbps)

Com

pu

tin

g

pow

er

(Tfl

op

s)

Clusters

Internets

Grids North European Grid

NEESgrid

TeraGrid

Surfnet

http://www.surfnet.nl/info/en/network/nationaal/map.jsp


3

Need for High Throughput Throughput is a grid resource: Uniform

grid growth requires growth in throughput Throughput growth requires improvement

in bandwidth, delay and availability Examples of throughput requirements

– GridFTP applications– Large Hadron Collider (LHC) at CERN

Brookhaven National LabLong Island, NY

OC-48 link to ESNET

ESNET outage?

Source: www.cerncourier.com/main/article/45/7/15

Year Production Experimental Remarks

2001 0.155 0.622-2.5 SONET/SDH

2002 0.622 2.5 SONET/SDH DWDM; GigE Integ.

2003 2.5 10 DWDM; 1 + 10 GigE Integration

2005 10 2-4 X 10 Switch; Provisioning

2007 2-4 X 10 ~10 X 10; 40 Gbps

1st Gen. Grids

2009 ~10 X 10 or 1-2 X 40

~5 X 40 or ~20-50 X 10

40 Gbps Switching

2011 ~5 X 40 or

~20 X 10

~25 X 40 or ~100 X 10

2nd Gen Grids Terabit Networks

2013 ~Terabit ~MultiTbps ~Fill One Fiber

PHENIX experiment – Used GridFTP to transfer 270 TB of data from Long

Island, NY to Japan

Source: Larry Smarr, “The Optiputer - Toward a Terabit LAN ,” The On*VECTOR Terabit LAN Workshop Hosted by Calit2,University of California, San Diego - January 2005

Transpacific 10 Gbps line to

SINET in Japan

Relativistic Heavy Ion Collider RHIC at Brookhaven:600 Mbps peak250 Mbps average


4

Photonic Switching: Key to End-to-End Transparency

WDM + Photonic switching– End-to-end transparency

• Bitrate transparency (10 Gbps, 40 Gbps, …)• Payload transparency (SONET, SDH, Ethernet, …)

– Transmission robustness• Simplification or even elimination of windowing• No packet loss due to congestion/buffer overrun• Simpler transport protocols, higher throughput

From: “Development of a Large-scale 3D MEMS Optical Switch Module,” T. Yamamoto, J. Yamaguchi and R. Sawada, NTT Technical Review, Vol. 1, No. 7, Oct. 2003

Electrical Cross-

Connect (EXC)

Photonic Cross-

Connect (PXC)

Photonic Cross-

Connect (PXC)WDM and electrical

switchingSeparate WDM and

optical switchingIntegrated WDM

and optical switching

~O(102) waveleng

ths

O-E-O O-E-O

Full transparency

O-O-O

~O(102) wavelength

s~O(102) Gbps per

wavelength


5

Wavelength Switching Scalability Grid-scale applications will ultimately press

even wavelength switching – Example:

Year Production Experimental Remarks

2001 0.155 0.622-2.5 SONET/SDH

2002 0.622 2.5 SONET/SDH DWDM; GigE Integ.

2003 2.5 10 DWDM; 1 + 10 GigE Integration

2005 10 2-4 X 10 Switch; Provisioning

2007 2-4 X 10 ~10 X 10; 40 Gbps

1st Gen. Grids

2009 ~10 X 10 or 1-2 X 40

~5 X 40 or ~20-50 X 10

40 Gbps Switching

2011 ~5 X 40 or

~20 X 10

~25 X 40 or ~100 X 10

2nd Gen Grids Terabit Networks

2013 ~Terabit ~MultiTbps ~Fill One Fiber

Source: Larry Smarr, “The Optiputer - Toward a Terabit LAN ,” The On*VECTOR Terabit LAN Workshop Hosted by Calit2,University of California, San Diego - January 2005

Require too many

optical ports to provide

non-blocking

connectivity!

Similar to any other switching technology, aggregation is essential for scalability of wavelength switching – hence the emergence of transparent multigranular (wavelength and waveband) switching architectures

PXC PXC PXCPXC

Wavelength switching4 wavelengths over 4 hops 32 optical ports

PXC PXC PXCPXC

Waveband

multiplexer

Waveband demultiplex

er

Waveband switching4 wavelengths over 4 hops 8 optical ports

From: “A Graph Model for Dynamic Waveband Switching in WDM Mesh Networks,” M. Li and B. Ramamurthy, IEEE ICC 2004, Vol. 3, June 2004, pp. 1821-1825.


6

Waveband Switching Efficiency

buh

h

n

ne

w

b 1

1

11

0 2 4 6 8 100

2

4-100

-80

-60

-40

-20

0

20

40

60

Physical hops in waveband

path (h)

Waveband-

switched circuits (bu)

Sw

itch

ing

eff

icie

ncy

(%)

Waveband switching efficiency: Relative saving in the total number of optical ports in a network when waveband switching is used instead of wavelength switching

nw = number of ports under wavelength switching

nb = number of ports under waveband switching

h = average number of physical hops in each wavebandb = average number of wavelengths in a wavebandu = average waveband utilization (used wavelengths)

1 2 3 4 5 6 7 8 9 101

2

3

4

Wave

ban

d-s

wit

ch

ed

cir

cu

its

(bu

)

Waveband-switching

efficient region

Physical hops in waveband path (h)

Increased

waveband

utilization

Increased waveband

path length (hops)

Waveband

switching gets more

efficient

Waveband switching becomes only more efficient (more saving in optical ports) as more wavelength circuits are carried over longer paths

Example: GridFTP using 4 parallel TCP streams over 4x40 Gbps circuits carried over 6 hops More than 0.1 Tbps throughput over 6 hops using only 30 ports


7

2085 circuits@40Gbps ~ 85 Tbps~28400 ports - wavelength switching~21800 ports - waveband switching

998 circuits@40Gbps ~ 40 Tbps~11800 ports - wavelength switching~9700 ports - waveband switching

615 circuits@40Gbps ~ 2.5 Tbps~6200 ports - wavelength switching~5500 ports - waveband switching

5000

10000

15000

20000

25000

30000

0 20 40 60 80 100Network throughput (Tbps)

Req

uired

optica

l por

ts

Wavelength-switching

Waveband-switching

More on Waveband Switching Efficiency

Example: WDM WAN ~80 nodes, ~140 links This simple analysis

does not consider the extra scalability from the increase in bitrate (160Gbps and beyond, OTDM).

1 2 3 4 5 6 7 8 9 101

2

3

4

Wave

ban

d-s

wit

ch

ed

cir

cu

its

(bu

)

Waveband-switching

efficient region

Physical hops in waveband path (h)

Waveband

switching gets more

efficient

2.5 Tbps

40 Tbps 80 TbpsTransmission breakthrough

s Increase in throughput

without increase in

ports

More to appear in:P. Torab and V. Hutcheon, “Waveband switching efficiency in all-optical networks: analysis and case study,” in preparation for OFC 2006.


8

Hierarchical Transparent Switching Waveband switching adds another level of

switching to the transparent switching hierarchy Multigranular switching Logical WDM

topologies

Wavelength XC

Wavelength Interfaces

Waveband XC

Wavelength XC

Waveband XC


Waveband XC

1 2 h

h physical hops – one logical hop

Waveband Multiplexer

Bandpathbp1

bp1

Waveband Demultiplexer

Two lightpaths with the same routes

Node A Node B

Node A Node B

Wavelength XC


Waveband XC

Wavelength XC

Waveband XC


Waveband XC

1 2 h1

h1 physical hops – one logical hop

Bandpathbp1

bp1

Node A Node B

Node A Node B

Wavelength XC

Waveband XC


Waveband XC

1 2 h2

Bandpathbp2

Waveband Demultiplexer

Node C

Waveband Multiplexer

h2 physical hops – one logical hop

Node C

bp2

lp1 lp2Two lightpaths with partially overlapping routes

FiberWavebandWavelengt

hIP/TDM

Payload-transpar

ent Switchin

g

Several physical hops are lumped into one logical WDM link, requiring switching only at the link endpoints Fast and still flexible dynamic wavelength service over reduced number of hops


9

Logical (Virtual) WDM Combined wavelength and waveband switching allows dynamic

configuration of transparent optical topologies supporting dynamic lambdas (from connection on-demand to topology on-demand)

Example: During the next 14 days, computing facility at site A, the storage center at site B, and the visualization room at site C will participate in an experiment that will require multiple dynamic lambdas (e.g., timescale in seconds)

Computing - AStorage - B

Visualization - C

Computing - AStorage - B

Visualization - CDynamic lambdas(fast setup and

teardown)

Logical WDM Topology

Waveband

connections


10

Lambda OpticalSystems Solutions Dedicated to transparent switching

technology

Addressing research community and carrier needs

Deployed at U.S. Naval Research Lab (NRL) and Starlight

LambdaNode 200Transparent 64x64 full duplex portsGMPLS, CLI and web interface5.25 inches tall

LambdaNode 2000Integrated WDM and photonic switchingMultigranular switching for maximum scalabilityProvides waveband and wavelength switchingGMPLS, CLI, TL1 and web interface


11

NL 101, NL 103 Demos at iGrid 2005

vangogh 6

E600

nud05

x y

a b

iGRID A iGRID B

HDXc

2 x GbE

circuits

Qwest / other wave service

Qwest / other wave service

CENICCENIC

2 x GbE

circuits

VMT Controller

San Diego/UCSD (SAN) Chicago/SL (CHI) Amsterdam/NL (AMS)

**E600

nud06

**or other L2 switch

AAA/DRAC AAA/DRACAAA/DRAC

GbEOC192STM64

vangogh 5

VMT visualization host

HDXc

/2

vm vmvm

vh

4003(2)

HDXc

X /2

X

LambdaNode200

2/12 2/13

2/18 2/19

E120012/2 12/3

VLAN 350 VLAN 350

OME1 3

2 4

5 6

GbEOC192STM64


12

Control Plane: Enabler of Transient Services

Grid’s balanced growth needs dynamic on-demand high network throughput

What do we need to provide high throughput?1. Dynamism: Make optimum use of all network resources for the tasks at

hand• Example: If 1.0 Tbps throughput is needed between A and B for one hour, fill up

the network with 25x40Gbps connections and kill them an hour later.

2. Availability: The ability to maintain high throughput through fast recovery• Network failures do happen, therefore high bandwidth does not guarantee high

throughput• In a transient service environment protection is not as expensive

– Telco thinking: 1+1 protection is expensive- I need to plan for twice the capacity, therefore I need to charge my customer twice as much (bronze service, silver service, platinum service, …)

– Grid thinking: Provide as much protection that your schedule allows. The connections will not be there in an hour. The more network resources the more protected circuits.

• (Dynamic) restoration can also add to reliability when (dedicated) protection is unavailable

Key effort needed: Integrating traditional service levels (1+1 protection, 1:N protection, shared mesh restoration, …) into Grid services– Can a GridFTP application ask for transfer over 1+1 connection?– Trade-off between replication/migration and network recovery– Where does the optimal performance stand?

Application

intelligence

(replication,

migration)

Network intellige

nce (protecti

on, restorati

on)


13

IP-based control plane paradigm to control packet, time slot (TDM), wavelength, waveband and space (fiber) switching across multiple switching layers, and across multiple domains.

Developed by IETF – CCAMP workgroup with liaison work with OIF and ITU-T

Mature standard now (RFC 3945) with various extensions for different switching technologies (Layer 2, wavelength/waveband, SONET/SDH,…)

Basic functionalities/protocols– Neighbor discovery/link management (Link Management Protocol -

LMP)– Routing with traffic engineering extensions (OSPF-TE, ISIS-TE)– Signaling (RSVP-TE with GMPLS extensions)

Applications/solutions– Recovery (protection, restoration)– Make-before-break– Layer 1 VPN (L1VPN working group)

Generalized Multiprotocol Label Switching (GMPLS)

Cross-connect set upon receiving the PATH message

Bidirectional LSP

PATH PATH

RESV RESVIngress Node A

Transit Node B

Egress Node C

Bidirectional data plane

Cross-connect set upon receiving the RESV message

Both cross-connects set upon receiving the PATH message

Bidirectional data plane

RFC 3473 bidirectional

LSP setup

More efficient bidirectional

LSP setup

PATH

PATH

RESV

RESVCONF (optional)


14

Generalized Multiprotocol Label Switching (GMPLS)

New directions– Separation of path computation as a service– Attention to Ethernet as a Layer2 transport– Inter-domain traffic-engineering

• Good work at NSF’s DRAGON project– Inter-domain circuit setup, path computation element (Network Aware

Resource Broker –NARB)– The next step is interoperability with other networks

NARB

End Syste

m

NARB

NARB

End System

AS 1

AS 2

AS 3

Transport Layer Capability Set Exchange

Source: Jerry Sobieski, Tom Lehman, Bijan Jabbari, “Dynamic Resource Allocation via GMPLS Optical Networks (DRAGON),” Presented to the NASA Optical Network Technologies Workshop, August 8, 2004


15

Key word for Grid networks is high throughput Lambda Grids are the only way to keep up with throughput demand

– Reality When is access to dark fiber going to be cheap? Dream

– Starting as islands of transparency• Regional Optical Networks (RONs)• Fiber sharing is critical, RONs have to have transparent access to each other• Wavebands as highways between RONs

– Islands growing as optical reach/transmission improves• Digital wrapper, FEC

High throughput needs end-to-end transparency– Data plane transparency

• WDM and photonic switching

– Control plane transparency• Inter-domain end-to-end circuit setup

Availability and recovery are the new QoS for lambda grids Ethernet will be the dominant end-to-end payload

– Transparent networks are ready for payload change

Conclusions: Dream or Reality?

Photonic access to super highway for

RONs?

HOPI Node

Date post:	06-Jan-2016
Category:	Documents
Upload:	moke
View:	39 times
Download:	0 times

All-Optical Networks for Grids: Dream or Reality?

Documents