DevOps and Software Defined Networking (SDNs) and

Software Defined Exchanges (SDXs)

Joe Mambretti, Director, (j-mambretti@northwestern.edu)

International Center for Advanced Internet Research (www.icair.org)

Northwestern University

Director, Metropolitan Research and Education Network (www.mren.org)

Director, StarLight, PI StarLight IRNC SDX,Co-PI Chameleon, PI-iGENI, PI-

OMNINet (www.startap.net/starlight)

MAGIC Meeting

Washington DC

October 3, 2018

• Creation and Early Implementation of Advanced Networking Technologies - The Next Generation Internet All Optical Networks, Terascale Networks, Networks for Petascale and Exascale Science

• Advanced Applications, Middleware, Large-Scale Infrastructure, NG Optical Networks and Testbeds, Public Policy Studies and Forums Related to Optical Fiber and Next Generation Networks

• Three Major Areas of Activity: a) Basic Research b) Design and Implementation of Prototypes and Research Testbeds, c) Operations of Specialized Communication Facilities (e.g., StarLight, Specialized Science Networks/Science Research Platforms)

Accelerating Leading Edge Innovation

and Enhanced Global Communications

through Advanced Internet Technologies,

in Partnership with the Global Community

Introduction to iCAIR:

StarLight – “By Researchers For Researchers”

Abbott Hall, Northwestern University’s

Chicago CampusView from StarLight

StarLight is an experimental optical infrastructure andproving ground for network services optimized forhigh-performance applicationsMultiple10GE+100 GbpsStarWaveMultiple 10GEsOver Optics –World’s “Largest”10G/100G ExchangeFirst of a KindEnabling InteroperabilityAt L1, L2, L3

Global Research Platform: Based On The Global Lambda

Integrated Facility

Visualization courtesy of Bob Patterson, NCSA; data compilation by Maxine Brown, UIC.

www.glif.is

IRNC: RXP: StarLight SDX A Software Defined

Networking Exchange for Global Science Research and

Education

Joe Mambretti, Director, (j-mambretti@northwestern.edu)

International Center for Advanced Internet Research (www.icair.org)

Northwestern University

Director, Metropolitan Research and Education Network (www.mren.org)

Co-Director, StarLight (www.startap.net/starlight)

PI IRNC: RXP: StarLight SDX

Co-PI Tom DeFanti, Research Scientist, (tdefanti@soe.ucsd.edu)

California Institute for Telecommunications and Information Technology (Calit2),

University of California, San Diego

Co-Director, StarLight

Co-PI Maxine Brown, Director, (maxine@uic.edu)

Electronic Visualization Laboratory, University of Illinois at Chicago

Co-Director, StarLight

Co-PI Jim Chen, Associate Director, International Center for Advanced Internet

Research, Northwestern University

National Science Foundation

International Research Network Connections Program

r !CT Testbed Researdl and I OevelopmentPromot.lon Cente, I

I ~ -JGN 1 I ••~ I I • I I I 1 (Japan) 1 1 (Los Angeles) 1

10G

SINET

(Japan) (Los Angeles)

;-- ------, I .A.rH~r&- 1 : : - -Research- - - _ N_E_AA_ -R- - : [-Ge~;()~~-: l (Chicago) I On Demand

---F~.:'.C~P-N: (::;:::) (NowYo<k) •i•--•9-1! GE~ !14•....-------,! -------- : I (Ottawa) (London) I I

[ T.@.D§J:AC .: • ] I l (Chicago) 1 _ J ~ ~o~e_!:) _ _ 1 _ __ _ _ ___ 1 1 1 (London) _ _ _ _ __

: ~ Cl~ : 1 : 1=--~ ~---_...,_ ,~----------,~----2_x_1_o_o_G_~,-~,~--,--i ~ ! : (Tokyo) : I I I ,~ I I (Seattle) I I I 40G I I - - - - - - - - y 'f. I l I (G ) I - - -------- 1 eneva 1

NX100G I 1 ~ '+----+' man Lan. :1••--+•: • :Mon~ Open :1i~G (- - -- 11 111 ___ : I~ Light :1

- I canane I 1 I I I (New York) I I (Canada) I '- - - - - - - -l_ j ~ ~ ag~ __ I

[.u 1 : I I I (Chicago) I CERN ET I 1 0G ,- - - - -1 - -- ----I lfJ I 14•....----)1,9-11 ~ I : (China) : 1 PACIFIC WAVE I .... 1-2x_1_o_O_G __ .....1J'------'T'---'

(Amsterdan) [- l~t; r~a~i~n~I-] I SON Networks I I I 1 & Testbeds 1.,_ 1 I I

1 (Los Angeles) 1 (Chicago) I (Seattle) 1 NX100G I M 1

I (Chicago) I I i ________ , I

r l 1 (L(Sos Angel1es)) ~ - __ ~~1~~G- 3~~0_9~ _ ~~100G 1 - - - - 1 1,..,. ... T\.ANOCCROSSAOM>S 1 unnyva e 1 1 \ 1 p r I

S~n gA,R EN : , I

I

: (Singapore)

1 (Los Angeles)

r I I I I I 1 ( Honolulu) J (Mauna) ----i---,---- ---, :O aarnet: I I 1 ( Australia) J

I (Seattle) 1 • I (Los Angeles) I I I

'----+---- 1 S T~f,: R L I GHT ... :2x 1ooG: 1 NT NET® 1:1••--•1 MAX/WIX : .. I The Opt ical STAR TAP - I• • J I ~- _ J~~} ___ I

NORDUnet I I I I I

r------- -,~ SOX Switches 1 1 ~--------... I I I (Chicago) I I (Chicago) I 2x1 0G (Nordic) I GRPNet I '-- - -- - - - - - - -- _J 1--------1 (Chicago) I I t ---- ----1 .... _.._ ____ _

: (Chicago) : I (Seattle) I .-------'

___ J __ _ _ l

RE AN N Z : I I I (New Zealand) 1

(Auckland) I ___ t ___ _ (Taiwan)

(Chicago)

2x10G

(Taiwan) (Chicago)

10G I l f l I r/.iii I

: KRt.e_)JET : I I

:@ :!~~~: : CSTNET :

: (S. Korea) : I (Chicago) I

: (China) : 1 (Chicago) 1

r------------------------------------, 1 Federal Agency Networks, Regional Network 1

1 State Wide Networks, University Neworks, Testbed Networks 1

I (Chicago) I ~------------------------------------

10 f l

: OpenTransits : 1 Network 1

I I I I 1 (France) 1 1 (Chicago) 1

] [ Southern Light ] AtlanticWave 1 1 • 1

(Miami) ~ : RN p: (Atlanta) I I I

(D.C.) I 1 (Brazil) I _ J_N~~y~~) __ I l_ i_S~o!~u~l_ _r

ANA300 • ---• I Overlay Network [

IRNC SDX

IRNC SDX

IRNC SDX

IRNC SDX

GENI

SDX

UoM SDX

Emerging US SDX Interoperable Fabric

GENI

SDX IRNC SDX

; I..,_..,.,,..\. g •' CERNetZ/ \

f ,1 CSTNet ! C', , China _, ~ '- IHEP-CMS -- _.,

~ ' -W IHEP-ATLAS r I .I

'"' -........ , N n

., ........ ( nm. ' f .J_Proposed}}

"-w.1-'

CERN ~ ~

Qtflal"llli .. OARR

~ '"' _.,._

LHCONE: A global infrastructure for the High Energy Physics (LHC and Belle II) data management

.,~~\ ,. .. CUDI "..,

l Mexico \ -- ' \ 1 UNAM - ~ ...__ ..,_,r

---\-- >NET .,,,,.,

PurU

Vanderbilt

Also see ht1p://lhcone.ne1 for details

UWisc

CANARIE Canada

TRIUMF-Tt

UFlorida

__ ,_.,,.---.. ,,, t I_ _ _ RNP __ , .,. ,

" jH EPGridl Brazil pAMP~ ~ <, L(UER!l I -- 1l U~ J., .;'

- ~' ... - ~p~ -- - , .,

PIC-T1

5 Ap ril 2015 - WEJohnston, ESnet, wej@es.net

I LPHIIE 1 Distributed site C) LHCONE VRF domain/aggresrator

i_'.=~_:_J Pending LHCONE 9 Network provider / e)(change point router connection • Cross-border router .,., _ Sites connected at 40G-100G 0 Regional R&E communication nexus

0 Broadcast VlAN w/ switch providing VLAN connections

l!!.!Y Catania CNAF-Tl Frascati Legnaro

INFN INFN Milano

Pisa Napoli Roma1 Torino

UChi LHC Tier l/2/3 ALTAS and CMS } yellow ouWne indicates

"" Belle II Tier 1/2 LHCtBelle II

N LHC ALICE or LHCb site

~ Sites that are standalone VRFs,

~ Communicat ion links: 1/10. 20/30/40. and 100Gb/s

Peta Trans: Petascale Sciences Data Transfer Siingapore

/ - ' t \ I I I I I I

'

DTN @100G

------ ·--- --✓

SeaUle/L .AJS un nyVille

f I I I I I

.,,,, ---------,

I I I

1, PRP Siites I -----· --- ✓

-·---------/ ---- \ I

I I I I

\ ~ ) ..._ __ .5 _____ _ Washington D.C.

Sydney/Brisbane Daejeon CIERN ~- -------- , / ---------

{ ( \ __ _,.I___,.. I I :

I I I I I I I I

I -..Vi~ NC .. 1(!)•·- ', ~ I : ~ : I '!(·,,Si' ·. . . ' . ·. . . ,, ., 1, 11 1 " ... , '· . . L--~ . '--------- - , ____ ----✓

/ - ---·----·- ,

( 1 I I I ,.__.,.,. ...... T.. I I I I j I I

J

......_, __ , ____ , ____ _ ,. Warsaw

,, - --·------ - , \

I I I ~'11":'1 I I l~~ I I I

~-t II J

Chicago

I I I

I ·• I 1 _ cuzna J ' ----------

Denver Ottawa

Persistence 1 OOG IDTNI Services Bevond SC17

---------✓ Am1sterdam

/ -- ...,, t \ J I

I I I I I I , ~ . I I C -,-..-., ~ , _ _ .,;;_T_' _"-!!.. __ ✓

Montreal

Emerging Topics In Advanced Networking

• Transition From Legacy Networks To Networks That Take Full Advantage

of IT Architecture and Technology

• Extremely Large Capacity (Multi-Tbps Streams)

• Specialized Network Services, Architecture and Technologies for Data

Intensive Science

• High Degrees of Communication Services Customization

• Highly Programmable Networks

• Network Facilities As Enabling Platforms for Any Type of Service

• Network Virtualization

• Tenet Networks

• Network Virtualization

• Network Programming Languages (e.g., P4) API (e.g., Jupyter)

• Disaggregation

• Orchestrators

• Highly Distributed Signaling Processes

• Network Operations Automation (Including Through AI/Machine Learning)

• SDN/SDX/SDI/OCX/SDC/SDE

~ ~

CHAMELEO A LARG SCALE RE . ONFIGU · BLE EX~E ·1ME TA INST U _ENT FOR O _p TER SCI ENCE

K te Keah y

Joe Ma mb ett , · i erre Rltea , Pa u R th, Da · Sta Zli10 ne

I_) T ·CC ursA

track 15date

your

logo

G n mi u ivi

201

~- - -~ ? ~

\ ~

r I I I I I I I 1..-, i

L3 Euro

· (T iiw . n~

DWDM Transi anders

Op1tical LiniH Systems

DWDM Tirans, onde111's

DWD

ti} .c 15 Q 1•D1 Ill

A Disagg111e_gatedl SCinet Op1tical Layrer

?

1 :

DD

1nl2 1• 00 1D04 1 JS

OLS5 Vendor C

OLS6

IReconfigura · on o;p.tions A. Booth to boolli 1oonneotions B. Booth to WAN conneotiollS C. Booth to switch or muteir 1oonneotio:llS D. WAN to S'll\l1h::h Olr rower COfllllleciiOfllS

IE:xam;p'.les

Nlo'lies

A. B-B a. Boo1h UUI 1-1 to 11 [1[12- 1 via

optical laye ir b. 1Boo1h 11 [l[l l -1 to 11 [1[14-3 via

optical la),11e-ir (asSl!lm es OLS2 to OLS4 path)

B. Booth to WAN a. 1Boo1h 11 [l[l l -2 to PoP1-1 via

OLS2-2 and OLS1-1---b. Boo1h 1 [1[11-2 to PoP2-B1 ·via

OLS2-2 , OlS4., OLS6 and OLS5-11

C. Booth to switcWrooter a. 1Boo1h 1[1[11-3 to S 1-A 1-b. 1Boo1h 11[1[13-1 to RTR·1[-A5

(asSl!lme-s OlS4 to, OLS6 p.alli) D. WAN to s'll\l1tchlrouter

a. PoP2-A.1 (WAN2) to SW1 -3 via OLSJ-11 and OLS4-3 -

b. IPoP2-A2 (WAN2) to RlR11-3 ·via OLSJ-2 and OLS4-7

1) TroHponders· could be. from mult1iple veru::fors [but for near term the Ii ll'il.lics would Med to be bui It wirilih 1i101tch i ~ tmnspo:m:1.ers:.

2) Corrtrotllers and Ol"'CMS"hr'ation s,s,tems, arc not shown lb1!!11t ,oll Tp111dr/ 'OLS 5Ystems must be ,conlil.ected

. ST RLIGHT SM

Starlight SOX Geoscience Research Workflow

lnptJt para mete1r 1.Date 2 .Tim e(00/06/12/18)

n.etCDF ... png ..

1--oo,;10 li t :•'

Convert to video

mp4 Display

Sele· t Data

Data Transfer To Other DTN

ESA Service Prototype

-.. ~ -- -

1,ooobps

10Gbps.

ANA.300

·~ ST RLIGHT SM

Source Corby Schmidt

~ 9;18 Oatla!i,lhpc

Still Largely Reflects SC 17 Conflg in SL booth.!!

ifX r!$J)l!e$.

tin~ r1 .com/SC 18,-JB DT To request changes, please leave a co ment

See also

hPP'lftinwd com'SC 1 B-NBE

..

1000-l.1!14

1000-BR4

- 1000-DAC

-- olOO --!IW

-- 100

-.. 1&

09/02/20 8

I

~ '\ l>.I\IJ\al{lS\~ \

Arista 400G SW

.z . I­~ ,Cl

starligh Boo h 2851

SCine~

TA TESTBED

SCinet L1I 11

SCinet L1I

~ <rnCLJil1fil WW U Ul.lll Ll.llJ WW

Oiena 8180

•••

SC 7 Demonstrations, of 4,00 Gbps D skMto lsk WA lie 17ransfers us, ng IWAR and VMe Drives,

An SCl 7 Col lalborative In iti .a tive A1mong1 NASA and Severa I Pa rtlne rs

...-- SC17 @ • e:nver, co SCl1net NOC

13ootti # 061

NA:S..A. demo ·S.tarlLigh lr.lOCCJiCAIR Exlh I B0(ltt) #11653

C2 -C30

C 4 -CU

SC inet

HEN X400NVME

24 -NVME DISKS X'.14 DCI x• I

BROCADE LX-9240

( A'&D Pa,rtners )

ESnet

Cl4•Cl.6

:K3

ST~ .R LIGHT · .A A MeUa ·ox .~ r r:-r.t-1 1r.1n1 nn 1F"R

~ ~ CenturyL. ink - 11,:ii\~liiao SAJA.C~~Lef ~ ~ · ·. · :;:;.:? tr, E ~~ ~ "r,,gii;; Sl;it1,r:1;11-c;

S:inet ARISTA • 1:!{1f11il»

WASH-CR.5

(

X2

Ec kingt,on ·

-....

L@V•@B/ I

MAX

( Sta1rUg ht/M REN

X200NVME 24-NVME DISKS.

\.....__ _____________ _

!.egend

lrOO-GE

NA AJ~ FC-ow ned

( N 5 A/G SF C 24-NVM E DISKS - ::-,,1 Gree nbe It HECN MD ' X200NVME

l '­

X2

DELL 2 100

NASAIGSFC Hlr;i h Bnd Co put er Netwo I r;i (HECN I Te Dla . am b~ B" Fin I Paul La g - B/1412017

.Add'I Data

INRL will d,emonst rat e:

"' Dynamic diistribut,ed p rocess1i ng of large !.l'ol:uimes of data

"' Across. geogrn:phirolly dispersed HPC an di ne ork reso'U roes

"'Ab! e, to rapidly· change resources to mee a ppl'ication demandis.

"' Software Defined 1etworlcii n.g (SID 1)

"' Pii p61 ined 2 Gbps, reall-time, 11j deo pmcess:i n.g workflows coexi1st iin.g wi h bul · da a transf,ers between proc,essi111,g nodes.

"' 3, - mo 1G bps ,connections, DC­to-SC 7 flo m p!uiS 00 G b P5 connections to Chicag:o and Oakland!

!Dynamic Distributed ID1ata Proce,ssing Chicago 1L

12 G'bps-4Kx60fps

Processing &

Dat a1Tr,amfer

Software De fined Exchange

P ooes1i g1 oodemancl

elective Dat a Disp lay & User Interaction

Nava l R,es,earcti Laboratory

Cente r for Comp tat im11a l Scienoe

SCl7 Demonstration

Pipelines

U.S.NAVAL - ESEARc ~-

ORA OR

RL

P'rocessi ng & Da1talr,an.sfeir

NRL, Waslm1gton, DC

D,eirve·l', c,o, Booth 165,3, "lnten::mmecteri am:J interlocking problems:"

call for a high periormance dynamic distributed data Gentric infiastructuire

vFerm1lab

~OAKRIDGE ~National Laboratory

ET

~l ~

KISTi C l'fWl'I , li.e.

fN ' .

~ ST RLIGHT SM

Transferring Lar1ge,Scale Airline Data E2E A1cr10.ss WANs U1sing DTN.s •' , :21 E 20

I 'IL.,l)j~_,;!

till _, !;, -EM

'i,11,-. b .. DTN

1'1.Ml 2ro8 'A/IN~ VIA~20J!I

DTN06r.l1U ii. 1/0XY ir.l;r -HJ

I \II.AN :!l:>311 -tl K ,Tl

~

-'I •=, ,._,, ~

IJ'.IA ~T'S MXj: ID.200.a!t 11

'111\.N-t 21}3.!.

'ANl2:ro!

AdlM,1u1l ,urn li i;..gn,o,

- eur111 n!I:, ll'wll Ail»-dil«!!ln !In.• n INMlli' ae~

· ""'0G~ 111111 !:rmfl..-• E11.p n;i "11 tu• QI.F n1 &:

I~ _ ,

It : /oil rnll

~ ,_ IQ

10.!i:02l!. 07

I D"IN&'l.nll,5

10", . J81

t i or

~ ST RLIGHT ""'

GENI – iCAIR P4 Testbed – Integrated With GENI

StarLight SDX

• In Partnership With The GENI Initiative, iCAIR Is Developing a P4

Testbed for Computer Science Research.

• The Testbed Will Be Integrated With the GENI SDX At StarLight

• P4 (Programming Protocol-Independent Packet Processors).

• An Emerging Networking Programming Language,

• A Domain Specific Language for Network Protocols.

• Highly Flexible In Contrast To OpenFlow

• Testbed Based on Tofino (Barefoot Networks) Switches

• Compiler (V16) Enables Rules To Be Dynamically Implemented In

Chip

Other Notable iCAIR Suported Testbeds

• International AI Testbed (Currently Being Designed)

• International DTN Testbed

• LHC P2P Service

• High Performance Digital Media Network (HPDMnet)

• Geophysical Sciences Testbed

• Content Routing Network

• AutoGOLE

• MEICAN/Network Service Interface (NSI)

• AmoebaNet

• Cisco Information Centric Networking Testbed (ICN)

• SD-WAN Testbed

• ToMaTo

• Et Al

1111 • • a ~ 1. • • • • •I • • • I

jl • •' •~ • I• •! ' "~ •• •

Bu Id ng the Open Storage Netvvork

nslllute or Data lnlerslve Englnee Ing and Science

idit2::1

Alex Szalay The Johns Hoplkins University

www.startap.net/starlight

Thanks to the NSF, DOE, DARPA,

NIH, USGS, NASA,

Universities, National Labs,

International Partners,

and Other Supporters

"Any opinions, findings, conclusions or recommendations

expressed in this material are those of the author(s) and do not

necessarily reflect the views of the Networking and Information

Technology Research and Development Program."

The Networking and Information Technology Research and Development

(NITRD) Program

Mailing Address: NCO/NITRD, 2415 Eisenhower Avenue, Alexandria, VA 22314

Physical Address: 490 L'Enfant Plaza SW, Suite 8001, Washington, DC 20024, USA Tel: 202-459-9674,

Fax: 202-459-9673, Email: nco@nitrd.gov, Website: https://www.nitrd.gov