EXPERIMENTING WITH VIRTUAL SDXS USING CHAMELEON AND … · SDSN GENI Phys DMZ DTN IP egress BEN...

transcript

www. chameleoncloud.org

AP RIL 6 , 2 0 1 8 1

EXPERIMENTING WITH VIRTUAL SDXS USING CHAMELEON AND EXOGENI

Paul RuthRENCI – University of North Carolina

pruth@renci.org

OUTLINE

� Background� ExoGENI testbed (wide footprint edge cloud)

� NSF Cloud Chameleon testbed (mid-scale cloud)

� Experiments Spanning Testbeds � Inter-slice stitching

� Campus stitching

� Inter-testbed stitching

� Software Defined eXchange (SDX) Experiments� SAFE SDX (RENCI, DUKE, US DOE/Esnet)

� SciDAS (Clemson, RENCI, Washington State University)

OUTLINE

NSF GLOBAL ENVIRONMENT FOR NETWORK INNOVATIONS (GENI)

Virtual laboratory for networking and distributed systems research and education

GENI FEDERATION� Federated identity

� InCommon

� X.509 identity certificates

� Common APIs� Aggregate Manager

� Clearinghouse

� Agreed upon resource description language� RSpec

� ExoGENI translates relevant portions from NDL-OWL to RSpec and back as needed

� Several major portions� ExoGENI, InstaGENI, WiMax, Internet2 AL2S, ESnet

� Federation with EU FIRE effort

Cloud Providers

Virtual Compute and Storage Infrastructure

Network Transit Providers

Cloud APIs (Amazon EC2 ..) Network Provisioning APIs (DOE ESNetOSCARS, Internet2, OESS, OGF NSI …)

Virtual Network Infrastructure

EXOGENI

Mutually Isolated Virtual Networks

Edge Providers(Compute Clouds and Network Providers)

Mutually Isolated Slicesof Virtual Resources

Workflows

EXOGENI

� Relationship to GENI� One of two computational testbeds built for GENI

� Implements GENI API

� Accepts GENI users

� Notable features:� Wide scale footprint (20 sites)

� Edge clouds (OpenStack)

� Dynamic layer 2 circuits between sites

� Stitchports: layer 2 connections to external resources

� Limitations� Small scale computational sites

� No core network control

EXOGENITOPOLOGY

EXOGENITOOLS

EXOGENI: STITCHING

OUTLINE

CHAMELEON PHASE 1 IN A NUTSHELL� Deeply reconfigurable: “As close as possible to having it in your lab”

� Deep reconfigurability (bare metal) and isolation

� Power on/off, reboot from custom kernel, serial console access, etc.

� But also – modest KVM cloud for ease of use

� Large-scale: “Big Data, Big Compute research”

� ~650 nodes (~15,000 cores), 5 PB of storage distributed over 2 sites connected with 100G network…

� …and diverse: ARMs, Atoms, FPGAs, GPUs, etc.

� Blueprint for a sustainable production testbed: “cost-effective to deploy, operate, and enhance”

� Powered by OpenStack with bare metal reconfiguration (Ironic)

� Open production testbed for Computer Science Research

� Project started in 10/2014, testbed available since 07/2015

� Currently 1,600+ users, 300+ projects

CHAMELEON: PHASE 1 HARDWARE

SCUs connect tocoreandfullyconnected toeachother

HeterogeneousCloudUnits

ARMs,Atoms,lowpowerXeions, FPGAs,GPUs,SSDs, etc.

SwitchStandardCloudUnit42compute4storagex10

Chicago

To UTSA, GENI, Future Partners

AustinChameleonCoreNetwork

100Gbps uplink publicnetwork(eachsite)

CoreServices3.6PBCentralFileSystems, FrontEndandDataMovers

CoreServicesFrontEndandData

MoverNodes 504x86ComputeServers48Dist.StorageServers102HeterogeneousServers16Mgt andStorageNodes

SwitchStandardCloudUnit42compute4storagex2

NEW HARDWARE� 4 new Standard Cloud Units (32 node racks in 2U chassis)

� 3x Intel Xeon “Sky Lake” racks (2x @UC, 1x @TACC)

� 1x future Intel Xeon rack (@TACC) in Y2

� Corsa DP2000 series switches� 2x DP2400 with 100Gbps uplinks (@UC)

� 1x DP2200 with 100Gbps uplink (@TACC)

� Each switch will have a 10 Gbps connection to nodes in the SCU

� Optional Ethernet connection in both racks

� More storage configurations� Global store @UC: 5 servers with 12x10TB disks each

� Additional storage @TACC: 150 TB of NVMes

� Accelerators: 16 nodes with 2 Volta GPUs (8@UC, 8@TACC)

� Maintenance, support and reserve

NEW HARDWARE� 4 new Standard Cloud Units (32 node racks in 2U chassis)

� 3x Intel Xeon “Sky Lake” racks (2x @UC, 1x @TACC)

� 1x future Intel Xeon rack (@TACC) in Y2

� Corsa DP2000 series switches� 2x DP2400 with 100Gbps uplinks (@UC)

� 1x DP2200 with 100Gbps uplink (@TACC)

� Each switch will have a 10 Gbps connection to nodes in the SCU

� Optional Ethernet connection in both racks

� More storage configurations� Global store @UC: 5 servers with 12x10TB disks each

� Additional storage @TACC: 150 TB of NVMes

� Accelerators: 16 nodes with 2 Volta GPUs (8@UC, 8@TACC)

� Maintenance, support and reserve

CORSA DP2000 SERIES SWITCHES� Hardware Network Isolation

� Sliceable Network Hardware

� Tenant controlled Virtual Forwarding Contexts (VFC)

� Software Defined Networking (SDN)� OpenFlow v1.3

� User defined controllers

� Performance� 10 Gbps within a site

� 100 Gbps between UC/TACC (Aggregated)

StandardCloudUnit

NETWORK HARDWARE

Chicago

Internet 2 AL2S, GENI, Future Partners

Austin

ChameleonCoreNetwork100Gbps uplink publicnetwork

(eachsite)

StandardCloudUnit

Corsa DP2400Corsa DP2400

StackedSwitches(LogicallyOne)

StandardCloudUnit

Corsa DP2200

100Gbps(Aggregate)

ISOLATED VIRTUAL SDN SWITCH� Isolated Tenant Networks

� BYOC– Bring your own controller: isolated user controlled virtual OpenFlowswitches (coming soon)

StandardCloudUnit

Corsa Switch

ComputeNode

(TenantA)

ComputeNode

(TenantA)

ComputeNode

(TenantB)

ComputeNode

(TenantB)

VFC(TenantA)

VFC(TenantB)

OpenFlowController(TenantB)

OpenFlowController(TenantA)

StandardCloudUnit

CHAMELEON: SDN EXPERIMENTS

� Chameleon Networking

� RENCI added to the team� Hardware Network Isolation

� Corsa DP2000 series

� OpenFlow v1.3 � Sliceable Network Hardware� Tenant controlled Virtual Forwarding

Contexts (VFC)

� Isolated Tenant Networks� BYOC – Bring your own controller

� Wide-area Stitching

� Between Chameleon Sites (100 Gbps)� ExoGENI� Campus networks (ScienceDMZs)

CorsaDP2400Switch

Internet 2 AL2S, GENI, Future Partners

Chicago

Austin

ComputeNode

(TenantA)

OpenFlowController(TenantB)

OpenFlowController(TenantA)

VFC(TenantA)

ComputeNode

(TenantA)

ChameleonCoreNetwork100Gbps uplink publicnetwork

ComputeNode

(TenantB)

ComputeNode

(TenantB)

VFC(Tenantb)

OUTLINE

EXOGENI: INTER-SLICE STITCHING

Public Internet

Starlight

Service Slice Client Slice

OUTLINE

EXOGENITO CAMPUS STITCHING

IPcore(L3) Circuit fabric

providers

DMZDTN

IPegress

I2/A2LS

OtherGENIsitesOthercampusesOtherfacilities

ControlplaneAPIs

Dukecampusboundary

L2egress

e.g.GENI-APIe.g.OSCARSe.g.Plexuse.g.ORCA

Duke University Software Defined Science Network (SDSN)Science DMZ

EXOGENITO CAMPUS STITCHING

Stitchport: Named meeting point linking a layer 2 circuit between ExoGENI and

external resources.

Stitchport Duke SDSN

MULTI-TESTBED EXPERIMENTS

Starlight

Client SliceService Slice

Client Campus

OUTLINE

CHAMELEON TO EXOGENI STITCHING

• Dynamic VLANs• Connectivity to

ExoGENI Stitchport

• ExoGENI slice• Dynamic Chameleon

Stitchport

Stitched L2 path

CHAMELEON TO EXOGENI STITCHING

StitchPort

Stitching between ExoGENI and Chameleon nodes

INTER-TESTBED EXPERIMENTS

Starlight

Client SliceService Slice

Client Campus

Starlight

Client CampusClient Slice

Starlight

Virtual SDXService Slice Client Slice

OUTLINE

CICI SAFE PROJECT

“Creating Dynamic Superfacilities the SAFE Way”Paul Ruth, Cong Wang, Mert Cevik, RENCI

Jeff Chase, YuanjunYao, Qiang Cao, Victor Orlikowski. Charley Kneifel, Duke Univeristy

Nick Buraglio, ESnet

NSF CICI Award #1642142

SUPERFACILITY

� Definition� Two or more existing facilities (e.g. instruments, compute resources, data repositories) using

high-performance networks and data management software in order to increase scientific output.

� Currently manually created � Superfacilities are purpose-built manually for a specific scientific application or community.

� Trust: “handshake model”

� Ideally automated� Advanced Science DMZs and federated Infrastructure-as-a-Service provide the technical

building blocks to construct dynamic superfacilities on demand.

SUPERFACILITY

� Definition� Two or more existing facilities (e.g. instruments, compute resources, data repositories) using

high-performance networks and data management software in order to increase scientific output.

� Currently manually created� Superfacilities are purpose-built manually for a specific scientific application or community.

� Trust: “handshake model”

� Ideally automated� Advanced Science DMZs and federated Infrastructure-as-a-Service provide the technical

building blocks to construct dynamic superfacilities on demand.

Trust also needs to be automated

SUPERFACILITIES THE SAFE WAYDuke Science DMZ Other Campus

ExoGENI Slice

IDS IDS IDS IDS

Virtual SDX

• Automating Superfacilites– Multiple domains– Friction free L2 paths

• Naked L2 paths are not secure– Handshake model of trust is not possible

• Virtual SDX (vSDX)– Distributed– Enforces SDX connectivity policy– Enforces client’s forwarding policy

(security, BGP, etc.)– Intrusion Detection System (Bro)

• SAFE: Secure Authorization for Federated Environments– Isolates applications from logic concerns

• Certificate discovery (DAGs)• Logic inference• Cryptography

– Logic scripting language• Slang (SAFE Language)• Based on Datalog

– Shared certificate repository• Stores statements and DAGs

SCIDAS

1PBStge/FIONA 1PBStge./FIONA 1PBStge./FIONA

Cost-AwareOptimize

iRODSShim (aaS)

PerfSONARShim (aaS)

API PerfSONARmapping

Requester

Orchestrator

Network

SCIDAS

Automated vSDX superfacility

THANK YOUpruth@renci.org

EXPERIMENTING WITH VIRTUAL SDXS USING CHAMELEON AND … · SDSN GENI Phys DMZ DTN IP egress BEN...

Documents