Post on 11-Apr-2018
transcript
Grid Tutorial
Networking
Laukik ChitnisSanjay Ranka
Outline
MotivationKey Issues and ChallengesEmerging protocols
– DWDM– MPLS, GMPLS
Network Infrastructure– Internet2 Abilene and HOPI– NLR and FLR– Gloriad
End-to-end utilization of line speeds– Data transfers– Provisioning bandwidth– scheduling usage of network resources
Co-scheduling with compute and storage resources
Motivation
Exchange of large amounts of data
– Collaborative data analysis in HEP
For LHC experiments, CERN would host only about 11% of the data
Large quantities of data flow in the Gridprocessing power at CERN would comprise only about 17% of the total
Data needs to be dispersed geographically
– Multiple applications sharing the same infrastructure, but requiring QoS guarantees
Experiments in Data Fusion (ITER), Astronomy, Medical Sciences
– High Definition video conferencingRequires provisioning and QoSguaranteese.g. VRVS
Types of data transfers in e-Science
Scheduled transfer of data– Explicit request to transfer data
by end-user– Automatic replication of files
and datasets by storage managers
– Data produced at source (like LHC experiments) to be disseminated to various destinations
E.g. bulk data transfers in e-Science
– LHC– e-VLBI
Unscheduled data movement– Interactive analysis of
experiment results– Remote visualization of
datasets
Amount of data transferred not usually large
– Else, it makes sense to schedule the transfer
e.g.– GAE (Grid enabled analysis
using distributed services)– Root
Key Issues for Networking in Grid Computing
1. Traffic engineering
Co-existence of circuit-oriented services (like protocols specialized for long-haul, high-speed bulk data transfers) with best-effort TCP-based transport protocols used in the IP network.
2. Decentralized ownership of Networks
Networks not owned by specific groupsManaged by multiple organizations like Internet2, ESNetData flows have to co-exist with other data flows in multiple domain networks
Key Challenges for Networking in Grid Computing
1. Deploying networks with large wire-speeds
– Encompassing geographically distributed sites– Technologies that enable fast transfer of bulk data
2. Effective end-to-end utilization of these speeds
– Enabling data transfers to achieve speeds as close to the wire-speeds as possible
– QoS guarantees for services
Key protocolsin emerging networks
Layer 1–SONET/SDH–DWDM
IP over DWDMLayer 2/3
–MPLS and GMPLS
Early years
SONET/SDH at Layer 1– infrastructure introduced in the early 1990s to support
traditional time-division multiplexing (TDM)-based data and voice services.
– efficient multiplexing of lower-speed TDM circuits such as T1/E1 and T3/E3 to higher-speed OC-3 and OC-12 trunks for transport across service providers' core networks.
– Supports three critical functionsGrooming
– aggregating IP traffic to utilize the available bandwidthprotection and restoration; and thorough operational support (such as alarming and performance monitoring).
Multiplexing Wavelengths
In the latter part of the 1990s, DWDM emerged as a way to significantly increase the efficiency of the installed fiber plant by allowing transmission of multiple wavelengths over a single physical fiber. Another level of multiplexing and demultiplexing at the optical level to support greatly increased bandwidth at the core of the networkThe SONET/SDH layer mapped into wavelengths at the DWDM transport layer to be carried across the core long-haul networks spanning regions and countries in many cases.
Wavelength Division Multiplexing
WDM enables the utilization of a significant portion of the available fiber bandwidth
Allows many independent signals to be transmitted simultaneously on one fiber, with each signal located at a different wavelength
The more number of different wavelengths (colors) are transmitted at once, the more complex the components (Mux, switch, DeMux) need to be
– Coarse WDM systems usually support 16 channels per fiber
– DWDM (Dense WDM) can transmit 40 or 80 channels
Source: rad.com
Routing in Optical networks
Two types of traffic entering and exiting a typical service provider point of presence (POP) – Router-terminated traffic
IP traffic that needs a Layer 3 lookup at the POPriding a wavelength that will terminate on a router.
– "pass-through" (or transient) trafficstays in the transport domain and bypasses the router to travel on to an adjacent POP in the service provider's core network.
Typical operations at PoP
Typical operations at PoP– Incoming traffic composed of colored wavelengths multiplexed through
DWDM on to a physical fiber– Fed to DWDM demultiplexers that convert it to different wavelengths– Each wavelength passed through a transponder that converts it to short
reach wavelength (grey light)optical-to-electrical-to-optical (OEO) conversion is used because historically short-reach optics have been used for connectivity inside the POP environment.
– Grey light is then typically fed into a short-reach interface on a SONET/SDH cross-connect
– The SONET/SDH cross-connect then feeds the 10 Gbps to the router – Router does the following
performs performance monitoring at Layer 1 through Layer 3, monitors for LOS so it can perform MPLS Fast Reroute (FRR) restoration, and performs a Layer 3 and above lookup to route the packet to its destination.
IP-over-DWDM Interconnect Model in Today's Network
Cross-connects v/s Manual patching
– Capacities are utilized today. Since aggregation is not required, cross connects function only as patch panels
– However, with manual patching, any changes require human interference for reconfiguration
In both cases, OEO conversions and the associated electrical processing result in an additional cost in terms of space
Source: Cisco whitepaper
IP over DWDM
IP over DWDM
Cisco CRS-1– Family of routers offering IPoDWDM
Advantages– Integrated transponders
Lesser independent components that can fail better reliability– Provides S-GMPLS
Single IP control plane management providing service flexibility and lower operational expenditure
– Segmented or Integrated Management modelBetter provisioning
– ROADMs (Reconfigurable Optical Add Drop Multiplexers)Eliminate O/E/OBetter reliability, more flexibilityLower Operational expenditure
Current routers offering similar capabilities– Cisco 7600 series, Force10
Layer 2/3 convergence
Layer 2– Switching of frames
Layer 3– Routing of IP packets based on destination IP address
Motivation for convergence– Lookup of destination IP address to determine next hop in the core (even
within an Autonomous System) is expensive at each router– ATM-like flows desirable guaranteeing certain Quality of Service
Connection oriented service over connectionless networks
Recent trends– MPLS and GMPLS– Recent work in progress for dynamic provisioning of Inter-domain paths
MPLS
Motivation– Speed up packet forwarding – Traffic Engineering in IP networks
How to do it?– Convert the connectionless IP into a
connection-oriented network– Path between source and
destination is pre-calculated based on specifications provided by the user
MultiProtocol Label Switching– Use labels rather than address look-
up for determining the next hop of a packet/frame
To speed up forwarding– Use tables to store QoS parameters
For traffic Engineering
MPLS
LSP – Label switch pathLSR – Label switch routerLIB – Label Information base
Picture source: netcraftsmen.net
Advantages– Ability to place IP traffic on
predefined paths– Guarantees can be given
for Bandwidth and other differentiated QoS
MPLS protocol suite
Traditional IP routing protocols and their extensions for traffic engineering– Open Shortest Path First (OSPF)– Intermediate System to Intermediate System (IS-
IS)
Extensions to existing signalling protocols– Label Distribution protocol (CR-LDP) – Resource Reservation protocol (RSVP)
Generalized MPLS (GMPLS)
Extends control plane to other network types– Provides Signaling and Routing to dissimilar network types
such as packet (IP), time (TDM networks) and optical (WDM networks)
Advantages– Automating end-to-end provisioning of connections across
multiple networksManage establishment and release of Label Switched Paths (LSPs) spanning across networks
– Simplifying network managementAllows admins to automate the provisioning and management of networks and lowers the cost of operations
Source: www.iec.org
GMPLS protocol suite
Signaling CR-LDP, RSVP-TE
For establishment of LSPs(support for generalized labels, and bidirectional LSPs)
Routing OSPF-TE, IS-IS-TE
Routing protocols for auto-discovery of network topology, advertisement of parameters such as bandwidth availability
Link Management
LMP Control channel management, Link connectivity verification, fault isolation
MPLS and GMPLSin a nutshell
Multiprotocol Label Switching– Route at edge, switch in core– Ultra fast forwarding – IP Traffic Engineering
Constraint-based Routing– Virtual Private Networks
Controllable tunneling mechanism– Voice/Video on IP
Delay variation + QoS constraints
Generalized Multiprotocol Label Switching– enhances MPLS architecture by the complete separation of the
control and data planes of various networking layers.
Network InfrastructureDeploying networks with large wire-speeds
–The National LambdaRail (NLR) initiativeAnd the Florida LambdaRail (FLR)
–Initiatives by the Internet2 communityAbileneHOPI
–Gloriad (Global Ring Network for Advanced Application Development)
NLR
1st transcontinental Ethernet networkHigh-speed network over fiber-optic lines
Florida LambdaRail
Over 1,540 miles of dark fiberDense wave division multiplexing (DWDM)-based optical footprint using Cisco Systems’ 15454 optical electronic systems
– capacity of 32 wavelengths per fiber pair.
– each wavelength supports transmission upto 10 Gbps.
Ethernet based MPLS built on top of the optic infrastructure
Internet2and Abilene
Internet2– Consortium to provide
scalable, sustainable, high-performance networking in support of the research universities of the United States.
Abilene– OC-192c over unprotected
DWDM waves with SONET framing
– Backbone – Shared packet infrastructure
Abilene provides an IP connection over infrastructure rented from commercial backbone providers
– NRL offers a complete fiber infrastructure on which researchers can build their own Internet Protocol networks.
– an arrangement that ultimately limits research possibilities.
Hybrid Optical and Packet Infrastructure
Motivation: Increased demands for deterministic paths
Hybrid of shared IP packet switching and aggressive use of dynamically provisioned optical lambdas.
– The packet based infrastructure is the usual IP infrastructure
– The circuit infrastructure can be viewed as a set of paths between nodes
may or may not connect to the packet infrastructure
HOPIPossible types of circuit infrastructure
Node type Path Attributes
Optical switches Raw lambdas (light paths)
Color, spacing (No digital data)
SONET Add-Drop Multiplexer
SONET channels Number of channels, channel size
Ethernet switches VLANs Dedicated bandwidths
IP routers MPLS tunnels Dedicated bandwidths
Current HOPI network topology
Full scale optical switching on raw waves does not existUtilizes Ethernet switches and VLANs (may be with MLPS tunnels) to model optical capabilities
GLORIAD
Fiber-optic ring of networks around the northern hemisphereHigh-speed computer network
– connects scientific organizations in Russia, China, United States, the Netherlands, Korea and Canada.
Bandwidth of up to 10 Gbit/svia OC-192 links
– e.g. between KRLight in Korea and the Pacific NorthWest GigaPOP in the United States.
Source: Gloriad
Global Lambda Integrated Facility
International VO promoting the paradigm of lambda networking. Integrated facility
– support data-intensive scientific research
– supports middleware development for lambda networking.
End-to-end utilization of network resources
DRAGONUltralight
–Vinci–LISA–FDT–MonALISA
TeraPaths
Dynamic Resource Allocation in GMPLS Optical Networks (DRAGON)
DRAGON is a research and experimental framework for high performance networks
– Motivation: e-Science applications need network services beyond the typical best-effort infrastructures
Deploying infrastructure that enables dynamic provisioning of network resources to establish deterministic paths
– Multi-domain provisioning of traffic engineering paths– Distributed control plane across heterogeneous network technologies– Includes mechanism for Authentication, Authorization and Accounting
(AAA) – Enables Scheduling across domains
Reference implementation in Washington, DC area
DRAGON Virtual Label Switch Router
Provides GMPLS protocols to switching elements without native GMPLS capabilityA small unix-based PC running GMPLS control planeActs as proxy agent for switching device
– Converts events into commands native to the local switching element
Primarily used in DRAGON for controlling Ethernet switches via the GMPLS control plane
DRAGON Network Aware resource Broker (NARB)
Network Aware Resource Broker (NARB)– Exchanges interdomain topology – Stores topology in OSPF-TE database
Allows for abstraction of topology– Performs intra-domain and inter-domain path
First calculates path based on interdomain abstractionsInitial path computation result can be expanded to high-fidelity path via coordination of NARBs in different domains
– Maintain Traffic Engineering Database and associated AAA and scheduling information
3D TEDB– GMPLS TE constraints, AAA constraints, scheduling constraints
Policy-based provisioning using 3D Resource Computation element
DRAGON NARBs
Inter-domain path computation
NARB
End System
NARB
NARB
End System
AS 1AS 2
AS 3
Transport Layer Capability Set ExchangeControl plane
Data plane More on DRAGON: http://dragon.maxgigapop.net/twiki/bin/view/DRAGON/Overview
UltraLight
Promotes network as an actively managed component
– Along with storage and compute resources
Single interface for end applications to access collection of network servicesOptimize transfer performance for individual applicationsGlobal optimization for all applications transferring data at a given time
Dynamically evolving network core on other backbones such as NLR, HOPI, Abilene and ESnet.
Virtual Intelligent Networks for Computing Infrastructures
Multi-agent system for light path provisioning based on dynamic discovery of the topology in distributed networksAdds a new level of data transfer predictibility
– Use provisioning and reservation of network resources
Path Discovery service– Provides the scheduler with the best suited
path(s) between source and destination withBandwidth informationEarliest reservation timeMaximum duration of b/w guarantee
Responsible for setting up the required path at the request of the scheduler
VINCI Architecture
VINCI
Components– Transfer scheduler
Provides uniform interface for requesting network resourcesProvides transfer classes and prioritiesInteracts with Authentication, Authorization and Accounting services
– End host agents utilize LISA
– Network servicesPath discovery
– Distributed set of Monitoring and Alert toolsMonALISA end-to-end monitoring
– Transfer ManagementEfficient data transfers across networks using FDT
The Functionality of the VINCI System
Layer 3
Layer 2
Layer 1
Site A Site B Site C
MonALISAML AgentML Agent
MonALISAML AgentML Agent
MonALISAML AgentML Agent
ML proxy servicesML proxy services
Agent
Agent
Agent
Agent
ROUTERS
ETHERNETLAN-PHYor WAN-PHY
DWDMFIBER
Agent
Source: GridNets06 Ultralight talk
Fast Data Transfer
Application for Efficient Data Transfers – capable of reading and writing at disk
speed over wide area networks with standard TCP
can be used to stream a large set of files across the network Uses appropriate-sized buffers for disk I/O and for the network Transfers data in parallel on multiple TCP streams, when necessary Uses independent threads to read and write on each physical device Streams a dataset (list of files) continuously, using a managed pool of buffers through one or more TCP sockets.
Work in progress for integrating FDT with dCache
Experiments at SC07 demonstrated sustained data transfer speeds of 18 Gbps
Other initiatives
OSCAR, CheetahTeraPaths
– Route planning with MPLS
BNL TeraPaths project
General key issuefor doing e-Science across multi-domain networks
Control plane interaction between domains for scheduling, routing and circuit provisioning
Multi-domain provisioning flowOverview
Source: Internet2.edu