Date post: | 19-Dec-2015 |
Category: |
Documents |
View: | 218 times |
Download: | 4 times |
• E-Science
• Research Challenges
• Examples of Infrastructure for Network Research Experimentation• NLR• GENI• GLIF
• International Collaboration Experiences w/EnLIGHTened Computing
• Conclusions
Outline
Motivation
• E-science: global, large scale scientific collaborations enabled through distributed computational and communication infrastructure
• Combines scientific instruments and sensors, distributed data archives, computing resources and visualization to solve complex scientific problems
• In physics, molecular biology, environmental, Health, Entertainment, etc.
• E-Science Definitions Oxford E-Science Center• The Department of Trade and Industry defines e-Science as: "Science increasingly
performed through distributed global collaborations enabled by the Internet, using very large data collections, terascale computing resources and high performance visualizations"
• This essentially means that many areas of science currently using computing resources as part of their research, will soon have the ability to utilize more powerful computing resources across a new infrastructure commonly described as the 'grid'. Scientists will have access to very large data sets and perform real time experiments on this data. This will ultimately lead to scientists tackling the 'big scientific questions' hitherto unexplorable.
E-science
• Grid computing: main enabler of E-science• Grid is concerned with "coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations." (Ian Foster)
• Migration of the E-science community towards Grid Computing emerged from three converging trends;
i) Advances in optical networking technologies. Widespread deployment of the fiber infrastructure has led to low-cost, high-capacity optical connections.
ii) Affordability of the required computational resources through sharing. The increasing demand of computational power and bandwidth by the new e-science applications is proving to be a financially difficult and nearly impossible task unless resources are shared across research institutions.
iii) Need for interdisciplinary research. The growing complexity of scientific problems is driving the need for increasing numbers of scientists from diverse disciplines and locations to work together in order to achieve breakthrough results.
E-Science and Grid computing
• Korea’s HVEM • One of a kind in the world -
• Provide global access to
unique instruments for the purpose of advancing science for humanity • WEB service interface• High capacity optical network for output
Developing a Global E-science Laboratory (GEL)
The tasks that the HVEM users can perform:• Requesting the general operations of the goniometer, TEM, and CCD camera• Viewing the real-time video from the CCD camera• Accessing or manipulating the 2-D or 3-D images• Generating the workflow specification and requesting the
workflow to be executed• Searching the images or video files, papers, and
experiments in the databases or storages
Hyuck Han, Hyungsoo Jung, Heon Y. Yeom, Hee S. Kweon, and Jysoo Lee ”HVEM Grid: Experiences in Constructing an Electron Microscopy Grid”
• Governments world-wide promoting global E-science research programs• New Zealand, $43 million to establish the Advanced Network• United States: from Optiputer over Cyber-infrastructure to DDDAS , and DoE ($60M)
• Canada: CA*net4 and i-Infrastructure• Australia: e-Research Initiative (Started Oct 2004)• Europe: 7th framework (€1B/year), GEANT2 (€93M), Netherlight, Geodise, UK E-science, etc.
• Private sector setting a strong pace • Google building their own L1/L2 network• Multi $B investments from large businesses: IBM, HP, Intel,…
• Smaller businesses: Grid-based enterprise applications, data centers solution, commercialization of Globus toolkit,…
E-Science and Grid Opportunity
Network Requirements
• High bandwidth connectivity between supercomputers (teraflops+)
• Large file transfers, over long distances - rethink TCP (FAST)
• Applications/end-users/sensors/instruments requesting optical networking resources host-to-host connections
• Determinism (QoS), jitter and latency requirements (difficult to do with today’s Internet, and may be unfair to transfer terabyte of data to other apps)
• Coordination of network with computational and non-computational resources (CPU, databases, sensors, instruments )
New Demands on NetworksEmerging High-End applications
• Exchange data with sensors via potentially other physical resources. wireless
• Destination may not be known initially rather only a service is requested from source and the destination is derived from the request information
• Mechanisms for retrieving near-real-time information about network resources and network states
• Mechanism for both advance and fast on-the-fly reservation and set-up
• Low latency on-demand connection requests
• Policy and security enforcement in open scientific environments
New Demands on NetworksEmerging High-End applications (cont’d)
• 1000 channels per fiber….. Experimentation with 160G per channel
• Dark Fiber every where …. Paradigm shift on ownership– Fiber is much cheaper…US Headlines: Google buys Fiber
• All-optical switches are getting faster and smaller (ns switch reconfiguration)
• Control Plane protocols, SOA, continue to mature - but should be revisited
• Layer one Optical switches relatively cheaper than other technologies
• Electronic Dispersion Compensation • Fiber, optical impairments control, and
transceiver technology continue to advance while reducing prices
Advances in Optical Technologies(How do we take advantage?)
Research Challenges
• Coordination of resources per request for both on-the-fly and advanced reservations - Network resources is an integral part of the application’s request for shared resources
• Advanced reservation in distributed form - Borrow from ATM research
• Optimization of Resource Allocation
• Interdomain across Global Grid networks - network interdomain protocols, policies (management plane and control plane, Grid … WEB services )
• Dynamic and Adaptive on-demand use of end-to-end networking resources (requires near real-time feedback loop)- Identification of functions and interactions between the control plane, management plane, and Grid middleware
Research Challenges
• Monitoring information of resources - i) identification of information, ii) abstraction of information, and iii) frequency of updates
• Software algorithms to support multiple classes of software including highly-dynamic, workflow engines, data-driven and event-driven applications
• Rethinking the Behavioral Control of Networks • Control/management planes interacting
with middleware• Centralized vs. distributed functionality
Research Challenges (cont’d)
Resource Allocation
Resource ManagerCo-Scheduler
Resource Monitoring
ApplicationsEdge
RoutersWorkflowEngines
Application Abstraction Layer (API)
Policy
Translate app request to policy
•Discovery
•Performance
•Policy
For SLA
MonitoringPolicy
Feedback LoopAbstraction
Distributed Control: Control Plane• Infrastructure and distributed intelligence
that controls the establishment and maintenance of connections in the network, including protocols and mechanisms to disseminate this information; and algorithms for engineering an optimal path between end points.
Control Plane vs. Management Plane
Centralized Control: Management Plane •Management plane mechanisms rely on client/ server model, usually involving one or more management applications (structured hierarchically) communicating to each network element in its domain via a management protocol, (i.e., SNMP, Tl1, XML, etc).
Centralized vs. Distributed…
Key areas for Today’s Control Plane are:1) Provisioning 2) Recovery
Network Management
(Hierarchical )
NENENE
Migration
Centralized (vertical)
Network Management
NENENE
Distributed (Horizontal)
Protocols Protocols
NetworkBehavioral
Control
• Routing - Intra-domain and Inter-domain1) automatic topology and resource discovery 2) path computation (How do we use the infrastructure)
• Signaling - standard communications protocols between network elements for the establishment and maintenance of connections
• Neighbor discovery - NE sharing of details of connectivity to all its neighbors (very powerful tool)
• Local resource management - accounting of local available resources
Control Plane Functions
• Re-thinking control functionality in terms of (centralized or Distributed):
• Information exchanged
• Algorithms for path computation and recovery (CPU power vs. fast reaction time)
• Discovery and advertising of resources
• Scalability (frequency and amount of data exchange)
• Timing (reaction time of events)
• Interdomain interactions (Is BGP the solution or should it be centralized?)
• Policy enabled (where is it residing vs. executing)
Centralized vs. Distributed Behavioral Control of Networks
Examples of Infrastructure for Network Research Experimentation
•NLR•GENI•GLIF
NLRwww.nlr.net
courtesy of Tom West, CEO NLR
To advance the research, clinical and educational goals of members and other institutions by establishing and maintaining a nationwide advanced network infrastructure.
courtesy of Tom West, CEO NLR
National LambdaRail Mission
• Support experimental and production networks
• Foster networking research
• Promote next-generation applications
• Facilitate interconnectivity among high-performance research and education networks
courtesy of Tom West, CEO NLR
National LambdaRail Goals
GENIwww.geni.net
Facility Design: Key ConceptsGENI (Global Environment Network Innovations)
Slicing, Virtualization, Programmability
Mobile Wireless Network
Sensor Network
Edge Site
FederatedInternational Facility
Slide from: Guru Parulkar, CISE, NSF
GENI Facility GoalsEnable exploration of new network architectures, mechanisms, and distributed system capabilities
A shared facility that allows • Concurrent exploration of a broad range of
experimental networks and distributed services
• Interconnection among experimental networks & the commodity Internet
• Users and applications to “opt-in”
• Observation, measurement, and recording of outcomes enabled
Develop stronger scientific base
Slide from: Guru Parulkar, CISE, NSF
GLIFwww.glif.is
GLIF is a collaboration of institutions, organizations, consortia and country National Research and Education Networks (NRENs) who voluntarily share optical networking resources and expertise for the advancement of scientific collaboration and discovery.
GLIF's mission : to create and sustain a Global Facility that supports leading-edge capabilities based on new and emerging technologies and paradigms related to advanced optical networking to enable high-performance applications and services.
What is GLIF?
Global Lambda Integrated Facility
www.glif.is
Visualization courtesy of Bob Patterson, NCSA.
GLIF Automation?
NRENControl Plane
NRENControl Plane
NRENControl Plane
Network Management
Network Management
Network Management
Client A Client B
Grid middleware
Grid middleware
WEBServices ?
Mission: To agree on the interfaces and protocols to automate and use the control planes of the contributed Lambda resources to help users on a global scale access optical resources on-demand or pre scheduled.
GLIF Control Plane and Grid Middleware Integration wg
1. Work with GLIF Tech group top establish what are GLIF resources (GOLEs)
2. Defined Network Elements in RDF3. Software that reads RDF description4. Need to write to Google MAP APIs to draw resources
on a global bases5. Provide algorithm to compute path from broker
information6. Establish WEB services for connection services
GLIF Control Plane and Grid Middleware Integration wg
Thanks to Jereon Van Der Ham
EnLIGHTened ComputingInternational Collaboration
Experiences
EnLIGHTened Computing connectivity diagram with partners
Cisco/UltraLight wave
EnLIGHTened wave (Cisco/NLR)
LONI wave
Members:- MCNC GCNS- LSU CCT-NCSU-(Subcontract) RENCI
Official Partners:- AT&T Research- SURA- NRL- Cisco Systems- Calient Networks- IBM
NSF Project Partners- OptIPuter- UltraLight- WAN-in-LAB- DRAGON- Cheetah
International Partners •LUCIFER - EC•G-Lambda - Japan-GLIF
CHI
HOU
DAL
TUL
KAN
PIT
WDC
OGD
BOI
CLE
POR
DEN
SVL
SEA
Baton Rouge
Raleigh
To Asia To Canada To Europe
L.A.
San Diego
CAVE wave
Chicago
3 EU NRENs are partners + 3 national test-beds + 3 research networks in US and Canada + 5 expressed interest through LoIs
These community representatives are willing to monitor project progress, collaborate and exploit its results
Enlightened/LUCIFER
Sister Projects - Similar Goals
Testbeds
Japan’s G-Lambda research collaboration
Slide: Courtesy of Michiaki HayashiKDDI R&D Laboratories Inc.
Japan’s G-Lambda research collaboration
Slide: Courtesy of Michiaki HayashiKDDI R&D Laboratories Inc.
computeresource
GL Grid EL Grid
GLCRM
computeresource
HARCCRM
networkresource
GLNRM network
resource
HARCNRM
GLGRS
HARCAcceptor
GNS-WSI
GL-CRMI/F
GNS-WSIWrapper
GNS-WSI
HARC-CIFWrapper
GNS-WSI
GL-CRMI/F
HARC NIF
HARC NIF HARC CIF
HARC CIF
HARC NIF
GL-CRMI/F
HARC CIF
GL Request
ELRequest
Slide: Courtesy of Lina Battestilli
Two interfaces emerging1) For Network resources2) For Compute resources
Conclusions
Further Reference
IEEE Communications Magazine
Feature Topic
Optical Control Plane for Grid Networks: Opportunities,
Challenges and the
Vision
Guest Editors: Admela Jukan and Gigi Karmous-Edwards
March, 2006
Vol.44 No.3March 2006
An Optical Control Plane for The Grid Community
A Book written by the Community
Coming soon
Conclusions• Control Plane research is vital to meeting future generation NRENs - with
a strong focus on algorithms to meet the needs of driving applications
• Dynamic reconfigurability of L1/2 is essential to bring down cost and meet application requirements.
• Paradigm Shifts: i) ownership and control of network infrastructure, ii) network resources are treated as an integral Grid resource. - affect Interdomain policies
• The Research networks are taking these bold steps on GLIF, testbed infrastructures… apply lessons learned to production quickly.
• International Collaboration is a very Key ingredient for the future of Scientific discovery - The Optical network plays the most critical role in achieving this!
Yufeng Xin, Steve Thorpe, Bonnie Hurst, Lina Battestilli, Mark Johnson , John Moore, Ed Seidel, Gabriele Allen, Seung Jong (Jay) Park , Jon Maclaren, Andrei Hutanu, Lonnie Leger, Lavanya Ramakrishnan, Joel Dunn, Savera Tanwir, Harry Perros, Javad Boroumand, Russ Gyurek, Wane Clark, Kevin McGrattan, Rick Schlichting, John Strand, Matti Hiltunen, Gary Crane, Hank Dardy, Olivier Jerphagnon, Ron Mackey, John Bowers,Carla Hunt, Andrew Mabe, Gigi Karmous-Edwards
Acknowledgments
The Enlightened Team