11
THOUGHTS ON SDN IN DATA INTENSIVE SCIENCE APPLICATIONS Artur Barczyk/Caltech Internet2 Technology Exchange Indianapolis, October 30 th , 2014 October 29, 2014 [email protected] 1
THOUGHTS ON
SDN
IN DATA INTENSIVE SCIENCE
APPLICATIONS
Artur Barczyk/Caltech
Internet2 Technology Exchange
Indianapolis, October 30th, 2014
October 29, 2014 [email protected] 1
• LHC experiments now moving 100+ PB per year
• Main driver of R&E network bandwidth utilization over the
past decade
• LHC restart in Spring 2015, expect traffic to grow
HEP context - for this talk
October 29, 2014 [email protected]
Th
rou
gh
pu
t(
MB
/s)
TransferThroughput20 0 9-0 1-0 10 0 :0 0 t o20 14-0 2-0 8 0 0 :0 0 UT C
CA CERN DE ES FR IT ND NL RU TW UK US
Destinations
2009-0
1
2009-0
2
2009-0
3
2009-0
4
2009-0
5
2009-0
6
2009-0
7
2009-0
8
2009-0
9
2009-1
0
2009-1
1
2009-1
2
2010-0
1
2010
-02
2010
-03
2010-0
4
2010
-05
2010
-06
2010
-07
2010
-08
2010
-09
2010-1
0
2010
-11
2010
-12
2011
-01
2011
-02
2011
-03
2011
-04
2011
-05
2011
-06
2011
-07
2011
-08
2011
-09
2011
-10
2011
-11
2011
-12
2012
-01
2012-0
2
2012-0
3
2012
-04
2012-0
5
2012-0
6
2012-0
7
2012-0
8
2012-0
9
2012
-10
2012
-11
2012
-12
2013
-01
2013-0
2
2013-0
3
2013
-04
2013-0
5
2013-0
6
2013-0
7
2013-0
8
2013-0
9
2013
-10
2013
-11
2013
-12
2014
-01
0 k
10 k
2.5k
5k
7.5k
ATLAS data transfer rates, monthly average
LHCONE VRF
• Change in Computing Models towards
– more flexibility (source/destination pairings)
– more dynamic (popularity based caching)
– remote access
• Different flow characteristics and patterns
– Data production
– Data processing ; cached vs remote access
• One constant: Will remain massively distributed, WAN performance
will be key to success!
New data movement and access patterns
October 29, 2014 [email protected]
T1
T1
T1
T1
T2T2
T2
T2T2
Tier-1
Tier-1
Tier-1Tier-1
Tier-2Tier-2
Tier-2
Tier-2 Tier-2
Tier-0
Tier-2 Tier-2Tier-2 Tier-2
Tier-2
CAF
Tier-2 Tier-2
Tier-1 Tier-1 Tier-1
• Leveraging provider diversity in LHCONE:
– many NRENs, providing multiple paths with multiple
• Standard network forwards allpackets for a given subnet onthe same path– Multipath configurations not trivial
– Especially when coupled with multiple administrative domains
• SDN approach allows to build a flexible yet robust and deterministic forwarding scheme
• Similarities with SDX (general) concept
New capability through SDN example
October 29, 2014 [email protected]
Slides from LHCONE workshop February 2013
• Provision capacity between OpenFlow switches based on
real-time requirements
• Approach in the OLiMPS project: flow management in
OpenFlow controller
triggers circuit requests
to OSCARS controller
• I.e. create a topology
optimizing the load
distribution in the network
• Future: couple with data
transfer application
OpenFlow + Dynamic Circuits
October 29, 2014 [email protected]
• Fact: Not all flows/users/application_requirements are the same
• A (the?) key value proposition of SDN in R&D networks:
– it allows to provide not only differentiated, but tailored services;customized to the particular need of a research community
– Can be done with minimal amount of effort
• (once the system is built)
• HEP: Example of a vision could be to differentiate between data production and transfer flows (“elephants”) and analysis data flows in remote access scenarios (“mice”), e.g.
– “old model”: not the same sites (Tier1/2/3 definition!)
– “new model”: roles defined on temporal varying requirements
– not the same capacity and access latency requirements
– not the same resiliency requirements
Another SDN Value Proposition
October 29, 2014 [email protected]
• At the computing sites: Network Virtualization
– Scalability, flexibility, robustness, security (think
ScienceDMZ here)
• In the WAN
– flexible services
• But also through
A programmatic interface (TBD) between application and
the network
– Tight integration; direct feedback loop -> reactive system
– Increased predictability; reduced distribution tails
– Dynamic workflow optimization
– Increased efficiency
How can HEP profit from SDN?
October 29, 2014 [email protected]
• BoD aka Dynamic Circuit Networks, aka Lightpaths, aka…
• Is a form of Software Defined Networking
– Circuits are configured by a controller
– Typically implemented as a (central) Domain Controller
• A proof-of-concept “experiment” is being built by LHCONE
– in collaboration with GLIF AutoGOLE efforts
– Interconnect a small number of sites initially
– Interface into CMS and ATLAS data movement and workflow management
• But really, it will be a first step towards a tight integration between network and applications through broader SDN concepts
SDN and Bandwidth on Demand
October 29, 2014 [email protected]
• For data intensive science applications, SDN has the
potential to enable
– Better application performance
– More determinism in workflows
– Resource optimization
• In order to harness the full potential of SDN, we need
– a good application-network interface
– multi-domain capability
– support in the R&E networks end-to-end
– coordinated effort between the service providers and the
scientists developing their applications
Conclusions
October 29, 2014 [email protected]
