Post on 16-Jul-2020
transcript
Performance investigation and comparison
between virtual networks and physical networks
based on Sea-Cloud Innovation Environment
Website: http://scie.ac.cn E-mail: scie@cstnet.cn
CANS 2015, Chengdu, Sep 21, 2015
2015/9/28
Outline
• Background
• Architecture
• Software & Hardware
• Deployment
• Demonstration
• Performance investigation
Background
• Sea-Cloud Innovation Environment, a national wide
testbed supported by the “Strategic Priority
Research Program - New Information and
Communication Technology”(SPRP-NICT) of the
Chinese Academy of Sciences, is aiming to build
an open, general-purpose, federated and large-
scale shared experimental facility to foster the
emergence of new ICT.
Background
•Providing shared and sliceable experimental facilities for academia and
industry to bridge the gap between visionary research and large-scale
experimentation.
•Establishing and practicing the methodology of experimentally -driven
innovation for the clean-slate architecture of ICT.
•Evaluating and validating new protocols, devices and research
achievements of SPRP-NICT.
Objective
2015/9/28
Outline
• Background
• Architecture
• Software & Hardware
• Deployment
• Demonstration
• Experimentation
Architecture
Experiment Topology Requests • SCIE portal
• Scie.ac.cn
• Resource control
framework
• Experiment
measurement
system
• SDN/VLAN-based
network slicing
2015/9/28
Outline
• Background
• Architecture
• Software & Hardware
• Deployment
• Demonstration
• Performance investigation
Software--Overview
SCIE Portal
Control Center
Resource Site
Experiment
Measurement
System
Resource
Management &
Control System
Topology
Editor
Experiment
Playground
Resource
Management
Authorization
& Accounting
Site manager
Resource
Control
module
Measurement
Module Site manager…….
Experiment Service System
SoftwareSCIE Resource Control Architecture
• Distributed resource control framework with one control center and many site managers
• Defining resource control interfaces, measurement interfaces to integrate different resource
• Light-weight VM management tool
SoftwareSCIE measurement System
• External VM
measurement without
any plug-in in VMs
• AMQP based control
message &
measurement data
transfer
• sFlow based network
traffic measurement
• MongoDB as Storage
Engine
SoftwareExperiment Service System
• Experiment life cycle Management
• Java & Python based experiment control library
• Topology and experiment process visualization
Hardware
Smart-Flow Switch
• OpenFlow 1.2
• GRE tunnel
• QoS supported
• 24*GE
• 1*10GE
• Four slots
• Line Card & UTM Card
Hardware
SCIE Rack
•Integrated network,
computing and storage
• Built-in site management
module
• Virtualization
• Dynamic scheduling
2015/9/28
Outline
• Background
• Architecture
• Software & Hardware
• Deployment
• Demonstration
• Performance investigation
Deployment
•Four contries&Seven cities &22 sites
•Data plane via GRE tunnel; Control plane via L3 network
•2234 cores, 1510TB storage, 512TB experimental data
2015/9/28
Outline
• Background
• Architecture
• Software & Hardware
• Deployment
• Demonstration
• Performance investigation
2015/9/28
Outline
• Background
• Architecture
• Software & Hardware
• Deployment
• Demonstration
• Performance investigation
• Performance investigation and
comparison between virtual
networks and physical networks
based on an advanced testbed
network
Deployment
• Three cities & 4 sites
• VM based on KVM
• Data transmission via GRE tunnel built
based on OVS
GRE Tunnel
VM1
VM8
VM2
...
VM1
VM8
VM2
...
Beijing, China
Beijing, China
Xinjiang, China
Michigan, US
Deployment
• Scenario– single-thread vs. multi-thread
• For each scenario
– the intra-domain case, from Beijing to Xinjiang in China
– the inter-domain case, from Beijing in China to Michigan in US
• Extensive performance evaluation tests
– UDP and TCP traffic in idle and non-idle period
• Key performance metrics– For UDP traffic
• round trip time (RTT), throughput, packet loss, and jitter
– For TCP traffic
• RTT and throughput
Experimental results and analysis
• UDP traffic
– throughput0
10
20
30
40
50
60
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Thro
ugh
pu
t (M
bit
s/se
con
d)
Seconds
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
0
10
20
30
40
50
60
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Thro
ugh
pu
t (M
bit
/s)
Seconds
VM(single-thread,non-Idle)
PHY(single-thread,non-Idle)
VM(multi-thread,non-Idle)
2
3
4
5
6
7
8
9
10
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Thro
ugh
pu
t (M
bit
/sec
on
d)
Seconds
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
2
3
4
5
6
7
8
9
10
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Thro
ugh
pu
t (M
bit
/sec
on
d)
Seconds
VM(single-thread,non-Idle)
PHY(single-thread,non-Idle)
VM(multi-thread,non-Idle)
(a) UDP Bandwidth BJ-XJ (Idle) (b) UDP Bandwidth BJ-XJ (Non-Idle)
(c) UDP Bandwidth BJ-MI (Idle) (d) UDP Bandwidth BJ-MI (Non-Idle)
• The virtual network is very similar to single-thread physical network very similar to single-thread physical network scheme in intra-domain
and in inter-domain
the deviation less than 0.35% in intra-domain
about 0.21% in inter-domain
• The deviation is stable (multi-thread virtual network vs. single-thread
physical network)
Experimental results and analysis
• UDP traffic
– packet loss rate(a) UDP Packet Loss BJ-XJ (Idle) (b) UDP Packet Loss BJ-XJ (Non-Idle)
(c) UDP Packet Loss BJ-MI (Idle) (d) UDP Packet Loss BJ-MI (Non-Idle)
• Single-thread virtual network and single-thread physical network are
better than multi-thread virtual network scheme in all cases
• The deviation is less than 1% (multi-thread virtual network vs. single-
thread physical network) 0.17% in intra-domain
about 0.23% in inter-domain
0
5
10
15
20
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Pa
cket
Lo
st R
ate
(%
)
Seconds
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
0
5
10
15
20
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Pa
cket
Lo
st R
ate
(%
)
Seconds
VM(singel-thread,non-Idle)
PHY(singel-thread,non-Idle)
VM(multi-thread,non-Idle)
0
5
10
15
20
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Pac
kets
Lo
st R
ate
(%)
Seconds
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
0
5
10
15
20
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Pack
ets
Lost
Rat
e (%
)
Seconds
VM(single-thread,non-Idle)
PHY(single-thread,non-Idle)
VM(multi-thread,non-Idle)
Experimental results and analysis
• UDP traffic
– jitter (a) UDP Jitter BJ-XJ (Idle) (b) UDP Jitter BJ-XJ (Non-Idle)
(c) UDP Jitter BJ-MI (Idle) (d) UDP Jitter BJ-MI (Non-Idle)
• The jitter of inter-domain environment was higher than that of intra-
domain environment with about 13.5%
• Jitter in Multi-thread virtual network is higher than single-thread physical
network 2.6% higher in intra-domain environment
10.2 % higher in inter-domain environment
• The distance is the main factor affecting the jitter
0
0.2
0.4
0.6
0.8
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Jitt
er (
ms)
Seconds
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
0
0.2
0.4
0.6
0.8
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Jitt
er (
ms)
Seconds
VM(single-thread,non-Idle)PHY(single-thread, non-Idle)VM(multi-thread, non-Idle)
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Jitt
er (
ms)
Seconds
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
Jitt
er (
ms)
Seconds
VM(single-thread,non-Idle)
PHY(single-thread,non-Idle)
VM(multi-thread,non-Idle)
Experimental results and analysis
• UDP traffic
– RTT(a) a UDP RTT BJ-XJ (Idle) (b) UDP RTT BJ-XJ (Non-Idle)
(c) UDP RTT BJ-MI (Idle) (d) UDP RTT BJ-MI (Non-Idle)
• Single-thread physical network is more stable and smaller than
single-thread and multi-thread virtual network • RTT is most stable and lowest in all cases in single-thread physical network
• The RTT of single-thread physical network is about 3.3% smaller than single-
thread and multi-thread virtual network schemes
• The RTT is not so stable in non-idle scenario, the deviation between
idle and non-idle cases is 0.1%
• The background traffic is the key influence factor of RTT and the
performance of network experiment in virtual network environment
50
55
60
65
70
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
RTT
(m
s)
Seconds
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
50
55
60
65
70
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
RTT
(m
s)
Seconds
VM(single-thread,non-Idle)
PHY(single-thread,non-Idle)
VM(multi-thread,non-Idle)
200
210
220
230
240
250
260
270
280
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
RTT
(m
s)
Seconds
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
200
220
240
260
280
300
320
340
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
RTT
(m
s)
Seconds
VM(single-thread,non-Idle)
PHY(single-thread,non-Idle)
VM(multi-thread,non-Idle)
Experimental results and analysis
• TCP traffic
– throughput(a) TCP Throughput BJ-XJ (Idle) (b) TCP Throughput BJ-XJ (Non-Idle)
(c) TCP Throughput BJ-MI (Idle) (d) TCP Throughput BJ-MI (Non-Idle)
• Throughput in Idle case is better than non-idle case
• Throughput in intra-domain environment is higher than that in inter-
domain environment
• After slow start, the throughput of single-thread physical network is
higher than that of single-thread and multi-thread virtual network
scheme
• The throughput of single-thread virtual network is the most stable and
is very similar to single-thread physical network in non-idle case
0
50
100
150
200
250
300
1 11 21 31 41 51
Thro
ugh
pu
t (M
bit
/sec
on
d)
Samples
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
0
50
100
150
200
250
300
1 11 21 31 41 51
Thro
ugh
pu
t (M
bit
/sec
on
d)
Samples
VM(single-thread,non-Idle)
PHY(single-thread,non-Idle)
VM(multi-thread,non-Idle)
0
20
40
60
80
100
1 11 21 31 41 51
Thro
ugh
pu
t (M
bit
/sec
on
d)
Samples
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
0
20
40
60
80
100
1 11 21 31 41 51
Thro
ugh
pu
t (M
bit
/sec
on
d
Samples
VM(single-thread,non-Idle)
PHY(single-thread,non-Idle)
VM(multi-thread,non-Idle)
Experimental results and analysis
• TCP traffic
– RTT(a) TCP RTT BJ-XJ (Idle) (b) TCP RTT BJ-XJ (Non-Idle)
(c) TCP RTT BJ-MI (Idle) (d) TCP RTT BJ-MI (Non-Idle)
• Stability of RTT single-thread physical network > single-thread virtual network > multi-thread
virtual network
• In idle traffic case, the deviation: 0.23%(single-thread physical network vs. single-thread virtual network)
0.55%(single-thread physical network vs. multi-thread virtual network)
• In non-idle traffic case, the deviation: 3.7% (single-thread physical network vs. single-thread virtual network)
4.5% (single-thread physical network vs. multi-thread virtual network)
50
52
54
56
58
60
62
64
66
68
70
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
RTT
(m
s)
Seconds
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
50
52
54
56
58
60
62
64
66
68
70
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
RTT
(m
s)
Seconds
VM(single-thread,non-Idle)
PHY(single-thread,non-Idle)
VM(multi-thread,non-Idle)
220
230
240
250
260
270
280
290
300
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
RTT
(m
s)
Seconds
VM(single-thread,Idle)
PHY(single-thread,Idle)
VM(multi-thread,Idle)
220
230
240
250
260
270
280
290
300
1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341
RTT
(m
s)
Seconds
VM(single-thread,non-Idle)
PHY(single-thread,non-Idle)
VM(multi-thread,non-Idle)
Experimental results and analysis
• Conclusion• 1) the RTT and jitter of virtual networks have little deviation
from physical networks,
• 2) the throughput and packet loss rate of virtual networks are
similar to physical networks,
• 3) the performance of single-thread virtual network is more
similar to the existing physical network than the multi-thread
virtual networks,
• 4) the multi-thread virtual networks have certain deviation from
physical networks, but the deviation is stable and shows
certain characteristics.
• Thus, it is possible to get the performance
of real physical networks in virtual
networks