Date post: | 07-Jan-2017 |
Category: |
Technology |
Upload: | opnfv |
View: | 420 times |
Download: | 1 times |
Agenda
• Network Latency on NFV
• KVM4NFV and Network Latency
• Low Latency Testing Enhancement on OPNFV
Low network latency is important
• Voice quality for VOIP service - 150ms (tolerable by ears)
• LTE Cloud RAN (Radio Access Network): 1 ms
• - With 5G, the latencies need to be 1/10th of that
• A VNF that receives 10 Gbit/s network traffic of 256 byte packets will receive 3900 packets in 1 ms
• Profit or loss in algorithmic trading requires microsecond latencies
HW
Network Latency and NFV
Open vSwitch VNF1
DPDK Apps
virtio NIC
Linux
HWNIC NIC
vHost
PMD
QEMU
KVMNIC
DPDK Apps
Hypervisor and Network Latency
• Virtualization overhead
• vCPU pre-emption/scheduling
• Virtual switch scheduling/service
• Resource allocation
KVM4NFV Project Introduction
• Join OPNFV on B release
• 3 focus area• Low Latency
• Fast live migration
• Inter-VM communication
• Upstream based effort
Today’s focus
Network Latency with KVM4NFV – B release
• Timer latency (Average: 5 us, Maximum: 15 us)
• Interrupt latency (Maximum: 20 us)
• Maximum guest vCPU pre-emption period (Maximum: 10 us)
• Pre-emption periods in total within 1 ms window (Maximum: 20 us)
Network Latency with KVM4NFV – C release
• Real network application• SR-IOV Packet forwarding
• DPDK Packet forwarding
• Add complexity to the test environment• Stress workload/Noicy neighbor
• Kernel update• From 4.1 LTS to 4.4 LTS
VNF1
DPDK L2 FWD
Linux
HWNIC1 NIC2
TrafficGen
NIC1 NIC2
Packet Forwarding Latency Enhancement
Kernel Average Latency
Maximum Latency
General Kernel
289.6 us 373.9 us
KVM4NFV 19.6 us 101.8 us
• Guest
• NIC devices assigned to guest directly
• DPDK BasicFwd workload
• Host: KVM4NFV
• TrafficGen: 64 byte packet with 10G line rates throughput for 24 hours
VNF1
DPDK Apps
virtio NIC
Hypervisor
Open vSwitch
User Level User Level
virtio NIC
HWNIC NIC
vHost
PMD PMD
vHost
Packet Latency Enhancement (Phy-VM-Phy)
Test Environment:• VM with 4 vCPU• 2 vCPU for data path• PMD and vCPU threads are pinned to isolated CPUs• 2 rx/tx queues per NIC port• 2 rx/tx queues per vHost user port
Testpmd on VNF• 10 minutes testing• 30% line rates and 100% line rates
Packet Latency Enhancement (Phy-VM-Phy)
64 128 256 512 1024 1280 1518
Average (30% ) 14.2 11.2 10.8 10.6 11.4 10 10
Maxium (30%) 31.3 29.9 33.7 29.2 36.1 33 35.8
Average(100%) 25.9 28.8 39.5 46.1 37.7
Maxium(100%) 38.5 103.4 109.5 115 115.2
0
20
40
60
80
100
120
140
Average (30% ) Maxium (30%) Average(100%) Maxium(100%)
Based on 2 hours of tests
HW
CAT and Isolation
Linux KVM
core core
LLCCPU
Memory
VNFStress• LLC (Last Level Cache) is shared
by all Cores on the CPU
• LLC noise impact VNF• Local LLC hit: 40~75 cycles *
• Local DRAM access: 60ns
*http://software.intel.com/sites/products/collateral/hpc/vtune/performance_analysis_guide.pdf
LLC LLC
CAT and Isolation
0
1000000
2000000
3000000
4000000
5000000
6000000
7000000
8000000
0 3 6 9 1215182124273033363942454851545760636669727578818487
Occ
ura
nce
s
Latency
Cyclictest on Idle System
0
200000
400000
600000
800000
1000000
1200000
1400000
1600000
0 3 6 9 1215182124273033363942454851545760636669727578818487
Occ
ura
nce
s
Latency
Cyclictest + stress load with CAT disabled
0
1000000
2000000
3000000
4000000
5000000
6000000
7000000
8000000
0 3 6 9 1215182124273033363942454851545760636669727578818487
Occ
ura
nce
s
Latency
Cyclictest + stress load with CAT
Low Latency Testing on OPNFV
• A automatic latency test system is important• Cover different scenarios
• Catch regression
• But challenges• Complex environment setup
• Openstack or no openstack• Not ready for environment setup
• Analysis not easy with openstack
• Resource requirement• 24 hours testing for each scenario
Target of latency test
• Easy to support different scenarios• Cyclictest->packet forwarding->…..
• Flexible testing scenarios• 24 hours release testing
• Short-term patch testing
• Automatic data publish
Low Latency Test Status
Patch Submission
Artifact Build
Gerrit/Jekins Octopus
Verify or Testing
Job Dispatch
OctopusPharos
Yardstickvsperf
See discussion on https://etherpad.opnfv.org/p/nfv-kvm-test
Data publish
?