1
Performance Evaluationof WebRTC-basedVideo Conferencing
IFIP WG 7.3 Performance 2017Bart Jansen
Delft University of Technology– Bart Jansen– Fernando Kuipers
Columbia University– Timothy Goodwin– Varun Gupta– Gil Zussman
2
WebRTC
• Web Real Time Communications• Audio, Video and Data communication• Standard by W3C and IETF since 2012
• Benefits– Plugin less– Peer to peer– Cross browser/platform– Easy to use (API)
3
Research motivations
• WebRTC is a new emerging technology– Easy to use– Adobe Flash dying out
• Fast changing protocol under active development
• Study WebRTC’s performance for both simulated and real world conditions
4
WebRTC – Topologies
Peer to Peer
Thewaynetworknodesarearrangedinanetwork
SFU
Meshed SFU
Server to Client
5
WebRTC – Interoperability• Adoption
• Media codecs
• Browser compatibility– Support for older browsers with plugin
SupportedAnnounced SupportUnsupported
6
Congestion Control
• Google Congestion Control Algorithm– Dynamically adjusts send and receive rate
• Receiver: delay based• Sender: loss based
Arrival-time filter
Over-usedetector
Ratecontroller
Adaptive threshold
signal
Ar
mi
mi
As
Receiver-sideSender-side
Sender sidecontroller
RTP
REMBRTCP
Congestionoccurswhenanodecarriesmoredatathanitcanhandle
7
Congestion Control
• Receiver side (delay based):
• Sender side (packet loss based):
• Send rate never exceeds receive rate:
8
Experimental test setup
• Custom video stream• Network limiter• Statistics via RTCStatsReport
Networklimiter
WebRTC
bw delay pl
Video stream1920 x 1080
Gatherstatistics
everysecond
RTCStatsReport
WebRTC
Video stream1920 x 1080
Gatherstatistics
everysecond
RTCStatsReport
Node 1 Node 2
Synthetic network conditions
9
Experimental test setup
• Avoid CPU and memory limitations• 5 tests averaged
• Statistics:– Data rate (kbps)– Framerate (fps)– Resolution (width * height) (pixels)– Round-trip-time (ms)
• Source code available at: https://github.com/Wimnet/webrtc_performance
10
Performance Evaluation
• Baseline experiments• Cross traffic• Multi-party topologies• Video codecs• Mobile browsers• Real wireless networks
11
Performance Evaluation
• As expected– 5% converges to maximum data rate– > 10% converge to minimum data rate
Network characteristics – Packet loss
00:00 00:30 01:00 01:30 02:00 02:30 03:00 03:30 04:00 04:30 05:00
Time (mm:ss)
0
500
1000
1500
2000
2500
3000
Da
ta r
ate
(kb
ps) no loss
5% packet loss
10% packet loss
20% packet loss
12
Performance Evaluation
• 77% utilization• Follows GCC rates after minute 1:
1000 ∗ 1.05&' = 2300𝑘𝑏𝑝𝑠• Also after minute 3:
500 ∗ 1.05&0 = 1200𝑘𝑏𝑝𝑠
Network adaptability – Bandwidth
00:00 01:00 02:00 03:00 04:00 05:00Time (mm:ss)
0
500
1000
1500
2000
2500
3000
Da
ta r
ate
(kb
ps)
set data rate
data rate #1
data rate #2
00:00 01:00 02:00 03:00 04:00 05:00
Time (mm:ss)
0
200
400
RT
T (
ms)
RTT #1
RTT #2
13
Performance Evaluation
• Three separate calls with 2Mbps limit• Fairness is reached with delay
Cross traffic – Intra-protocol fairness
00:00 00:30 01:00 01:30 02:00 02:30 03:00 03:30 04:00 04:30 05:00
Time (mm:ss)
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Da
ta r
ate
(kb
ps)
call #1
call #2
call #3
total
14
Performance Evaluation
• 2Mbps limit• Competing TCP flows• Fairness can be improved
Cross traffic – Inter-protocol fairness
00:00 02:00 04:00 06:00 08:00 10:00 12:00
Time (mm:ss)
0
500
1000
1500
2000
Da
ta r
ate
(kb
ps)
WebRTC flowTCP flow
15
Performance Evaluation
• 2,3 and 4 person calls• Results averaged on uplink and downlink• CPU limitations
Multi-party - Meshed
00:00 00:30 01:00 01:30 02:00 02:30 03:00 03:30 04:00 04:30 05:00
Time (mm:ss)
0
2000
4000
6000
8000
10000
12000
Da
ta r
ate
(kb
ps)
2 persons
3 persons
4 persons
16
Performance Evaluation
• Less uplink bandwidth required• Start up delay
Multi-party - SFU
00:00 00:30 01:00 01:30 02:00 02:30 03:00 03:30 04:00 04:30 05:00
Time (mm:ss)
0
1000
2000
3000
Da
ta r
ate
(kb
ps)
Average data rates - 2p SFU
uplink rate
downlink rate
00:00 00:30 01:00 01:30 02:00 02:30 03:00 03:30 04:00 04:30 05:00
Time (mm:ss)
0
2000
4000
6000
Da
ta r
ate
(kb
ps)
Average data rates - 3p SFU
uplink rate
downlink rate
00:00 00:30 01:00 01:30 02:00 02:30 03:00 03:30 04:00 04:30 05:00
Time (mm:ss)
0
2000
4000
6000
8000
10000
Da
ta r
ate
(kb
ps)
Average data rates - 4p SFU
uplink rate
downlink rate
SFU
17
Performance Evaluation
• VP8 (default), VP9 and H.264• Room for improvement:
– H.264 needs to balance between framerate and resolution
– VP9 needs to scale up when congestion disappears
Video codec comparison
00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00
Time (mm:ss)
0
500
1000
1500
2000
2500
3000
Da
ta r
ate
(kb
ps)
H.264
VP8
VP9
limit
00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00
Time (mm:ss)
0
100
200
300
400
500
RT
T (
ms)
H.264
VP8
VP9
limit
00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00
Time (mm:ss)
0
2
4
6
8
10
Re
solu
tion
(p
ixe
ls)
× 105
H.264
VP8
VP9
00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00
Time (mm:ss)
0
10
20
30
40
50
60
Fra
me
rate
(F
PS
)
H.264
VP8
VP9
18
Experimental test setup
• 3 nodes:– Local wireless node (NYC)– Local wired node (NYC)– Remote wired nodes (Oregon or Sydney)
• Varying:– AP transmission power (to 1mW)– Distance from AP (5ft – 25ft)– MAC retry limit
Wireless performance
19
Performance Evaluation
• 5ft vs 25ft AP distance
• Results– Higher RTTs results in more packet loss
and lower quality
Wireless network conditions
20
Performance Evaluation
• Limit retransmissions in AP– From MAC retry limit 1 (min) and 15 (max)
• Results– Trade off between RTT and packet loss– Less video freezes– GCC relies too heavy on packet loss
Wireless network conditions
21
Summary
• Improved browser interoperability
• Thorough evaluation of WebRTC– Custom video stream– Similar results
• Improved cross traffic fairness
• Poor performance over wireless due to:– Bursty losses– Packet retransmissions
22
Conclusions
• Heavily reliant on packet loss
• Multi-party– When more than 3 people: Add SFUs
• Room for improvement– Mobile performance– Video codecs
23
Future Work
• Take CPU limitations into account– Energy consumption (battery)– Call characteristics when CPU is
exhausted
• Simulate Google Congestion Control
• Tests with cellular connections
