Achieving Best Quality Video on Mobile with Scalability

. © Copyright 2016 Vidyo Inc., confidential, proprietary and patent pending information

Achieving Best Quality Video on Mobile with ScalabilityGene GershanokVidyoKranky Geek, 2016



• Multiparty Video API

• Cross Platform Support

• Customizable layout compositing

Vidyo.io

Video API Platform as a Service

https://vidyo.io@Vidyo_io

https://vidyo.io/

© copyright 2016 Vidyo Inc., confidential, proprietary and patent pending information

• Video Routing Platform• Video Codec• Audio AEC• Device Management • Rendering

Full picture

Components


• Fluctuating Network Bandwidth• Packet Loss & Jitter• Varying Device Capabilities• Bitrate vs. Battery Life• Maintaining Low Latency

Experience with building a platform

Mobile Video Challenges


A set of video frames


Let’s keep it simple

F1

F2

F3

F4

F5


Traditional single-layer coding

F1

F2

F3

F4

F5

----


E

A

Transcoding MCU

B

C

D

MCU

- High Delay- Transcoding Loss- Complexity & Cost- Limited Layouts


a

c

B

Simulcast Router

- Better error resiliency- Custom layouts- No server-side decode

A

B

C

a

b

c


Simulcasting

For 2:1 resolution ratios:- ~50% overhead vs. single

layerHigh Resolution (A)

Low Resolution (a)

Simulcast


ac

Bb

Scalable Router

A

B

C

VR

a

b

c

- Error concealment on downlink- Less bandwidth- Synchronized streams


Scalable Video Coding - SVC


A set of video frames


Let’s keep it simple

F1

F2

F3

F4

F5


Traditional single-layer coding

F1

F2

F3

F4

F5

----


Temporally Scalable Coding

F1

F2

F3

F4

F5


Temporally Scalable Coding

F1

F2

F3

F4

F5

TO TO


Layer 0 (7.5 fps)

F1

F5

TO TO ----


Layers 0 and 1 (15 fps)

F1

F5

TO

TO

T1

F3


Layers 0, 1, and 2 (30 fps)

T1

TO

TO

T2 T2

F3

F1

F2

F4

F5


Adaptation Example: 2 to 1 to 0

T1

TO

TO

T2 T2

F3

F1

F2

F4

F5


Error Resilience

XT1

TO

TO

T2 T2

F3

F1

F2

F4

F5


Error Resilience (cont’d)

XT1

TO

TO

T2 T2

F3

F1

F2

F4

F5



X

XT1

TO

TO

T2 T2

F3

F1

F2

F4

F5



XT1

TO

TO

T2 T2

TO

F3

F1

F2

F4

F5

F1


Spatially Scalable Coding

TO

SO


Scalable Video Coding

TO

SO

TO

SOT1

S1T2

S2

T2

S2



TO

SO

TO

SOT1

S1T2

S2

T2

S2

360p@ 30 fps



TO

SO

TO

SOT1

S1T2

S2

T2

S2

720p @ 15 fps


Fluctuating Network Bandwidth

Mobile Video Challenges Addressed

Bit stream will scale as packets are dropped without needing IFrames


Fluctuating Network Bandwidth Packet Loss and Jitter



Only need to FEC and NACK T0 packets


Fluctuating Network Bandwidth Packet Loss and Jitter Varying Device

Capabilities




Send and Receive based on screen real estate



Bitrate vs Battery Life

Varying Device Capabilities





Only ask for what you can decode



Bitrate vs Battery Life

Varying Device Capabilities


Maintaining Low Latency




Only ask for what you can decode

No transcoding, only forwarding packets


What did we gain?

~20%Fewer bits than simulcast

can tolerate more than 20% packet loss rate(using both temporal and spatial scalability)

ADAPTABILITY & SUPERB ERROR RESILIENCE


Scalability in Video Codecs

• Already in:– H.264 (Annex G, SVC – temporal, spatial/quality)– VP8 (temporal only)– HEVC (H.265) v.1 (temporal) and v.2 (SHVC – spatial/quality)– VP9 (Vidyo & Google)

• Work in progress:– Alliance for Open Media - AV1 (temporal mandatory, spatial

under review)


Error Resilience Support in Standards

• Error resilience features supported in the RTP payload formats of all modern video codecs:– VP8 (RFC 7441)– H.264 SVC (RFC 6190)– H.265 HEVC (RFC 7798)– VP9 (draft-ietf-payload-vp9)

https://tools.ietf.org/html/rfc7741



https://tools.ietf.org/html/draft-ietf-payload-vp9-02


Vidyo WebRTC

Vidyo is active in the evolution of WebRTC:• Google / Vidyo are co-developing VP9, the next generation WebRTC

codec• Contributing software for both the WebM and WebRTC open source

projects• Co-Chairing IETF Group on AVT Extensions• Co-authored numerous RFCs related to signaling and transport

Vidyo participates in many standardization bodies:


Scalable VP9 Status

• Chrome has full VP9 scalability in decoder– Encoder configure through URL flags

• Ongoing work with WebRTC Editors to provide ways to configure the video encoder for different use cases


Questions?

Gene Gershanok

@GeneGershanok@Vidyo_io

Date post:	17-Jan-2017
Category:	Technology
Upload:	madeline-gooding
View:	140 times
Download:	0 times

Achieving Best Quality Video on Mobile with Scalability

Technology