Department of Electrical EngineeringStanford University

Yi Liang, Eric Setton and Bernd Girod

Channel-Adaptive Video Streaming Using Packet Path Diversity andRate-Distortion Optimized Reference Picture Selection

Liang, Setton and Girod: Packet Path Diversity and Rate-Distortion Optimized RPS

Motivation

Channel-adaptive packet dependency control: Higher error-resilience; no packet

retransmission Low-latency streaming (<1

second) for interactive applications

[Liang, Flierl, Girod, ICIP’02]

Path diversity Uncorrelated channel statistics Mitigate negative impact on data

This work Low-latency streaming using

multiple paths for further improvement

R-D optimization for multiple streams

1 2

Datatraffic Data

traffic

ContentServer

Client

Relay Server

Relay Server

Liang, Setton and Girod: Packet Path Diversity and Rate-Distortion Optimized RPS

Outline

Prior work Channel-adaptive reference picture selection Optimal packet dependency control for multiple

streams Results and video demo

Liang, Setton and Girod: Packet Path Diversity and Rate-Distortion Optimized RPS

Prior Schemes

Video redundancy coding (VRC) [H.263++]

Multiple state coding [Apostolopoulos]

RPS-NACK[Lin et. al., ICME’01]

1 3

2 4S S

0 2 8

1 3 5 7 9

4 6 10

NACK (1) ACK (7)

Path 1

Path 2

Path 1

Path 2

Liang, Setton and Girod: Packet Path Diversity and Rate-Distortion Optimized RPS

Dependency Control over Multiple Paths

vvopt

vvv

Jv

In

n-vV

vRDJ

min arg

as channel same the

oversent frame

for ,

}{}

|,...3,2{}1{

Path 1

Path 2

Long-Term Memory V=5

n-5 n-4 n-3 n-2 n-1 n

Prediction modes: v=1, 2, … V, I

},5,3,2,1{ I

V=1

,1J

V=2

,2J

V=3

,3J

V=5

,5J IJ

Liang, Setton and Girod: Packet Path Diversity and Rate-Distortion Optimized RPS

Path Selection

Path selection: to minimize the distortion of next frame and take advantage of path diversity

Alternate when both channels are good

Send small probe packets over the channel in bad state

Liang, Setton and Girod: Packet Path Diversity and Rate-Distortion Optimized RPS

Results (1)

Channel loss_rate_1=loss_rate_2 =15%

LTM len=12Avg burst len=8Feedback delay=6

Liang, Setton and Girod: Packet Path Diversity and Rate-Distortion Optimized RPS

Results (2)

Channel loss_rate_1=loss_rate_2 =15%

LTM len=12Avg burst len=8Feedback delay=6

Liang, Setton and Girod: Packet Path Diversity and Rate-Distortion Optimized RPS

Video Demo (1)

Proposed scheme RPS-NACK

Mother-Daughter 318 Kbps20% loss

Liang, Setton and Girod: Packet Path Diversity and Rate-Distortion Optimized RPS

Video Demo (2)

Proposed scheme RPS-NACK

Foreman 185 Kbps20% loss

Liang, Setton and Girod: Packet Path Diversity and Rate-Distortion Optimized RPS

Conclusions

Eliminated the need for retransmission; low-latency (<1 second) streaming over multi-

path

Increased error-resilience by using Packet path diversity Feedback-based path selection Optimal packet dependency control

Improved tradeoff between error-resilience (independent streams) and coding efficiency using R-D optimization

Liang, Setton and Girod: Packet Path Diversity and Rate-Distortion Optimized RPS

Estimate of the Distortion

q

q

pq

p

p

),( 11 DR

),( 22 DR

),( VV DR

…

q

p

D11, p11=q3

D12, p12=q2p

D18, p18=p3

…

…

8

1111

iii pDD

P1

4

1222

iii pDD

D21, p21=q2pq

p D22, p22=up

D23, p23=up

D24, p24=p2

P2n-3 n-2 n-1 n