Tammam Tillo, Macro Grangetto, and Gabriella Olmo
CSVT, Jan 2008
1
Introduction◦ Multiple description coding◦ Redundant slice
Proposed Algorithm Optimal Allocation Problem Algorithm Implementation Experimental Results
2
Multiple Description Coding (MDC)◦ Single source Multiple Descriptions
◦ D0 < D1, D2
◦ Rsingle_description = R1 + R2 – Rredundancy
Encoder 1
Encoder 2
Central decoder
Side decoder 1
Side decoder 2
Channel I
Channel 2
R1
R2
D1
D2
D0S
S’1
S’2
S’3
Introduced by MD coding 3
I I2i+1
2i
Example
Redundant slice in H.264◦ A slice can be encoded and transmitted twice Called primary slice and redundant slice
◦ QPp < QPr ◦ The redundant slice is used at the decoder side if
the primary slice is lost
QPr
QPp Primary slice
redundant slice
Encoder Decoder
4
MDC with redundant slices
◦ Simple post and pre-processing + H.264 decoder
I P P I
Primary Slice
Redundant Slice
Source
I P P I Description 1
Description 2
5
Redundant slice◦ QPr > QPp
◦ Not for reference◦ Interlaced with primary slices◦ Drift problem◦ QPr drift redundancy◦ QPr drift (under loss rate p) Dtotal = (1-p)dprimary_slice + p(1-p)(dredundant_slice+dpropagation) + p2dall_loss Rtotal = Rprimary_slice(QPp) + Rredundant_slice(QPr)
6
Impact of loss of a primary slice k on the total distortion of the current GOP with size N ◦
N
ijkmkrkt jddd
1.,, ][Distortion caused by
loss of primary slice k
Distortion of redundant slice k at frame i
Propagated distortion cause by redundant slice k after frame i
...
i i+1
Loss occurs
k
N
7
Decay of propagated distortion [22]◦
dm,k = dr,k -dp,k dr,k when p,k << r,k
◦ f[n]: Power transfer function Representation of distortion attenuation Reasons
Spatial filter such as de-blocking filter Spatial interpolation such as fractional pel ME Intra blocks such as random intra replacement
iN
nkm
N
ijkm nfdjd
1,
1. ][][
]1[,, idd kmkm
...
i i+1
krd , kmd , ]1[, fd km ][, iNfd km
[22] N. Farber, K. Stuhlmuller, and B. Girod, “Analysis of error propagation in hybrid video coding with application to error resilience,” in ICIP, 1999. 8
Evaluation of dm,k ◦
◦ Aligned quantizer
◦
deefeeEd km )(}{ 22,
e: Difference between the primary and redundant reconstruction (drift)f(e): pdf of e
n
jkpkp
kr
kp
kr
kp jejeeef1
,,,
,
,
, )]()([)()(
: Delta function
r
pe
Primary reconstruction
Redundant reconstruction
Source x
Source x
e
f(e)
)( prp je
p
)( prp je e = 0,p, 2p,…f(e) = p/r at p|e
9
Evaluation of dm,k ◦
◦ p,k << r,k dm,k dr,k
2
,
,2
,
1
2
,
3,
1,,
2
,
,2,
112
)]()([)(
kr
kpkr
n
jkr
kp
n
jkpkp
kr
kpkm
jc
dejejeedeefed
Assumption:r,k/p,k = 2n+1dr,k = r,k
2/12
10
Evaluation of dm,k ◦ Not aligned quantizer
◦
r
p e
Reconstructed level
Reconstructed level
Source x
Source x
e
f(e)
r
r1
12
1)(
2,
2/
2/ ,
22,
,
,
kr
krkm
kr
kr
deedeefed
e can be any numberf(e) = 1/r
11
Accuracy of evaluated dm,k
Real distortion – evaluated dm,kReal distortion
12
Total GOP distortion dt,k ◦
Expected GOP distortion caused by loss of slice k at loss rate p◦
Expected GOP distortion caused by random loss of a slice◦
ikr
iN
nkr
iN
nkrkr
iN
nkmkrkt dnfdnfddnfddd .
0.
1.,
1.,, ][][][
ktkp
kktkpk
dppdp
dpdppdpd
,,
,02
,,
)1()1(
)1()1(
N
i
N
iiirip
kk DpDpdD
1 1,,)1(
f[0] = 1
iN
ni nf
0
][
low loss rate
Loss of primary sliceUsing primary slice Loss of primary and redundant slice
ik
krir dD ,,
ik
kpip dD ,,
Expected frame distortion
13
Optimization problem◦
◦ Lagrangian approach: min J = D + R ◦
Dmin
GOP1
,, )( subject to RRRN
iirip
0)1(
0)1(
,
,
.
,
,
.
ir
iri
ir
ip
ip
ip
R
Dpp
R
J
R
Dp
R
J
ir
iri
ip
ip
R
Dp
R
D
,
,
,
,
1,
1,1
,
,
1,
1,
1,
1,
r
r
iir
ir
r
ri
p
p
p
p
R
D
R
D
R
Dp
R
D
R
D
Since Dp,i/ Rp,i is independent of i
14
Optimal conditions◦
Redundant data < primary data i > i+1
1,
1,1
,
,
1,
1,
r
r
iir
ir
r
ri
p
p
R
D
R
D
R
Dp
R
D
15
Encoder
Decoder
QPp QPr,i
Post processing
Channel 1
Channel 2
Channel 1
Channel 2
Preprocessing
Merge
H.264 Standard decoding
Redundant slice
16
Determination of RD parameters◦
◦ H.264 reference software: QPp = QPr,1 + 3log(p1)
QPr,i = QPr,1 + 3log(1/i)
◦ In [22], assuming , deducing
Taking
3
12
285.0QP
RD1,
1,1
,
,
1,
1,
r
r
iir
ir
r
ri
p
p
R
D
R
D
R
Dp
R
D
-
nenf ][
yxyxuuyxtv ddHt ),(),(4
1][
2
22
22
1
1][ uv t
t
Variance of drift v[x,y,t]
Power spectral density of u[x,
y]= v[x,y,0]
Linear system (decoder)
)1()1( )1( ee iNi
Small for serious packet loss17
(QPi) = QPr,i – QPp ◦ i QPr,i◦ p QPr,i◦ QPr,i
18
Parameters◦ H.264 reference software JM9.4◦ P-slice with 5 reference pictures◦ 5 slices for a CIF picture and 3 slices for a QCIF
picture◦ Bernoulli model for PLR = p ◦ foreman and coastguard at CIF and QCIF format
are used
19
Selection of ◦ Not sensitive to
20
RD comparisons under different p and N (=0.4)◦ Shorter GOP size for larger PLR
p=0.01p=0.05
p=0.1
N=11
N=21N=45
21
Redundancy (=0.4)◦
i irip
i ir
RR
R
)( ,,
,
p
p
R
D
1,
1,
r
r
R
D
22
Comparisons with other MDC◦ Four descriptions by subsampling CIF to QCIF◦ Two descriptions by separating odd and even
rows Other MDC schemes:N = 21 with P and B picturesExtra computation is needed at the decoder side
23
Central decoder versus side decoder◦ p redundancy (central decoder) and (side
decoder)
No loss
24
Central decoder versus side decoder (p=0.05)
25