Date post: | 31-Dec-2015 |
Category: |
Documents |
Upload: | zenia-chaney |
View: | 40 times |
Download: | 2 times |
Distributed Video CodingDistributed Video CodingDistributed Video CodingDistributed Video Coding
VLBVVLBV, Sardinia, September 16, 2005, Sardinia, September 16, 2005
Bernd GirodBernd Girod
Information Systems LaboratoryInformation Systems LaboratoryStanford UniversityStanford University
B. Girod: Distributed Video Coding 2
Outline
Foundations of distributed coding– Slepian-Wolf Theorem and practical Slepian-Wolf coding– Wyner-Ziv results and practical Wyner-Ziv coding
Low-complexity video encoding– Pixel-domain and transform-domain coding– Hash-based receiver motion estimation– Wyner-Ziv residual coding
Error-resilient video transmission– Systematic lossy joint source-channel coding– Improving the error-resiliency of MPEG (or anything else)
by Wyner-Ziv coding
B. Girod: Distributed Video Coding 3
Outline
Foundations of distributed coding– Slepian-Wolf Theorem and practical Slepian-Wolf coding– Wyner-Ziv results and practical Wyner-Ziv coding
Low-complexity video encoding– Pixel-domain and transform-domain coding– Hash-based receiver motion estimation
Error-resilient video transmission– Systematic lossy joint source-channel coding– Improving the error-resiliency of MPEG by Wyner-Ziv coding
B. Girod: Distributed Video Coding 4
Compression of Dependent Sources
Source X
Source X
Source Y
Source Y
JointDecoder
JointDecoder
X
Y
,R H X YJoint
Encoder
JointEncoder
,XY
X Y
p x y
p x p y
B. Girod: Distributed Video Coding 5
Distributed Compression of Dependent Sources
Source X
Source X
Source Y
Source Y
Encoder X
Encoder X
Encoder Y
Encoder Y
JointDecoder
JointDecoder
X
Y
??XR
??YR
,XY
X Y
p x y
p x p y
B. Girod: Distributed Video Coding 6
Slepian Wolf Theorem
[bits]XR
[bits]YR
H X
H Y
Independent decoding
Achievable rate region for i.i.d sequences
B. Girod: Distributed Video Coding 7
Slepian Wolf Theorem
[bits]XR
[bits]YR
H X
H Y
|H Y X
|H X Y
,X YR R H X Y
Joint decoding:Vanishing error probabilityfor long sequences
Independent decoding:No errors
[Slepian, Wolf, 1973]
Achievable rate region for i.i.d sequences
B. Girod: Distributed Video Coding 8
Lossless Compression with Receiver Side Information
[bits]XR
[bits]YR
H X
H Y
|H Y X
|H X Y
,X YR R H X Y
Source Encoder Decoder
Y
X
Y
|R H X Y
X
B. Girod: Distributed Video Coding 9
Distributed Compression and Channel Coding
Idea
Interpret Y as a “noisy” version of X
with “channel errors” Encoder generates “parity bits” P to
protect against errors Decoder concatenates Y and P and
performs error-correcting decoding
Idea
Interpret Y as a “noisy” version of X
with “channel errors” Encoder generates “parity bits” P to
protect against errors Decoder concatenates Y and P and
performs error-correcting decoding
SourceX|Y
Encoder Decoder
Y
X
YX Y
01001100010101
01001101010101
00000001000000
X
Y
P
B. Girod: Distributed Video Coding 10
Towards Practical Slepian-Wolf Coding
Convolution coding for data compression [Blizard, 1969, unpublished] Convolutional source coding [Hellman, 1975] Coset codes [Pradhan and Ramchandran, 1999] Trellis codes [Wang and Orchard, 2001] Turbo codes
[Garcia-Frias and Zhao, 2001]
[Bajcsy and Mitran, 2001]
[Aaron and Girod, 2002] LDPC codes [Liveris, Xiong, and Georghiades, 2002] . . . . . .
B. Girod: Distributed Video Coding 11
Slepian-Wolf Coding Using Turbo Codes
Systematic Convolution
al Code
X
Systematic Convolution
al Code
Interleaver
SISO Decode
r
SISO Decode
rDecision X
Parity bits
Parity bits
Systematicbits X . . . . . . . Y
X Y“Correlation channel”
[Aaron and Girod, 2002]
B. Girod: Distributed Video Coding 12
X
Lossy Compression with Side Information
'XSource Encoder Decoder
Y Y
X 'XSource Encoder Decoder
Y Y Y
[Wyner, Ziv, 1976] For mse distortion and Gaussian statistics, rate-distortion functions of the two systems are the same.
[Wyner, Ziv, 1976] For mse distortion and Gaussian statistics, rate-distortion functions of the two systems are the same.
B. Girod: Distributed Video Coding 13
Practical Wyner-Ziv Encoder and Decoder
Wyner-Ziv Decoder
QuantizerSlepian-
Wolf Encoder
Wyner-Ziv Encoder
Slepian-
WolfDecode
r
Minimum Distortion
Reconstruction
Y Y
X 'XQ Q
B. Girod: Distributed Video Coding 14
Non-Connected Quantization Regions
Example: Non-connected intervals for scalar quantization
Decoder: Minimum mean-squared error reconstruction with side information
x
1q 2q 3q
x
| |X Yf x y 2
ˆ
conditional centroid
ˆ ˆ | ,arg min x
x E X x y q
B. Girod: Distributed Video Coding 15
Outline
Foundations of distributed coding– Slepian-Wolf Theorem and practical Slepian-Wolf coding– Wyner-Ziv results and practical Wyner-Ziv coding
Low-complexity video encoding– Pixel-domain and transform-domain coding– Hash-based receiver motion estimation
Error-resilient video transmission– Systematic lossy joint source-channel coding– Improving the error-resiliency of MPEG by Wyner-Ziv coding
B. Girod: Distributed Video Coding 16
Interframe Video Coding
PredictiveInterframe Decoder
PredictiveInterframe Encoder
X’
Side Information
YX Y
B. Girod: Distributed Video Coding 17
Video Coding with Low Complexity
Wyner-ZivInterframe Decoder
Wyner-ZivIntraframe Encoder
X’
Side Information
YX
[Witsenhausen, Wyner, 1978][Puri, Ramchandran, Allerton 2002][Aaron, Zhang, Girod, Asilomar 2002]
B. Girod: Distributed Video Coding 19
Low Complexity Encoding and Decoding
B. Girod: Distributed Video Coding 20
Pixel Domain Wyner-Ziv Coder
Interframe DecoderIntraframe Encoder
Reconstruction X’
Y
Video frame
XScalar
QuantizerTurbo
EncoderBuffer Turbo
Decoder
Request bits
Slepian-Wolf Codec
InterpolationKey frames
previous
next[Aaron, Zhang, Girod, Asilomar 2002][Aaron, Rane, Zhang, Girod, DCC 2003]
B. Girod: Distributed Video Coding 21
Decoder side informationgenerated by motion-
compensated interpolationPSNR 30.3 dB
After Wyner-Ziv Decoding16-level quantization – 1.375 bpp
11 pixels in errorPSNR 36.7 dB
Pixel Domain Wyner-Ziv Coder
B. Girod: Distributed Video Coding 22
Pixel Domain Wyner-Ziv Coder
Decoder side informationgenerated by motion-
compensated interpolationPSNR 24.8 dB
After Wyner-Ziv Decoding16-level quantization – 2.0 bpp
0 pixels in errorPSNR 36.5 dB
B. Girod: Distributed Video Coding 23
Stanford Camera Array
Courtesy Marc Levoy, Stanford Computer Graphics Lab
B. Girod: Distributed Video Coding 24
…WZ-ENC
WZ-DEC
WZ-ENC
WZ-DEC
…
GeometryReconstruction
Rendering
Wyner-Ziv Cameras Conventional Cameras
Distributed Compression
Distributed Encoding
Joint Decoding
[Zhu, Aaron, Girod, 2003]
B. Girod: Distributed Video Coding 25
Light Field Compression
Rate: 0.11 bppPSNR 39.9 dB
Rate: 0.11 bppPSNR 37.4 dB
Wyner-Ziv, Pixel-Domain JPEG-2000
B. Girod: Distributed Video Coding 26
DCT-Domain Wyner-Ziv Video Encoder
For each low frequency coefficient band k
level Quantizer
DCTkM2 Turbo
EncoderExtract bit-
planes
bit-plane 1
bit-plane 2
bit-plane Mk
…Inputvideo frame
QuantizerEntropy Coder
Comparison
Previous-frame quantized high freq coefficients
Wyner-Ziv parity bits
High frequency
bits
Low freq coeffs
High freq coeffs
B. Girod: Distributed Video Coding 27
IDCT
Wyner-Ziv Video Decoder with Motion Compensation
Reconstruction
DCT
Entropy Decoder and
Inverse Quantizer
Side information
Wyner-Ziv parity bits
High frequency
bits
Turbo Decoder
Motion-compensated Extrapolation
Previous frame
DCT
Refinedside information
ExtrapolationRefinement
For each low frequency band Decoded frame
Reconstructed high frequency coefficients
B. Girod: Distributed Video Coding 28
Rate-Distortion Performance - Salesman
Every 8th frame is a key frame
Salesman QCIF sequence at 10fps 100 frames
6 dB
3 dB
B. Girod: Distributed Video Coding 29
Rate-Distortion Performance – Hall Monitor
8 dB
3 dB
Every 8th frame is a key frame
Hall Monitor QCIF sequence at 10fps 100 frames
B. Girod: Distributed Video Coding 30
Rate-Distortion Performance – Foreman
2 dB
1.5 dB
Every 8th frame is a key frame
Foreman QCIF sequence at 10fps 100 frames
DCT-based Intracoding 149 kbps
PSNRY=30.0 dB
Wyner-Ziv DCT codec 152 kbps
PSNRY=35.6 dB GOP=8
Salesman at 10 fps
DCT-based Intracoding 156 kbps
PSNRY=30.2 dB
Wyner-Ziv DCT codec 155 kbps
PSNRY=37.1 dB GOP=8
Hall Monitor at 10 fps
DCT-based Intracoding 290 kbps
PSNRY=34.4 dB
Wyner-Ziv DCT codec 293 kbps
PSNRY=35.5 dB GOP=8
Foreman at 10 fps
B. Girod: Distributed Video Coding 34
Wyner-Ziv Residual Coding
Wyner-ZivDecoder
Wyner-Ziv Encoder
Side Information
Y X’n
X’n-1
Side Information
-Xn
X’n-1
Frame differenceXn
B. Girod: Distributed Video Coding 35
Rate-Distortion Performance – Foreman
Every 8th frame is a key frame
Foreman QCIF sequence at 30fps 100 frames
B. Girod: Distributed Video Coding 36
Rate-Distortion Performance – Foreman
Every 8th frame is a key frame
Foreman QCIF sequence at 30fps 100 frames
B. Girod: Distributed Video Coding 37
Outline
Foundations of distributed coding– Slepian-Wolf Theorem and practical Slepian-Wolf coding– Wyner-Ziv results and practical Wyner-Ziv coding
Low-complexity video encoding– Pixel-domain and transform-domain coding– Hash-based receiver motion estimation
Error-resilient video transmission– Systematic lossy joint source-channel coding– Improving the error-resiliency of MPEG by Wyner-Ziv coding
B. Girod: Distributed Video Coding 38
Systematic Lossy Error Protection (SLEP)of Compressed Video
Any OldVideo
Encoder
Video Decoder with Error
Concealment
Err
or-P
rone
cha
nnel
S S’
Wyner-Ziv Decoder A S*
Wyner-Ziv Encoder A
Wyner-Ziv Decoder B S**Wyner-Ziv
Encoder B
Graceful degradation without a layered signal representation
Analog channel (uncoded)
[Aaron, Rane, Girod, ICIP 2003]
B. Girod: Distributed Video Coding 39
MPEG with Systematic Lossy Error Protection
Cha
nnel
Slepian-WolfEncoder
Wyner-Ziv Encoder
ED T-1Q-1 +
MC
S*MPEGEncoder
main
S
Side information
MPEGEncoder
coarse
T-1q-1ED +
MC
S’
R-SDecoder
ReconstructedFrame atEncoder
MPEGEncoder
coarse
R-SEncoder
[Rane, Aaron, Girod, VCIP 2004]
Parityonly
B. Girod: Distributed Video Coding 40
Reed-Solomon Coding Across Slices
1 byte in slice
filler byte
parity byte
RS code across slices
Transmit parity slices only
B. Girod: Distributed Video Coding 41
Results: CIF Foreman
Main Stream @ 1.092 MbpsFEC (n,k) = (40,36) FEC bitrate = 120 KbpsTotal = 1.2 Mbps
WZ Stream @ 270 KbpsSLEP (n,k) = (52,36)WZ bitrate = 120 KbpsTotal = 1.2 Mbps
SLEP
FEC
FECSLEP
B. Girod: Distributed Video Coding 42
MPEG with Systematic Lossy Error Protection
Cha
nnel
Slepian-WolfEncoder
Wyner-Ziv Encoder
ED T-1Q-1 +
MC
S*MPEGEncoder
main
S
Side information
MPEGEncoder
coarse
T-1q-1ED +
MC
S’
R-SDecoder
ReconstructedFrame atEncoder
MPEGEncoder
coarse
R-SEncoder
[Rane, Aaron, Girod, VCIP 2004]
Parityonly
B. Girod: Distributed Video Coding 43
Quantized transformed Prediction error
Coarse Quantizer
Entropy CodingQ1
Q-1
Quantization parameter (Q)
MPEG2Encoder
Conventionally encoded streamInputVideo
Err
or-
pro
ne
Ch
an
ne
l
Entropy Decoding
MPEG2Decoder
T-1 +
MC
LEGACY BROADCASTING SYSTEM
WYNER-ZIV ENCODER WYNER-ZIV DECODER
RS Decoder
Fallback to coarser version
Decoded motion vecs
Entropy Decoding
-11Q
Parityonly
RS Encoder
Side Information (motion vectors,mode decisions)
SLEP MPEG Codec with Simple Decoder
Q1
Entropy Coding
Sid
e I
nfo
(mo
tion
ve
cs,
mo
de
de
cisi
on
s)
B. Girod: Distributed Video Coding 44
Performance at Symbol Error Rate 10-4
MPEG-2 video: 2 Mbps+ FEC 222 Kbps(PSNR 29.78 dB)
MPEG-2 video: 2 Mbps+ Wyner-Ziv 222 Kbps
(PSNR 35.45 dB)
B. Girod: Distributed Video Coding 45
Distributed Coding of Video:Why Should We Care?
Chance to reinvent compression from scratch– Entropy coding– Quantization– Signal transforms– Adaptive coding– Rate control– . . .
Enables new compression applications– Very low complexity encoders– Compression for networks of cameras– Error-resilient transmission of signal waveforms– Digitally enhanced analog transmission– Unequal error protection without layered coding– . . .
The EndThe EndFurther interest:Further interest:
B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, "Distributed Video Coding," B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, "Distributed Video Coding," Proceedings of the IEEE,Proceedings of the IEEE, Special Issue on Video Coding and Delivery. Special Issue on Video Coding and Delivery. January 2005.January 2005.
http://www.stanford.edu/~bgirod/pdfs/DistributedVideoCoding-IEEEProc.pdf http://www.stanford.edu/~bgirod/pdfs/DistributedVideoCoding-IEEEProc.pdf