Digital Media Lab - Sharif University of Technology2
Outlines
Problems of video streaming
Error concealment
Error resilient video coding
Error-induced problems
Multiple description video coding
What Is Problem?
Internet: best effort service
No guarantees for:
Bandwidth
Loss rates
Goal: reliable delivery of high quality video over the Internet
Digital Media Lab - Sharif University of Technology3
Problem Solution
Bandwidth Rate control
Loss Error control
Rate Control
Two main steps:
Estimation of the available bandwidth
Matching the video rate to available bandwidth
It can be perform in both sides.
Digital Media Lab - Sharif University of Technology4
Sender ReceiverDynamic
Bandwidth
Error Control
Forward Error Correction (FEC)
Retransmission
Error-resilient video coding
Error concealment
Digital Media Lab - Sharif University of Technology5
Source Coding (Current lecture)
Channel Coding
Error Types
Type Example
Few bit errors and packet loss Videophone over PSTN
Packet loss up to 30% Video streaming over the Internet
Burst Error Wireless video
Digital Media Lab - Sharif University of Technology6
Error Concealment: Motivation
Problem
Lost information caused by transmission errors.
Goal
Estimating lost information in order to conceal (no correct) the error.
Observation
High amount of temporal/spatial correlation in video.
General Approach
Performing interpolation/regression to estimate the lost information based on
correctly received neighboring ones.
Digital Media Lab - Sharif University of Technology7
Error Concealment: Approaches
There are three approaches with respect to considered correlation:
Spatial
Temporal
Spatio-Temporal
Digital Media Lab - Sharif University of Technology8
Spatial Error Concealment
Estimate the values of
the lost pixels f*
based on the correctly
received neighboring
pixels y.
Main Assumption:
Spatial Smoothness
Digital Media Lab - Sharif University of Technology9
Motion-Compensated Temporal Error Concealment
Estimate lost block as
motion-compensated
block from prior frame
Main Assumption:
Temporal Smoothness
Digital Media Lab - Sharif University of Technology10
Error Concealment: Challenges
Usually lose more than a single
macroblock
E.g. lose a row of macroblocks or an
entire frame
Low correlation in spatial domain
for some videos
Low correlation in temporal domain
for some others.
Digital Media Lab - Sharif University of Technology11
Error Resilient Coding
Two basic error-induced problems:
Loss of bitstream synchronization
Decoder: what bits correspond to what parameters?
Error in Huffman codeword
Incorrect state and error propagation
Different states in encoder and decoder
Design the compression algorithm in the way that generates bitstreams
resilient to channel errors.
Digital Media Lab - Sharif University of Technology12
Loss of Bitstream Synchronization
Decoder does not know what bits correspond to what parameters
In variable length coding (VLC), single bit error can result in significant
subsequent loss
Fixed length code does not have this problem but suffer from low
compression ratio.
However, VLC is widely used.
Digital Media Lab - Sharif University of Technology13
Loss of Bitstream Synchronization
First Solution
Use resync markers
Use a marker codeword different from all codewords.
Restart decoding after marker.
Location of Markers:
Strategic locations such as beginning of frame, slice, …..
Periodic locations
Digital Media Lab - Sharif University of Technology14
Loss of Bitstream Synchronization
Second Solution:
Reversible variable length coding (RVLC):
Conventional VLC codes are only decodable in forward direction
RVLC codes provide the ability of decoding also in backward direction
If an error is detected, jump to the next resync position and start decoding
backwards, enabling partial recovery of data.
Digital Media Lab - Sharif University of Technology15
Loss of Bitstream Synchronization
Third Solution:
Data Partitioning
Bits near the resync markers are more likely to be correct.
Place most important information immediately after the resync
marker.
Digital Media Lab - Sharif University of Technology16
Summary: Loss of Bitstream Synchronization
Solutions:
Resync markers at strategic locations
Resync markers at periodic locations
Reversible VLC’s (RVLC’s)
Data partitioning
Digital Media Lab - Sharif University of Technology17
Incorrect state and error propagation
Incorrect state and error propagation at decoder:
An error causes the reconstructed frame (state) at the decoder to be
incorrect
Decoder’s state is different from encoder’s, leading to mismatched
predictions and often significant error propagation.
Digital Media Lab - Sharif University of Technology18
Incorrect state and error propagation: Solution
Limit effect of error propagation: for example reinitialize prediction loop.
How?
All I-frames
Usage of intra-coded frames degrades compression ratio.
Periodic I-frames
This approach leads to existence of group of pictures (GOP).
Partial intra-coding of frames
Individual macroblocks are intra-coded
Periodic
Preemptive
Digital Media Lab - Sharif University of Technology19
Intra-Coding Approaches
Advantages
Limits the effect of error propagation
Disadvantages
High bit rate required for intra coding limits its use
Optimal tradeoff between on intra/inter mode, depends on accurate knowledge
of channel characteristics
Digital Media Lab - Sharif University of Technology20
Scalable Video Coding
Codes video into a base layer and one or more enhancement layers
Temporal
I and P frames in base layer and B frames in enhancement layer.
Spatial
Down-sampling and up-sampling in encoder and decoder
respectively.
SNR
Combination of coarse and fine quantization
Best match for networks which support different qualities of service
Digital Media Lab - Sharif University of Technology21
Multiple Description Video Coding
Code video into a number of descriptions, each of roughly equal
importance
Properties:
Receiving either bitstream leads to good quality video
Receiving both bitstreams leads to highest quality video
Digital Media Lab - Sharif University of Technology22
Single Description versus Multiple-State Approach
Single description coding:
Multiple description coding:
Digital Media Lab - Sharif University of Technology23