Statistical Multiplexer of VBR video streams

31.12.2002

By Ofer Hadar

Statistical Multiplexer of VBR video streams

31.12.2002

By Ofer Hadar

Outline of the talk Introduction and motivation for Statistical

Multiplexing VBR vs. CBR coding Statistical multiplexing of video streams Joint rate control

Current Piecewise CBR multiplexer Future true VBR multiplexer

Conclusions

Case in Point: Movies Huge bandwidth consumer Without compression: Unthinkable (up to 1

Gbps for HDTV stream) With compression: highly bursty

temporal and spatial information: frames vary in sizes, scenes vary in bandwidth rate

Burstiness complicates server & network provisioning

MPEG Compression

Fixed GoP pattern such as IBBPBBP IBBPBBP I frames are JPEG encoded B and P frames exploit temporal redundancy

I B B PB B P B

B I

Burstiness

One simple definition: Ratio between peak and average bit rate (PAR)

What to do ? (1)

Conservative rich: reserve peak required bandwidth. Wasteful, costly: ratio of peak to

average rate may be over 5:1.

Conservative poor: Compress to specified bandwidth

Peak Bandwidth Reservation

0 200 400 600 800 1000 12000.8

1

1.2

1.4

1.6

1.8

2

2.2x 10

4

Reserved peak-rate

Wasteful of Bandwidth

CBR Video Encoding b = target encoding rate

q(n) is a function of buffer contents quantization becomes very coarse as buffer contents approaches B

Why is it always blurry exactly when it starts getting interesting ?

Encoder Buffer

Digitizedvideo

B

q(n) = quantization scale

Xn

Storage

b/F bits per frame

What to do ? (2) Practical solution: trade bandwidth for

memory. With large storage space: trickle the

stream, playback from memory need huge space how about live broadcast?

What to do ? (3) Using Statistical Multiplexing

Multiple channels fill a CBR pipeMultiple channels fill a CBR pipe They fill it completelyThey fill it completely Each component is VBR or Each component is VBR or Piecewise CBRPiecewise CBR Not reallyNot really statisticalstatistical multiplexing,multiplexing,

butbut dynamicdynamic, , centrally controlledcentrally controlled

bitrate

time

What is Statistical MultiplexingWhat is Statistical Multiplexing? ?

Properties of Video Communication Properties of Video Communication ApplicationsApplications

Constant bit rate (CBR) or variable bit rate (VBR) coding

Constant bit rate leads to variable qualityVariable bit rate can enable constant quality Example of CBR: Digital TV, videoconferencing

over ISDN

Example of VBR: DVD, StatMux

Basic Video Coding Question:Basic Video Coding Question:

VBR vs. CBR codingVBR vs. CBR coding

Question: How many bits should we allocate to code each frame?

How to Allocation Bits Among Frames ?How to Allocation Bits Among Frames ?

Digitized (uncompressed) video has a constant rate

Question: Compress at a constant bit rate? Variable rate? Observations:

• Some frames are more complex than others, or are less predictable than others, and therefore require more bits• E.g. to achieve constant quality for every frame, a high

complexity frame would require more bits than a low complexity frame

Program complexity and bits

11 22 33

VBR vs. CBR CodingVBR vs. CBR Coding

Ordinary fixed rate encoding is insufficient

Statistical multiplexing into CBR channel

Stat Mux

Channelbuffer

Rc

MPEG-2Enc. 1

Bitstream , R1

Statistics

Source 1

MPEG-2Enc. 3

Bitstream, R3

Statistics

MPEG-2Enc. 2

Bitstream, R2

Statistics

.

.

.

MPEG-2Enc. 3

Bitstream, Rn

Statistics

Joint rate control

Bit rates

…Statistics

Source 2

Source 3

Source n

Main steps of a Statistical Multiplexer

Joint rate control algorithm is based on the feedback concept Statistics are produced by the encoders along with the

compressed bitstream. These statistics are continuously fed into the joint rate

controller from each encoder after compression of a picture. These coding statistics, together with the information on

channel buffer fullness, are used to dynamically compute the bit rate allocation for the individual encoders.

The bit rate of program i is obtained according to the complexity of that program and the sum of the complexities of all programs :

ii

ici X

XRR

Considerations Decoder buffers must not overflow or

underflow. How to distribute the total bitrate Channels are not synchronous - turn

down before up.

Joint rate control Sum of elementary

stream rates may not exceed allowed total

Distortion measured as a weighted average over the channels should be minimized

Distortion measure: average value of quantizer_scale over a frame. 0

1

2

3

4

5

6

7

8

ch 1 ch 2 ch 3 ch 4

rate

Joint rate control (Cont.) One simple assumption:

q(i)/k(i)=q(j)/k(j), all i,j

Assumption leads to both TM5 and StatMux solution:

r(j) = Rc(X(j)/k(j) / X(i)/k(i)) where

X(i) = b(i)*q(i)

Description of the communication commands between each encoder and the

StatMux

ack

Command (Rnew_i)

Response (Ri, Qi)

query

StatMux Encoderi

Program bit-rate for encoding the first set of video sources at a channel bit rate

of 16 Mb/s

Frame number0.1 s

4 Mbps

Mbit/sEnc. 1

Enc. 2

Enc. 3

Enc. 4

Total bit-rate requirements

CBR channel bandwidth16 Mbps

Frame number

Summation

Full VBR multiplexer design The expectations are that variable bit-rate (VBR)

encoding with joint bit-allocation will give some gain in quality and bandwidth efficiency over the previous scheme of piecewise constant bit-rate (CBR) encoding.

The global control system controls the bit rate on a global level, i.e. for each picture it computes a global target using the concept of picture complexity.

Multiplexing and Intelligent rate

control In a stat-mux system, we wish to distribute the channel

bandwidth among the program such that

The bit rate allocation can be achieved by two schemes: Changing the quantization factor (Qi). The StatMux sends a new vector of Q for all the encoders.

changing the video resolution i.e., reducing the picture resolution of some streams in the case when the aggregated bit rate is above the channel bit rate, and increasing the resolution when the aggregated bit rate is bellow the channel bit-rate

ci

i RR

Criterions for StatMux What is better ?

To allocate bits so that the picture quality is the same for all programs.

To allocate bits so that the picture quality of each program is the same as function of time.

Evaluation the PSNR statistics both as function of

time and as function of the stream index i.

PSNR_1

Histogram_1

PSNR_1

PSNR_1

Histogram_T1

Time

PSNR_1

Time

PSNR_2

Histogram

Histogram

Time

PSNR_N

PSNR_2

Histogram_2

Histogram_1

Histogram

PSNR_n

Histogram_Tn

Conclusions

Statistical Multiplexer improves video quality. Statistical Multiplexer improves data efficiency

Overall Bandwidth Wasted. The most important feature of VBR coding is its

ability to achieve similar or better video coding quality over random complexity video content while using fewer bits compared to CBR coding.

This amount of bit saving is a very significant gain for any digital communication system.