A Sequence-Based Rate Control Framework for Consistent Quality Real-Time Video

Bo Xie and Wenjun Zeng

CSVT 2006

Outline

Introduction Global Bit-allocation Target Bit Calculation Quantization Parameter Determination Simulation Results

Introduction (1)

Rate control Bit-allocation Bit-allocation achievement (QP determination)

MPEG-2 TM5, MPEG-4 Annex L, and H.263+ TMN8 GOP-based bit-allocation Fixed size GOP Constant bit-allocation among GOPs and among the sam

e type frames Some assumptions are necessary

Stationary video sequence Similar characteristics of all GOPs

Fixed-size GOPs do not match the scene structure

Introduction

QP determination Different R-Q models are developed There is no guarantee that a model is

always accurate (model mismatch) Resulting in buffer overflow or underflow

Introduction (3)

Problem Formulation

Subject to

seq

seq1seq

* |)()(|1

minargN

iii

Qqqd-qd

Nq

N

seq,...,2,1 ,),(0 NiBqiB s

TNqqqq },...,,{ **

2*1

*

seq

# of framesDistortion for the ith frame with QP qi

Average distortion

seq

1seq

)(1

)(N

iii qd

Nqd

)0,)(),1(max(),( CqrqiBqiB ii

Actual bits for the ith frame with QP qi Bit rate/Frame rate

Buffer sizeBuffer fullness

introduction (4)

Goal: Constant quality Allocating more bits to high complexity

scenes/frames, and less bits to low complexity scenes/frames Coding complexity: The number of bits required to

encode a frame

Three stages Global bit-allocation model Target bit calculation QP determination

Global Bit-allocation (1)

Relation between rate and complexity

R-J relation

N

ijijS

1

2,

The variance of the ith MB in the jth frameOptimal target bit rate for the jth frame

Bit budget for that GOP

# of pixels for each MB # of MBs for each frame

Average rate for the mth frame (bit/pixel)

The energy of the jth frame

N

iimdev

Nmdev

1

1

A

jiiji xx

Amdev

1, ||

1

Intensity of the jth residue pixel of the ith MB

Average residue intensity of the ith MB

Mean deviation

Average bits for motion vectors of a frame

Global Bit-Allocation (2)

Problems of traditional models They cannot differentiate Intra and Inter

frames (Too less bits for Intra frames) Proposed R-MAD model

Value of original pels for intra MBs

Value of residue pels for inter MBsConstant Shift factor

Is chosen as ½ R = K*(MAD)½

If a frame has MAD > average MAD, and calculated QP < average QP, set QP = average QP

Target Bit Calculation (1)

Calculation of K

represents the complexity of a scene/frame

Scene change detection Stationary assumption is no more necessary Most existing GOP-based bit-allocation schemes use

only past source data Worst case: the scenes get more and more complex or

simpler

Buffer output rate (bit rate/frame rate)

MAD of the current frame

Average MAD of all previous encoded frames

1MAD

n

CK

1MAD

MAD

n

n

Target bit count of the current frame

Target Bit Calculation (2)

Adjustment by actual bit account

Achievement of constant quality Buffer constraint

Target bit count for the ith frameActual bit count for the ith frame

Bs VBV_fullness

Quantization Parameter Determination (1)

Bit-allocation guarantee A R-Q model is used to determine QP for the tar

get bits A traditional model-based QP determination has

no bit-allocation guarantee Resulting in “error propagation” e.g. MPEG-4 Annex L

QP re-adjustment If |Actualbit - Targetbit|/Targetbit > Threshold, re-quant

ization is performed to achieve bit-allocation guarantee

Quantization Parameter Determination (2)

Proposed R-Q model Initial QP determination

(similar to )

2Q

X

MAD

R …

n

Window size

n-M

# of re-quantization timesActual # of bits used for the jth re-quantization of the p*th frame in the past

Actual QP used for the q*th re-quantization of the p*th frame in the past

221

Q

X

Q

X

MAD

R

R-Q model

Most similar MAD

Most similar QP

Quantization Parameter Determination (3)

Proposed R-Q model QP re-adjustment

Check convergence

If re-adjusted QP cannot converge to the target bit count,

MAD/Targetbit

MAD/prevActualbit_prev_new_ QPQP

QP_left QP_rightQP_prev

QP_left QP_right

QP_rightQP_left

Actualbit < Targetbit:

Actualbit > Targetbit:

QP_left QP_right QP_prev

QP_new

Quantization Parameter Determination (4)

Proposed R-Q model Re-quantization algorithm

QP final sanity check If then )QPQP && MAD(MAD 11 nnnn 1QPQP nn

)Targetbit

prevActualbit_prev_new_( QPQP

Quantization Parameter Determination (5)

Proposed R-Q model Buffer overflow checking

Simulation Results (1)

Sequences (QCIF) Foreman (medium motion, one scene

change) Sea World (high motion, two scene

changes) Glasgow (lots of scene cuts) Charlie’s Angels (high motion, lots of

scene changes)

Simulation Results (2)

Model failure rate and re-quantization times

|Actualbit-Targetbit| / Targetbit > 30% MPEG-4 Annex L

Simulation Results (3)

Frame dropping and PSNRMPEG-4 Annex LProposedProposed without re-quantizationR- model*

*Z. He and S. K. Mitra, “A unified rate-distortion analysis framework for transform coding,” CSVT 2001.

Simulation Results (4)

Quality smooth Smaller buffer (0.5s)

Simulation Results (5)

Buffer fullness (0.5s)

Simulation Results (6)

Quality smooth Larger buffer (2s)