Overview of Fine Granularity Scalability inMPEG-4 Video Standard

Weiping Li, Fellow, IEEE

System configuration for Internet Streaming Video

FGS Scalability Structure

I P/B P/B P/B

MPEG-4 base layer

Fine-granular scalable enhancement layer

P/B

n e(n) n+1 e(n+1) n+2 e(n+2) n+3 e(n+3)

Layered scalable coding Tech.

SNR scalability Temporal scalability Spatial scalability

Illustration of video coding performance

SNR scalability decoder defined in MPEG-2

Layered scalable coding Tech.

SNR scalabilityIf the encoder uses the enhancement-layer information in

the motion-prediction loop and the enhancement-layer information is (not) received by the decoder, the enhancement-layer coding efficiency is high (low).

If the encoder does not use the enhancement-layer information in the motion-prediction loop and the enhancement-layer information is (not) received by the decoder, drift happens in the enhancement layer and coding efficiency is low. (the result is the same as using the base layer only.)

Layered scalable coding Tech.

Temporal scalability

Layered scalable coding Tech.

Spatial scalability

BIT-PLANE CODING OF THE DCT COEFFICIENTS

BIT-PLANE CODING OF THE DCT COEFFICIENTS

← CCIR-601 Sequences

SIF and QCIF Sequences →

FGS USING BIT-PLANE CODING OF DCT COEFFICIENTS

Overall Coding Structure of FGS Some Details of FGS Coding Profile Definitions in the Amendment of

MPEG-4

Overall Coding Structure of FGS

FGS encoder structure

Overall Coding Structure of FGS

FGS decoder structure

Some Details of FGS Coding

1) Different Numbers of Bit-Planes for Individual Color Components

2) Variable-Length Codes3) Decoding Truncated Bitstreams4) Variations to the Standardized FGS C

oding Structure

Different Numbers of Bit-Planes for Individual Color

Components

Variable-Length Codes Statistics of the (RUN, EOP) symbols in the four VLC

tables

Coding patterns for syntax element fgs_cbp

Coding patterns for syntax element fgs_cbp

Decoding Truncated Bitstreams

Decoding of the truncated bitstream is not standardized in MPEG-4.

One possible method To look ahead 32 bits at every byte-aligned position in t

he bitstream. If the 32 bits are not fgs vop start code, the first 8 bits o

f the 32 bits are information bits of the FGS frame to be decoded. The decoder slides the bitstream pointer by one byte and looks ahead another 32 bits to check for fgs vop start code.

Variations to the Standardized FGS Coding

Structure

A possible variation of FGS encoder structure

DCT Domain

Variations to the Standardized FGS Coding

Structure

A possible variation of FGS decoder structure

“Find Reference”

To generate the reference signal in the DCT domain to be subtracted in the encoder and added in the decoder.

The operation of taking residue is slightly different in the variation method.

The distribution of such a residual signal

Taking Residue between Original and Reconstructed DCT Coefficients.

The distribution of such a residual signal

Taking residue between original and lower boundary point of quantization bin.

Comparison of two different ways of generating the DCT

residues

Encoder structure without the DCT unit in the enhancement

layer

Experiment results on mismatch errors

Profile Definitions in the Amendment of MPEG-4

Advanced Simple Profile contains a subset of nonscalable video-coding to

ols to achieve high coding efficiency at any given bit rate within a wide range of bit rates.

The base-layer coding tools defined in the Advanced Simple Profile include both P-VOP (forward pre-diction only) and B-VOP (bi-directional prediction) for coding motion-compensated residues.

The FGS profile is defined to meet the requirement of optimizing video quality over a given bit rate range.

CODING EFFICIENCY PERFORMANCE OF FGS

FGS versus Multilayer SNR Scalability FGS versus Non-Scalable Coding FGS versus Simulcast

Compare FGS with multilyer SNR scalability

Compare FGS with non-scalable coding

Compare FGS with simulcast

ADVANCED FEATURES IN FGS

Frequency Weighting Selective Enhancement Error Resilience FGS Temporal Scalability

Frequency Weighting

Selective Enhancement

Error Resilience

FGS Temporal Scalability

FGST organized into a separate layer from FGS

FGST and FGS organized into a single enhancement layer.

Conclusion

Advantages of using FGS for Internet streaming video applications It allows separation of encoding and tran

smission. The server can transmit enhancement lay

er at any bit rate without transcoding. It provides a solution to the video server

overload problem.