MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding

Authors from: University of GeorgiaSpeaker: Chang-Kuan Lin

2

Reference

S. Chattopadhyay, S. M. Bhandarkar, K. Li, “FGS-MR: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding,” ACM NOSSDAV 2006.

W. Li, “Overview of Fine Granularity Scalability in MPEG-4 Video Standard,” IEEE Trans. on Circuits and Systems for Video Technology, Vol. 11, No. 3, pp. 301-317, Mar. 2001.

H. Radha, M. van der Schaar, and Y. Chen, “The MPEG-4 fine-grained scalable video coding method for multimedia streaming over IP,” IEEE Trans. on Multimedia, vol.3, pp. 53–68, Mar. 2001.

3

Outline

Introduction MPEG-4 Fine Grained Scalability Motivation

FGS-AQ vs. FGS-MR Experimental Results Conclusion

4

Introduction MPEG4 Fine Grained Scalability (FGS) profile for

streaming video Base Layer Bit Stream

must exist at the decoder has coarsely quantized DCT coefficients provides the minimum video quality

Enhancement Layer Bit Stream can be absent at the decoder contains encoded DCT coefficient differences provides higher quality can be truncated to fit the target bit rate

5

FGS Encoding Block Diagram

6

Motivation Base Layer video quality is usually not

satisfactory in order to provide a wide range of bit rate adaptation

MPEG4 FGS Adaptive Quantization (FGS-AQ) for Base Layer video does not provide good rate-distortion (R-D) performance parameter overhead at the decoder

Proposed FGS-MR no parameter overhead to transmit transparent the codec better rate-distortion performance

7

Outline Introduction

MPEG-4 Fine Grained Scalability Motivation

FGS-AQ vs. FGS-MR FGS-AQ FGS-MR

MR-Mask Creation MR-Frame

Experimental Results Conclusion

8

FGS Adaptive Quantization (AQ) Goals

To improve visual quality To better utilize the available bandwidth

Method Define different quantization step sizes for differen

t transform coefficients within a macro-block (low freq. DCT coeff. => small step

size) for different macro-blocks (different quantization factors)

Disadvantages R-D performance degrades due to FGS-AQ param

eter overhead

9

Proposed Multi-Resolution FGS (FGS-MR)

Goal To improve the visual quality To better utilize the available bandwidth No transmission overhead and hence maintaining

the R-D performance Method

Apply a low-pass filter on “visually unimportant” portion of the original video frame before encoding.

10

Two Equivalent Operations

Apply a low-pass filter on the spatial domain of an image

Truncate DCT coefficients in the corresponding transform domain of an image

11

FGS-MR Process (Step 1)

MR-Mask creation Use Canny edge detector to detect edges Weight Mask

an weight parameter wi, j for each pixel p(i, j) of an image, 0 ≦ wi, j 1≦

wi, j = 1, if p(i, j) is on the edge 0 < wi, j 1, if ≦ p(i, j) is near edge wi, j = 0, if p(i, j) is in non-edge region

12

Original (5.12Mbps)

13

MR-Mask

14

FGS-MR Process (Step 2)

MR-Frame Creation VI = (I-W) VL +W VH

VF = Iteration( VI, G(σI))

Note VI contains abrupt changes i

n resolution VF is a smooth version of VI

Parameters Vo: original video frame VL : low resolution frame from the co

nvolution of Vo and G(σL) VH : high resolution frame from the c

onvolution of Vo and G(σH) VI : intermediate video frame VF : final multi-resolution frame I: matrix with all entries as 1 W: MR-mask weight matrix G(σ): Gaussian filter with standard d

eviation of σas LPF σL >σH

15

Original (5.12Mbps)

16

FGS-AQ (0.17Mbps, PSNR = 22.77dB)

17

FGS-MR (0.17Mbps, PSNR = 26.5dB)

18

Determine Parameters σL, σH, and σI

to control the bit rate W (weight matrix)

to control the quality of the encoded video frame Figure of merit function: δ=Q/C

Q = 2^( PSNR(σL, σH, σI)/10 ) or PSNR = 10log(Q) C: compression ratio

The authors empirically determine the parameters σL = 15, σL = 3, and varying σI

19

Outline Introduction

MPEG-4 Fine Grained Scalability Motivation

FGS-AQ vs. FGS-MR FGS-AQ FGS-MR

Experimental Results Rate Distortion Resource Consumption

Conclusion

20

Experiments

Video 1 320x240, fps = 30 A single person walking in a well lighted room

Video 2 176x144, fps = 30 A panning view across a poorly lighted room. No moving object

21

Rate Distortion Performance

Vary σI from 3 to 25 to adjust the target bit rate

22

Power Consumption

Energy used and hence power consumed by wireless network interface card (WNIC):

T: time durationS: data sizeb: the bit rate of streaming videoB: available BWER: energy used by WNIC during data reception

Es: energy used by WNIC when sleeping

23

Power Consumption Comparison

24

Conclusion

The rate distortion performance of FGS-MR is better than FGS-AQ.

FGS-MR can be seamlessly integrated into existing MPEG4 codec.

My comment Processing time issue of FGS-MR Empirical determined filter parameters