Fine Granularity Video Compression and Optimal FEC Assignment for FG Video Streaming over Burst...

Fine Granularity Video Compression and Optimal FEC Assignment for FG Video Streaming over Burst Error Channel

Yih-Ching Su

Department of Computer Science and Engineering, National Sun Yat-Sen University

2

Contents

1. Introduction2. Gilbert Channel with Loss Rate

Feedback3. Optimal FEC Assignment for FG Video4. HSDD Motion Estimation Metric5. HMRME Motion Estimation Algorithm6. ABEC Embedded Coder7. Conclusions & Future Works

3

Introduction 4

Research Focuses

Optimal FEC assignment scheme for FG video transmission over burst error channel (as wireless Internet) with or without loss rate feedback.

Wavelet domain video compression algorithms with high-performance or low-complexity features.

Introduction 5

Research Focuses (cont.)

MotionEstimation

TransformQuantization

& EntropyCoding

FECProtection

Raw Video

Error-Resilient Video Packets

Source Coder

Channel Coder

HSDDHMRME

ABEC

Optimal FEC Assignment

Introduction 6

Definition of Fine Granularity Video Stream

Bit stream is scalable (layered). Rate can be precisely controlled.

R

min.(base layer length)

max.

arbitrary enhancement layerlength (in bits)

Introduction 7

Merits of Fine Granularity Video Stream

Precise rate control Bandwidth adaptation

HeterogeneousInternet

Environment

MediaServer

FGVideo

Encoder

Client

BL

ELFG

ClientBL

EL

BL

ELNo transcoding!

8

Merits of Fine Granularity Video Stream (cont.) Content-adaptive error protection

EL

BL BL

EL

Equal Error Protection Unequal Error Protection

Introduction 9

Fine Granularity Video Compression Systems

DCT based: MPEG-4 FGS

ISO/IEC 14496-2:2001/Amd 2:2002 Base layer plus enhancement layer

DWT based: “Multirate 3-D subband coding of video”,

D. Taubman et al., 1994. “3D SPIHT”, B.-J. Kim et al., 2000. “HSDD”, Y.-C. Su et al., 2003.

10

Gilbert Channel with Loss Rate Feedback

11

Packet Loss

Packet loss can severely affect the quality of delay sensitive multimedia applications.

FEC (Forward Error Correction) technique can be used when delay time is strictly restricted.

data FEC redundancy

data len = k pkts

BOP len = n pkts),( nmP : the probability of m lost packets within a block of n packets.


12

Gilbert Channel Model

The ability of the application to react is enhanced by the availability of simple and efficient loss models.

A two state Markov model or Gilbert-model is often used to simulate burst loss patterns over wired/wireless channel.

C. C. Tan, N. C. Beaulieu, ”On first-order Markov modeling forthe rayleigh fading channel,” IEEE Commun., 2000.


13

Enhanced Video Transmission over Gilbert Channel

Feedback loss rate. Decide FEC protection ratio relying

on a new probability function which is conditioned on loss rate feedback.

BOP0BOP-bBOP-b-1BOP-b-v+1

mm1m2mv

feedback delay b


14

Renewal Error Process Packet loss over

Gilbert-model can be modeled with a renewal error process.

The lengths of consecutive inter-error intervals (also called gaps) are independently and identically distributed.

Gap probabilities:

Probability that m-1 packet losses occur in thenext n-1 packets following an error:

Probability that m packet losses occur withina block of n packets:

E. N. Gilbert, "Capacity of a burst-noise channel," Bell Syst.Tech. J., vol.39, pp.1253-1265, Sept. 1960.E. O. Elliott, "A model of the switched telephone networkfor data communications," Bell Syst. Tech. J., 1965.


15

Probability Toolbox

E EE#(m-1)

n-1

E SE#(m-1)

n-1

16

Probability Toolbox (cont.)

S S#(m-1)

n-1

S S#(m-1)

n-1

S

S S#(m-1)

n-1

E


17


S

n

S

S

n

E

E

n

S

E

n

E

18


E SE#(m)

n-1

E EE#(m-1)

n-1

S EE#(m-1)

n-1

S SE#(m)

n-1


19

Iterative Equation Set

BOP-bBOP-b-iBOP-b-v+1


20

Initial Conditions

.1,),()1(

,1),,()1(5.0),(

)1(0,00 btnmnTP

bnmnTPnmp

nbB

B

.1,),()1(

,1),,(5.0),(

)1(0,01 btnmnTP

bnmRPnmp

nbB

B

.1,),(

,1),,()1(5.0),(

)1(0,10 brnmRP

bnmnTPnmp

nbB

B

.1,),(

,1),,(5.0),(

)1(0,11 brnmRP

bnmRPnmp

nbB

B


21

Conditional Probability Function


22

Validation of Correctness


23

Performance Evaluation

.),...,,|Pr(

,),(

11

1

n

iv

n

i

mmbiimE

nipimE

24

Optimal FEC Assignment for FG Video

25

FEC Assignment Schemes

Equal error protection Content-adaptive unequal error

protection Content-adaptive plus channel-

adaptive unequal error protection

B. Hong and A. Nostratinia, "Rate-constrained scalable video transmission over the internet," Packet Video 2002.

Y.C. Su, C.S. Yang, and C.W. Lee, "Optimal FEC Assignment for Scalable Video Transmissionover Burst Error Channel with Loss Rate Feedback," Packet Video 2003.

26

Block of Packets (BOP) Structure

Layer 0 Layer 1 Layer lLayer i

k0

packet

numberof

packetsn

k1ki

kl

s0 s1 si sl

FEC overhead

packet size s


27

Complete Expected Quality


28

Simplified Expected Quality

content adaptive

content+channel adaptive


29

The Optimization Problem

Constrained by


30

Dynamic Programming


31

Validation of Correctness

(i) frame resolution = CIF format (352x288)

(ii) constant stream rate = 256 Kbps

(iii) 1 GOP = 1 intra frame accompanied with 14 inter frames and frame rate = 15 fps

(iv) sequence length = 9 GOPs


32

Performance Discrepancy between Complete & Simplified Models


33



34

Performance Evaluation (cont.)


35


)3()4( UEPPSNREUEPPSNRE


36




37



38

HSDD Motion Estimation Metric 39

Bit-Plane Coding The Core of FGS

or Embedded Coder

Just bit-plane coding!


Zero-Tree Coding Natural images in general have a

low pass spectrum. Large wavelet coefficients are

more important than small wavelet coefficients.

A zero-tree is a quad-tree of which all nodes are equal to or smaller than the root.


Hierarchical Sum of Double Difference Metric

Zero-tree coding aware Jointly constrain motion vector

searching for both temporal and spatial (quad-tree) directions

Fewer bits are spent later for describing isolated zeros


Sum of Absolute Difference Metric

2p+1

Current Block

Reference Block

2p+

1

.,,,),(),(

),(

1 1ppvujiCvjuiR

vuSADn

i

n

j

:),( jiC Current block's pixel(block size nxn)

:),( vjuiR Reference block's pixelwithin search area (2p+1)x(2p+1)

SAD metric conflicts with the zerotree rule often,because the goal of SAD metric is just to minimizethe temporal difference, and it is irrelevant to themagnitude hierarchy of the spatial quad-trees.


HSDD Metric Calculation

.,,,),(),()2/,2/(

),(

1 11 ppvujiCvjuiRji

vuHSDDn

i

n

j

ol

ol

ol

ol

:),( jiCol Current block's pixel (block size nxn)

:),( vjuiRol Reference block's pixel within

search area (2p+1)x(2p+1)

:)2/,2/(1 jiol Corresponding parent pixel information

in the upper level of motion compensation pyramid

Double Difference

Sum

Hierarchy


Observations on HSDD Metric

HSDD value may be negative, but a larger positive one is preferred.

Given any parent pixel information, the maximal HSDD(MV) occurs if and only if the perfect SAD matching exists, that is SAD(MV)->0.


Motion Estimation Applying HSDD Metric

,),(maxarg

,,vuHSDDMV o

lppvu

,),()2/,2/(1 1

1

n

i

n

j

ol

ol jiCjiTH

otherwise. mode,intra

TH, HSDD(MV)if mode,inter MOD


Layered Magnitude Distributions for HSDD & SAD



48

HMRME Motion Estimation Algorithm

49

Half-Pixel Multi-Resolution Motion Estimation

Combine transform-adapted half-pixel interpolation with anti-aliasing under complexity constraints.

Avoid multiple inverse transforms. Can be united with the

conventional wavelet domain motion estimation algorithms.


50

H-Transform

dy

xo

hh

hh

aa

aa

1

1110

0100

H

H

11

10

01

00

a

a

a

a

a

d

y

x

o

h

h

h

h

h

1111

1111

1111

1111

21

H h = H a


51

Aliasing


52

Half-Pixel Interpolation

i

topo

oh

h levelmax_2

ˆ

d

y

x

o

h

h

h

h

h

ˆ

ha ˆHˆ 1

ttt hah ˆHHˆHˆ 1


53

Horizontal Interpolation

2/)(

2/)(

2/)ˆˆ(

1122

1122

1122

dydyxd

dydyxy

xoxoxx

hhhhh

hhhhh

hhhhh


54

Vertical Interpolation

2/)(

2/)ˆˆ(

2/)(

1133

1133

1133

dxdxyd

yoyoyy

dxdxyx

hhhhh

hhhhh

hhhhh


55

Diagonal Interpolation

4/)ˆˆˆˆ(

4/)ˆˆˆˆ(

4/)ˆˆˆˆ(

2/)ˆ(ˆ

2/)ˆ(ˆ2/)ˆ(ˆ

2/)ˆ(ˆ

4444

3333

2222

1111

dgjmh

dgjmh

dgjmh

hhhhm

hhhhj

hhhhg

hhhhd

dd

dy

dx

dyxo

dyxo

dyxo

dyxo


56


MRME: Y. Q. Zhang, S. Zafar, “Motion-Compensated Wavelet Transform Codingfor Color Video Compression,” IEEE CSTV, 1992.

AMRME: M. K. Mandal, E. Chan, X. Wang and S.Panchanathan, “Multiresolution Motion EstimationTechniques for Video Compression,” OpticalEngineering, 1996

57

ABEC Embedded Coder 58

Array-Based Embedded Coder Performance similar to SPIHT (Amir S

aid and William A. Pearlman, ”A New Fast and Efficient Image Codec Based on Set Partitioning in Hierarchical Trees,” IEEE CSVT, 1996)

One pass processing & no link lists Hardware implementation friendly R.O.C. patent no. 141267, 2001

59

ABEC Encoding FlowRaw

Image

WaveletTransform

RemoveDC Gain

EstablishSignificance

Map

Predict BitExpenditure

LastRound?

ABECFinal

Processing

YES

Stop

ABECNormal

Processing

NO


Max values

Max value

Significance Map


ABEC Encoder Structure

S

R

C

P

B i t B u dgetC on tro ller

W av eletT ran sform

C oeffi cien ts

O u tpu t B i tS tream

S ign ifi cen ceM ap

Definitions of ABEC Status Bits•P: parent’s significance bit•S: parent’s sign bit•R: parent’s refinement bit•C: children’s significance bit

“Zero-tree”

62

Conclusions & Future Works 63

Conclusions Joint optimization for wavelet domain

ME & zero-tree coding can raise the compression performance significantly (HSDD).

According to the prediction for DC coefficients in wavelet domain, the ideas of fast anti-aliasing & transform-adapted half-pixel interpolation can be combined (HMRME).


Conclusions (cont.)

One pass processing & no link lists; fast & hardware friendly zero-tree coding is possible (ABEC).

The loss probability function for Gilbert channel conditioned on past loss rates can be calculated out by an iterative equation set.


Conclusions (cont.)

Content-adaptive plus channel-adaptive (loss rate feedback) unequal error protection can further enhance FG video transmission efficiency.

Simplified quality prediction formulas can be used with trivial performance degradation while significant speeding up.


Future Works

Exploit possible optimal or sub-optimal weighting rules for the two difference terms in HSDD metric.

Extend HMRME (by lifting scheme?) to be available for overlapped transforms.

Try to find some other better estimation method for ho in HMRME.


Future Works (cont.)

Upgrade to an context-based entropy-constrained version of ABEC coder.

Investigate the affection of packet length to FG video transmission over bit-error channel.

68

Date post:	21-Dec-2015
Category:	Documents
View:	217 times
Download:	0 times

Fine Granularity Video Compression and Optimal FEC Assignment for FG Video Streaming over Burst...

Documents