Decision Trees for Error Concealment in Video Decoding

Song Cen and Pamela C. Cosman, Senior Member, IEEE

IEEE TRANSACTION ON MULTIMEDIA, VOL. 5, NO. 1, MARCH 2003

Outline

Introduction Error Concealment Methods

Panning Top/botMV

Classification tree design CART algorithm

Experiments and Results conclusions

Introduction(1/2)

Error control or concealment Forward error correction added at the encoder Post-processing methods employed by the decoder

Three main approaches for error concealment Frequency： using the corresponding DCT coefficient of

neighboring blocks (low frequency) Spatial： bilinear interpolation from the nearest MBs

(computational complexity is large) Temporal： searching for a block from other frames

Introduction(2/2)

Hybrid methods： combine more than one of the three approachs Ex： In temporal concealment, the referenced

block can be improved by spatial smoothing at its edges (additional complexity)

Adaptive methods： using different approachs in different situations Ex： Temporl concealment is used for most

blocks while spatial concealment is used when scene changes (additional complexity)

Error Concealment Methods(1/3)

spatial： interpolate linearly from boundary pixels in top/bottom MBs

frequency： weighted average of first 9 DCT coefficients of top/bottom MBs

panning： use the camera panning motion vector

top/botMV： use top MV for top 8*16 sub-MB,use bottom MV for bottom 8*16 sub-MB

Error Concealment Methods(2/3)

averageMV： use the average motion vectors of top and bottom MBs

useonlyMV： top or bottom MB is Intra-coded =>use the only MV available

spat+onlyMV： use only available MV for nearest half, spatial interpolation for rest

copyPmb： copy co-sited MB from previous P frame if it’s Intra-coded or has MV=0

Error Concealment Methods(3/3)

Classification tree design(1/4)

A classification tree was built at the encoder. It was then transmitted to the receiver.

The true class of every MB is defined to be the best concealment method in table I, which provides the minimum MSE.

The decoder will classify each lost MB using the decision trees

sample

Classification tree design(2/4)

The classification parameters are measurements which describe the spatial, temporal, and frequency domain context of a MB.

They must be parameters which are available to the decoder even if the MB is lost.

The parameters include both ordinal and categorical variables.

Classification tree design(3/4)

CART algorithm was used to design the tree at the encoder.x： the vector of measurements associated with the missing MB.

C： 1,2,…8 be the set of eight EC methods

d(x)： assigns to every vector x a class j from C

L： a training sequence consists of data (x1,j1), (x2,j2),…,(xn,jn)

The root node of the tree contains all the N training cases

Each split depends on the value of only a single variable.

Classification tree design(4/4)

For each variale, we find the split which provides the greatest decrease in node impurity. (using the Gini index to measure the purity of a set of data)

Comparing all of these,and find the best overall split of the data.

Assigning an EC method to every terminal node.

Experiments and Results(1/5)

For any node, 1 bit to indicate which type of this node.

For a internal node, 5 bits to specify which variable to split on, and 7 bits to specify the splitting threshold.

For a terminal node, 2~3 bits to specify which EC methods to use.

Experiments and Results(2/5)

If the tree grows large enough, eventually the classification will be perfect => “omniscient minimum” MSE

“maximum” MSE is the MSE that results from using a single fixed and best method from table I.

“relative MSE” is the MSE compared to that of the best of the fixed concealment methods. (best fixed = 1)

Experiments and Results(3/5)

panningspatialcopyPmb

0.08%

Experiments and Results(4/5)

0.45%

Experiments and Results(5/5)

Conclusions

Two new temporal EC methods Panning Top/botMV

The use of a decision tree provided lower distortion than any fixed method alone.

The memory and computational requirements are quite asymmetric