Real Time Transcoding

Soam AcharyaJuly 30, 1997

Cornell University

Why Transcode?

Random access vs. Good compression M-JPEG vs. MPEG

Editing Processing Gateways

Roadmap

Brief introduction to MPEG and JPEG

Transcoding: Spatial Domain Approach Transcoding: Compressed Domain

Experiments Results CMT/Demo

JPEG Summarized

DCT Quantize

Zigzag Scan

Run-length CodeHuffman Code

Block

0101.

MPEG Summarized Sequence of images/frames reference frames: past, future frames consist of various types of blocks

I P P IB B B B B B

I frame

P frame

B frame

Dependence

Inter-frame dependencies in MPEG

MPEG Summarized (cont)I

to P

pro

cess

ing

Compressed block

Residual block: correction coefficients

Predicted Block

Decompressed Block

decom

decom

comp

compressed I frame partially restored P framerestored I frame

Add

Spatial Domain Transcoding

Simple Method– Decompress each MPEG frame to pixel image– JPEG compress each pixel image

public MPEG decoders, JPEG encoders readily available

“straw man” for comparisons

MPEG decompress

JPEG compress

Spatial Domain Transcoding II Optimizations

– I frames: direct conversion– lazy decompression, cache decompressed blocks

Quality: OK Speedup: a few percent (except for I frames)

I P P IB B B B B B

Source Block Extraction Problem

predicted block

overlaps reference blocks

portionof reference

image

reference blocks required toreconstruct predicted block

Compressed Domain Transcoding

Develop compressed domain translation

Compressed Domain Source Block Extraction Process

dx

Using Translation For Block Extraction

Translating on each axis separately

dx

dy

Add

dx

Add

dx

Add

A

Deriving the Translation Operator

011011...

RLEvector IDCT, etc

EntropyDecoding 72717073

73737273

75747372

70707069

011011...

FDCT, etc EntropyCoding 72727274

7373747572747375

71706968

36.21.30.4-0.8

0.0

3610-1

0

36

1

-1

0

0

0

1

0

36

1

-1

0

0

0

1

0

Translation Transform Calculation

Techniques to– skip multiplies– avoid unnecessary computations– unroll loops– hardcode multiplies

All transcoding implementations run on HP 735 PA-RISC machine

MPEG decoder from Berkeley Performance measured in terms of frames

transcoded per second Frames were grayscale Run on various MPEG sequences Quality in PSNR

– 20 is bad quality, 30 is good, 40 is perfect

Experiments

Experiments on “bike.mpg”

frames/sec PSNR PSNR PSNR Speedup I P B Method I 5.7 36.3 37.5 37.8 1.0Method II 5.9 36.3 37.4 37.7 1.0

Method IIImaxerr=0 6.5 36.3 35.3 35.7 1.1maxerr=5 8.1 36.3 35.0 35.3 1.4maxerr=10 9.6 36.3 34.5 34.7 1.7maxerr=15 10.5 36.3 33.5 33.6 1.8

Picture Quality Comparisons

Decoded to grayscale

Maxerr = 0 Maxerr = 10

Results On Various Streams

Transcoding Performance

05

10152025303540

methods used

fram

es/s

ec

alesi.mpgbike.mpgbus.mpgcannon.mpgus.mpg

Performance Charaterizations Sparseness of MPEG stream

– sparse: us.mpg, cannon.mpg– dense: bike.mpg, alesi.mpg – non-standard quantization tables: bike.mpg

Good for sparse streams (conferencing)

Potential for improvement:– 50% of code spent on matrix multiplies– rest divided between Huffman encoding and run

length encoding matrices to vectors

Integration into CMT

MPEG File MPEG File

Transcoder Transcoder

JPEG Mixer

JPEG Player