Video TranscodingVideo Transcodingwith Intel IPPwith Intel IPP

Eric ShufroEric Shufro

April 27, 2004April 27, 2004

COT6930COT6930

Introduction to TranscodingIntroduction to Transcoding

What is transcoding?What is transcoding?Why transcode?Why transcode?What is involved?What is involved?PerformancePerformanceQualityQuality Intel IPPIntel IPPApplicationsApplications

Transcoding OverviewTranscoding Overview

• Reduced bitrate

• Reasonable Quality

• Performance Gain

MPEG-2 bit stream

MPEG-4 bit stream

MPEG-2 bit stream

MPEG-4 bit stream

Decode Encode

Partial Decodeand Encode

1

2

ApplicationsApplications

Streaming video for both broadband and Streaming video for both broadband and narrow band networks.narrow band networks.

Decreased video bitrate for playback on Decreased video bitrate for playback on mobile or other small embedded devices.mobile or other small embedded devices.

Conversion and modification of pre-Conversion and modification of pre-encoded bit streams.encoded bit streams.

Perhaps steganography? (jpg, mp3…)Perhaps steganography? (jpg, mp3…)

Integrated Performance PrimitivesIntegrated Performance Primitives

Provides source code and libraries for Provides source code and libraries for media types such as MP3, MPEG-2, media types such as MP3, MPEG-2, MPEG-4, H.263, JPEG, JPEG2000, GSM-MPEG-4, H.263, JPEG, JPEG2000, GSM-AMR, G.723, and computer vision. [Intel]AMR, G.723, and computer vision. [Intel]

Well documented.Well documented.Easy to use.Easy to use.

MPEG-2 DecoderMPEG-2 Decoder

Runs in two separate threads.Runs in two separate threads.Responsible for splitting the input stream Responsible for splitting the input stream

into two separate parts, audio and video into two separate parts, audio and video and then decoding the video stream into and then decoding the video stream into YUV components.YUV components.

YUV buffers are available through the YUV buffers are available through the transcoder class to the input of the transcoder class to the input of the encoder.encoder.

MPEG-4 EncoderMPEG-4 Encoder

Uses input data from the decoder YUV frame Uses input data from the decoder YUV frame buffers on a frame by frame basis.buffers on a frame by frame basis.

Creates an MPEG-4 bit stream file, out.cms, Creates an MPEG-4 bit stream file, out.cms, between 800KB and 1MB in size, video only.between 800KB and 1MB in size, video only.

Motion Estimation can be disabled for testing Motion Estimation can be disabled for testing purposes.purposes.

Requires input parameters via a parameter file, Requires input parameters via a parameter file, though some elements can be ignored.though some elements can be ignored.

Transcoder ArchitectureTranscoder Architecture

MPEG-2 decoder and MPEG-4 encoder MPEG-2 decoder and MPEG-4 encoder based on the IPP.based on the IPP.

Transcoder class encapsulates both the Transcoder class encapsulates both the encoder and decoder.encoder and decoder.

Memory is accessible between the Memory is accessible between the encoder and the decoder.encoder and the decoder.

Transcoder runs in three separate threads.Transcoder runs in three separate threads.

Transcoder InitializationTranscoder Initialization

Read application parameters, source file, Read application parameters, source file, encoder parameter file and output file name.encoder parameter file and output file name.

Read the encoder parameters into memory.Read the encoder parameters into memory. Create decoder and splitter thread.Create decoder and splitter thread.

DecoderInit()DecoderInit() EncoderInit(), overwrite parameters.EncoderInit(), overwrite parameters. Splitter – Open bit streamSplitter – Open bit stream

Encode MPEG-4 header.Encode MPEG-4 header. Begin transcoding.Begin transcoding.

Input StreamInput Stream

MPEG-2.MPEG-2.Contains both audio and video.Contains both audio and video.147 frames @ 720x480.147 frames @ 720x480.Source file is 3.57 MB.Source file is 3.57 MB. Intra and Inter coded frames (I, P, B).Intra and Inter coded frames (I, P, B).

Encoder ModificationsEncoder Modifications

ExpandFrame() bypassed.ExpandFrame() bypassed.StepLuma and StepChroma artificially set StepLuma and StepChroma artificially set

after decoder init.after decoder init.Parameters overwritten after encoder init.Parameters overwritten after encoder init.mp4_MacroBlock changed to a public mp4_MacroBlock changed to a public

member of ippVideoEncoderMPEG4.member of ippVideoEncoderMPEG4.Motion Estimation can be disabled.Motion Estimation can be disabled.

(avoids Sum of Absolute Difference)(avoids Sum of Absolute Difference)

Modified ParametersModified Parameters

Input filename (ignored).Input filename (ignored).Resolution (ignored).Resolution (ignored).Frame count (ignored).Frame count (ignored).Frame rate = 30.Frame rate = 30.ME algorithm and accuracy.ME algorithm and accuracy.Number of motion vectors, 1 or 4.Number of motion vectors, 1 or 4.

Decoder / Encoder InterfacingDecoder / Encoder Interfacing

memcpy(mp4enc.mCurrPtrmemcpy(mp4enc.mCurrPtrYY, frame-, frame->Y_comp_data, mp4par.Width * >Y_comp_data, mp4par.Width * mp4par.Height);mp4par.Height);

memcpy(mp4enc.mCurrPtrmemcpy(mp4enc.mCurrPtrUU, frame-, frame->U_comp_data, mp4par.Width * >U_comp_data, mp4par.Width * mp4par.Heightmp4par.Height/4/4););

memcpy(mp4enc.mCurrPtrmemcpy(mp4enc.mCurrPtrVV, frame-, frame->V_comp_data, mp4par.Width * >V_comp_data, mp4par.Width * mp4par.Heightmp4par.Height/4/4););

Transcoder is hard coded to work with 4:2:0 only

Motion VectorsMotion Vectors16x16, 1MV 8x8, 4MV

• No ME for I-VOP’s

• Better quality with 4MV

• With ME disabled, all MV copied, but correct number coded.

mp4enc.MBinfo->mv[0].dx = this->context->macroblock.vector_luma[0]; //xmp4enc.MBinfo->mv[0].dy = this->context->macroblock.vector_luma[1]; //ymp4enc.MBinfo->mv[1].dx = this->context->macroblock.vector_luma[2]; //xmp4enc.MBinfo->mv[1].dy = this->context->macroblock.vector_luma[3]; //ymp4enc.MBinfo->mv[2].dx = this->context->macroblock.vector_luma[4]; //xmp4enc.MBinfo->mv[2].dy = this->context->macroblock.vector_luma[5]; //ymp4enc.MBinfo->mv[3].dx = this->context->macroblock.vector_luma[6]; //xmp4enc.MBinfo->mv[3].dy = this->context->macroblock.vector_luma[7]; //y

Output StreamOutput Stream

MPEG-4.MPEG-4.Contains only video.Contains only video.147 frames @ 720x480.147 frames @ 720x480.Output file is 824 KB.Output file is 824 KB. Intra and Inter coded VOP’s (I, P).Intra and Inter coded VOP’s (I, P).

Image ComparisonImage ComparisonMPEG-2 to MPEG-4 with 1 MV and ME Enabled

Mpeg-2

Mpeg-4

Peak Signal to Noise RatioPeak Signal to Noise Ratio

2

2)],(),([

N

jiFjiFRMSE

)(log20 25510 RMSEPSNR

• F(i,j), The Average Decoded Luminance Pixel Shade (0-255)

• N^2 = The Number of pixels present

128)],().([2),( jiFjiFjiE

Error Image

PSNR of Sample ClipPSNR of Sample Clip

1 MV 4 MV

Graphs created by CalcPSNR, a freely distributed product of VideoSoft Inc.

Motion Estimation Enabled

PSNR of Sample ClipPSNR of Sample Clip

1 MV 4 MV

Graphs created by CalcPSNR, a freely distributed product of VideoSoft Inc.

Motion Estimation Disabled

ConclusionConclusion

Transcoding is practical for many applications Transcoding is practical for many applications and quality can be maintained.and quality can be maintained.

PSNR is reasonable measure of quality, but PSNR is reasonable measure of quality, but does reveal everything.does reveal everything.

Partial decoding and encoding along with motion Partial decoding and encoding along with motion vector reuse can save execution time (168%)!vector reuse can save execution time (168%)!

Dramatic difference in execution time between Dramatic difference in execution time between AMD and Intel processors of near equivalent AMD and Intel processors of near equivalent speed due to the use of the Intel IPP.speed due to the use of the Intel IPP.

LimitationsLimitations

Resolution (Input = Output)Resolution (Input = Output)Format (4:2:0)Format (4:2:0)Audio (None)Audio (None)

ReferencesReferences

Intel - - http://www.intel.com/software/products/ipp/overview.htmhttp://www.intel.com/software/products/ipp/overview.htm VideoSoft - VideoSoft - http://www.videosoftinc.comhttp://www.videosoftinc.com (for PSNR) (for PSNR) HK – H. Kalva, A. Vetro, and H. Sun, HK – H. Kalva, A. Vetro, and H. Sun,

"Performance Optimization of the MPEG-2 to MPEG-4 Video Tra"Performance Optimization of the MPEG-2 to MPEG-4 Video Transcoder“nscoder“, May 2003., May 2003.

GIT - Seong Hwan Jang, Nikil Jayant GIT - Seong Hwan Jang, Nikil Jayant (Georgia Institute of Technology)(Georgia Institute of Technology)