Video TranscodingVideo Transcodingwith Intel IPPwith Intel IPP
Eric ShufroEric Shufro
April 27, 2004April 27, 2004
COT6930COT6930
Introduction to TranscodingIntroduction to Transcoding
What is transcoding?What is transcoding?Why transcode?Why transcode?What is involved?What is involved?PerformancePerformanceQualityQuality Intel IPPIntel IPPApplicationsApplications
Transcoding OverviewTranscoding Overview
• Reduced bitrate
• Reasonable Quality
• Performance Gain
MPEG-2 bit stream
MPEG-4 bit stream
MPEG-2 bit stream
MPEG-4 bit stream
Decode Encode
Partial Decodeand Encode
1
2
ApplicationsApplications
Streaming video for both broadband and Streaming video for both broadband and narrow band networks.narrow band networks.
Decreased video bitrate for playback on Decreased video bitrate for playback on mobile or other small embedded devices.mobile or other small embedded devices.
Conversion and modification of pre-Conversion and modification of pre-encoded bit streams.encoded bit streams.
Perhaps steganography? (jpg, mp3…)Perhaps steganography? (jpg, mp3…)
Integrated Performance PrimitivesIntegrated Performance Primitives
Provides source code and libraries for Provides source code and libraries for media types such as MP3, MPEG-2, media types such as MP3, MPEG-2, MPEG-4, H.263, JPEG, JPEG2000, GSM-MPEG-4, H.263, JPEG, JPEG2000, GSM-AMR, G.723, and computer vision. [Intel]AMR, G.723, and computer vision. [Intel]
Well documented.Well documented.Easy to use.Easy to use.
MPEG-2 DecoderMPEG-2 Decoder
Runs in two separate threads.Runs in two separate threads.Responsible for splitting the input stream Responsible for splitting the input stream
into two separate parts, audio and video into two separate parts, audio and video and then decoding the video stream into and then decoding the video stream into YUV components.YUV components.
YUV buffers are available through the YUV buffers are available through the transcoder class to the input of the transcoder class to the input of the encoder.encoder.
MPEG-4 EncoderMPEG-4 Encoder
Uses input data from the decoder YUV frame Uses input data from the decoder YUV frame buffers on a frame by frame basis.buffers on a frame by frame basis.
Creates an MPEG-4 bit stream file, out.cms, Creates an MPEG-4 bit stream file, out.cms, between 800KB and 1MB in size, video only.between 800KB and 1MB in size, video only.
Motion Estimation can be disabled for testing Motion Estimation can be disabled for testing purposes.purposes.
Requires input parameters via a parameter file, Requires input parameters via a parameter file, though some elements can be ignored.though some elements can be ignored.
Transcoder ArchitectureTranscoder Architecture
MPEG-2 decoder and MPEG-4 encoder MPEG-2 decoder and MPEG-4 encoder based on the IPP.based on the IPP.
Transcoder class encapsulates both the Transcoder class encapsulates both the encoder and decoder.encoder and decoder.
Memory is accessible between the Memory is accessible between the encoder and the decoder.encoder and the decoder.
Transcoder runs in three separate threads.Transcoder runs in three separate threads.
Transcoder InitializationTranscoder Initialization
Read application parameters, source file, Read application parameters, source file, encoder parameter file and output file name.encoder parameter file and output file name.
Read the encoder parameters into memory.Read the encoder parameters into memory. Create decoder and splitter thread.Create decoder and splitter thread.
DecoderInit()DecoderInit() EncoderInit(), overwrite parameters.EncoderInit(), overwrite parameters. Splitter – Open bit streamSplitter – Open bit stream
Encode MPEG-4 header.Encode MPEG-4 header. Begin transcoding.Begin transcoding.
Input StreamInput Stream
MPEG-2.MPEG-2.Contains both audio and video.Contains both audio and video.147 frames @ 720x480.147 frames @ 720x480.Source file is 3.57 MB.Source file is 3.57 MB. Intra and Inter coded frames (I, P, B).Intra and Inter coded frames (I, P, B).
Encoder ModificationsEncoder Modifications
ExpandFrame() bypassed.ExpandFrame() bypassed.StepLuma and StepChroma artificially set StepLuma and StepChroma artificially set
after decoder init.after decoder init.Parameters overwritten after encoder init.Parameters overwritten after encoder init.mp4_MacroBlock changed to a public mp4_MacroBlock changed to a public
member of ippVideoEncoderMPEG4.member of ippVideoEncoderMPEG4.Motion Estimation can be disabled.Motion Estimation can be disabled.
(avoids Sum of Absolute Difference)(avoids Sum of Absolute Difference)
Modified ParametersModified Parameters
Input filename (ignored).Input filename (ignored).Resolution (ignored).Resolution (ignored).Frame count (ignored).Frame count (ignored).Frame rate = 30.Frame rate = 30.ME algorithm and accuracy.ME algorithm and accuracy.Number of motion vectors, 1 or 4.Number of motion vectors, 1 or 4.
Decoder / Encoder InterfacingDecoder / Encoder Interfacing
memcpy(mp4enc.mCurrPtrmemcpy(mp4enc.mCurrPtrYY, frame-, frame->Y_comp_data, mp4par.Width * >Y_comp_data, mp4par.Width * mp4par.Height);mp4par.Height);
memcpy(mp4enc.mCurrPtrmemcpy(mp4enc.mCurrPtrUU, frame-, frame->U_comp_data, mp4par.Width * >U_comp_data, mp4par.Width * mp4par.Heightmp4par.Height/4/4););
memcpy(mp4enc.mCurrPtrmemcpy(mp4enc.mCurrPtrVV, frame-, frame->V_comp_data, mp4par.Width * >V_comp_data, mp4par.Width * mp4par.Heightmp4par.Height/4/4););
Transcoder is hard coded to work with 4:2:0 only
Motion VectorsMotion Vectors16x16, 1MV 8x8, 4MV
• No ME for I-VOP’s
• Better quality with 4MV
• With ME disabled, all MV copied, but correct number coded.
mp4enc.MBinfo->mv[0].dx = this->context->macroblock.vector_luma[0]; //xmp4enc.MBinfo->mv[0].dy = this->context->macroblock.vector_luma[1]; //ymp4enc.MBinfo->mv[1].dx = this->context->macroblock.vector_luma[2]; //xmp4enc.MBinfo->mv[1].dy = this->context->macroblock.vector_luma[3]; //ymp4enc.MBinfo->mv[2].dx = this->context->macroblock.vector_luma[4]; //xmp4enc.MBinfo->mv[2].dy = this->context->macroblock.vector_luma[5]; //ymp4enc.MBinfo->mv[3].dx = this->context->macroblock.vector_luma[6]; //xmp4enc.MBinfo->mv[3].dy = this->context->macroblock.vector_luma[7]; //y
Output StreamOutput Stream
MPEG-4.MPEG-4.Contains only video.Contains only video.147 frames @ 720x480.147 frames @ 720x480.Output file is 824 KB.Output file is 824 KB. Intra and Inter coded VOP’s (I, P).Intra and Inter coded VOP’s (I, P).
Image ComparisonImage ComparisonMPEG-2 to MPEG-4 with 1 MV and ME Enabled
Mpeg-2
Mpeg-4
Peak Signal to Noise RatioPeak Signal to Noise Ratio
2
2)],(),([
N
jiFjiFRMSE
)(log20 25510 RMSEPSNR
• F(i,j), The Average Decoded Luminance Pixel Shade (0-255)
• N^2 = The Number of pixels present
128)],().([2),( jiFjiFjiE
Error Image
PSNR of Sample ClipPSNR of Sample Clip
1 MV 4 MV
Graphs created by CalcPSNR, a freely distributed product of VideoSoft Inc.
Motion Estimation Enabled
PSNR of Sample ClipPSNR of Sample Clip
1 MV 4 MV
Graphs created by CalcPSNR, a freely distributed product of VideoSoft Inc.
Motion Estimation Disabled
ConclusionConclusion
Transcoding is practical for many applications Transcoding is practical for many applications and quality can be maintained.and quality can be maintained.
PSNR is reasonable measure of quality, but PSNR is reasonable measure of quality, but does reveal everything.does reveal everything.
Partial decoding and encoding along with motion Partial decoding and encoding along with motion vector reuse can save execution time (168%)!vector reuse can save execution time (168%)!
Dramatic difference in execution time between Dramatic difference in execution time between AMD and Intel processors of near equivalent AMD and Intel processors of near equivalent speed due to the use of the Intel IPP.speed due to the use of the Intel IPP.
LimitationsLimitations
Resolution (Input = Output)Resolution (Input = Output)Format (4:2:0)Format (4:2:0)Audio (None)Audio (None)
ReferencesReferences
Intel - - http://www.intel.com/software/products/ipp/overview.htmhttp://www.intel.com/software/products/ipp/overview.htm VideoSoft - VideoSoft - http://www.videosoftinc.comhttp://www.videosoftinc.com (for PSNR) (for PSNR) HK – H. Kalva, A. Vetro, and H. Sun, HK – H. Kalva, A. Vetro, and H. Sun,
"Performance Optimization of the MPEG-2 to MPEG-4 Video Tra"Performance Optimization of the MPEG-2 to MPEG-4 Video Transcoder“nscoder“, May 2003., May 2003.
GIT - Seong Hwan Jang, Nikil Jayant GIT - Seong Hwan Jang, Nikil Jayant (Georgia Institute of Technology)(Georgia Institute of Technology)