Fast Motion Estimation AlgorithmsMaster:Prof. Hashemi

By:Amir Shahrokhi

ProgressWhat has been done:Download Papers about ME Specially Block MatchingSurvey Papers and Their References

What will be done:Simulate Some of Proposed Algorithms Using MATLABComparison of different algorithms

OutlineFast Motion Estimation Algorithms:IntroductionBlock Matching AlgorithmsFull Search Method2-D Logarithmic Search3-Step SearchConjugate Direction SearchDiamond Search

IntroductionTemporal correlation between successive framesAn easy way to remove this redundancy:Frame Replenishment:A threshold for difference in pixel intensity from previous frameIf less than threshold Send nothingIf more than threshold Send position & intensity differenceAdvantage: Easy to implement, Low complexityDisadvantage: Low compression when rapid changes In low bit-rates may cause scene cutsBetter way: Motion Compensated Coding

Introduction (Continue)Motion Compensated CodingFrame changes are considered as object movements so:Estimate motion of current frame respect to previousCalculate difference between motion compensated & current framesCode motion vectors and differences as this frameTo have better compression we need to good motion estimationDifferent algorithms for motion estimationBlock Matching (Our Discussion)Pel Recursive TechniqueOptical Flow

OutlineFast Motion Estimation Algorithms:IntroductionBlock Matching AlgorithmsFull Search Method2-D Logarithmic Search3-Step SearchConjugate Direction SearchDiamond Search

Block MatchingDivide current frame to small rectangular blocksMotion of each block is assumed to be uniformFind the best match for each block in previous frameCalculate motion vector (MV) between current block and its counterpart in previous frameTypical size for blocks: 16x16 pixelsMaximum movement: w: typically 8, 16 or 32Matching Criteria:Mean Absolute Error (MAE)Mean Square Error (MSE)Sum of the Squared Error (SSE)MAE is preferred due to its simplicity

Block MatchingSearch Window (in previous frame)Rectangle with the same coordinate as current block in current frame, extended by w pixels in each directionswwwwqpq+2wp+2w

Block MatchingFull SearchAll candidates within search window are examined(2w+1)2 positions should be examinedAdvantage: Good accuracy, Finds best matchDisadvantage: Large amount of computation: (2w+1)2 matches, 16x16 MAE for each match Impractical for real-time applicationsIn order to avoid this complexity, we should reduce search positions Fast Block Matching Algorithms

Block Matching (Fast Algorithms)Improvements to Full SearchSuccessive Elimination Algorithm (SEA)Li, SalariUsing SAD as matching criterionIdea: excluding many search positions while finding still best matchLet f(x,y) be the intensity of each pixel in current block and f(x,y) be intensity of each pixel in a current search position, and (x,y) be the counterpart of (x,y), so from triangular inequality:|S f(x,y) S f(x,y)|

Block Matching (Fast Algorithms)2-D Logarithmic SearchJain and JainExamine central point & its four surroundingsDistance from center: w/2Find best matchIf best match is in center or on boundaries, half distance from centerExamine five new points centering previous best When distance is 1, use all 9 matches, find best. Stop111112223344444444

Block Matching (Fast Algorithms)Three-Step Search (3SS)Koga et al9 Points: Central point & its 8 surroundingsDistance: w/2Find the best matchUse previous best as centerHalf distance, select 8 newRepeat algorithm 3 timesExamines 25 pointsAssumes a uniform distribution of MVs1111111112322222223333333

Block Matching (Fast Algorithms)Improvements to 3SSNew Three-Step Search (N3SS)Li, Zeng, LiouExperiments show that most of best matches are near centerMore points in center: 8 neighbors of center of search area (33 Search points)More robust & Smaller error in motion compensationFour-Step Search (4SS)Po, MaThe idea is same as N3SS, a hybrid of 3SS and 2DL, (For w=8)Central point and 8 surrounding with distance = 2If best match on center, distance = 1 select 8 new points and end else select best match as new center with previous distance (2)If following step repeated 3 times, distance=1, select 8 new points and end2 to 4 steps 17-27 search points, Less search points on averageBetter MSE than 3SS & less search points than N3SS, Better prediction

Block Matching (Fast Algorithms)Conjugate Direction SearchSrinivasan, RaoStart from center point, find best match between this point & its 2 horizontal neighborsSearch 1 point in the direction of best match & ExamineIf no change in best match in horizontal, repeat for verticalWhen no change in vertical, select the point as best matchworst case 2w+3 points1234115567

Block Matching (Fast Algorithms)Improvement to CDSFast One-Step Search (FOSS)Srinivasan, Ramachandran5 macro-blocks : center of search area and 4 points between center and 4 borders of search areaCalculate number of search points for these macro-blocks in eight directionsDirection with least number of search points is used as optimal for CDSThis is done for first P-frame after I-frameThis direction is unique for all P-frames in a GOPReduces search points by 2, with the cost of 0.4% overhead of overall computeBetter direction select and faster than CDS

Block Matching (Fast Algorithms)Diamond Search (DS)Zhu, MaStart with 9 points creating a large diamond with center of search window in centerFind best match, repeat with this best match as centerIf best match is in center, use 5 points creating small diamond, find best match, this is targetBetter performance than 3SSLess complex than N3SS and 4SS, with same performance

Block Matching (Fast Algorithms)DS Example111111111222333334444

ReferencesFPGA Implementation of a Novel, Fast Motion Estimation Algorithm for Real-Time Video Compression, FPGA 2001, CA. USA, S. Ramachandran and S. Srinivasan, Feb. 2001Image & Video Compression for Multimedia Engineering, Y.Q. Shi and H. Sun, 2000A New Diamond Search Algorithm for Fast Block-Matching Motion Estimation, IEEE Trans. Image Processing, S. Zhu and K. K. Ma, Feb. 2000A Novel Four-Step Search Algorithm for Fast Block Motion Estimation, IEEE Trans. Circuits System, Video Technology, L. M. Po and W. C. Ma, June 1996Successive Elimination Algorithm for Motion Estimation W. Li and E. Salari IEEE Trans. , Jan. 1995A New Three-Step Search Algorithm for Block Motion Estimation, IEEE Trans. Circuits System, Video Technology, R. Li, B. Zeng, and M.L. Liou, Aug. 1994Predictive Coding Based on Efficient Motion Estimation, IEEE Trans. on communications, R. Srinivasan, K.R. Rao, Aug. 1985Motion Compensated Inter-Frame Coding for Video-Conferencing, T. Koga, K. Iinuma, A. Hirano, Y. Iijima, and T. Ishiguro, Proc. NTC81, Nov. 1981Displacement Measurement and its Applications, IEEE Trans. on communications, J.R. Jain and A.K Jain, Dec. 1981

Any Question?