Fast disparity motion estimation in Fast disparity motion estimation in MVC based on range predictionMVC based on range prediction
Xiao Zhong Xu, Yun He
ICIP 2008
OutlineOutlineDisparity estimation in MVCDisparity search range decisionSearch range reductionExperimental ResultsConclusion
Analysis(1/3)Analysis(1/3)Disparity of two views is decided by
• distance between objects and cameras• displacement between two cameras
Disparity could be 30~50 pixels for background and up to 100 pixels for foreground
40 pixels
75 pixels
Analysis(2/3)Analysis(2/3)The directions of disparity are identical for
all blocks as long as the relative positions between cameras are not changed.• Global disparity vector
The majority of encoding time is consumed by motion estimation(ME) and disparity estimation(DE).
Over 99% encoding time is spent on ME and DE for full search.
Analysis(3/3)Analysis(3/3)Comparison of corresponding bitrate
reduction using different search ranges
• Search range(SR) = ±16 , QP=22/27/32/37• The larger search range is, the more bitrate
can be saved.• computation time increases as well
MVC Motion skip mode (1/2)MVC Motion skip mode (1/2)Proposed by LG Co., JVT-W081(April, 2007).Motion information of current MB can be
inferred from the corresponding MB in the picture with the same temporal index of the neighboring view.• mb_type• motion vector (MV)• reference indices
MVC Motion skip mode (2/2)MVC Motion skip mode (2/2)Two main stages
1. Search for corresponding MB• Global disparity vector (by using SAD) is used.
2. Derivation of motion information• Comparing all the MB mode results, and then
choose the best.
Disparity map storageDisparity map storageDisparities of temporal static objects will
be the same of the same view.After encoding inter-view picture, the
disparity map is then refreshed by the disparities of the current picture.
The disparity map of the current picture is recorded as a disparity predictor.
For disparity prediction, the disparity map is stored.
Disparity map
Disparity estimation
Input picture
Local disparity feature decisionLocal disparity feature decisionThree already obtained disparity are
used:• Co-located MB in the disparity map (dis_col)• The disparity of the current 16*16 block (dis_16)• The disparity predicted by spatial neighboring
blocks (dis_pred)In 16*16 mode, dis_16 is set to 0.
Disparity intensityDisparity intensityDisparity intensity:
Categorizing the current block into 3 different modes:• Low disparity mode – background (DV < 2)• High disparity mode – foreground (DV > 3)• Intermediate disparity mode (other cases)
( _ _ _16 )
( _ _ _16 )x x x x
y y y y
DV abs dis col dis pred dis
DV abs dis col dis pred dis
Search range reduction(1/2)Search range reduction(1/2)The directions of disparity are identical
for all blocks• Symmetrical search window is redundant.• Once global disparity direction is detected,
the opposite direction could be paid less attention to.
Global disparity: summing up the already obtained disparity vectors and check its sign.
Search range reduction(2/2)Search range reduction(2/2)Using an unsymmetrical window.
• 4 directional factors, 2 scale factors:• Negative vertical scale (NVS)• Positive vertical scale (PVS)• Negative horizontal scale (NHS)• Positive horizontal scale (PHS)
• Vertical scale (VS)• Horizontal scale (HS)
directional factors
scale factors
16
16
NVS PVS
NHS PNS
Initial search window
Global disparity direction
new search window
Low disp.
Inter. disp.
High disp.
VS=4 VS=2 VS=1
HS=8 HS=4 HS=1
16
16
NVS PVS
NHS PNS
• Global disparity is positive: PVS = VS (PHS=HS)
• Global disparity is negative: NVS=VS (NHS=HS)
±16pixel
±16pixel
• search range:
( , ) ( , )
( , ) ( , )
SR SRSR SR
VNS VPSSR SR
SR SRHNS HPS
16SR
16 16: ( 16,16) ( , )
4 416 16
: ( 16,16) ( , )8 2
vertical
horizontal
2 pixel 8 pixel
4 pixel
4 pixel
PVS
NVS
PHSNHS
4
4, 4
8
8, 2
PVS VS
PVS NVS
PHS HS
PHS NHS
NHS PHS
NVS
PVS
Simulation results(1/3)Simulation results(1/3)Only inter-view prediction is allowed.
I II I
PPP P
Time
View
• Initial search range = ±96• QP=22,27,32,37•50 frames are coded in each view•Sequence resolution: 640*480
Simulation results(2/3)Simulation results(2/3)ISRP: Inter-view search range predictionJMVM uses extended diamond searchDE speedup comparison:
Simulation results(3/3)Simulation results(3/3)Encoding time comparison of DE:
◦The rest parts of encoder are recognized as 1 unit.
ConclusionConclusionCombination of ISRP and JMVM
performs well.Search range of disparity is
properly reduced.retaining certain quality while
significantly improving encoding speed of MVC.