Computer Vision November 2003 L1.1© 2003 by Davi Geiger
Shape Representation and SimilarityOcclusion Articulation
Shape similarities should be preserved under occlusions and articulation.
Two equal stretching along contours can affect shape similarity differently.
Shapes can be represented by contours, or by a set of interior points, or other ways. What is the shape representation that can be best used for shape similarity?
Stretching Stretching
Computer Vision November 2003 L1.2© 2003 by Davi Geiger
Shape Axis (SA)
SA-Tree
Shape
Symmetry Axis Representation
Computer Vision November 2003 L1.3© 2003 by Davi Geiger
A variational shape representation model based on self-
similarity of shapes.
For each shape contour, first compute its shape axis then derive a unique shape-axis-tree (SA-tree) or shape-axis-forest (SA-forest) representation.
Shape Axis (SA) SA-TreeShape Contour
Shape Representation via Self-SimilarityBased on work of Liu, Kohn and Geiger
Computer Vision November 2003 L1.4© 2003 by Davi Geiger
Use Use twotwo different parameterizations to compute the different parameterizations to compute the represention of a shape.represention of a shape.
Construct a cost functional to measure the Construct a cost functional to measure the goodnessgoodness of a of a match between the two parameterizations.match between the two parameterizations.
The cost functional is decided by the self-similarity criteria The cost functional is decided by the self-similarity criteria of of symmetrysymmetry..
The Insight
counterclockwise clockwise
Computer Vision November 2003 L1.5© 2003 by Davi Geiger
Two different parameterizations:Two different parameterizations:
counterclockwisecounterclockwise
clockwiseclockwise
When the curve is closed we haveWhen the curve is closed we have
Matching the curves is described as the match of functionsMatching the curves is described as the match of functions
Parameterized Shapes
}10:)({1 ssx}10:)(~{2 ttx )1()(~: txtxnote
(1)x~(0)x~ and )1()0( xx
))((x~ ))(x(or[0,1];)()( tsts
Computer Vision November 2003 L1.6© 2003 by Davi Geiger
A Global Optimization Approach We seek “good” matches over the possible We seek “good” matches over the possible
correspondencescorrespondences
[0,1]where))((x~ ))(x(or)()( tsts
)1()(~: txtxnote
Computer Vision November 2003 L1.7© 2003 by Davi Geiger
Co-Circularity
Mirror Symmetry
Similarity criteria: Symmetry
)('))((~)()(~))((~
)())(()()())((where
)())((~))(())((~)((
)())((~))(())((~)((
of valuessmall give shoulderror y thisIntuitivelror.measure/ersymmetry
a define nowcan but we symmetry,perfect havelonger nomay we~ tangent extra the
Given y.circularit-osymmetry/cmirror defines
~point another and, tangent its ,point One
ttd
dtdt
txddtxd
ssd
dsds
sdxdsdx
bdtxd
dsdxtxsx
adtxd
dsdxtxsx
xx
)(~ t
)(~ tx)(~ t
)(~ t
)(~ tx)(~ t
)(sx)(s
)(~ tx
)(sx)(s
)(~ tx
Computer Vision November 2003 L1.8© 2003 by Davi Geiger
nt vectorsunit tange oriented are))((~and))((where),()( escorresponc possible allover
)(),(,))((~)),((x~)),(()),(x( ))(t,)E(s(
Minimize1
0
tstσs
dtsttssF
A Global Optimization Approach
Computer Vision November 2003 L1.9© 2003 by Davi Geiger
Constraints on the form of F
).( and)( maps of choice on thenot and)()(match on theonly dependscost that theassuring
~)~(),~(,~,x~,,x
~,~,~,x~,,x~~,,~,x~,,x
)(),(,~,x~,,x
)(~ :ationparametriz of changeany under invarianceobtain To)(),(,~,x~,,x)(),(,~,x~,,x Condition Scaling 2.
roles equivalentplay ~and i.e.,
)(),(,,x,~,x~)(),(,~,x~,,xmatch theofSymmetry 1.
1
0~
1
0
1
0
1
0
tsts
dtsF
ddd
ddt
dd
ddsFd
dd
ddt
ddsF
dtsF
tsFtsF
stFtsF
Computer Vision November 2003 L1.10© 2003 by Davi Geiger
Constraints on the form of F (cont.)
. )(amount known andt independen shape aby cost thechange and matching thepreserve will
factorcommon by the ~ sand scaling Thus, ).(function somefor
)(),(,,~,x~,,x)()(),(,~,x~,,x Invariance Scaling 4.
cost affect thenot willations transformrigid that so )(),(,,~,x~,,x)(),(,~,x~,,x
InvarianceRotation 3.
g
g
tsFgtsFPartial
tsFtsRRRRF
Computer Vision November 2003 L1.11© 2003 by Davi Geiger
1
02
2
1
0
1
0)(
)(')()('))((~)())(())((~))((
)(')()('))((~)())(())((~)((
)(),(,~,x~,,x
as defined is encecorrespondany for cost symmetry The).)( giveslly automatica(which )( mappings possible allover
cost theminize we)2constraint()(zation parameteriarbitrary any for where
)(),(,~,x~,,xmin)~,(
as measuresymmetry thewrite We
d
tsttsstxsx
tsttsstxsx
dtsF
stts
dtsFS t
Similarity criteria: Symmetry
Computer Vision November 2003 L1.12© 2003 by Davi Geiger
Similarity criteria: Symmetry II
(a)
(b)
dFstE JumpCost )()similarity-self())(),((1
0
solution theof cost) (jump nsbifurcatiofor cost a add we(d), and (c) see parts,object for nsbifurcatio allow andty irregulari shape todues
nbifurcatio ofcreation theavoid To small. be should parameter The
)(')()(')(
))((~))((
)(')()('))((~)())(())((~))((
)(')()('))((~)())(())((~)((
)(),(,~,x~,,x
solutions (b)over (a) bias topoints, closedbetween encescorrespond bias to term thirda add We
1
02
2
2
2
1
0
c
d
tsts
txsxc
tsttsstxsx
tsttsstxsx
dtsF
(c)
(d)
Computer Vision November 2003 L1.13© 2003 by Davi Geiger
Structural propertiesStructural propertieso SymmetricSymmetrico Parameterization IndependentParameterization Independent
Geometrical propertiesGeometrical propertieso Translation invariantTranslation invarianto Rotation invariantRotation invarianto Scale “almost” invarianceScale “almost” invariance
Self-Similarity propertiesSelf-Similarity properties
Summary: Cost Functional/Energy Density
Computer Vision November 2003 L1.14© 2003 by Davi Geiger
A Dynamic Programming Solution
Computer Vision November 2003 L1.15© 2003 by Davi Geiger
A Dynamic Programming
SolutionThe recurrence is on the “square boxes”. Each box is either
linked to one inside square box or linked to
a pair of inside and well aligned square
boxes (leading to bifurcations).
Assumption, that will not be needed later: The final state is a
match (0,0).
…?
!!
!!
!!
bifurcation
bifurcationt
s
1-s
1-t
tt
1-tts+1/N
t+1/N
!!
! !! !
Computer Vision November 2003 L1.16© 2003 by Davi Geiger
)],(),1[(tt],1BestCost[smin
)],()1,[(]1ts,BestCost[smin
Jumpt]tt,-BestCost[1tt],BestCost[smin
min
t],BestCost[s
}11,...,1{tt
}11,...,1{ss
}1,11,...,1,{
tsttN
sFN
tsN
tssFN
Ns
Nt
Nt
Ns
sN
sN
tttt
t
1-s
tt
t+1/N …?
!!
!!
!!!
s 1-t1-tts+1/N
A Dynamic Programming Solution
s
t
Computer Vision November 2003 L1.17© 2003 by Davi Geiger
A Dynamic Programming Solution
A few more steps …
iteration direction
s
t
Starting states
(matching candidates).
All corresponding to
“self-matches”
Computer Vision November 2003 L1.18© 2003 by Davi Geiger
A General Dynamic Programming Solution
No need to force a match at (0,0) or any particular node/match.
s
t
Extended Graph
Starting states
(matching candidates).
All corresponding to
“self-matches”
Ending states
Extend the parameters s and t such that negative values x s,t=1+x.
x
x
Computer Vision November 2003 L1.19© 2003 by Davi Geiger
A Dynamic Programming Solution
Computer Vision November 2003 L1.20© 2003 by Davi Geiger
Experimental Results for Closed Shapes
Shape Axis
SA-Tree
Shape
Computer Vision November 2003 L1.21© 2003 by Davi Geiger
Experimental Results for Open Shapes(the first and last points are assumed to be a match)
Shape Axis (SA)
SA-Tree
Shape
Computer Vision November 2003 L1.22© 2003 by Davi Geiger
Shape Axis (SA)
SA-Forest
Experimental Results for Open Shapes
Computer Vision November 2003 L1.23© 2003 by Davi Geiger
Matching Trees with deletions and merges for articulation and occlusions
Computer Vision November 2003 L1.24© 2003 by Davi Geiger
Experimental Results for Open Shapes
Computer Vision November 2003 L1.25© 2003 by Davi Geiger
Convexity v.s. Symmetry
White Convex Region Black Convex Region