Date post: | 31-Dec-2015 |
Category: |
Documents |
Upload: | rosalyn-hebert |
View: | 24 times |
Download: | 0 times |
CSE554 Laplacian Deformation Slide 1
CSE 554
Lecture 8: Laplacian Deformation
CSE 554
Lecture 8: Laplacian Deformation
Fall 2012
CSE554 Laplacian Deformation Slide 2
ReviewReview
• Alignment
– Registering source to target by rotation and translation
• Rigid-body transformations
• Methods
– Aligning principle directions (PCA)
– Aligning corresponding points (SVD)
– Iterative improvement (ICP)
• Combines PCA and SVD
Input
After PCA
After ICP
Source
Target
CSE554 Laplacian Deformation Slide 3
Non-rigid RegistrationNon-rigid Registration
• Rigid alignment cannot account for shape variance
• Non-rigid deformation can give a better fit
After non-rigid deformationRigid alignment
Source
Target
CSE554 Laplacian Deformation Slide 4
Non-rigid RegistrationNon-rigid Registration
• A minimization problem
– Minimizing the distance between the deformed source and the target
• “Fitting term”
– Minimizing the distortion to the source shape
• “Distortion term”
CSE554 Laplacian Deformation Slide 5
Intrinsic vs. ExtrinsicIntrinsic vs. Extrinsic
• Intrinsic methods
– Deforms points on the source curve/surface
– App: boundary curve or surface matching
• Extrinsic methods
– Deforms all points on and interior to the source curve/surface
– App: image or volume matching
CSE554 Laplacian Deformation Slide 6
Laplacian-based DeformationLaplacian-based Deformation
• An intrinsic method
– Simple, efficient, producing reasonable results
• Preserving local shape features
– Widely used in graphics applications for interactive deformation
Reference: “Laplacian surface editing”, by Sorkine et al., 2004 (citation ~ 500)
CSE554 Laplacian Deformation Slide 7
• Input
– Source with n points: p1,…,pn
• Let the first m points be “handles”
– Target location of handles: q1,…,qm
• Output
– Deformed locations of source points: p1’,…,pn’
SetupSetup
p1=q1
p3=q3
q2
p2
An example with 3 target points, two of which are stationary (red)
Source
Deformed
When deforming the source to fit a target shape: m=n and qi is the point on the target closest to pi.
CSE554 Laplacian Deformation Slide 8
OverviewOverview
• Finding deformed locations pi’ that minimize:
– Ef: fitting term
• Measures how close are the deformed source to the target
– Ed: distortion term
• Measures how much the source shape is changed
E Ef Ed
CSE554 Laplacian Deformation Slide 9
Fitting TermFitting Term
• Sum of squared distances to target handle locations
Ef i1
m pi' qi2q2
p2
CSE554 Laplacian Deformation Slide 10
Distortion TermDistortion Term
• Q: How to measure shape?
• A: By “bumpiness” at each vertex
– Laplacian: vector from the centroid of neighbors to the vertex
• Recall that in fairing, we reduced this vector to “smooth out” bumps
• A linear operator over point locations
Lpi pi 1Ni
jNi
pj
pi
pi1pi2
pi1 pi2
2
where Ni are indices of neighboring vertices of pi
CSE554 Laplacian Deformation Slide 11
Distortion TermDistortion Term
• Minimizing changes in Laplacians during deformation
– Over all source points
Ed ni1
Lpi' i2pi
Lpi'pi'
i
i: Laplacian at pi before deformation
CSE554 Laplacian Deformation Slide 12
Putting TogetherPutting Together
• Finding deformed locations pi’ that minimize:
– A quadratic equation in terms of variables (pix’, piy’, piz’)
• qi, i are constants
• L[] is a linear operator
E Ef Ed
i1
m pi' qi2 ni1
Lpi' i2
CSE554 Laplacian Deformation Slide 13
Quadratic MinimizationQuadratic Minimization
• A general form of quadratic minimization:
– There are s variables: x=(x1,…,xs)T
– Each a1,…, ak is a length-s column vector (linear coefficients)
– Each b1,…, bk is a scalar (constant coefficients)
– k should be greater than s (so that the problem is over-constrained)
min ki1
aiT x bi2
CSE554 Laplacian Deformation Slide 14
Quadratic MinimizationQuadratic Minimization
• To solve:
• Re-write in matrix form:
min A x B2where A
a1T
akTis a k by s matrix
min ki1
aiT x bi2
B b1bk
is a length-k vector
CSE554 Laplacian Deformation Slide 15
Quadratic MinimizationQuadratic Minimization
• The minimizer is where the partial derivatives are all zero
– To solve for x in this equation:
• Taking matrix inverse (good for small s, but numerically unstable for large s)
• Using specialized linear system solver (LinearSolve in Mathematica, TNT/LAPACK in C)
0 Ax B2
x
2 AT A x 2 AT B AT A x AT B
x AT A 1 AT B
CSE554 Laplacian Deformation Slide 16
Quadratic MinimizationQuadratic Minimization
• Re-writing our minimization in the general form
– In 2D, there are 2n variables: x = (p1x’,…, pnx’, p1y’,…, pny’ )T
• In 3D, there are 3n variables
– We will next re-write each quadratic term in 2D as (aix-bi)2
• Can be extended easily to 3D
E Ef Ed
i1
m pi' qi2 ni1
Lpi' i2
CSE554 Laplacian Deformation Slide 17
Quadratic MinimizationQuadratic Minimization
• The ai and bi in the fitting term
– There are 2m quadratic terms
– In the first set of m terms:
• For i=1,…,m, bi=qix, ai contains all zero, except its (i)th entry is 1.
– In the second set of m terms:
• For i=1,…,m, bi+m=qiy, ai+m contains all zero, except its (i+n)th entry is 1
Ef i1
m pi' qi2 i1
m pix ' qix2 i1
m piy ' qiy22 mi1
aiT x bi2 x p1x ', …, pnx ', p2y ', …, pny '
CSE554 Laplacian Deformation Slide 18
Quadratic MinimizationQuadratic Minimization
• The ai and bi in the fitting term
– There are 2m quadratic terms
– Example with 3 vertices and 2 fitting constraints (n=3; m=2):
Ef i1
m pi' qi2 i1
m pix ' qix2 i1
m piy ' qiy22 mi1
aiT x bi2x
p1x 'p2x 'p3x 'p1y '
p2y '
p3y '
b1 q1xb2 q2xb3 q1yb4 q2y
a1T 1 0 0 0 0 0
a2T 0 1 0 0 0 0
a3T 0 0 0 1 0 0
a4T 0 0 0 0 1 0
CSE554 Laplacian Deformation Slide 19
Quadratic MinimizationQuadratic Minimization
• The ai and bi in the distortion term:
– There are 2n quadratic terms
– The first set of n terms:
• For i=1,…,n, ai is all zero except the (i)th entry is 1, the (j)th entries are -1/|Ni|
for all jNi, and bi=ix
– The second set of n terms:
• For i=1,…,n, ai+n is all zero except the (i+n)th entry is 1, the (j+n)th entries are
-1/|Ni| for all jNi, and bi+n=iy
Ed ni1
Lpi' i2 ni1
Lpix ' ix2 ni1
Lpiy ' iy2Lpi pi
1Ni jNi
pj
2 ni1
aiT x bi2
CSE554 Laplacian Deformation Slide 20
Quadratic MinimizationQuadratic Minimization
• The ai and bi in the distortion term:
– There are 2n quadratic terms
– Example with 3 vertices (n=3):
Ed ni1
Lpi' i2 ni1
Lpix ' ix2 ni1
Lpiy ' iy2Lpi pi
1Ni jNi
pj
x
p1x 'p2x 'p3x 'p1y '
p2y '
p3y '
b1 1xb2 2xb3 3xb4 1yb5 2yb6 3y
2 ni1
aiT x bi2
p1 p2
p3a1T 1 1
2 12
0 0 0 a2
T 12
1 12
0 0 0 a3
T 12
12
1 0 0 0 a4
T 0 0 0 1 12
12
a5T 0 0 0 1
21 1
2
a6T 0 0 0 1
2 12
1
CSE554 Laplacian Deformation Slide 21
SummarySummary
• Compute Laplacians (i)
• Construct coefficients (ai, bi)
– Stick them into matrices (A,B)
• Solve (x)
– AT A x AT B
CSE554 Laplacian Deformation Slide 22
ResultsResults
Deformed
A small deformation
CSE554 Laplacian Deformation Slide 23
ResultsResults
Deformed
A larger deformation
CSE554 Laplacian Deformation Slide 27
DiscussionDiscussion
• Limitations
– Local features are “skewed”, and they don’t scale with the model
• Reason: Laplacian changes with rotation or scale
– Two bumps that differ by rotation or scale have different Laplacians
• Which will be penalized by our distortion term
pipi'
Lpi Lpi'Lpi Lpi'
CSE554 Laplacian Deformation Slide 28
A Better Distortion TermA Better Distortion Term
• Not penalizing rotation and scaling of local features
– Transforming the original Laplacian vectors before comparing to the deformed Laplacians
• Ti is a matrix that describes how the local shape around pi is deformed
Ed ni1
Lpi' Ti i2pi
pi'Ti
CSE554 Laplacian Deformation Slide 29
Key QuestionsKey Questions
• How to represent transformations as matrices?
• How to compute Ti?
• We will focus in the derivations of the 2D case
– 3D results will be briefly presented at the end
CSE554 Laplacian Deformation Slide 30
Transformation Matrices (2D)Transformation Matrices (2D)
• Homogeneous coordinates
– A 2D point: (x,y,1)
• A 2D vector: (x,y,0)
– A 3D point: (x,y,z,1)
• A 3D vector: (x,y,z,0)
CSE554 Laplacian Deformation Slide 31
Transformation Matrices (2D)Transformation Matrices (2D)
• Translation
– Cartesian coordinates: vector addition
– Homogeneous coordinates: matrix product
px'
py' vx
vy px
py
px'
py'
1
1 0 vx0 1 vy0 0 1
pxpy1
CSE554 Laplacian Deformation Slide 32
Transformation Matrices (2D)Transformation Matrices (2D)
• Isotropic scaling
– Cartesian coordinates: vector scaling
– Homogeneous coordinates: matrix product
px'
py' s px
py
px'
py'
1
s 0 10 s 10 0 1
pxpy1
CSE554 Laplacian Deformation Slide 33
Transformation Matrices (2D)Transformation Matrices (2D)
• Rotation
– Cartesian coordinates: matrix product
– Homogeneous coordinates: matrix product
px'
py' Cos Sin
Sin Cos pxpy
px'
py'
1
Cos Sin 0Sin Cos 0
0 0 1
pxpy1
CSE554 Laplacian Deformation Slide 34
Transformation Matrices (2D)Transformation Matrices (2D)
• Summary of elementary similarity transformations
– To combine transformations: take the product of these matrices
px'
py'
1
M pxpy1
Trsv 1 0 vx0 1 vy0 0 1
Scls s 0 10 s 10 0 1
Rot Cos Sin 0Sin Cos 0
0 0 1
Translation by vector v
Scaling by scalar s
Rotation by angle
CSE554 Laplacian Deformation Slide 35
Similarity Transforms (2D)Similarity Transforms (2D)
• General similarity transformations
– The product of any set of elementary matrices can be written this way
– Any choice of (a, w, tx, ty) can be written as a sequence of rotation,
isotropic scaling and translation
• Note that a and w can’t be both zero
T a w txw a ty0 0 1
a w txw a ty0 0 1
Trstx, ty.Scl a2 w2 .RotArcTan a
w
CSE554 Laplacian Deformation Slide 36
Computing Ti (2D)Computing Ti (2D)
• Suppose we know the deformed locations p i’
• Compute Ti as the similarity transform that best fits the
neighborhood of pi to that of pi’
min Ti pi pi'2 jNi
Ti pj pj'2pi
pi'Ti
CSE554 Laplacian Deformation Slide 37
Computing Ti (2D)Computing Ti (2D)
• Suppose we know the deformed locations p i’
• Compute Ti as the similarity transform that best fits the
neighborhood of pi to that of pi’
• This is a quadratic minimization problem for entries of T i
– E.g., a, w, tx, ty
min Ti pi pi'2 jNi
Ti pj pj'2
CSE554 Laplacian Deformation Slide 38
Computing Ti (2D)Computing Ti (2D)
• The matrix form of the minimization is:
where is a 2|Ni|+2 by 4 matrix, and Ni={i1, i2,…} are indices of neighboring vertices of pi
C
pix piy 1 0
piy pix 0 1
pi1x pi1y 1 0
pi1y pi1x 0 1
min C
awtxty
pix 'piy '
pi1x 'pi1y '
2
CSE554 Laplacian Deformation Slide 39
Computing Ti (2D)Computing Ti (2D)
• By quadratic minimization:
– Linear expressions of variables (pix’ , piy’)
awtxty
CT C 1 CT
pix 'piy '
pi1x 'pi1y '
CSE554 Laplacian Deformation Slide 40
Distortion Term (2D)Distortion Term (2D)
• Two parts of each distortion term:
– Transformed Laplacian:
– Laplacian of the deformed locations:
Ti i D
awtxty
D CT C 1 CT
pix 'piy '
pi1x 'pi1y '
Lpi' L
pix 'piy '
pi1x 'pi1y '
L 1 0 1
Ni0 ...
0 1 0 1Ni
...
where
where is a 2 by 2|Ni|+2 matrix
Lpi' Ti i2D
ix iy 0 0
iy ix 0 0
CSE554 Laplacian Deformation Slide 41
Distortion Term (2D)Distortion Term (2D)
• Putting together:
– They form 2n quadratic terms (aix-bi)2 for x = (p1x’,…, pnx’, p1y’,…, pny’ )T
• All bi are zero
• Each ai can be extracted from H
H L D CT C 1 CT
H1, H2where
are its rows
Ed ni1
Lpi' Ti i2 n
i1
H1 pix 'piy '
pi1x 'pi1y '
2
ni1
H2 pix 'piy '
pi1x 'pi1y '
2
and
CSE554 Laplacian Deformation Slide 42
Results (2D)Results (2D)
New distortion term
Old distortion term
CSE554 Laplacian Deformation Slide 43
Results (2D)Results (2D)
New distortion termOld distortion term
CSE554 Laplacian Deformation Slide 44
Results (2D)Results (2D)
New distortion term
Old distortion term
CSE554 Laplacian Deformation Slide 45
Results (2D)Results (2D)
New distortion term
Old distortion term
CSE554 Laplacian Deformation Slide 46
RegistrationRegistration
• Use nearest neighbors as corresponding target locations
– Assuming the source is already close to the target
• Iterative closest point (ICP)
– 1. For each point on the source, assign its closest point on the target as its corresponding point. Compute Laplacian-based deformation.
• A threshold on the closest distance can be used to throw away unlikely correspondences
– 2. Repeat step (1) until a termination criteria is met.
• Maximum iteration or minimum RMSD improvement
CSE554 Laplacian Deformation Slide 47
ResultResult
After rigid alignment 1 iteration of Laplacian
7 iterations of Laplacian Overlaying all curves
CSE554 Laplacian Deformation Slide 48
ResultResult
• Weighting the distortion term
E Ef w Ed
small w
medium w
large w
CSE554 Laplacian Deformation Slide 49
Similarity Transforms (3D)Similarity Transforms (3D)
• Elementary transformation matrices
– To perform a sequence of transformations: take the product of these matrices
Translation by vector v
Scaling by scalar s
Rotation by angle around X axis
px'
py'
pz'
1
M
pxpypz1
Trsv
1 0 0 vx0 1 0 vy0 0 1 vz0 0 0 1
Scls
s 0 0 10 s 0 10 0 s 10 0 0 1
RotX,
1 0 0 00 Cos Sin 00 Sin Cos 00 0 0 1
CSE554 Laplacian Deformation Slide 50
Similarity Transforms (3D)Similarity Transforms (3D)
• General similarity transformations in 3D
– Approximates the product of a set of elementary matrices
• Up to a small rotation angle
• May introduce skewing for large rotations
T
s h3 h2 txh3 s h1 tyh2 h1 s tz0 0 0 1
CSE554 Laplacian Deformation Slide 51
Computing Ti (3D)Computing Ti (3D)
• Assuming known deformation, by quadratic minimization:
– Linear expressions of the deformed points pi’
• C is a 3|Ni|+3 by 7 matrix
sh1h2h3txtytz
CT C 1 CT
pix 'piy '
piz 'pi1x 'pi1y '
pi1z '
C
pix 0 piz piy 1 0 0
piy piz 0 pix 0 1 0
piz piy pix 0 0 0 1
pi1x 0 pi1z pi1y 1 0 0
pi1y pi1z 0 pi1x 0 1 0
pi1z pi1y pi1x 0 0 0 1
where
CSE554 Laplacian Deformation Slide 52
Distortion Term (3D)Distortion Term (3D)
• Constructing transformed Laplacian:
whereTi i D
sh1h2h3txtytz
D CT C 1 CT
pix 'piy '
piz 'pi1x 'pi1y '
pi1z '
D
ix 0 iz iy 1 0 0
iy iz 0 ix 0 1 0
iz iy ix 0 0 0 1where