Post on 25-Feb-2016
description
transcript
Modeling the Shape of People from 3D Range Scans
Dragomir AnguelovAI Lab
Stanford University
Joint work withPraveen Srinivasan, Hoi-Cheung Pang,
Daphne Koller, Sebastian Thrun
The Dataset Scans
4 views, ~125k polygons ~65k points each
Problems Missing surface Drastic pose changes
Space of Human Shapes movie
[scape movie]
The Modeling Pipeline
Talk outline Scan registration
Rigid Nonrigid Correlated correspondence
Decomposition into approximately rigid parts
Learning a deformable human model
Talk outline 3D Scan registration
Rigid Nonrigid Correlated correspondence
Decomposition into approximately rigid parts
Learning a deformable human model
Rigid Scan Registration Aligning point clouds
X and Z Match each point in Z
to its closest point in X Compute the best
rigid transformation T which aligns the matching pairs
Transform the points of cloud X by T
Repeat
Iterative Closest Point (ICP) algorithm [Besl et al ’92]
ICP matching step Objective
Setting obtains
Substituting the above for s yields a system linear in R Choose a parameterization of R (quaternions q) and solve
Exact solution Overall, ICP converges to a local minimum of the
energy
(using )
Nonrigid Registration Example Deformable template
model X Set of deformable springs
Nonrigid registration Deforms X to match a scan Z Minimizes the deformation of
X
Generative Model
Model mesh X Transformed mesh X’
Deformation / Transformati
on
Goal: Given meshes X and Z, recover transformation and correspondences C
Data mesh Z
Data Generation /
Correspondences
C
Data mesh generation
ZX’
kzix'
),';()',|( Zikikk xzNxiczP
Correspondence ck specifies which point in X’ generates point zk:
DataTransformed
Model
X Z
Nonrigid ICP failure example
Local Minima Poor transformation and correspondence estimates
reinforce each other
Nonrigid ICP attempts to maximize
Correlated correspondences Nonrigid-ICP decorrelates the correspondences
Correspondences are conditionally independent given
If is unknown, correspondences are correlated
E.g. nearby points in Z should be mapped to nearby points in X
The search space is exponential for the general registration problem
c1c
2
Correlated Correspondence
Compute an embedding of mesh Z into model X
The domain of ci is the set of possible (discrete) matches for zi on the surface of X
The solution is a consistent assignment to all correspondence variables C
Z
X
Correlated CorrespondenceZ
X
Edge is generated from some edge in X
Prefer values of c1, c2 according to the deformation of X they induce
Doing this for all edges in Z results in a Markov network over C
c2c1
c3
(c1, c2)
(c2,c3)(c1,c3)
(c2)
(c3)
(c1)
{1, …, K}
Pairwise Markov Networks
Inference is Markov Nets is generally intractable Good approximate algorithms exist (belief
propagation)
yi yjiij
K
…
1
K K
…
1 2
1 1
CC Potentials
Deformation potentials Spin image* potentials
Quantify the similarity of the local surface around two matching points
Geodesic Potentials
)( ick
),( jcic lkG
*[Johnson+Hebert ’97]
c2c1
c3
(c1, c2)
(c2,c3)(c1,c3)
(c2)
(c3)
(c1)
Geodesic potentials
Nearby points in Z must be nearby in X Constraint between each pair of
adjacent points zk, zl
Distant points in Z must be distant in X Constraint between each pair of distant
points zk, zl (farther than 5r)
Z X Z X
otherwise
rxxDistjcic jiG
lk 13),(0
),(
otherwise
rxxDistjcic jiG
lk 12),(0
),(
r resolution of mesh X
Local surface signatures
Use spin-images [Johnson ’97] 2D Histogram of distances from an oriented reference point Rotationally-invariant / Robust under clutter and occlusion / Compressible
(PCA) Potential (ck = i) encodes how well the signature of point zk matches
the signature of point xi in the model:
),;()( Sikk SSNic
Results: Human poses datasetModel
ArtModel I ArtModel II
• 4 markers were used on each scan to avoid the need for multiple initializations of Loopy-BP• ArtModel I found using 20 registered scans• ArtModel II found using 44 registered scans
Cyberware scans
Registrations
CC movie
[movie]
Talk Outline Scan registration
Rigid Nonrigid Correlated correspondence
Recovering the articulated skeleton Learning a deformable human model
Recovering the skeleton
Input: models, correspondences
Output: rigid parts, skeleton
Probabilistic Generative Model
Transformations
T
TransformedModel y1 yN…
Model a1
x1
aN
xN
……
Part labels
Points
aj = Part (xj)
z1
b1
zK
bK
Instance …
Point labels
Points
bk = ModelPoint (zk)
Part Properties Parts are preferably contiguous regions
Adjacent points on the surface should have similar labels
Enforce this with a Markov network: a1 a3
a2
ji
jiji
:1:
),(
If i, j are connected in model mesh
1 > > 0.5
Iterative Optimization
Hard EM Given transformations , perform min-cut
inference to get
Given labels , solve for rigid transformations
Results: Puppet articulation
Results: Arm articulation
Parts movie
[parts.gif]
Talk Outline Scan registration
Rigid Nonrigid Correlated correspondence
Recovering the articulated skeleton Learning a deformable human model
Modeling Human ShapePo
se v
aria
tion
Body-shape variation
Representation of pose deformation
Pose deformation
Rigid articulateddeformation
Given estimates of R, Q, synthesizing the shape is straightforward :
Learning non-rigid pose deformation For each polygon, predict entries of from
rotations of nearest 2 joints (represented as twists ).
Linear regression parameters :
Learning body-shape deformation Include also change in shape due to different
people:
Do PCA over body-shape matrices :
Shape completion Process:
Add a few markers (~6-8)
Run CC algorithm to get >100 markers
Optimize for pose/body-shape parameters and non-rigid surface
Mocap shape completion
Recap Algorithms for registration between two surfaces
ICP algorithm iterates between finding corresponding points between two scans and computing the transformation between them
Rigid variant (6 unknown DOF) Nonrigid variant (6 Number of polygons DOF)
CC explicitly encodes correlations between correspondence variables
Essential when we have poor initial transformation estimate Exponential complexity, approximate algorithm
Modeling humans Decouple effects of pose and different physique on shape
deformation
The end
Geodesic Potentials: closeclose
Nearby points in Z must be nearby in X Constraint between each pair of adjacent points
zk, zl
Z X
Geodesic Potentials: farfar
Distant points in Z must be distant in X Constraint between each pair of distant points
zk, zl
Z X