Probabilistic Point Cloud Regsitration Part Isiavashk/teaching/UBC_October_2016/presentatio… ·...

$Page 1: Probabilistic Point Cloud Regsitration Part Isiavashk/teaching/UBC_October_2016/presentatio… · Bogo et al., \FAUST: Dataset and evaluation for 3D mesh registration", CVPR 2014$
Probabilistic Point Cloud RegsitrationPart I

Siavash Khallaghi

October 13, 2016

Siavash Khallaghi Probabilistic Point Cloud Registration October 13, 2016 1 / 26

Introduction

Outline

Point cloud registration, iterative closest point and coherent pointdrift (25 minutes)

Applications and related literature (25 minutes)

Example Python and Matlab code (25 minutes)


Introduction

Outline





Introduction

Outline





Introduction

Computer Vision

Bogo et al., “FAUST: Dataset and evaluation for 3D mesh registration”, CVPR 2014


Introduction

Image Guided Interventions

Rasoulian et al., “Augmentation of Paramedian 3D Ultrasound Images of the Spine”, IPCAI 2013


Point Cloud Registration

Problem Definition

Suppose we are given a fixed XN×3 = {x1, x2, x3, . . . , xN} and amoving YM×3 = {y1, y2, . . . , yM} point cloud.

We would like to know the correspondences between X and Y , e.g.,{x1 → y2, x2 → y1, x3 → y3, . . .},and also the transformation T with parameters θ that maps Y intoX , i.e. X = T (Y , θ).



Problem Definition


We would like to know the correspondences between X and Y , e.g.,{x1 → y2, x2 → y1, x3 → y3, . . .},

and also the transformation T with parameters θ that maps Y intoX , i.e. X = T (Y , θ).



Problem Definition


We would like to know the correspondences between X and Y , e.g.,{x1 → y2, x2 → y1, x3 → y3, . . .},and also the transformation T with parameters θ that maps Y intoX , i.e. X = T (Y , θ).



Problem Definition

x2 x1

x3

y2 y1

y3



Problem Definition

x2 x1

x3

y2 y1

y3



Problem Definition

x2 x1

x3

y2 y1

y3



Estimating the Transformation Between Two Point Clouds

We start by assuming that the point clouds have an equal N numberof points and correspondences are known in advance.

Notations:

We define X and Y as the center of the two point clouds.xn = xn − X and yn = yn − Y are centered coordinates.The transformation is determined by a scale factor s, a rotation matrixR and a translation vector t.

We can transform one point in one cloud to the other usingxn = sRyn + t





Notations:







Notations:

We define X and Y as the center of the two point clouds.

xn = xn − X and yn = yn − Y are centered coordinates.The transformation is determined by a scale factor s, a rotation matrixR and a translation vector t.






Notations:

We define X and Y as the center of the two point clouds.xn = xn − X and yn = yn − Y are centered coordinates.

The transformation is determined by a scale factor s, a rotation matrixR and a translation vector t.






Notations:







Notations:





Least Squares Minimization

The transformation parameters can be determined by minimizing:E =

∑Nn=1 ‖xn − sRyn − t‖2

It can be shown that the minimization problem has a closed-formsolution.

Optimal translation: t∗ = X − sRY .

Optimal scale: s∗ =∑

n(Ryn)T xn∑n yn

T yn.

Optimal rotation1: R∗ = VUT where Y XT = USV T is the singularvalue decomposition of Y XT .









n(Ryn)T xn∑n yn

T yn.










n(Ryn)T xn∑n yn

T yn.










n(Ryn)T xn∑n yn

T yn.










n(Ryn)T xn∑n yn

T yn.


1Hint: R∗ = argmaxR tr(RY XT )Siavash Khallaghi Probabilistic Point Cloud Registration October 13, 2016 9 / 26



Now, we assume that point-to-point correspondences are not known.The minimization criterion becomes:E =

∑Nn=1

∑Mm=1 Pmn‖xn − sRym − t‖2

Possible representations for the correspondence variables Pmn:

1 Binary: Pmn = 1 if ym corresponds to xn and Pmn = 0 otherwise.2 Soft:

∑n Pmn =

∑m Pmn = 1, and 0 ≤ Pmn ≤ 1

3 Probabilistic: Pmn = P(xn = m|xn) or the posterior probability ofassigning point ym to xn.





∑Nn=1


Possible representations for the correspondence variables Pmn:

1 Binary: Pmn = 1 if ym corresponds to xn and Pmn = 0 otherwise.2 Soft:

∑n Pmn =

∑m Pmn = 1, and 0 ≤ Pmn ≤ 1






∑Nn=1


Possible representations for the correspondence variables Pmn:1 Binary: Pmn = 1 if ym corresponds to xn and Pmn = 0 otherwise.

2 Soft:∑

n Pmn =∑

m Pmn = 1, and 0 ≤ Pmn ≤ 13 Probabilistic: Pmn = P(xn = m|xn) or the posterior probability of

assigning point ym to xn.





∑Nn=1


Possible representations for the correspondence variables Pmn:1 Binary: Pmn = 1 if ym corresponds to xn and Pmn = 0 otherwise.2 Soft:

∑n Pmn =

∑m Pmn = 1, and 0 ≤ Pmn ≤ 1






∑Nn=1


Possible representations for the correspondence variables Pmn:1 Binary: Pmn = 1 if ym corresponds to xn and Pmn = 0 otherwise.2 Soft:

∑n Pmn =

∑m Pmn = 1, and 0 ≤ Pmn ≤ 1



Iterative Closest Point

The ICP Algorithm

The iterative closest point algorithm is the most used pointregistration method:

1 Initialize the scale, rotation and translation parameters2 Transform each point ym and search its closest point

xk = argminn‖xn − sRym − t‖2 and set Pkn = 1

3 Estimate the transformation parameters by minimizing:E =

∑Mk=1 Pmk‖xk − sRym − t‖2

4 Repeat steps 2 and 3 until convergence



The ICP Algorithm


1 Initialize the scale, rotation and translation parameters

2 Transform each point ym and search its closest pointxk = argmin

n‖xn − sRym − t‖2 and set Pkn = 1






The ICP Algorithm









The ICP Algorithm









The ICP Algorithm









Drawbacks of ICP

It is very sensitive to initial transformation parameters and often getstrapped in a local minima.

It is not robust to large transformations.

It cannot deal with anisotropic noise, i.e. the noise has to have thesame variance across all coordinates.



Drawbacks of ICP






Drawbacks of ICP





Coherent Point Drift

Probabilistic Point Cloud Registration

We assume that points on the moving point cloud are the centroids ofGaussians with covariance matrices {Cm}Mm=1

The minimization criterion is now generalized to:E =

∑Nn=1

∑Mm=1 Pmn

(‖xn − sRym − t‖2 + log |Cm|

)The posterior probabilities are defined as:

Pmn =|Cm|−

12 exp(− 1

2‖xn−sRym−t‖2

Cm)∑M

k=1 |Ck |−12 exp(− 1

2‖xn−sRyk−t‖2

Ck)+∅3D

Mahalanobis distance: ‖a− b‖2C = (a− b)TC−1(a− b)






∑Nn=1

∑Mm=1 Pmn


)

The posterior probabilities are defined as:

Pmn =|Cm|−

12 exp(− 1


Cm)∑M

k=1 |Ck |−12 exp(− 1


Ck)+∅3D







∑Nn=1

∑Mm=1 Pmn



Pmn =|Cm|−

12 exp(− 1


Cm)∑M

k=1 |Ck |−12 exp(− 1


Ck)+∅3D







∑Nn=1

∑Mm=1 Pmn



Pmn =|Cm|−

12 exp(− 1


Cm)∑M

k=1 |Ck |−12 exp(− 1


Ck)+∅3D




Point Registration with Expectation Maximization (EM)

1 Initialize the transformation parameters s, R, t and covariancematrices Cm

2 Expectation step: Estimate posterior probabilities Pmn

3 Maximization step: Estimate transformation parameters andcovariance matrices

4 Repeat steps 2 and 3 untile convergence
























The ”Rigid” Coherent Point Drift Algorithm

We consider the special case where all convariance matrices arespherical Gaussians with variance σ2.

This removes the dependency between covariance matrices androtation.

There is a closed-form solution for transformation parameters(s, R, t) and σ2 to similar to the least squares minimization (slide 9).
















Rigid + scaling registration of a moving blue point cloud to a fixed red point cloud using CPD. Both point clouds have been

corrupted with removed points and random spurious outlier points.



The Non-rigid Coherent Point Drift Algorithm

This is similar to the rigid case, only we apply a differenttransformation to our point clouds

In the rigid case, we used a transformation model thatT (ym, s,R, t) = sRym + t subject to RRT = I and det(R) = 1

In the non-rigid case, we assume moving cloud points that are closewithin a β radius, move together

This coherent motion is regularized with a constant λ

This translates to the following transformation model:T (ym,W ) = ym + GW where:

1 Gij = exp(− 12β2 ‖yi − yj‖2) is a Guassian kernel matrix that is

calculated once from the moving cloud points2 And W is an unknown Number− of − points×Dimension matrix of

transformation parameters

The transformation parameters are found using a linear equation:(diag(P1)G + λσ2

)W = PX − diag(P1)Y






























































calculated once from the moving cloud points

2 And W is an unknown Number− of − points×Dimension matrix oftransformation parameters
































Non-rigid registration of a moving blue point cloud to a fixed red point cloud using CPD. Both point clouds have been corrupted

with removed points and random spurious outlier points.



Why Does EM Perform So Well?

CPD is basically a mixture density estimation algorithm.

We assume that fixed points belong to a Gaussian Mixture Model(GMM) with unknown parameters that is constructed from themoving point cloud.

We have to solve two problems simultaneously:

1 Correspondence, i.e. which Gaussian the fixed point belongs to2 and GMM parameters, i.e. the transformation that best aligns the

GMM centroids

We have to solve this problem without knowing the correspondencesin advance, i.e. missing data.

The EM algorithm was specifically designed to solve the maximumlikelihood with missing data problem

This is why EM performs so well for point cloud registration








GMM centroids











GMM centroids









We have to solve two problems simultaneously:1 Correspondence, i.e. which Gaussian the fixed point belongs to

2 and GMM parameters, i.e. the transformation that best aligns theGMM centroids









We have to solve two problems simultaneously:1 Correspondence, i.e. which Gaussian the fixed point belongs to2 and GMM parameters, i.e. the transformation that best aligns the

GMM centroids










GMM centroids










GMM centroids










GMM centroids





References

References

Itertive closest pointhttp://www.cs.virginia.edu/~mjh7v/bib/Besl92.pdf

Coherent point drift https://arxiv.org/pdf/0905.2635.pdf


http://www.cs.virginia.edu/~mjh7v/bib/Besl92.pdf

https://arxiv.org/pdf/0905.2635.pdf

Extra Slides




Let’s expand the quadratic error using RTR = I :

E =∑

n (xn − sRyn − t)T (xn − sRyn − t)

E = s2∑

n yTn yn + 2s

(∑n y

Tn

)RT t − 2s

∑n y

Tn RT xn −

2(∑

n xTn

)t +

(∑n x

Tn xn

)+ NtT t


Extra Slides





E =∑


E = s2∑

n yTn yn + 2s

(∑n y

Tn

)RT t − 2s

∑n y

Tn RT xn −

2(∑

n xTn

)t +

(∑n x

Tn xn

)+ NtT t


Extra Slides





E =∑


E = s2∑

n yTn yn + 2s

(∑n y

Tn

)RT t − 2s

∑n y

Tn RT xn −

2(∑

n xTn

)t +

(∑n x

Tn xn

)+ NtT t


Extra Slides





E =∑


E = s2∑

n yTn yn + 2s

(∑n y

Tn

)RT t − 2s

∑n y

Tn RT xn −

2(∑

n xTn

)t +

(∑n x

Tn xn

)+ NtT t


Extra Slides

The Optimal Translation

t∗ = argmint

E

The derivative of the error w.r.t t is:∂E∂t = 2sR

∑n yn − 2

∑n xn + 2Nt

Setting the derivative to zero we obtain:t∗ = X − sRY


Extra Slides


t∗ = argmint

E


∑n yn − 2

∑n xn + 2Nt



Extra Slides


t∗ = argmint

E


∑n yn − 2

∑n xn + 2Nt



Extra Slides

A Simplified Criterion

By substituting the optimal translation we obtain:

E =∑N

n=1 ‖xn − sRyn‖2

E = s2∑

n ynT yn − 2s

∑n yn

TRT xn +∑

n xnT xn


Extra Slides



E =∑N


E = s2∑

n ynT yn − 2s

∑n yn

TRT xn +∑

n xnT xn


Extra Slides



E =∑N


E = s2∑

n ynT yn − 2s

∑n yn

TRT xn +∑

n xnT xn


Extra Slides

The Optimal Scale Factor

s∗ = argmins

E

which yields: s∗ =∑

n(Ryn)T xn∑n yn

T yn


Extra Slides

The Optimal Scale Factor

s∗ = argmins

E

which yields: s∗ =∑

n(Ryn)T xn∑n yn

T yn


Extra Slides

The Optimal Rotation

From the expansion of the error function we obtain that:R∗ = argmax

R

(∑n yn

TRT xn)

= argmaxR

(∑n xn

TRyn)

R∗ = argmaxR

tr(RY XT

)= argmax

Rtr (RCXY )

Let CXY = USV T be the singular value decomposition of the datacovariance matrix, the optimal rotation is given by

R∗ = VUT


Extra Slides



R

(∑n yn

TRT xn)

= argmaxR

(∑n xn

TRyn)

R∗ = argmaxR

tr(RY XT

)= argmax

Rtr (RCXY )


R∗ = VUT


Extra Slides



R

(∑n yn

TRT xn)

= argmaxR

(∑n xn

TRyn)

R∗ = argmaxR

tr(RY XT

)= argmax

Rtr (RCXY )


R∗ = VUT


Extra Slides



R

(∑n yn

TRT xn)

= argmaxR

(∑n xn

TRyn)

R∗ = argmaxR

tr(RY XT

)= argmax

Rtr (RCXY )


R∗ = VUT


Extra Slides

Point Registration with Expectation Maximization

1 The EM algorithm for point registration uses a Gaussian mixturemodel (GMM) augmented with a uniform-distribution componentthat captures outliers.

2 The estimation of the transformation parameters do not need explicitpoint-to-point correspondences.

3 However, such correspondences can be easily established by searchingthe probabilities Pmn

4 The algorithm is robust to initialization, Gaussian noise and uniformlydistributed gross errors (outliers), it may however be trapped in localminima.

5 It is less efficient than ICP because of the computation of theprobabilities Pmn.


Extra Slides








Extra Slides








Extra Slides








Extra Slides








Date post:	04-Aug-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Probabilistic Point Cloud Regsitration Part Isiavashk/teaching/UBC_October_2016/presentatio… ·...

Documents