Fast and Global 3D Registration of Points, Lines and ...

Fast andGlobal 3D

Registrationof Points,Lines and

Planes

Jesus Briales

Problem

FormulationUnification

Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Fast and Global 3D Registrationof Points, Lines and Planes

Adapted from "Convex Global 3D Registration with Lagrangian Duality"(CVPR17)

Jesus Briales

MAPIR GroupUniversity of Malaga

LPM WorkshopSep 28, 2017

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Generalized Absolute Pose problem

3D-3D registration: Find optimal pose T ? aligning measured points to model

2D-3D registration Generic registration

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



2D-3D registration

Generic registration

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



2D-3D registration Generic registration

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Problem formulation

Optimization problem

T ? = arg minT∈SE(3)

m∑i=1

dPi (T ⊕ x i)2

where• x i : Measured points• T ⊕ x i : Transformed point• Pi : Primitive in the model• d(·, ·): Euclidean distance

Assumptions:

• Known correspondences{x i ↔ Pi}mi=1

• No outliers

Challenge:

• Non-convexity of R ∈ SO(3)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Problem formulation



m∑i=1

dPi (T ⊕ x i)2

Assumptions:


• No outliers

Challenge:


Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Problem formulation



m∑i=1

dPi (T ⊕ x i)2

Assumptions:


• No outliers

Challenge:


Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Problem formulation



m∑i=1

dPi (T ⊕ x i)2

Assumptions:


• No outliers

Challenge:


Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Unified formulation

Optimization objective

f (T ) =m∑

i=1

dPi (T ⊕ x i)2

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Unified formulation


f (T ) =m∑

i=1

dPi (T ⊕ x i)2

Mahalanobis distance:

Pi ≡ {y i ,C i}dPi (x)

2 = ‖x − y i‖2C i

= (x − y i)>C i(x − y i),

‖x − y‖2I3‖x − y‖2(I−vv>) ‖x − y‖2nn>

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Unified formulation


f (T ) =m∑

i=1

(T ⊕ x i − y i)>C i(T ⊕ x i − y i)

Mahalanobis distance:

Pi ≡ {y i ,C i}dPi (x)

2 = ‖x − y i‖2C i

= (x − y i)>C i(x − y i),

‖x − y‖2I3‖x − y‖2(I−vv>) ‖x − y‖2nn>

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Unified formulation

Optimization objective:

f (T ) =m∑

i=1

(T ⊕ x i − y i)>C i(T ⊕ x i − y i)

Vectorization:

T ⊕ x i − y i =[x̃>i ⊗ I3| − y i

]τ̃

x̃ i =

[x i1

]τ̃ =

[vec(T )

1

]vec(T ) =

[vec(R)

t

]

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Unified formulation


f (T ) =m∑

i=1

τ̃>M̃ i τ̃ = τ̃>

(m∑

i=1

M̃ i

)︸︷︷︸

M̃

τ̃ .

M̃ i =[x̃>i ⊗ I3| − y i

]>C i

[x̃>i ⊗ I3| − y i

]

Vectorization:

T ⊕ x i − y i =[x̃>i ⊗ I3| − y i

]τ̃

x̃ i =

[x i1

]τ̃ =

[vec(T )

1

]vec(T ) =

[vec(R)

t

]

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Unified formulation


f (T ) =m∑

i=1

τ̃>M̃ i τ̃ = τ̃>

(m∑

i=1

M̃ i

)︸︷︷︸

M̃

τ̃ .

M̃ i =[x̃>i ⊗ I3| − y i

]>C i

[x̃>i ⊗ I3| − y i

]

Vectorization:

T ⊕ x i − y i =[x̃>i ⊗ I3| − y i

]τ̃

x̃ i =

[x i1

]τ̃ =

[vec(T )

1

]vec(T ) =

[vec(R)

t

]

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Unified formulation


f (T ) =m∑

i=1

τ̃>M̃ i τ̃ = τ̃>

(m∑

i=1

M̃ i

)︸︷︷︸

M̃

τ̃ .

M̃ i =[x̃>i ⊗ I3| − y i

]>C i

[x̃>i ⊗ I3| − y i

]

Vectorization:

T ⊕ x i − y i =[x̃>i ⊗ I3| − y i

]τ̃

x̃ i =

[x i1

]τ̃ =

[vec(T )

1

]vec(T ) =

[vec(R)

t

]

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Translation marginalization

Complete problem

f ? = minT∈SO(3)

τ̃>M̃ τ̃ , τ̃ =

vec(R)t1

Marginalized problem

f ? = minR∈SO(3)

r̃>Q̃r̃ , r̃ =

[vec(R)

1

]

Schur complement

Q̃ = M̃ !t ,!t − M̃ !t ,tM−1t ,t M̃ t ,!t ,

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Complete problem

f ? = minT∈SO(3)

τ̃>M̃ τ̃ , τ̃ =

vec(R)t1


f ? = minR∈SO(3)

r̃>Q̃r̃ , r̃ =

[vec(R)

1

]

Schur complement

Q̃ = M̃ !t ,!t − M̃ !t ,tM−1t ,t M̃ t ,!t ,

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Complete problem

f ? = minT∈SO(3)

τ̃>M̃ τ̃ , τ̃ =

vec(R)t1


f ? = minR∈SO(3)

r̃>Q̃r̃ , r̃ =

[vec(R)

1

]

Schur complement

Q̃ = M̃ !t ,!t − M̃ !t ,tM−1t ,t M̃ t ,!t ,

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Complete problem

f ? = minT∈SO(3)

τ̃>M̃ τ̃ , τ̃ =

vec(R)t1


f ? = minR∈SO(3)

r̃>Q̃r̃ , r̃ =

[vec(R)

1

]

Schur complement

Q̃ = M̃ !t ,!t − M̃ !t ,tM−1t ,t M̃ t ,!t ,

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Rotation-Constrained Quadratic Program:RCQP

f ? = minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. R ∈ SO(3)

• Quadratic objective• Single rotation constraint

Flexible formulation: We could also consider

• Non-isotropic measurement noise• 3D normal-to-normal correspondences• 3D line-to-plane correspondences• 3D plane-to-plane correspondences

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? = minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. R ∈ SO(3)

• Quadratic objective• Single rotation constraint



Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? = minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. R ∈ SO(3)

• Quadratic objective

• Single rotation constraint→ non-convex→ how to solve globally?



Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

QCQP and Lagrangian relaxationQCQP problem

f ? =minx

x̃>Q̃x̃ , x̃ ≡[x1

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C

, y2 = 1

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

QCQP and Lagrangian relaxationPrimal problem

f ? =minx

x̃>Q̃x̃ , x̃ ≡[x1

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C

, y2 = 1

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C, y2 = 1

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1

Lagrangian relaxation

L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = minx̃

L(x̃ , λ̃)

≤ f ?

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = minx̃

L(x̃ , λ̃) ≤ f ?

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Dual problem

d? = maxλ̃

d(λ̃)

• Weak duality (always): d? ≤ f ?

• Strong duality (sometimes): d? = f ?

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Dual problem

d? = maxλ̃

d(λ̃)



Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Dual problem

d? = maxλ̃

d(λ̃)



Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Solution recovery

If d? = f ? (strong duality):

x̃? = arg minx̃

L(x̃ , λ̃?)

Dual problem

d? = maxλ̃

d(λ̃)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Solution recovery

If d? = f ? (strong duality):

x̃? ∈ null(Q̃ +∑i∈C̃

λ?i P̃ i)

Dual problem

d? = maxλ̃

d(λ̃)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Solution recovery

If rank(null(Q̃ +∑

i∈C̃ λ?i P̃ i)) = 1:


λ?i P̃ i)

Dual problem

d? = maxλ̃

d(λ̃)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Solution recovery


i∈C̃ λ?i P̃ i)) = 1:


λ?i P̃ i), y? = 1

Dual problem

d? = maxλ̃

d(λ̃)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Solution recovery


i∈C̃ λ?i P̃ i)) = 1:


λ?i P̃ i), y? = 1

Dual problem

d? = maxλ̃

d(λ̃)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Solution recovery


i∈C̃ λ?i P̃ i)) = 1:


λ?i P̃ i), y? = 1

Dual problem

d? = maxλ̃

γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

• Linear objective• Lin. mat. ineq. (LMI) constraint

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Solution recovery


i∈C̃ λ?i P̃ i)) = 1:


λ?i P̃ i), y? = 1

Dual problem

d? = maxλ̃

γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

• Linear objective

• Lin. mat. ineq. (LMI) constraint

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


f ? =minx

x̃>Q̃x̃ , x̃ ≡[xy

],

s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃

, y2 = 1


L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃

λi P̃ i)x̃ ,

d(λ̃) = γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

Solution recovery


i∈C̃ λ?i P̃ i)) = 1:


λ?i P̃ i), y? = 1

Dual problem

d? = maxλ̃

γ s.t. Q̃ +∑i∈C̃

λi P̃ i < 0

• Linear objective• Lin. mat. ineq. (LMI) constraint

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Semidefinite Programming (SDP)

Primal SDP

f ? = minX∈Symn

tr(CX )

s.t. tr(AiX ) = bi , ∀i ∈ CX < 0.

Dual SDP

d? = maxy∈Rm

b>y , m = #(C)

s.t. C +∑i∈C

yiAi < 0

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Primal SDP

f ? = minX∈Symn

tr(CX )

s.t. tr(AiX ) = bi , ∀i ∈ CX < 0.

Dual SDP

d? = maxy∈Rm

b>y , m = #(C)

s.t. C +∑i∈C

yiAi < 0

LP domain: Polytope

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Primal SDP

f ? = minX∈Symn

tr(CX )

s.t. tr(AiX ) = bi , ∀i ∈ CX < 0.

Dual SDP

d? = maxy∈Rm

b>y , m = #(C)

s.t. C +∑i∈C

yiAi < 0

LP domain: Polytope SDP domain: Spectrahedron

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Primal SDP

f ? = minX∈Symn

tr(CX )

s.t. tr(AiX ) = bi , ∀i ∈ CX < 0.

Dual SDP

d? = maxy∈Rm

b>y , m = #(C)

s.t. C +∑i∈C

yiAi < 0

Interior Point Method solvers:

• SeDuMi• SDPT3• SDPA• Mosek• Etc.

SDP domain: Spectrahedron

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

RCQP problemUsual Lagrangian relaxation

RCQP problem

f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. R ∈ SO(3)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


RCQP problem

f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. R ∈ SO(3)

Rotation matrix constraints:

R ∈ SO(3)⇒ {R ∈ R3×3 : R>R = I3,det(R) = 1}.

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


RCQP problem

f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. R ∈ SO(3)


R ∈ SO(3)⇒ {R ∈ R3×3 : R>R = I3︸︷︷︸quadratic

, det(R) = 1︸︷︷︸cubic

}.

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


RCQP problem

f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. R ∈ SO(3)



, ��det(R) = 1︸︷︷︸cubic

}.

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


RCQP problem

f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. R>R = I3



, ��det(R) = 1︸︷︷︸cubic

}.

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


RCQP problem ≡ QCQP

f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,6

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,6

Dual problem SDP ?−→ R?

d? =maxλ,γ

γ, s.t. Q̃ +6∑

i=1

λi P̃ i + γP̃h < 0

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,6


d? =maxλ,γ

γ, s.t. Q̃ +6∑

i=1

λi P̃ i + γP̃h < 0

But is dual problem tight?

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,6


d? =maxλ,γ

γ, s.t. Q̃ +6∑

i=1

λi P̃ i + γP̃h < 0

But is dual problem tight?No, in general d? ≤ f ? [1].[1] Olsson & Eriksson, "Solving quadratically constrained geometrical problems usingLagrangian duality". ICPR08.

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Duality strengtheningBUT the dual problem is not intrinsic [2]!

Strengthening tools

• compose objective function with increasing function• introduce extra (constrained) variables• add (linearly independent) redundant constraints

[2] Boyd & Vandenberghe, "Convex optimization". Cambridge University Press (2004).

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Duality strengtheningBUT the dual problem is not intrinsic [2]!

Strengthening tools

• compose objective function with increasing function• introduce extra (constrained) variables• add (linearly independent) redundant constraints

[2] Boyd & Vandenberghe, "Convex optimization". Cambridge University Press (2004).

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Redundant rotation constraints

Desired properties:

• Linearly independent• Quadratic

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Ort. columns R>R = I3

Ort. rows RR> = I3

R(1) × R(2) = R(3) R(2) × R(3) = R(1) R(3) × R(1) = R(2)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Ort. columns R>R = I3 Ort. rows RR> = I3

R(1) × R(2) = R(3) R(2) × R(3) = R(1) R(3) × R(1) = R(2)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



R(1) × R(2) = R(3)

R(2) × R(3) = R(1) R(3) × R(1) = R(2)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



R(1) × R(2) = R(3) R(2) × R(3) = R(1)

R(3) × R(1) = R(2)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



R(1) × R(2) = R(3) R(2) × R(3) = R(1) R(3) × R(1) = R(2)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Minimal rotation constraints:Constraint type Constraint equation #

Orthonormal rows RR> = I3 6Determinant det(R) = +1 1

Redundant rotation constraints:Constraint type Constraint equation #

Orthonormal rows RR> = I3 6Orthonormal columns R>R = I3 6

HandednessR(1) × R(2) = R(3) 3R(2) × R(3) = R(1) 3R(3) × R(1) = R(2) 3

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Penalization patterns:

Ort. columns Ort. rows Handedness

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

RCQP problemStrengthened Lagrangian relaxation

RCQP problem

f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. R ∈ SO(3)


R ∈ SO(3)⇒ {R ∈ R3×3 : R>R = I3,RR> = I3,

R(1)×R(2) = R(3),

R(2)×R(3) = R(1),

R(3)×R(1) = R(2)}.

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,21

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,21


d? =maxλ,γ

γ, s.t. Q̃ +21∑

i=1

λi P̃ i + γP̃h < 0

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,21


d? =maxλ,γ

γ, s.t. Q̃ +21∑

i=1

λi P̃ i + γP̃h < 0

Is dual problem tight?

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary



f ? =minR

r̃>Q̃r̃ , r̃ =

[vec(R)

1

], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,21


d? =maxλ,γ

γ, s.t. Q̃ +21∑

i=1

λi P̃ i + γP̃h < 0

Is dual problem tight?Yes, d? = f ?, for any problem.Warning: Empirical evidence [3].

[3] Briales & Gonzalez, "Convex Global 3D Registration with Lagrangian Duality". CVPR17.

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Experiments

Experiment from Olsson and Eriksson [1]

[1] Olsson & Eriksson, "Solving quadratically constrained geometrical problems usingLagrangian duality". ICPR08.

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Experiments

0 0.1 0.2 0.3 0.4 0.5

σ (m)

0

10

20

30

40

50

60

70

80

90

100

%op

timal

BnB

Ours

Olsson

Synthetic problems (m̂ = 10)

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Experiments

7 8 9 10 11 12 13 14 15

m̂

0

10

20

30

40

50

60

70

80

90

100

%optimal

BnB

Ours

Olsson

Real measurements on model

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

Closure and questions

Conclusions:

• Generic 3D registration solved globally• Non-convexity of R ∈ SO(3) circumvented via convex (SDP) relaxation

Code available: http://mapir.isa.uma.es/work/rotlift

Or simply scan the QR code! Future directions:

• Theoretical proof of strong duality• Faster resolution of SDP problem• Optimality verification• Multiple global minima• Robust registration

http://mapir.isa.uma.es/work/rotlift

Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Conclusions:

• Generic 3D registration solved globally

• Non-convexity of R ∈ SO(3) circumvented via convex (SDP) relaxation





Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Conclusions:






Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Conclusions:



Or simply scan the QR code!

Future directions:



Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary


Conclusions:






Fast andGlobal 3D


Planes

Jesus Briales

Problem


Marginalization

RCQP

QCQP

SDP

Dual of RCQP

Experiments

Conclusion

Summary

References I

C. Olsson and A. Eriksson, “Solving quadratically constrained geometricalproblems using lagrangian duality,” in Pattern Recognition, 2008. ICPR2008. 19th Int. Conf., pp. 1–5, IEEE, 2008.

S. Boyd and L. Vandenberghe, Convex optimization.Cambridge University Press, 2004.

J. Briales and J. González-Jiménez, “Convex Global 3D Registration withLagrangian Duality,” in Int. Conf. Comput. Vis. Pattern Recognit., jul 2017.

Date post:	25-Dec-2021
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Fast and Global 3D Registration of Points, Lines and ...

Documents