Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Fast and Global 3D Registrationof Points, Lines and Planes
Adapted from "Convex Global 3D Registration with Lagrangian Duality"(CVPR17)
Jesus Briales
MAPIR GroupUniversity of Malaga
LPM WorkshopSep 28, 2017
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Generalized Absolute Pose problem
3D-3D registration: Find optimal pose T ? aligning measured points to model
2D-3D registration Generic registration
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Generalized Absolute Pose problem
3D-3D registration: Find optimal pose T ? aligning measured points to model
2D-3D registration
Generic registration
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Generalized Absolute Pose problem
3D-3D registration: Find optimal pose T ? aligning measured points to model
2D-3D registration Generic registration
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Problem formulation
Optimization problem
T ? = arg minT∈SE(3)
m∑i=1
dPi (T ⊕ x i)2
where• x i : Measured points• T ⊕ x i : Transformed point• Pi : Primitive in the model• d(·, ·): Euclidean distance
Assumptions:
• Known correspondences{x i ↔ Pi}mi=1
• No outliers
Challenge:
• Non-convexity of R ∈ SO(3)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Problem formulation
Optimization problem
T ? = arg minT∈SE(3)
m∑i=1
dPi (T ⊕ x i)2
Assumptions:
• Known correspondences{x i ↔ Pi}mi=1
• No outliers
Challenge:
• Non-convexity of R ∈ SO(3)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Problem formulation
Optimization problem
T ? = arg minT∈SE(3)
m∑i=1
dPi (T ⊕ x i)2
Assumptions:
• Known correspondences{x i ↔ Pi}mi=1
• No outliers
Challenge:
• Non-convexity of R ∈ SO(3)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Problem formulation
Optimization problem
T ? = arg minT∈SE(3)
m∑i=1
dPi (T ⊕ x i)2
Assumptions:
• Known correspondences{x i ↔ Pi}mi=1
• No outliers
Challenge:
• Non-convexity of R ∈ SO(3)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Unified formulation
Optimization objective
f (T ) =m∑
i=1
dPi (T ⊕ x i)2
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Unified formulation
Optimization objective
f (T ) =m∑
i=1
dPi (T ⊕ x i)2
Mahalanobis distance:
Pi ≡ {y i ,C i}dPi (x)
2 = ‖x − y i‖2C i
= (x − y i)>C i(x − y i),
‖x − y‖2I3‖x − y‖2(I−vv>) ‖x − y‖2nn>
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Unified formulation
Optimization objective
f (T ) =m∑
i=1
(T ⊕ x i − y i)>C i(T ⊕ x i − y i)
Mahalanobis distance:
Pi ≡ {y i ,C i}dPi (x)
2 = ‖x − y i‖2C i
= (x − y i)>C i(x − y i),
‖x − y‖2I3‖x − y‖2(I−vv>) ‖x − y‖2nn>
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Unified formulation
Optimization objective:
f (T ) =m∑
i=1
(T ⊕ x i − y i)>C i(T ⊕ x i − y i)
Vectorization:
T ⊕ x i − y i =[x̃>i ⊗ I3| − y i
]τ̃
x̃ i =
[x i1
]τ̃ =
[vec(T )
1
]vec(T ) =
[vec(R)
t
]
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Unified formulation
Optimization objective:
f (T ) =m∑
i=1
τ̃>M̃ i τ̃ = τ̃>
(m∑
i=1
M̃ i
)︸ ︷︷ ︸
M̃
τ̃ .
M̃ i =[x̃>i ⊗ I3| − y i
]>C i
[x̃>i ⊗ I3| − y i
]
Vectorization:
T ⊕ x i − y i =[x̃>i ⊗ I3| − y i
]τ̃
x̃ i =
[x i1
]τ̃ =
[vec(T )
1
]vec(T ) =
[vec(R)
t
]
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Unified formulation
Optimization objective:
f (T ) =m∑
i=1
τ̃>M̃ i τ̃ = τ̃>
(m∑
i=1
M̃ i
)︸ ︷︷ ︸
M̃
τ̃ .
M̃ i =[x̃>i ⊗ I3| − y i
]>C i
[x̃>i ⊗ I3| − y i
]
Vectorization:
T ⊕ x i − y i =[x̃>i ⊗ I3| − y i
]τ̃
x̃ i =
[x i1
]τ̃ =
[vec(T )
1
]vec(T ) =
[vec(R)
t
]
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Unified formulation
Optimization objective:
f (T ) =m∑
i=1
τ̃>M̃ i τ̃ = τ̃>
(m∑
i=1
M̃ i
)︸ ︷︷ ︸
M̃
τ̃ .
M̃ i =[x̃>i ⊗ I3| − y i
]>C i
[x̃>i ⊗ I3| − y i
]
Vectorization:
T ⊕ x i − y i =[x̃>i ⊗ I3| − y i
]τ̃
x̃ i =
[x i1
]τ̃ =
[vec(T )
1
]vec(T ) =
[vec(R)
t
]
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Translation marginalization
Complete problem
f ? = minT∈SO(3)
τ̃>M̃ τ̃ , τ̃ =
vec(R)t1
Marginalized problem
f ? = minR∈SO(3)
r̃>Q̃r̃ , r̃ =
[vec(R)
1
]
Schur complement
Q̃ = M̃ !t ,!t − M̃ !t ,tM−1t ,t M̃ t ,!t ,
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Translation marginalization
Complete problem
f ? = minT∈SO(3)
τ̃>M̃ τ̃ , τ̃ =
vec(R)t1
Marginalized problem
f ? = minR∈SO(3)
r̃>Q̃r̃ , r̃ =
[vec(R)
1
]
Schur complement
Q̃ = M̃ !t ,!t − M̃ !t ,tM−1t ,t M̃ t ,!t ,
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Translation marginalization
Complete problem
f ? = minT∈SO(3)
τ̃>M̃ τ̃ , τ̃ =
vec(R)t1
Marginalized problem
f ? = minR∈SO(3)
r̃>Q̃r̃ , r̃ =
[vec(R)
1
]
Schur complement
Q̃ = M̃ !t ,!t − M̃ !t ,tM−1t ,t M̃ t ,!t ,
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Translation marginalization
Complete problem
f ? = minT∈SO(3)
τ̃>M̃ τ̃ , τ̃ =
vec(R)t1
Marginalized problem
f ? = minR∈SO(3)
r̃>Q̃r̃ , r̃ =
[vec(R)
1
]
Schur complement
Q̃ = M̃ !t ,!t − M̃ !t ,tM−1t ,t M̃ t ,!t ,
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Rotation-Constrained Quadratic Program:RCQP
f ? = minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. R ∈ SO(3)
• Quadratic objective• Single rotation constraint
Flexible formulation: We could also consider
• Non-isotropic measurement noise• 3D normal-to-normal correspondences• 3D line-to-plane correspondences• 3D plane-to-plane correspondences
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Rotation-Constrained Quadratic Program:RCQP
f ? = minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. R ∈ SO(3)
• Quadratic objective• Single rotation constraint
Flexible formulation: We could also consider
• Non-isotropic measurement noise• 3D normal-to-normal correspondences• 3D line-to-plane correspondences• 3D plane-to-plane correspondences
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Rotation-Constrained Quadratic Program:RCQP
f ? = minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. R ∈ SO(3)
• Quadratic objective
• Single rotation constraint→ non-convex→ how to solve globally?
Flexible formulation: We could also consider
• Non-isotropic measurement noise• 3D normal-to-normal correspondences• 3D line-to-plane correspondences• 3D plane-to-plane correspondences
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationQCQP problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[x1
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C
, y2 = 1
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[x1
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C
, y2 = 1
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C, y2 = 1
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = minx̃
L(x̃ , λ̃)
≤ f ?
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = minx̃
L(x̃ , λ̃) ≤ f ?
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Dual problem
d? = maxλ̃
d(λ̃)
• Weak duality (always): d? ≤ f ?
• Strong duality (sometimes): d? = f ?
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Dual problem
d? = maxλ̃
d(λ̃)
• Weak duality (always): d? ≤ f ?
• Strong duality (sometimes): d? = f ?
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Dual problem
d? = maxλ̃
d(λ̃)
• Weak duality (always): d? ≤ f ?
• Strong duality (sometimes): d? = f ?
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Solution recovery
If d? = f ? (strong duality):
x̃? = arg minx̃
L(x̃ , λ̃?)
Dual problem
d? = maxλ̃
d(λ̃)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Solution recovery
If d? = f ? (strong duality):
x̃? ∈ null(Q̃ +∑i∈C̃
λ?i P̃ i)
Dual problem
d? = maxλ̃
d(λ̃)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Solution recovery
If rank(null(Q̃ +∑
i∈C̃ λ?i P̃ i)) = 1:
x̃? ∈ null(Q̃ +∑i∈C̃
λ?i P̃ i)
Dual problem
d? = maxλ̃
d(λ̃)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Solution recovery
If rank(null(Q̃ +∑
i∈C̃ λ?i P̃ i)) = 1:
x̃? ∈ null(Q̃ +∑i∈C̃
λ?i P̃ i), y? = 1
Dual problem
d? = maxλ̃
d(λ̃)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Solution recovery
If rank(null(Q̃ +∑
i∈C̃ λ?i P̃ i)) = 1:
x̃? ∈ null(Q̃ +∑i∈C̃
λ?i P̃ i), y? = 1
Dual problem
d? = maxλ̃
d(λ̃)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Solution recovery
If rank(null(Q̃ +∑
i∈C̃ λ?i P̃ i)) = 1:
x̃? ∈ null(Q̃ +∑i∈C̃
λ?i P̃ i), y? = 1
Dual problem
d? = maxλ̃
γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
• Linear objective• Lin. mat. ineq. (LMI) constraint
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Solution recovery
If rank(null(Q̃ +∑
i∈C̃ λ?i P̃ i)) = 1:
x̃? ∈ null(Q̃ +∑i∈C̃
λ?i P̃ i), y? = 1
Dual problem
d? = maxλ̃
γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
• Linear objective
• Lin. mat. ineq. (LMI) constraint
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
QCQP and Lagrangian relaxationPrimal problem
f ? =minx
x̃>Q̃x̃ , x̃ ≡[xy
],
s.t. x̃>P̃ i x̃ = 0, ∀i ∈ C̃
, y2 = 1
Lagrangian relaxation
L(x̃ , λ̃) = γ + x̃>(Q̃ +∑i∈C̃
λi P̃ i)x̃ ,
d(λ̃) = γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
Solution recovery
If rank(null(Q̃ +∑
i∈C̃ λ?i P̃ i)) = 1:
x̃? ∈ null(Q̃ +∑i∈C̃
λ?i P̃ i), y? = 1
Dual problem
d? = maxλ̃
γ s.t. Q̃ +∑i∈C̃
λi P̃ i < 0
• Linear objective• Lin. mat. ineq. (LMI) constraint
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Semidefinite Programming (SDP)
Primal SDP
f ? = minX∈Symn
tr(CX )
s.t. tr(AiX ) = bi , ∀i ∈ CX < 0.
Dual SDP
d? = maxy∈Rm
b>y , m = #(C)
s.t. C +∑i∈C
yiAi < 0
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Semidefinite Programming (SDP)
Primal SDP
f ? = minX∈Symn
tr(CX )
s.t. tr(AiX ) = bi , ∀i ∈ CX < 0.
Dual SDP
d? = maxy∈Rm
b>y , m = #(C)
s.t. C +∑i∈C
yiAi < 0
LP domain: Polytope
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Semidefinite Programming (SDP)
Primal SDP
f ? = minX∈Symn
tr(CX )
s.t. tr(AiX ) = bi , ∀i ∈ CX < 0.
Dual SDP
d? = maxy∈Rm
b>y , m = #(C)
s.t. C +∑i∈C
yiAi < 0
LP domain: Polytope SDP domain: Spectrahedron
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Semidefinite Programming (SDP)
Primal SDP
f ? = minX∈Symn
tr(CX )
s.t. tr(AiX ) = bi , ∀i ∈ CX < 0.
Dual SDP
d? = maxy∈Rm
b>y , m = #(C)
s.t. C +∑i∈C
yiAi < 0
Interior Point Method solvers:
• SeDuMi• SDPT3• SDPA• Mosek• Etc.
SDP domain: Spectrahedron
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemUsual Lagrangian relaxation
RCQP problem
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. R ∈ SO(3)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemUsual Lagrangian relaxation
RCQP problem
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. R ∈ SO(3)
Rotation matrix constraints:
R ∈ SO(3)⇒ {R ∈ R3×3 : R>R = I3,det(R) = 1}.
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemUsual Lagrangian relaxation
RCQP problem
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. R ∈ SO(3)
Rotation matrix constraints:
R ∈ SO(3)⇒ {R ∈ R3×3 : R>R = I3︸ ︷︷ ︸quadratic
, det(R) = 1︸ ︷︷ ︸cubic
}.
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemUsual Lagrangian relaxation
RCQP problem
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. R ∈ SO(3)
Rotation matrix constraints:
R ∈ SO(3)⇒ {R ∈ R3×3 : R>R = I3︸ ︷︷ ︸quadratic
, ������det(R) = 1︸ ︷︷ ︸cubic
}.
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemUsual Lagrangian relaxation
RCQP problem
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. R>R = I3
Rotation matrix constraints:
R ∈ SO(3)⇒ {R ∈ R3×3 : R>R = I3︸ ︷︷ ︸quadratic
, ������det(R) = 1︸ ︷︷ ︸cubic
}.
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemUsual Lagrangian relaxation
RCQP problem ≡ QCQP
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,6
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemUsual Lagrangian relaxation
RCQP problem ≡ QCQP
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,6
Dual problem SDP ?−→ R?
d? =maxλ,γ
γ, s.t. Q̃ +6∑
i=1
λi P̃ i + γP̃h < 0
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemUsual Lagrangian relaxation
RCQP problem ≡ QCQP
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,6
Dual problem SDP ?−→ R?
d? =maxλ,γ
γ, s.t. Q̃ +6∑
i=1
λi P̃ i + γP̃h < 0
But is dual problem tight?
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemUsual Lagrangian relaxation
RCQP problem ≡ QCQP
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,6
Dual problem SDP ?−→ R?
d? =maxλ,γ
γ, s.t. Q̃ +6∑
i=1
λi P̃ i + γP̃h < 0
But is dual problem tight?No, in general d? ≤ f ? [1].[1] Olsson & Eriksson, "Solving quadratically constrained geometrical problems usingLagrangian duality". ICPR08.
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Duality strengtheningBUT the dual problem is not intrinsic [2]!
Strengthening tools
• compose objective function with increasing function• introduce extra (constrained) variables• add (linearly independent) redundant constraints
[2] Boyd & Vandenberghe, "Convex optimization". Cambridge University Press (2004).
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Duality strengtheningBUT the dual problem is not intrinsic [2]!
Strengthening tools
• compose objective function with increasing function• introduce extra (constrained) variables• add (linearly independent) redundant constraints
[2] Boyd & Vandenberghe, "Convex optimization". Cambridge University Press (2004).
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Redundant rotation constraints
Desired properties:
• Linearly independent• Quadratic
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Redundant rotation constraints
Ort. columns R>R = I3
Ort. rows RR> = I3
R(1) × R(2) = R(3) R(2) × R(3) = R(1) R(3) × R(1) = R(2)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Redundant rotation constraints
Ort. columns R>R = I3 Ort. rows RR> = I3
R(1) × R(2) = R(3) R(2) × R(3) = R(1) R(3) × R(1) = R(2)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Redundant rotation constraints
Ort. columns R>R = I3 Ort. rows RR> = I3
R(1) × R(2) = R(3)
R(2) × R(3) = R(1) R(3) × R(1) = R(2)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Redundant rotation constraints
Ort. columns R>R = I3 Ort. rows RR> = I3
R(1) × R(2) = R(3) R(2) × R(3) = R(1)
R(3) × R(1) = R(2)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Redundant rotation constraints
Ort. columns R>R = I3 Ort. rows RR> = I3
R(1) × R(2) = R(3) R(2) × R(3) = R(1) R(3) × R(1) = R(2)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Redundant rotation constraints
Minimal rotation constraints:Constraint type Constraint equation #
Orthonormal rows RR> = I3 6Determinant det(R) = +1 1
Redundant rotation constraints:Constraint type Constraint equation #
Orthonormal rows RR> = I3 6Orthonormal columns R>R = I3 6
HandednessR(1) × R(2) = R(3) 3R(2) × R(3) = R(1) 3R(3) × R(1) = R(2) 3
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Redundant rotation constraints
Penalization patterns:
Ort. columns Ort. rows Handedness
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemStrengthened Lagrangian relaxation
RCQP problem
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. R ∈ SO(3)
Rotation matrix constraints:
R ∈ SO(3)⇒ {R ∈ R3×3 : R>R = I3,RR> = I3,
R(1)×R(2) = R(3),
R(2)×R(3) = R(1),
R(3)×R(1) = R(2)}.
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemStrengthened Lagrangian relaxation
RCQP problem ≡ QCQP
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,21
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemStrengthened Lagrangian relaxation
RCQP problem ≡ QCQP
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,21
Dual problem SDP ?−→ R?
d? =maxλ,γ
γ, s.t. Q̃ +21∑
i=1
λi P̃ i + γP̃h < 0
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemStrengthened Lagrangian relaxation
RCQP problem ≡ QCQP
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,21
Dual problem SDP ?−→ R?
d? =maxλ,γ
γ, s.t. Q̃ +21∑
i=1
λi P̃ i + γP̃h < 0
Is dual problem tight?
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
RCQP problemStrengthened Lagrangian relaxation
RCQP problem ≡ QCQP
f ? =minR
r̃>Q̃r̃ , r̃ =
[vec(R)
1
], s.t. r̃>P̃ i r̃ = 0, ∀i = 1, . . . ,21
Dual problem SDP ?−→ R?
d? =maxλ,γ
γ, s.t. Q̃ +21∑
i=1
λi P̃ i + γP̃h < 0
Is dual problem tight?Yes, d? = f ?, for any problem.Warning: Empirical evidence [3].
[3] Briales & Gonzalez, "Convex Global 3D Registration with Lagrangian Duality". CVPR17.
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Experiments
Experiment from Olsson and Eriksson [1]
[1] Olsson & Eriksson, "Solving quadratically constrained geometrical problems usingLagrangian duality". ICPR08.
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Experiments
0 0.1 0.2 0.3 0.4 0.5
σ (m)
0
10
20
30
40
50
60
70
80
90
100
%op
timal
BnB
Ours
Olsson
Synthetic problems (m̂ = 10)
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Experiments
7 8 9 10 11 12 13 14 15
m̂
0
10
20
30
40
50
60
70
80
90
100
%optimal
BnB
Ours
Olsson
Real measurements on model
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Closure and questions
Conclusions:
• Generic 3D registration solved globally• Non-convexity of R ∈ SO(3) circumvented via convex (SDP) relaxation
Code available: http://mapir.isa.uma.es/work/rotlift
Or simply scan the QR code! Future directions:
• Theoretical proof of strong duality• Faster resolution of SDP problem• Optimality verification• Multiple global minima• Robust registration
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Closure and questions
Conclusions:
• Generic 3D registration solved globally
• Non-convexity of R ∈ SO(3) circumvented via convex (SDP) relaxation
Code available: http://mapir.isa.uma.es/work/rotlift
Or simply scan the QR code! Future directions:
• Theoretical proof of strong duality• Faster resolution of SDP problem• Optimality verification• Multiple global minima• Robust registration
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Closure and questions
Conclusions:
• Generic 3D registration solved globally• Non-convexity of R ∈ SO(3) circumvented via convex (SDP) relaxation
Code available: http://mapir.isa.uma.es/work/rotlift
Or simply scan the QR code! Future directions:
• Theoretical proof of strong duality• Faster resolution of SDP problem• Optimality verification• Multiple global minima• Robust registration
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Closure and questions
Conclusions:
• Generic 3D registration solved globally• Non-convexity of R ∈ SO(3) circumvented via convex (SDP) relaxation
Code available: http://mapir.isa.uma.es/work/rotlift
Or simply scan the QR code!
Future directions:
• Theoretical proof of strong duality• Faster resolution of SDP problem• Optimality verification• Multiple global minima• Robust registration
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
Closure and questions
Conclusions:
• Generic 3D registration solved globally• Non-convexity of R ∈ SO(3) circumvented via convex (SDP) relaxation
Code available: http://mapir.isa.uma.es/work/rotlift
Or simply scan the QR code! Future directions:
• Theoretical proof of strong duality• Faster resolution of SDP problem• Optimality verification• Multiple global minima• Robust registration
Fast andGlobal 3D
Registrationof Points,Lines and
Planes
Jesus Briales
Problem
FormulationUnification
Marginalization
RCQP
QCQP
SDP
Dual of RCQP
Experiments
Conclusion
Summary
References I
C. Olsson and A. Eriksson, “Solving quadratically constrained geometricalproblems using lagrangian duality,” in Pattern Recognition, 2008. ICPR2008. 19th Int. Conf., pp. 1–5, IEEE, 2008.
S. Boyd and L. Vandenberghe, Convex optimization.Cambridge University Press, 2004.
J. Briales and J. González-Jiménez, “Convex Global 3D Registration withLagrangian Duality,” in Int. Conf. Comput. Vis. Pattern Recognit., jul 2017.