Post on 02-Jan-2016
transcript
Submodularization for Binary Pairwise Energies
Lena Gorelick
joint work with
O. Veksler
I. Ben Ayed
A. Delong
Y. Boykov
3
Binary Pairwise EnergyQuadratic Form
Submodular Energy global optimum with graphcut (Boros &
Hammer, 2002)
qp
qppqp
pp xxvxuE,
(x) 1,0x
pqpq vv 0
Potts Model
4
0, pqvpq
Non-Submodular Energy NP-hard
Binary Pairwise Energy Quadratic Form
const.)(,
qp
qppqp
pp xxvxuE x
Middlebury
Image credit: Carlos Hernandes
5
Standard Optimization Methods
General energy - NP-hardApproximate methods:
Global Linearization: QPBO, TRWS, SRMP (Kolmogorov et al. 2006, 2014)
Local Linearization: parallel ICM, IPFP (Leordeanu, 2009)
Message Passing: BP (Pearl 1989)
6
Related WorkGlobal Linearization
)(xE QPBO, TRWS, SRMP (Kolmogorov et al. 2006,
2014)
)(~
min..
yyE
Cts
Linearize introducing large number ofvariables and constraints
Solve relaxed LPor its dual
Integrality Gap
*relaxedyRounding
*integer x
Related WorkIterative Local Linearization
7
parallel ICM (Leordeanu, 2009) large steps weak min
IPFP (Leordeanu, 2009) controls step size by relaxation Integrality Gap
)(xE
x
tx
Et(x)~
1tx
N}1,0{Bounded domain of discrete configurations
8
Local Submodular Approximation LSA
Local Submodular Approximation model
Non-linear
Two ways to control step size
Et(x)~
)(xE
x
tx
1tx
N}1,0{Bounded domain of discrete configurations
9
Trust Region Local submodular approximation
Auxiliary Functions = Surrogate Functions = Upper Bounds = Majorize-Minimize Local submodular upper bound
Never leave the discrete domain
Iterative Optimization Framework
LSA-AUX
LSA-TR
10
Iterative Optimization Framework
Trust Region: Discrete High Order Energies Relaxed Quadratic Binary Energies Levenberg Marquardt
Auxiliary Functions=Surrogate Functions =Upper Bounds = Majorize-Minimize Discrete High Order Energies
Gorelick et al. 2012,2013
Ben Ayed et al. 2013
Olsson et al. 2008
Narasimhan & Bilmes 2005
Rother et al. 2006
Hartley & Zisserman 2004
13
Approximate around
Local Submodular ApproximationLSA
)()()( sup xxx EEE sub
)(xE tx
)(~
x(x) subt EE
Et(x)~
x
tx)(xE
14
Submodular functionLSA
Approximate around
Local Submodular ApproximationLSA
)()()( sup xxx EEE sub
)(xE tx
)()(~
xx(x) approxt
subt EEE
Linear Approximation
Et(x)~
x
tx)(xE
20
1
constyx 0
0
1
Linear Approximation of the Supermodular Term
x
y
1,0
0,0
1,1
Linear (Unary)
approximation
LSA-TRTrust Region Sub-Problem
24
)(xE
x
tx
Et(x)~
Trust Region
)()(~
xx(x) approxt
subt EEE
Trust Region Sub-Problem
td ||||s.t. txx
24
NP-hard!Constrained Submodular Optimization
25
fixed in each iteration inversely related to trust region size adjusted based on quality of approximation
LSA-TR: Approximate TR sub-problem
||||
)()(
tt
approxt
subt EEL
xx
xx(x)
Unary TermsBoykov et al. 2006
Gorelick et al. 2013
t
Submodular
27
Experiments & Results:Deconvolution
Binary De-convolution All pairwise terms supermodular
Original Img Convolved Convolved+Noise
?
Experiments & Results:Deconvolution
28
Noise:N(0,0.05)
SRMP
QPBOI
TRWS FTR-L
LBP
LSA-TR (0.3 sec.)E=21.13
LSA-AUX (0.04 sec)E=34.70
TRWS:5000 iter.E=65.07
LBP5000 iter.E=40.15
QPBO(0.1 sec.)
QPBO-I (0.2 sec.)E=66.44
IPFP(0.4 sec.)E=32.90
SRMP:5000 iter.E=39.06
Experiments & Results:Segmentation of Thin Structures
29
QPBO
QPBO-IE= -77.08
LBPE= -84.54
IPFPE= 163.25
Image
SRMP
Potts, v<0(submodula
r)
with edge repulsion, v>0(non-submodular)
TRWSE= -67.21
LSA-TRE= -175.05
LSA-AUXE= -120.03
SRMPE= -101.61
Repulsion = Reward different labels across high contrast edges
30
Experiments & Results:Inpainting
dtf-chinesechar database
LSA-TRInput ImgGround Truth
Kappes et al., 2013
32
Curvature Regularization
Efficient Squared Curvature model – (Nieuwenhuis et al. 2014, poster on Friday)
Potts Model Elastica
90-degree curvature
Heber et al. 2012
El-Zehiry&Grady, 2010
Our curvature Using LSA-TR
33
Summary
Two novel discrete optimization methods Simple, efficient, state-of-art results The code is available online -
http://vision.csd.uwo.ca/code/
Extensions: Find new applications▪ Convexity Shape Prior (in ECCV14)
Alternative optimization framework with LSA▪ Pseudo-Bounds (in ECCV14)
Please come by our poster