Date post: | 21-Dec-2015 |
Category: |
Documents |
View: | 215 times |
Download: | 0 times |
Measuring Uncertainty in Graph Cut Solutions
Pushmeet Kohli Philip H.S. Torr
Department of Computing Oxford Brookes University
Objective
Labelling problem
t
s
Graph Cut
st-mincut
Belief or Confidence
Most ProbableSolution
No uncertainty measureassociated with the solution
Outline
Inference in Graphical Models Inference using Graph Cuts Computing Min-marginals using Graph
Cuts Flow Potentials and Min-marginals Results
Outline
Inference in Graphical Models Inference using Graph Cuts Computing Min-marginals using Graph
Cuts Flow Potentials and Min-marginals Results
Inference in Graphical Models
x* = arg max Pr(x|D) x
E(x|D) = -log Pr(x|D) + constant
x* = arg min E(x|D) x
x: Set of latent variablesD: Observed Data
x*: Most Probable (MAP) SolutionPr: Joint Posterior Probability
Inference in Graphical Models
.2 .1 .05
.35 .05 .2
.01 .03 .01
A
B 0 1 2
0
1
2
Joint Distribution
Pr(A,B)
MAP Solution
arg max Pr(A,B) {A=1, B=0}
Inference in Graphical Models
Marginal- Sum the joint probability over all other variables.
.2 .1 .05
.35 .05 .2
.01 .03 .01
A
B 0 1 2
0
1
2
P(A=1) = 0.6
Inference in Graphical Models
.2 .1 .05
.35 .05 .2
.01 .03 .01
A
B 0 1 2
0
1
2
Max- Marginals (µ)- Maximum joint probability over all other variables.
µA,1 = 0.35
Inference in Graphical Models
.2 .1 .05
.35 .05 .2
.01 .03 .01
A
B 0 1 2
0
1
2
µA,1 = 0.35
µA,0 = 0.2
µA,2 = 0.03
Confidence or Belief (σ)- Normalized max-marginals σA,1 = µA,1 / Σx µA,x
= 0.35/ 0.58= 0.603
Inference in Graphical Models
Min-Marginals Energies(ψ)- Minimize joint energy over all other variables.
- Related to max-marginals as:
µj = (1/z)*exp(-ψj)
Inference in Graphical Models
Min-Marginals Energies(ψ)- Minimize joint energy over all other variables.
- Related to max-marginals as:
- Can be used to compute confidence as:
σj = µj / Σa µa = exp(-ψi) / Σa exp(-ψa)
µj = (1/z)*exp(-ψj)
Inference in Graphical Models
Min-Marginals Energies(ψ)- Minimize joint energy over all other variables.
- Related to max-marginals as:
- Can be used to compute confidence as:
How to compute min-marginal energies using Graph Cuts?
σj = µj / Σa µa = exp(-ψi) / Σa exp(-ψa)
µj = (1/z)*exp(-ψj)
Inference in Graphical Models
Graphical Model
Topology
Tree Graph with cycles
Belief Propagation and variants
Exact solution
True Marginals/ min-marginals
Approximate solution
Approximate Marginals/ min-marginals
Graph Cuts
No Marginals/
Min-Marginals
Class 1: Single Max-flow Computation, Exact Solution
Class 2: Expansions/ Swap Moves, Approximate Solution
Inference in Graphical Models
Graphical Model
Topology
Tree Graph with cycles
Belief Propagation and variants
Exact solution
True Marginals/ min-marginals
Approximate solution
Approximate Marginals/ min-marginals
Graph Cuts
No Marginals/
Min-Marginals
Class 1: Single Max-flow Computation, Exact Solution
Class 2: Expansions/ Swap Moves, Approximate Solution
Outline
Inference in Graphical Models Inference using Graph Cuts Computing Min-marginals using Graph
Cuts Flow Potentials and Min-marginals Results
• Given energy functions E(x|D),
Compute: arg min E(x|D)
Inference using Graph cuts
x
Inference using Graph cuts
• Certain E(x|D) can be minimized using graph cuts exactly.
• Given energy functions E(x|D),
Compute: arg min E(x|D)x
Inference using Graph cuts
• Class of energy function and graph contruction
- Binary random variables
- Submodular functions (Kolmogorov & Zabih, ECCV 2002)
- Multi-valued variables - Convex Pair-wise Terms (Ishikawa, PAMI 2003)
• Certain E(x|D) can be minimized using graph cuts exactly.
• Given energy functions E(x|D),
Compute: arg min E(x|D)x
EMRF(a1,a2)
Sink (0)
Source (1)
a1
a2
Graph Construction for Binary Random Variables
Inference using Graph cuts
a1
a2
EMRF(a1,a2) = 2a1
2t-edges
(unary terms)
Inference using Graph cuts
Sink (0)
Source (1)
EMRF(a1,a2) = 2a1 + 5ā1
a1
a2
2
5
Inference using Graph cuts
Sink (0)
Source (1)
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2
a1
a2
2
5
9
4
Inference using Graph cuts
Sink (0)
Source (1)
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2
a1
a2
2
5
9
4
2
n-edges(pair-wise term)
Inference using Graph cuts
Sink (0)
Source (1)
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
a1
a2
2
5
9
4
2
1
Inference using Graph cuts
Sink (0)
Source (1)
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
a1
a2
2
5
9
4
2
1
Inference using Graph cuts
Sink (0)
Source (1)
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
a1
a2
2
5
9
4
2
1
a1 = 1 a2 = 1
EMRF(1,1) = 11
Cost of st-cut = 11
Inference using Graph cuts
Sink (0)
Source (1)
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
a1
a2
2
5
9
4
2
1
a1 = 1 a2 = 0
EMRF(1,0) = 8
Cost of st-cut = 8
Inference using Graph cuts
Sink (0)
Source (1)
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
a1
a2
2
5
9
4
2
1
a1 = 1 a2 = 0
EMRF(1,0) = 8
Cost of st-cut = 8
Inference using Graph cuts
MAP Solutiona1,map = 1 a2,map = 0 Sink (0)
Source (1)
Outline
Inference in Graphical Models Inference using Graph Cuts Computing Min-marginals using Graph
Cuts Flow Potentials and Min-marginals Results
Computing Min-marginals using Graph Cuts
Computing Min-marginals using Graph Cuts
Instead of minimizing E(.), minimize a projection of E(.) where the value of latent
variable xv is fixed to label j.
All projections of a sub-modular function aresub-modular [Kolmogorov and Zabih, ECCV 2002]
Computing Min-marginals using Graph Cuts
Instead of minimizing E(.), minimize a projection of E(.) where the value of latent
variable xv is fixed to label j.
Problem Solved? Not Really!
Computing Min-marginals using Graph Cuts
• Computing a min-marginal requires computation of a st-cut.
• Typical Segmentation problem- 640x480 image, 2 labels- Variables = 640x480 = 307200- Number of Min-marginals = 307200x2 = 614400 - Time taken for 1 graph cut = .3 seconds- Total computation time = 614400x0.3 = 184320 sec
= 51.2 hours!!!
Energy Projections and Graph Construction
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
a1
a2
2
5
9
4
2
1
Sink (0)
Source (1)
Energy Projections and Graph Construction
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
a2= 1
EMRF(a1,1) = 2a1 + 5ā1+ 9 + ā1
a1
a2
2
5
9
4
2
1
Sink (0)
Source (1)
Energy Projections and Graph Construction
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
a1
a2
2
6
9
Sink (0)
Source (1) a2= 1
EMRF(a1,1) = 2a1 + 5ā1+ 9 + ā1
EMRF(a1,1) = 2a1 + 6ā1+ 9
Energy Projections and Graph Construction
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2 + Kā2
a1
a2
2
5
9
4
2
1
Sink (0)
Source (1)
∞
A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection.
Alternative Construction
K
Energy Projections and Graph Construction
a1
a2
2
5
9
4
2
1
Sink (0)
Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection.
• The minimum value of the energy projection can be calculated by using the same graph.
Alternative Construction
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2 + Kā2
∞K
Energy Projections and Graph Construction
a1
a2
2
5
9
4
2
1
Sink (0)
Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection.
• The minimum value of the energy projection can be calculated by using the same graph.
Alternative Construction
K> Sum of Outgoing/Incoming edges
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2 + Kā2
∞K
Energy Projections and Graph Construction
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2 + Kā2
A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection.
• The minimum value of the energy projection can be calculated by using the same graph.
• Results in a small change in the graph.
a1
a2
2
5
9
4
2
1
Sink (0)
Source (1)
∞K
Alternative Construction
Energy Projections and Graph Construction
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2 + Kā2
a1
a2
2
5
9
4
2
1
Sink (0)
Source (1)
∞
A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection.
K
• The minimum value of the energy projection can be calculated by using the same graph.
• Results in a small change in the graph.
Solve using Dynamic Graph Cuts
Alternative Construction
Dynamic Graph Cuts
- Kohli and Torr [ICCV 2005]
Graph 1
Graph 2
Energy function
Projection of Energy function
Dynamic Graph Cuts
- Kohli and Torr [ICCV 2005]
Graph 1
Graph 2
Energy function
Projection of Energy function
Graph 1 and 2 are similar
Dynamic Graph Cuts
- Kohli and Torr [ICCV 2005]
Graph 1
Graph 2
Dynamic Graph Cuts
- Kohli and Torr [ICCV 2005]
Graph 1 Residual Graph 1 st-cut 1
Graph 2 Residual Graph 2 st-cut 2
Computationally Expensive Procedure
Compute Max-flow
Compute Max-flow
Dynamic Graph Cuts
- Kohli and Torr [ICCV 2005]
Graph 1 Residual Graph 1 st-cut 1
Graph 2 Residual Graph 2 st-cut 2
Re-parameterized Graph 2
Compute Max-flow
Dynamic Graph Cuts
- Kohli and Torr [ICCV 2005]
Graph 1 Residual Graph 1 st-cut 1
Graph 2 Residual Graph 2 st-cut 2
Re-parameterized Graph 2same
solution
Dynamic Graph Cuts
- Kohli and Torr [ICCV 2005]
Graph 1 Residual Graph 1 st-cut 1
Graph 2 Residual Graph 2 st-cut 2
Re-parameterized Graph 2similar edge
weights
Dynamic Graph Cuts
- Kohli and Torr [ICCV 2005]
Graph 1 Residual Graph 1 st-cut 1
Graph 2 Residual Graph 2 st-cut 2
Re-parameterized Graph 2
Extremely Fast Operation
Compute Max-flow
Compute Max-flow
Dynamic Graph Cuts
- Kohli and Torr [ICCV 2005]
Graph 1 Residual Graph 1 st-cut 1
Graph 2 Residual Graph 2 st-cut 2
Re-parameterized Graph 2
Extremely Fast Operation
300 msec
0.002msec
Summary of the Algorithm
• Construct graph G for minimizing energy E
• For computing min-marginals do:
- Obtain graph G* ≡ energy function projection E*(By adding constraining edges)
- Find the maximum flow in G* using dynamic graph cut algorithm [Kohli and Torr, ICCV 2005]
Extension to Multiple Labels
Graph construction for multi-label random variables (Ishikawa PAMI 2003)
Labels: l1 …. ln
Latent variables: x1 …. xn
x1 = l3 x2 = l2
MAP Labels
x3 = l2 x4 = l1
Extension to Multiple Labels
Graph construction for multi-label random variables (Ishikawa PAMI 2003)
Labels: l1 …. ln
Latent variables: x1 …. xn
x4 = l3
Graph for Projection
Outline
Inference in Graphical Models Inference using Graph Cuts Computing Min-marginals using Graph
Cuts Flow Potentials and Min-marginals Results
Min-Marginals and Flow Potentials
• Flow Potentials- maximum amount of flow that can be passed from the node to a particular terminal.
Min-Marginals and Flow Potentials
1 2 3
4 5
s
t
3
29
7
1 1 4
3 1
2 11
Graph
• Flow Potentials- maximum amount of flow that can be passed from the node to a particular terminal.
Min-Marginals and Flow Potentials
1 2 3
4 5
s
t
3
29
7
1 1 4
3 1
2 11
Graph
• Flow Potentials- maximum amount of flow that can be passed from the node to a particular terminal.
Source (s) flow potential of node 4
Min-Marginals and Flow Potentials
1 2 3
4 5
s
t
3
29
7
1 1 4
3 1
2 11
Graph
• Flow Potentials- maximum amount of flow that can be passed from the node to a particular terminal.
Source (s) flow potential of node 4
Min-Marginals and Flow Potentials
1 2 3
4 5
s
t
3
29
7
1 1 4
3 1
2 11
Graph
Source (s) flow potential of node 4
1 2 3
4 5
s
t
1 1
1 1
• Flow Potentials- maximum amount of flow that can be passed from the node to a particular terminal.
Min-Marginals and Flow Potentials
1 2 3
4 5
s
t
3
29
7
1 1 4
3 1
2 11
Graph
• Flow Potentials- maximum amount of flow that can be passed from the node to a particular terminal.
Sink (t) flow potential of node 4
Min-Marginals and Flow Potentials
1 2 3
4 5
s
t
3
29
7
1 1 4
3 1
2 11
Graph
Sink (t) flow potential of node 4
1 2 3
4 5
s
t
2
29
• Flow Potentials- maximum amount of flow that can be passed from the node to a particular terminal.
Min-Marginals and Flow Potentials
Relationship between Min-marginal of binary latent variables and flow potential.
MAP Solution energy
Flow potential in residual graph
+Min-marginal =
For details: See theorem 1.
Implications: Any algorithm for computing flow potentials can be used to compute min-marginals.
Outline
Inference in Graphical Models Inference using Graph Cuts Computing Min-marginals using Graph
Cuts Flow Potentials and Min-marginals Results
• Typical Segmentation problem- 640x480 image, 2 labels
• Computation Times- Naïve Approach = 51.2 hours!!!- Our Algorithm (Dynamic Graph Cuts) = 1.2 seconds
Experimental Evaluation
Experimental Evaluation
Computation Times (in seconds) for binary variables
Problem Size
(No. of Variables/
Neighbourhood)
MAP solution
(single maxflow computation)
Computing all min-marginals
1x105, 4-neighbourhood 0.18 0.70
2x105, 4-neighbourhood 0.46 1.34
4x105 , 4-neighbourhood 0.92 3.15
8x105 , 4-neighbourhood 2.17 8.21
1x105 , 8-neighbourhood 0.40 1.53
2x105 , 8-neighbourhood 1.39 3.59
4x105 , 8-neighbourhood 2.42 8.50
6x105 , 8-neighbourhood 5.12 15.61
Min-Marginals in Image segmentation
Unary likelihood Contrast Term
Uniform Prior(Potts Model)
Image Segmentation Energy Boykov and Jolly [ICCV 2001], Blake et al. [ECCV 2004]
xi = binary variable representing label (‘fg’ or ‘bg’) of pixel i
Min-Marginals in Image segmentation
ImageMAP Solution Belief - Foreground
Lowsmoothness
Highsmoothness
Moderatesmoothness
Colour Scale
1
0
0.5
[MSR]
Min-Marginals in Image segmentation
encourage smoothness
Unary likelihood Contrast Term
Uniform Prior(Potts Model)
Image Segmentation Energy Boykov and Jolly [ICCV 2001], Blake et al. [ECCV 2004]
Min-Marginals in Image segmentation
Unary likelihood Contrast Term
Uniform Prior(Potts Model)
Image Segmentation Energy Boykov and Jolly [ICCV 2001], Blake et al. [ECCV 2004]
How smoothness effects solutions?
encourage smoothness
Min-Marginals in Image segmentation
Effect of increasing pair-wise terms of the energy function
Image
Min-Marginals in Image segmentation
Effect of increasing pair-wise terms of the energy function
MAP Segmentation Foreground Confidence Map
Concluding Remarks
• Efficient method for computing exact min-marginals for certain labelling problems• Relationship between Flow-potentials and min-marginals
Applications• Computing M Most Probable Solutions• Min-marginals for parameter learning• Hierarchical Segmentation
Future Work• Efficient min-marginal computation for general problems
Thank You