Combinatorial Optimization 1
✬
✫
✩
✪
CombinatorialOptimization
Guy Kortsarz
Combinatorial Optimization 2
✬
✫
✩
✪
Size of an instance
Running time. f(n) with n: Size of the input
If the input has no numbers then size of input
equals # of items in the input.
If input has large numbers size is # of items
· log(max).
Example:
Input: a number n
Question: is n a prime?
The size of the input is logn
Combinatorial Optimization 3
✬
✫
✩
✪
The naive algorithm
The naive algorithm tries all dividers of n
between 3 and n/2.
Actually, it is enough to stop at√n.
The algorithm makes√n divisions.
It runs in time EXPONENTIAL in the input
Combinatorial Optimization 4
✬
✫
✩
✪
Polynomially solvable problems
• Given G(V,E) and s ∈ V find the shortest
(least number of edges) path from s to
every v 6= s
• Given G(V,E), w(e) > 0 for every e ∈ E.
find a min-cost spanning tree.
• Given G(V,E) find if G admits a simple
path that covers all edges (Euler path).
• Minimum cost cut: Given a graph G(V,E)
split V into V1 ∪ V2 = V , V1 ∩ V2 = ∅ sothat e(V1, V2) is minimum
Combinatorial Optimization 5
✬
✫
✩
✪
Some problems that are probably
hard
• ∆−TSPA salesperson comes to the USA.
Wants to drive from state to state.
Wants to cover all states and get
back to its original location. How to
do it in minimum cost?
Remark: The weights usually obey
the triangle inequality. Even if this
holds, hard problem
• Vertex cover: find min-size (cost) collection
of vertices touching all edges
• k−center: given an undirected complete
graph with distance between every two
vertices and integer k, choose a subset of
size k of centers. Minimize the maximum
distance of any non-center to its closest
center.
Combinatorial Optimization 6
✬
✫
✩
✪
More problems that seem hard
• Maximum size (weight) independent sets.
An independent set is collection of vertices
which are pairwise non-neighbors
• Minimum size coloring. A coloring, a
disjoint partition of the graph to IS. The
size of the coloring is the number of sets in
the partition
• Maximum cost cut
• Steiner: Given an edge weighed graph and
a subset S ⊆ V of the vertices, find a
min-cost tree (or arborescence if directed)
containing S.
Combinatorial Optimization 7
✬
✫
✩
✪
The decision versions of TSP, VC,
k-Center are in NP
• Easy to verify a solution
• Too many candidates for exhaustive search.
For TSP: (n− 1)! candidate solutions
For IS (independent set): 2n
For VC: 2n
For MC: Roughly n!
For k = n/2 in k−center(
nn/2
)
which is roughly
2n
Combinatorial Optimization 8
✬
✫
✩
✪
An interesting example
The problem:
Given n find if it is a prime or not can be
solved in time O(log10.5 n), thus time
polynomial in the input.
The actual time may be O(log6 n).
Due to Agrawal Kayal Saxena. Solved (only) in
2003
BUT: Given n = p · q (its known that p, q are
primes) it seems hard to find the factors
Combinatorial Optimization 9
✬
✫
✩
✪
Why are we interested in NPC
problems?
• Industrial applications: Substantial
financial implications for a practical
polynomial algorithm for some NPC
problems
• Security and weapons: a practical
polynomial algorithm implies we can factor
large numbers (by binary search). Can
break most codes used....
• Who wants to be a millionaire? One
of the seven millennium problems. A
1000000 dollars reward.
I can even trace a relation to “Alice in
wonderland”: (Dodgson elections).
Combinatorial Optimization 10
✬
✫
✩
✪
A central problem: maximum flow
in networks
The input for the maximum flow problem is a
directed graph G(V,E) with capacity c(e) > 0
for every e ∈ E and two distinguished nodes: a
source s and a target t
We will assume no edge enters s and no edge
leaves t
A legal flow is a flow function f : E 7→ R+ that
obeys the following restrictions:
1. Capacity constraint: f(e) ≤ c(e) for every
e ∈ E
2. Flow conservation constraint: for every
v 6= s, v 6= t,∑
e∈in(v)
f(e) =∑
e∈out(v)
f(e).
Combinatorial Optimization 11
✬
✫
✩
✪
The goal in a maximum flow
problem
The scenario described is that some
commodities (could be fluids or electricity or
automobiles) from s to t
The middle vertices do not create nor waste
flow.
Hence the goal is to maximize:
∑
e∈in(t)
f(e).
Combinatorial Optimization 12
✬
✫
✩
✪
Example
3
8
1
5
1
6
1,
2,
2
1,
1,
91,
10
2,
1,
2,
a
b c
d
st
Figure 1: A legal flow
Combinatorial Optimization 13
✬
✫
✩
✪
A basic flow property
The flow can be measured at every cut.
A cut (S, S̄) is a partition of V into V = S ∪ S̄,
S ∩ S̄ = ∅ so that s ∈ S and t ∈ S̄.
The flow along the cut is:
∑
e∈e(S,S̄)
f(e)−∑
e∈e(S̄,S)
f(e)
This is called the net flow along the cut
Combinatorial Optimization 14
✬
✫
✩
✪
Example
3
8
1
5
1
6
1,
2,
2
1,
1,
91,
10
2,
1,
2,
a
b c
d
st
s
d
b
t
a
c
1
2
1
1
1+2+1−1−1=3
Figure 2: A cut and the net flow along the cut
Combinatorial Optimization 15
✬
✫
✩
✪
All net flows are equal
S
v
x w
z
y
y
xz
w
Z−Y=X−W
Z+W=X+Y
Figure 3: Moving one vertex from side to side
makes no difference
Combinatorial Optimization 16
✬
✫
✩
✪
The net flow does not depend on
the cut
Since moving a single vertex from left to right
does not change the flow (and similarly, moving
a vertex from right to left does not change the
flow) by moving vertices (except for s, t)
around we can get any proper partition.
Thus, for any S, S̄,
f(S) =∑
e∈S 7→S̄ −∑
e∈S̄ 7→S = |f |For example, the value of the flow also equals
the amount of flow that goes out of s
Combinatorial Optimization 17
✬
✫
✩
✪
The capacity of a cut
Given a cut S, S̄, s ∈ S, and t ∈ S̄ the capacity
of the cut is defined by:
c(S)∑
e∈S 7→S̄
c(e).
Remarks:
• The capacity of a cut depends on the
graph and the partition. In does not
depend on any specific flow
• In the capacity of a cut (unlike net flow)
we only take edges that go from S to S̄
Combinatorial Optimization 18
✬
✫
✩
✪
Every cut capacity bounds any
flow value
Claim 1 For every cut S, S̄, and legal flow f ,
|f | ≤ c(S).
Proof.
|f | =∑
e∈e(S,S̄)
f(e)−∑
e∈e(S̄,S)
f(e)
≤∑
e∈e(S,S̄)
f(e) ≤∑
e∈e(S,S̄)
c(e)
= c(S)
Combinatorial Optimization 19
✬
✫
✩
✪
The world of the dwarfs and giants
Every cut capacity bounds from above any flow
value.
c(S)
c(S’)
ff’
f’’f * S *
Figure 4: The capacity of every cut is at least
as large as the value of every flow
If there are f∗ and S∗ so that |f∗| = c(S∗) then
the flow is maximum and the capacity is
minimum
Combinatorial Optimization 20
✬
✫
✩
✪
The min-cut max-flow theorem
We are going to prove:
Theorem 1 For every network, the min-cut
equals the max-flow
This was proved by Ford and Fulkerson in
1955. The proof used the first ever algorithm
for finding the max-flow.
The min-cut is also discovered
Combinatorial Optimization 21
✬
✫
✩
✪
How do we compute a max-flow?
Improvements: the simple possibility:
s
a
b
t
M
M
M
1
M
+1
+1
+1
M
1,M
1,1
1,M
0,M
Figure 5: Any path of non-saturated edges from
s to t gives a unit improvement of flow
Combinatorial Optimization 22
✬
✫
✩
✪
Some time flow reduction on edges
help
s
a
b
t
M
M
M
1
1,
1,
1, M
+1
−1
+1
Figure 6: Flow on b 7→ a is redirected into b 7→ t
Combinatorial Optimization 23
✬
✫
✩
✪
The residual graph
s
a
b
t
M
M
M
1
1,
1,
1, M
M M−11
M−1
1M
1,M
1,M
1,M1,M
0,1
Figure 7: The residual graph
Combinatorial Optimization 24
✬
✫
✩
✪
The residual graph: definition
If an edge e = (u, v) has capacity c and flow f
then:
1. It has an edge in the same direction of e iff
f < c
2. It has an edge in the reverse direction of e
(this means the edge (v, u)) if and only if
f > 0
3. If f < c then the edge in the same
direction as e has capacity c− f .
4. If f > 0 the edge in the reverse direction to
e has capacity f .
Combinatorial Optimization 25
✬
✫
✩
✪
Intuition
Idea: we are splitting the edges into original
and reverse edges.
Original edges: how much can we increase f?
No more than c− f .
Reverse edges: how much can we decrease f?
By no more than f .
Instead of going in a “no entry” street (on
edges in the reverse direction) we simply add
the anti-parallel edge
Combinatorial Optimization 26
✬
✫
✩
✪
s to t paths in R(G, f)
Claim: if there is an s to t path in P in
R(G, f) we can increase f .
The idea: let
d1 = mine that is an original edge in P
c(e)− f(e),
d1 = mine that is a reverse edge in P
f(e),
d = min{d1, d2}.
It must be that d > 0 because there is a path in
R(G, f) between s and t.
Add d units of flow to every original edge in P
and subtract d flow units from every reverse
edge
Combinatorial Optimization 27
✬
✫
✩
✪
The flow is still legal
+d +d
+d −d
−d −d
−d +d
Figure 8: The balance is always preserved
Combinatorial Optimization 28
✬
✫
✩
✪
The flow has increased
Since the path is from s to t it starts with an
original edge (s, x).
The flow along (s, x) was increased by d.
Thus the value of the flow is larger: the value
of the flow can be measured at s.
But: if no s to t path exists in R(G, f) does
this mean that f is a maximum flow? Yes!
Combinatorial Optimization 29
✬
✫
✩
✪
Proof of the min-cut max-flow
theorem
Consider all vertices that are reachable in
R(G, f) from s:
a
b
c
s
f=c
f=c
f=0
f=0
Figure 9: The flow along the cut of reachable
from s vertices
Combinatorial Optimization 30
✬
✫
✩
✪
The cut that equals the flow
The vertex s with all the other vertices it can
reach in R(G, f) is a subset S ⊆ V
The subset S contains s but not t
All edges that go from S to S̄ are saturated
All edges that go from S̄ to S have flow zero
Therefore c(S, S̄) = |f |Thus, the flow is maximum and the cut is
minimum
We still need to show an algorithm that finds a
flow with no augmenting paths in polynomial
in the input time
Combinatorial Optimization 31
✬
✫
✩
✪
The Ford and Fulkerson algorithm
Find a directed path between s and t in
R(G, f). How? See later (BFS).
Augment the flow along the path by the
bottleneck.
Iterate.
The following example shows that this
algorithm can have bad running time:
Combinatorial Optimization 32
✬
✫
✩
✪
s
a
b
t
M
M
M
1
1,
1,
1, M
+1
−1
+1
Figure 10: Flow on b 7→ a is redirected into b 7→ t
Combinatorial Optimization 33
✬
✫
✩
✪
The running time of FF and better
algorithms
The above example shows the running time can
be O(M).
Thus not polynomial in the size of the input
Two choices give polynomial time algorithms:
1. Select the least number of edges paths
when you augment (later we see how to
find such path)
2. Select the highest bottleneck path
Combinatorial Optimization 34
✬
✫
✩
✪
The Edmonds-Karp algorithm
Select BFS paths. Requires O(|E|).Claim: The distance in R(G, f) from s to x can
never decrease
Try to prove it at home. Provide proof later
Claim: The number of augmenting paths is
O(|E| · |V |)Proof: the distance between s and t can be
only 1 or 2, and so on up to |V | − 1.
How much time can the distance stay at k?
Claim: at most |E| augmenting paths can the
distance stay at k.
If we prove it we get O(|E| · |V |) augmentations
for O(|E|2|V |) running time
Combinatorial Optimization 35
✬
✫
✩
✪
Proof
A single augmentation reverses an edge
This edge can no longer serve in any shortest
path
d=10
d=9
d>=10
d=9
Figure 11: An edge that no longer can be used
Therefore after O(|E|) augmenting paths there
is no length k path from s to t
Combinatorial Optimization 36
✬
✫
✩
✪
The bottleneck algorithm
Let R be the difference between the maximum
flow and the flow at the moment.
Claim: there is always a single path that
augments the flow by at least R/m for m = |E|.Before we prove it, let us see that this gives:
Claim: the bottleneck algorithm runs in
O (m · (logn+ logM)) time with M the
maximum capacity. Note: this is a polynomial
in the input time.
Combinatorial Optimization 37
✬
✫
✩
✪
Proof
Let f be the maximum flow value.
After the next augmentation,
Ri+1 = R
(
1− 1
m
)
.
Therefore after m · ln f augmentations:
Ri+1 ≤ f ·(
1− 1
m
)m·ln f
≤ 1.
We show this by noting that(
1− 1
m
)m
≤ 1
e.
So
f
(
1− 1
m
)m ln f
= f
((
1− 1
m
)m)ln f
≤ f ·(
1
e
)ln f
= 1
Combinatorial Optimization 38
✬
✫
✩
✪
Flow decomposition
The path claim: Given some flow f with value
|f | = x we show that we can select an s to t
path and a number 0 < d ≤ x so that:
: There exists a path P from s to t in G and
all the edges on the path carry at least d
units of flow
At least one edge in P carries d units of flow
exactly
If we can do that, we get the bottleneck claim
Combinatorial Optimization 39
✬
✫
✩
✪
The path claim implies the
bottleneck claim
Find P as promised in the path claim
Zero the flow on P
Iterate.
The process defines at most m paths because
for every P chosen the flow of at least one edge
is zeroed.
Conclusion: the reverse augmentation shows
that we may augment the flow to maximum in
at most m paths (always!).
Let |f | be the maximum flow and R the current
one. By averaging, at least one of the paths
carries (|f | −R)/m flow units or more
Remark: there are m paths that ONLY
INCREASES FLOW ALONG EDGES.
Combinatorial Optimization 40
✬
✫
✩
✪
The path claim
Start with |f |. Let e1 = (v, t) be an edge
entering t. Decrease the flow on this edge by 1.
There most be an edge entering v with
positive flow. Choose e2 = (u, v), f(e2) > 0 and
decrease the flow on e2 by 1.
There must be still flow entering u. Select e3
entering u and reduce the flow along e3 by 1,
etc.
A problem that may happen is the formation of
cycles
Combinatorial Optimization 41
✬
✫
✩
✪
A cycle
t
Figure 12: A cycle is formed
Combinatorial Optimization 42
✬
✫
✩
✪
Proof, continued
But cycles can be discarded and the process
can continue.
Until we get to s.
We found a path from s to t and define d to be
the least flow on this path.
Thus the path claim and the decomposition
claim both hold.
Question: why do we not get O(|E|2) time
algorithm?
Combinatorial Optimization 43
✬
✫
✩
✪
The BFS algorithm
BFS(G)
1. Q← {s}
2. For all v ∈ V \ {s} dist(s, v)←∞.
dist(s, s)← 0.
3. While Q 6= ∅ do(a) v ← Delete(Q).
(b) u← N(v)
(c) While u 6= Null do:
i. If dist(s, u) =∞ then
A. dist(s, u)← dist(s, v) + 1
B. Add(Q, u)
ii. u← next(u)
4. Output the distance vector.
Combinatorial Optimization 44
✬
✫
✩
✪
Maximum independent sets in
bipartite graphs
Given a graph G(V,E), a set W ⊆ V is an
independent set if there are no edges internal to
W .
A graph is bipartite if V can be split into two
independent sets V = V1 ∪ V2 and V1 ∩ V2 = ∅Let G(V1, V2, E) be a bipartite graph.
We show how to find a maximum independent
set in G.
Combinatorial Optimization 45
✬
✫
✩
✪
The flow network
Form the following flow network: direct the
edges of E from V1 to V2.
Add a source s and a directed edge from s to
each v1 ∈ V1 with capacity 1.
Add a directed edge from each vertex in V2 to t
with capacity 1.
Give all the edges of E capacity ∞.
Combinatorial Optimization 46
✬
✫
✩
✪
The minimum cut
Consider the minimum cut in the network.
Let V s1 be the vertices of V1 in the side of s in
the cut, and V t1 = V \ V s
1 be the vertices of V1
in the side of t in the cut.
Similarly for t.
V_1^s
V_2^s
s
V_1^t V_2^t
E
E
t
Figure 13: The cut
Combinatorial Optimization 47
✬
✫
✩
✪
A crucial property
The cut containing in the left side {s} and in
the right side V \ {s} has capacity of |V1|: thereare |V1| capacity 1 edge from s to all of V1.
Any edge from V S1 to V t
2 has capacity ∞.
Thus, it can not be the case that this edge will
participate in a minimum cut, as there are
finite cuts.
So there are no edges from any vertex in V S1 to
any vertex from V t2
Combinatorial Optimization 48
✬
✫
✩
✪
The size of the cut
We show that the capacity of the cut is
|V t1 |+ |V s
2 .
Check which edges go from the side of s in the
cut to the side of t. There are no edges from
V s1 to V t
2 by the previous answer.
The edges from V t1 to V s
2 are in the “wrong”
direction (from the side of t to the side of s and
not vice-versa).
There certainly are no edges from V s1 to V s
2 or
V s2 to V t
2 , because the graph is bipartite.
There are edges from s to V t1 , each edge of
capacity 1, so |V t1 | in total, and similarly, from
V s2 to t, with a capacity of V s
2 in total.
So, the minimum cut (or maximum flow)
equals |V t1 |+ |V s
2 |.
Combinatorial Optimization 49
✬
✫
✩
✪
V s
1∪ V t
2is a maximum independent
set
V s1 ∪ V t
2 is an independent set.
Under this restriction, we need to minimize
|V t1 |+ |V s
2 |, because this is the capacity of the
cut.
So, we maximize
(|V | − (|V t1 |+ |V s
2 |)) = |V s1 |+ |V t
2 |So V s
1 ∪ V t2 is a maximum independent set
Combinatorial Optimization 50
✬
✫
✩
✪
Finding min-cuts
The global minimum cut on an UNDIRECTED
graph is the partition S, .S̄, S, S̄ 6= ∅ so that
e(S, S̄) = {e = (u, v) : u ∈ S, v ∈ S̄} isminimum.
If we know x ∈ S, y ∈ S̄, the following flow
network gives the cut:
1. Every edge e = (u, v) is replaced by two
edges u 7→ v and v 7→ u
2. Let x be source and y be sink
3. Compute the max flow, and min-cut
This takes O(n2) cut computation.
Improvement: fix x to be any vertex. Search
only for y (n flow computations)
Combinatorial Optimization 51
✬
✫
✩
✪
A better algorithm, by Karger
Let G be an undirected graph with parallel
edges allowed.
1. If the graph has only 2 vertices a, b return
the edge(s) between a and b
2. Choose an edge at random
Remark: A pair of vertices is more likely
to be chosen if it has many parallel edges
3. Contract the pair of vertices. Retain
parallel edges but delete self loops
4. Let G′ be the new graph
5. Recurse on G′
Combinatorial Optimization 52
✬
✫
✩
✪
Example
A
B
C
D
EF
B
AF E
C
D
B
AF EDC
ABF
ED
C
p=6/7
p=5/6
p=4/5
p=2/3
ABF CDE
P(MINIMUM)=6/7*5/6*4/5*2/3
Figure 14: The Karger algorithm
Combinatorial Optimization 53
✬
✫
✩
✪
The probability that the min-cut
results
Use:
Pr(A1 ∩A2 . . . ∩AK) = Pr(A1) · Pr(A2 | A1)
· Pr(A3 | A1 ∩ A2) · . . .
Say that the min-size of the cut is k
So, deg(v) ≥ k for every v.
So |E| ≥ n · k/2Therefore, the probability that an edge in the
cut is not contracted is:
1− k
nk/2= 1− 2
n.
Combinatorial Optimization 54
✬
✫
✩
✪
The probability that the result is
the min-cut, continued
Claim: the degrees of vertices after
contractions are also at least k
Proof: The edges touching a “super-vertex”
give a cut.
Therefore: number of edges after the first
contraction, at least k · (n− 1)/2
Probability contracting a cut edge at least
1− 2
n− 1.
Combinatorial Optimization 55
✬
✫
✩
✪
Computation
(
1− 2
n
)
·(
1− 2
n− 1
)
·(
1− 2
n− 2
)
· . . .
·(
1− 2
4
)
·(
1− 2
3
)
·
=
(
n− 2
n
)
·(
n− 3
n− 1
)
·(
n− 4
n− 2
)
·
· 3
5· 24· 13
=2 · 1
n · (n− 1)
Combinatorial Optimization 56
✬
✫
✩
✪
A randomized alg with one sided
error probability
Run the algorithm n(n− 1)/2 times.
The probability that non of those runs give the
min-cut is at most:
(
1− 2
n(n− 1)
)n(n−1)/2
< 1/e
If we run it n2 · (n− 1)/2 times probability of
error at most exp(−n)