Algorithms and Theory of Computation
Lecture 21: NP-Completeness
Xiaohui Bei
MAS 714
November 6, 2017
Nanyang Technological University MAS 714 November 6, 2017 1 / 32
NP-Completeness
P: set of decision problems that have polynomial time algorithms.
NP: set of decision problems that have polynomial time non-deterministic
algorithms.
Many natural problems we would like to solve are in NP.
P ⊆ NP
Some problems in NP are in P (e.g. shortest path problem)
Big Question
Does every problem in NP have an efficient algorithm?
Same as asking whether P = NP.
Nanyang Technological University MAS 714 November 6, 2017 2 / 32
“Hardest” Problems
Question
What is the hardest problem in NP? How do we define it?
Towards a definition:
hardest problem must be in NP
hardest problem must be at least as “difficult” as every other problem
in NP.
NP-Completeness
A problem X is said to be NP-Complete if
X ∈ NP, and
(hardness) for any Y ∈ NP, Y ≤P X.
Nanyang Technological University MAS 714 November 6, 2017 3 / 32
Solving NP-Complete Problems
Proposition
Suppose X is NP-Complete. Then X can be solved in polynomial time if
and only if P = NP.
Proof.
=⇒: Suppose X can be solved in polynomial time
Let Y ∈ NP, we know Y ≤P X.
Then Y can be solved in polynomial time.
Thus every problem Y ∈ NP is such that Y ∈ P; NP ⊆ P
Also P ⊆ NP, we have P = NP.⇐=: Since P = NP, and X ∈ NP, we know X ∈ P.
Nanyang Technological University MAS 714 November 6, 2017 4 / 32
NP-Hard Problems
NP-Hardness
A problem X is said to be NP-Hard if
for any Y ∈ NP, we have that Y ≤P X.
An NP-Hard problem need not to be in NP.
Example: Halting problem is NP-Hard, but not NP-Complete.
Nanyang Technological University MAS 714 November 6, 2017 5 / 32
Consequences of proving NP-Completeness
If X is NP-Complete
since we believe P 6= NP,
and solving X efficiently implies P = NP.
X is unlikely to be efficiently solvable.
At the very least, many smart people before you have failed to find an
efficient algorithm for X.
Question
Are there any problems that are NP-Complete?
Yes! There are many of them.
Nanyang Technological University MAS 714 November 6, 2017 6 / 32
Cook-Levin Theorem
Theorem (Cook-Levin)
SAT is NP-Complete.
First, need to show SAT is in NP.
Easy task. What should be our “proof/solution” string?
Second, need to prove that every language L ∈ NP, L ≤P SAT.
Difficult task. Infinite number of languages in NP. Must
simultaneously show a generic reduction strategy.
Steve Cook won the Turing award for this theorem.
Nanyang Technological University MAS 714 November 6, 2017 7 / 32
High-level Plan
What does it mean that L ∈ NP?
L ∈ NP implies that there is a non-deterministic TM M and polynomial
p() such that
L = {x ∈ Σ∗ |M accepts x in at most p(|x|) steps}
We will describe a reduction fM that depends on M,p such that:
fM takes as input a string x and outputs a SAT formula fM(x)
fM runs in time polynomial in |x|
x ∈ L if and only if fM(x) is satisfiable
Nanyang Technological University MAS 714 November 6, 2017 8 / 32
Plan continued
fM(x) is satisfiable ⇐⇒ x ∈ L ⇐⇒ non-det M accepts x in p(|x|) steps.
Big Idea:
fM(x) will express “M on input x accepts in p(|x|) steps”
fM(x) will encode a computation history of M on x
fM(x) will be a carefully constructed CNF formula s.t if we have a
satisfying assignment to it, then we will be able to see a complete
accepting computation of M on x down to the last detail of where the
head is, what transition is chosen, what the tape contents are, at each
step.
Nanyang Technological University MAS 714 November 6, 2017 9 / 32
Tableau of Computation
Entire computation of M on x can be represented by a “tableau”
1 0 0 1 blanks state q00 0 0 1 blanks state q1
......
Row i gives contents of all cells at time i.
At time 0 tape has input x followed by blanks.
Each row long enough to hold all cells M might ever have scanned.
Nanyang Technological University MAS 714 November 6, 2017 10 / 32
Variable of fM(x)
Four types of variables to describe computation of M on x
T(b, h, i): tape cell at position h holds symbol b at time i.
1 ≤ h ≤ p(|x|), b ∈ Γ, 0 ≤ i ≤ p(|x|)H(h, i): read/write head is at position h at time i.
1 ≤ h ≤ p(|x|), 0 ≤ i ≤ p(|x|)S(q, i) state of M is q at time i
q ∈ Q, 0 ≤ i ≤ p(|x|)I(j, i): instruction number j is executed at time i
Since M is non-det, need to specify transitions in some way.
Assume there are ` possible transitions, where j’th transition is
〈qj, bj, q ′j , b ′j , dj〉, indicating (q ′j , b′j , dj) ∈ δ(qj, bj),
direction dj ∈ {−1, 0, 1}
Number of variables is O(p(|x|)2).
Nanyang Technological University MAS 714 November 6, 2017 11 / 32
Notation
Some abbreviations for ease of notation
∨mk=1 xk means x1 ∨ x2 ∨ . . .∨ xm∧mk=1 xk means x1 ∧ x2 ∧ . . .∧ xm⊕(x1, x2, . . . , xk) is a formula that means exactly one of
x1, x2, . . . , xm is true. Can be converted to CNF form.
Nanyang Technological University MAS 714 November 6, 2017 12 / 32
Clauses of fM(x)
fM(x) is the conjunction of 8 clause groups
fM(x) = ϕ1 ∧ϕ2 ∧ϕ3 ∧ϕ4 ∧ϕ5 ∧ϕ6 ∧ϕ7 ∧ϕ8
where each ϕi is a CNF formula. Described in subsequent slides.
Property
fM(x) is satisfiable iff there is a truth assignment to the variables that
simultaneously satisfy ϕ1, . . . , ϕ8.
Nanyang Technological University MAS 714 November 6, 2017 13 / 32
ϕ1
ϕ1 asserts (is true iff) the variables are set T/F indicating that M starts in
state q0 at time 0 with tape contents containing x followed by blanks.
Let x = a1a2 . . . an
ϕ1 = S(q0, 0) state at time 0 is q0
∧nh=1 T(ah, h, 0) at time 0 cells 1 to n have a1 to an
∧p(|x|)h=n+1 T(B, h, 0) at time 0 cells n + 1 to p(|x|) have blanks
∧H(1, 0) head at time 0 is in position 1
Nanyang Technological University MAS 714 November 6, 2017 14 / 32
ϕ2
ϕ2 asserts M in exactly one state at any time i
ϕ2 =
p(|x|)∧i=0
(⊕(S(q0, i), S(q1, i), . . . , S(q|Q|, i)
))
Nanyang Technological University MAS 714 November 6, 2017 15 / 32
ϕ3
ϕ3 asserts that each tape cell holds a unique symbol at any given time.
ϕ3 =
p(|x|)∧i=0
p(|x|)∧h=1
(⊕(T(b1, h, i), T(b2, h, i), . . . , T(b|Γ |, h, i)
))
For each time i and for each cell position h exactly one symbol b ∈ Γ at
cell position h at time i
Nanyang Technological University MAS 714 November 6, 2017 16 / 32
ϕ4
ϕ4 asserts that the read/write head of M is in exactly one position at any
time i.
ϕ4 =
p(|x|)∧i=0
(⊕(H(1, i), H(2, i), . . . , H(p(|x|), i))
)
Nanyang Technological University MAS 714 November 6, 2017 17 / 32
ϕ5
ϕ5 asserts that M accepts, i.e., enters accepting state at some time
let qa be unique accept state of M
ϕ5 =
p(|x|)∨i=1
S(qa, i)
Nanyang Technological University MAS 714 November 6, 2017 18 / 32
ϕ6
ϕ6 asserts M executes a unique instruction at each time
ϕ6 =
p(|x|)∧i=0
(⊕(I(1, i), I(2, i), . . . , I(m, i))
)
where m is the max instruction number.
Nanyang Technological University MAS 714 November 6, 2017 19 / 32
ϕ7
ϕ7 ensures that variables don’t allow tape to change from one moment to
next if the read/write head was not there.
ϕ7 =
p(|x|)∧i=0
p(|x|)∧h=1
∧b6=c
(H(h, i) =⇒ T(b, h, i)∧ T(c, h, i+ 1)
)“If head is not at position h at time i then at time i + 1 the symbol at cell h must be
unchanged”
(A =⇒ B) can be formulated as (A∨ B)
Nanyang Technological University MAS 714 November 6, 2017 20 / 32
ϕ8
ϕ8 asserts that changes in tableau/tape correspond to transitions of M
Let j’th instruction be 〈qj, bj, q ′j , b ′j , dj〉
ϕ8 =∧i
∧j(I(j, i) =⇒ S(qj, i))
If instr j executed at time i then state must be correct to do j∧i
∧j(I(j, i) =⇒ S(q ′j , i+ 1))
and at next time unit, state must be the proper next state for instr j∧i
∧h
∧j [(I(j, i)∧H(h, i)) =⇒ T(bj, h, i)]
if j was executed and head was at position h, then cell h has correct symbol for j∧i
∧h
∧j
[(I(j, i)∧H(h, i)) =⇒ T(b ′j , h, i+ 1)
]if j was done at time i with head at h then at next step b ′
j was written in position h∧i
∧h
∧j [(I(j, i)∧H(h, i)) =⇒ H(h+ dj, i+ 1)]
head is moved properly according to instr j.
Nanyang Technological University MAS 714 November 6, 2017 21 / 32
Proof of Correctness (sketch)
Given M,x, fM can be constructed in polynomial time
if fM(x) is satisfiable then the truth assignment completely specifies
an accepting computation of M on x
if M accepts x then the accepting computation leads to an “obvious”
truth assignment to fM(x). Simply assign the variables according to
the state of M and cells at each time i.
Thus M accepts x if and only if fM(x) is satisfiable.
Nanyang Technological University MAS 714 November 6, 2017 22 / 32
Proving that a Problem X is NP-Complete
To prove X is NP-Complete, we need to
show that X is in NP
give a polynomial-time reduction from a known NP-Complete problem
such as SAT to X.
SAT ≤P X implies that every NP problem Y ≤P X. Why?
transitivity of reductions, Y ≤P SAT and SAT ≤P X implies Y ≤P X
3SAT
3SAT is NP-Complete.
Nanyang Technological University MAS 714 November 6, 2017 23 / 32
NP-Completeness via Reductions
SAT is NP-Complete due to Cook-Levin Theorem.
SAT ≤P 3SAT
3SAT ≤P Independent Set
Independent Set ≤P Clique
Independent Set ≤P Vertex Cover
3SAT ≤P 3-Color
3SAT ≤P Hamiltonian Cycle
Hundreds and thousands of different problems from many areas of science
and engineering have been shown to be NP-Complete.
A surprisingly frequent phenomenon.
Nanyang Technological University MAS 714 November 6, 2017 24 / 32
More Reductions
We have shown NP-completeness of SAT and 3SAT.
We will show some reductions from 3SAT to a few problems.
We can use reductions from any of the problems for which we alreadyhave a reduction from 3SAT
I due to transitivityI often this is easierI you can also do this in your exams
Nanyang Technological University MAS 714 November 6, 2017 25 / 32
Reducing 3SAT to Independent Set
Problem: Independent Set
Instance: A graph G and an integer k.
Question: Is there an independent set in G of size k?
The reduction 3SAT ≤P Independent Set
Input: Given a 3CNF formula ϕ
Goal: Construct a graph Gϕ and number k such that Gϕ has an
independent set of size at least k if and only if ϕ is satisfiable.
Gϕ should be constructible in time polynomial in size of ϕ
Notice: We handle only 3CNF formulas
Nanyang Technological University MAS 714 November 6, 2017 26 / 32
Interpreting 3SAT
There are two ways to think about 3SAT
1 Find a way to assign 0/1 (false/true) to the variables such that the
formula evaluates to true, that is each clause evaluates to true.
2 Pick a literal from each clause and find a truth assignment to make
all of them true. You will fail if two of the literals you pick are in
conflict, i.e., you pick both xi and xi.
We will take the second view of 3SAT to construct the reduction.
Nanyang Technological University MAS 714 November 6, 2017 27 / 32
The Reduction
1 Gϕ will have one vertex for each literal in a clause.2 Connect the 3 literals in a clause to form a triangle; the independent
set will pick at most one vertex from each clause, which will
correspond to the literal to be set to true.3 Connect 2 vertices if they label complementary literals; this ensures
that the literals picked do not have a conflict.4 Take k to be the number of clauses.
x2 x3
x1
x1 x3
x2
x2 x4
x1
Figure: Graph for ϕ =(x1 ∨ x2 ∨ x3
)∧
(x1 ∨ x2 ∨ x3
)∧
(x1 ∨ x2 ∨ x4
)Nanyang Technological University MAS 714 November 6, 2017 28 / 32
Correctness
Proposition
ϕ is satisfiable iff Gϕ has an independent set of size k (= number of
clauses in ϕ).
Proof.⇒: Let a be a truth assignment satisfying ϕ
Pick one of the vertices, corresponding to true literals under a, from
each triangle. This is an independent set of the appropriate size.⇐: Let S be an independent set of size k
S must contain exactly one vertex from each clause
S cannot contain vertices labeled by conflicting literals
Thus, it is possible to obtain a truth assignment that makes literals in
S true; such an assignment satisfies one literal in every clause
Nanyang Technological University MAS 714 November 6, 2017 29 / 32
Vertex Cover
For an undirected graph G, a vertex cover is a subset S of the vertices
such that every edge has at least one endpoint in S
Problem: Vertex Cover
Instance: A graph G and an integer k
Question: Does G has a vertex cover of size at most k?
Theorem
Vertex Cover is NP-complete.
Part 1: Vertex Cover ∈ NP
easy, guess S and check all edges
Part 2: we reduce from Independent Set
Nanyang Technological University MAS 714 November 6, 2017 30 / 32
Key Observation
Theorem
if G = (V, E) is a graph, then S is an independent set of G if and only if
V − S is a set cover.
Proof.=⇒:
Suppose S is an independent set, and let e = (u, v) be some edge.
Only one of u, v can be in S. Hence, at least one of u, v is in V − S.
So, V − S is a vertex cover.⇐=:
Suppose V − S is a vertex cover, and let u, v ∈ S. There can’t be an
edge between u and v (otherwise, that edge wouldn’t be covered in
V − S). So, S is an independent set.
Nanyang Technological University MAS 714 November 6, 2017 31 / 32
Reducing Independent Set to Vertex Cover
Reducing Independent Set to Vertex Cover
Input: Given a graph G and an integer k
Goal: Construct a graph G ′ and k ′ such that G has an independent set of
size at least k if and only G ′ has a vertex cover of size at most k ′
Reduction algorithm: output G ′ = G, k ′ = |V |− k.
correctness directly followed from our previous theorem
Nanyang Technological University MAS 714 November 6, 2017 32 / 32