Master’s Programme in Computer Science
Juha kärkkäinen
Based on slides by Veli Mäkinen
DESIGN AND ANALYSIS OF ALGORITHMS (DAA 2019)
02/10/2019 1 Design and Analysis of Algorithms 2019 week 5
NP-hardness & approximability
Design and Analysis of Algorithms 2019 week 5 2
Week V
02/10/2019
NP-hardness
Definitions, reductions, examples
Book Chapter 34
Design and Analysis of Algorithms 2019 week 5 3 02/10/2019
Master’s Programme in Computer Science
DECISION VS OPTIMIZATION PROBLEM
02/10/2019 Design and Analysis of Algorithms 2019 week 5 4
Decision problem is a problem with yes/no answer.
• Hamiltonian Cycle Problem: Given a graph, is there a cycle that visits every vertex exactly once.
Optimization problem seeks a solution with a minimal or maximal value.
• Traveling Salesperson Problem (TSP): Given a weighted graph, find a Hamiltonian cycle with the smallest total weight.
Optimization problems have decisions versions:
• Traveling Salesperson Problem: Given a weighted graph and a value W, is there a Hamiltonian cycle with a total weight ≤W.
Obviously, if we can solve the optimization problem, we can solve the decision version, but the opposite is usually true too (blackboard).
Complexity classes are usually defined for decision problems.
• Hard decision version implies hard optimization version.
Master’s Programme in Computer Science
COMPLEXITY CLASSES P AND NP
P = decision problems that can be solved in O(nk) time
NP = decision problems that can be verified in O(nk) time
k constant
n input length (with appropriate encoding)
NP stands for nondeterministic polynomial time: The problems can be
“solved” using the following nondeterministic algorithm:
1. Nondeterministically ”guess” an optimal solution/proof/certificate
• For example, guess a list of edges for Hamiltonian cycle
2. Verify the solution/proof/certificate in polynomial time.
3. Return ”yes” if verified and ”no” otherwise
• Every ”yes” instances must have a certificate that can verified
(co-NP = problems with polytime verification of ”no” instances)
Design and Analysis of Algorithms 2019 week 5 5
Next
week
02/10/2019
Master’s Programme in Computer Science
NP-HARD AND NP-COMPLETE
NP-hard = problems s. t. a polynomial time algorithm for it implies polynomial time algorithm for every NP problem
• Proof by reduction from any NP-complete problem
• Optimization problems are not in NP but they are NP-hard when their decision version is NP-complete or NP-hard
• NP-hard problem could be harder then NP problems
NP-complete = NP-hard problems that are in NP
• Proof by reduction from any other NP-complete problem plus polynomial time verification
The unproven but generally accepted conjecture P≠NP implies
• NP contains problems that have no polynomial time algorithm
• No NP-hard problem has a polynomial time algorithm
Design and Analysis of Algorithms 2019 week 5 6 02/10/2019
Master’s Programme in Computer Science
REDUCTIONS
yes yes no
no L’ L
Design and Analysis of Algorithms 2019 week 5 02/10/2019
Master’s Programme in Computer Science
DEFINITIONS
yes yes no
no L’ L
Design and Analysis of Algorithms 2019 week 5 8 02/10/2019
Master’s Programme in Computer Science
TOOL TO PROVE NP-HARDNESS
Design and Analysis of Algorithms 2019 week 5 9 02/10/2019
Master’s Programme in Computer Science
BOOLEAN SATISFIABILITY: SAT
Design and Analysis of Algorithms 2019 week 5 10 02/10/2019
Master’s Programme in Computer Science
Design and Analysis of Algorithms 2019 week 5 11 02/10/2019
Master’s Programme in Computer Science
3CNF is in NP:
We need a solution/proof/certificate for any ”yes” instance and
a polynomial time algorithm for verifying the certificate.
‒ For 3CNF, the certificate is an assignment of truth values to variables
s.t. the formula is satisfied.
‒ In simple cases like this, writing down the actual algorithm is not
required… but for the sake of practice:
‒ Read the assignments to variables.
‒ Read the 3CNF and evaluate a clause at a time. Return false if any clause
evaluates false. Otherwise return true.
‒ (exact details left as exercise).
Design and Analysis of Algorithms 2019 week 5 12 02/10/2019
Master’s Programme in Computer Science
Design and Analysis of Algorithms 2019 week 5 13
¬x1
y1
x3
y6
y4 y3
02/10/2019
Master’s Programme in Computer Science
0 0 0 1
0 0 1 1
0 1 0 0
0 1 1 1
1 0 0 0
1 0 1 0
1 1 0 1
1 1 1 0
3CNF
Design and Analysis of Algorithms 2019 week 5 14
3DNF (D=disjunctive)
02/10/2019
Master’s Programme in Computer Science
Design and Analysis of Algorithms 2019 week 5 15 02/10/2019
Master’s Programme in Computer Science
REDUCTION TREE COVERED IN THIS COURSE
SAT (next week)
3CNF (done)
CLIQUE (blackboard) Subset sum (next week)
VERTEX COVER (blackboard)
HAM-CYCLE (see the book, this shows how complex reduction gadgets can be)
Independent set (study group)
Multi-LCS (study group)
Inapproximability of TSP (following slides)
Hamiltonian path (study group)
Design and Analysis of Algorithms 2019 week 5 16
(Un)bounded Knapsack (exercise)
02/10/2019
Master’s Programme in Computer Science
SOME NP-HARD PROBLEMS
Max-Clique: Given a graph G, find the maximum clique (fully connected
subgraph) in G.
CLIQUE: Does a graph G contain a clique of size k.
Min-Vertex-Cover: Given a graph G, find the smallest set V’ of vertices
s.t. every edge in G is incident to a vertex in V’.
VERTEX-COVER: Does a graph G have a vertex cover of size ≤k.
Design and Analysis of Algorithms 2019 week 5 17 02/10/2019
Master’s Programme in Computer Science
SUMMARY: PROVING NP-COMPLETENESS
02/10/2019 Design and Analysis of Algorithms 2019 week 5 18
Prove NP-completeness of L by reduction from L’
1. Prove L is in NP
• Certificate that can be verified in polynomial time
2. Describe conversion from L’ to L
• L’ is known NP-complete problem
• Conversion of input to input, not solution to solution
3. Prove that “yes”-instance maps to “yes”-instance
• Conversion of solution to solution
4. Prove that “no”-instance maps to “no”-instance
• Often proof by contradiction: “yes”-instance maps to “yes”-instance in
opposite direction (which contradicts “no”-instance mapping to “yes”-
instance).
Approximability
Definitions, examples
Book Chapter 35
02/10/2019 Design and Analysis of Algorithms 2019 week 5 19
Master’s Programme in Computer Science
APPROXIMATION ALGORITHMS
Design and Analysis of Algorithms 2019 week 5 20 02/10/2019
Master’s Programme in Computer Science
VERTEX COVER 2-APPROXIMATION
OPT=3
Design and Analysis of Algorithms 2019 week 5 21 02/10/2019
Master’s Programme in Computer Science
METRIC TSP 2-APPROXIMATION
4
2 1
3 1
1
1
2 2 2
a b
c d
02/10/2019 Design and Analysis of Algorithms 2019 week 5 22
Master’s Programme in Computer Science
METRIC TSP 2-APPROXIMATION
H cost
c
d 1
b 2
a 1
c + 2
6
c d 4
2 1
3 1
1
1 a
b
2 2 2
TSP is a spanning tree, after one edge is removed
preorder makes shortcuts to full walk and by triangle inequality the cost is less
Design and Analysis of Algorithms 2019 week 5 23 02/10/2019
Master’s Programme in Computer Science
INAPPROXIMABILITY
Some problems are hard to approximate well.
An example is general TSP (without triangle inequality).
Theorem. If P≠NP, then for any constant c≥1, there is no polynomial time
c-approximation algorithm for general TSP.
Proof. Reduction from Hamiltonian Cycle:
• Let G=(V,E) be the Hamiltonian Cycle instance.
• Let G’=(V,E’) be the complete graph on V.
• Let w be the edge cost function:
w(e)=1 if eϵE and w(e)=c|V|+1 otherwise.
• Then a Hamiltonian cycle in G has cost |V| in G’ and any other cycle
has cost at least c|V|+1+|V|-1 = (c+1)|V| > c|V|.
• Thus any c-approximation algorithm would have to find a Hamiltonian
cycle if it exists.
Design and Analysis of Algorithms 2019 week 5 24 02/10/2019