CSCE350 Algorithms and Data StructureLecture 21

Jianjun Hu

Department of Computer Science and Engineering

University of South Carolina

2009.12.

Chapter 11: How to tackle NP-hard problems...

There are two principal approaches to tackling NP-hard problems or other “intractable” problems:

Use a strategy that guarantees solving the problem exactly but doesn’t guarantee to find a solution in polynomial time

Use an approximation algorithm that can find an approximate (sub-optimal) solution in polynomial time

E.g. Genetic Algorithm----for large optimization problemNewton gradient descent method

Exact solutions

The exact solution approach includes the strategies:

1 exhaustive search (brute force)• useful only for small instances

2 backtracking• eliminates some cases from consideration

3 branch-and-bound• further cuts down on the search• fast solutions for most instances • worst case is still exponential

Backtracking

Construct the state space tree:• nodes: partial solutions• edges: choices in completing solutions

Explore the state space tree using depth-first search (DFS)

“Prune” non-promising nodes• DFS stops exploring subtree rooted at nodes leading to no

solutions and...• “backtracks” to its parent node

Example: The n-Queen problem

Place n queens on an n by n chess board so that no two of them are on the same row, column, or diagonal

State-space of the four-queens problem

State-space of the four-queens problem

Example: Hamiltonian Circuit Problem

d

a b

e

c fc

d

e

fdead end

e

d f

b

a

f

e

c

d

solution

dead end dead end

0

1

2

3

4

5

6

7 8

9

10

11

12

a

Subset-Sum Problem

Find a subset of a given set S={s1,s2,…,sn} of n positive integers whose sum is equal to a given positive integer d

For example: S={3,5,6,7} and d=15 solutions {3,5,7}0

0

05

11 5

3

38

3

with 3

with 5

with 6

w/o 3

w/o 5

w/o 6 with 6 w/o 6

w/o 5 with 5

X X X X

X

14+7>15 3+7<15 11+7>14 5+7<15

0+13<15with 6

X9+7>15

14 98

8

w/o 7

w/o 6

X8<15

solution

with 7

15

Branch and BoundAn enhancement of backtracking. Applicable to optimization problems

Uses a lower bound for the value of the objective function for each node (partial solution) so as to:

• guide the search through state-space• rule out certain branches as “unpromising”

 large subsets of fruitless candidates are discarded en masse, by using upper and lower estimated bounds of the quantity being optimized.

The key idea of the BB algorithm is: if the lower bound for some tree node (set of candidates) A is greater than the upper bound for some other node B, then A may be safely discarded from the search. This step is called pruning, and is usually implemented by maintaining a global variable m (shared among all nodes of the tree) that records the minimum upper bound seen among all subregions examined so far. Any node whose lower bound is greater than m can be discarded.

Select one element in each row of the cost matrix C so that: • no two selected elements are in the same column; and • the sum is minimized

For example:Job 1 Job 2 Job 3 Job 4

Person a 9 2 7 8Person b 6 4 3 7Person c 5 8 1 8Person d 7 6 9 4

Lower bound: Any solution to this problem will have total cost of at least:

Example: The assignment problem

Assignment problem: lower bounds

State-space levels 0, 1, 2

Complete state-space

Traveling salesman example:

Approximation algorithm

approximation algorithms are algorithms used to find approximate solutions to optimization problems

Approximation algorithms are often associated with NP-hardproblems

Unlike heuristics, which usually only find reasonably good solutions reasonably fast, Approximation algorithm wants provable solution quality and provable run time bounds. Ideally, the approximation is optimal up to a small constant factor (for instance within 5% of the optimal solution)