+ All Categories
Home > Documents > Design and Analysis of Algorithms – Chapter 31 Brute Force * Dr. Ying Lu [email protected] RAIK 283:...

Design and Analysis of Algorithms – Chapter 31 Brute Force * Dr. Ying Lu [email protected] RAIK 283:...

Date post: 26-Dec-2015
Category:
Upload: brianne-norris
View: 222 times
Download: 0 times
Share this document with a friend
59
Design and Analysis of Algorithms – Chapter 3 1 Brute Force * Dr. Ying Lu [email protected] RAIK 283: Data Structures & RAIK 283: Data Structures & Algorithms Algorithms *slides referred to http://www.aw-bc.com/info/levitin http://www.aw-bc.com/info/levitin September 20, 2012
Transcript

Design and Analysis of Algorithms – Chapter 3 1

Brute Force *

Dr. Ying [email protected]

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms

*slides referred to http://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin

September 20, 2012

Design and Analysis of Algorithms – Chapter 3 2

Brute forceBrute force

A straightforward approach usually based on problem A straightforward approach usually based on problem statement and definitionsstatement and definitions

Examples:Examples:

1.1. Selection SortSelection Sort

2.2. Graph TraversalGraph Traversal

3.3. Simple Computational Tasks Simple Computational Tasks

4.4. Exhaustive SearchExhaustive Search

Design and Analysis of Algorithms – Chapter 4 3

Graph traversalGraph traversal

Many problems in A.I. and operations research require the Many problems in A.I. and operations research require the systematic processing of vertices and edges of graphssystematic processing of vertices and edges of graphs

Graph traversal algorithms:Graph traversal algorithms:• Depth-first searchDepth-first search

• Breadth-first searchBreadth-first search

First, graph representations!First, graph representations!

Design and Analysis of Algorithms – Chapter 4 4

Graph representations using data Graph representations using data structuresstructures

Adjacency Matrix RepresentationAdjacency Matrix Representation• Let G = (Let G = (VV, , EE), ), nn = | = |VV|, |, mm = | = |EE|, |, VV = { = {vv1, 1, vv2, …, 2, …, vnvn))

• GG can be represented by an can be represented by an nn nn matrix C matrix C

Design and Analysis of Algorithms – Chapter 4 5

Adjacency list representationAdjacency list representation

Design and Analysis of Algorithms – Chapter 4 6

Traversing graphsTraversing graphs

Depth-First Search (DFS) and Breadth-First Depth-First Search (DFS) and Breadth-First Search (BFS)Search (BFS)• Two elementary traversal strategies Two elementary traversal strategies

• Both provide efficient ways to “visit” each vertex and Both provide efficient ways to “visit” each vertex and edge of a graphedge of a graph

• Both work on directed and undirected graphsBoth work on directed and undirected graphs

• They differ in the order of “visiting” verticesThey differ in the order of “visiting” vertices

Design and Analysis of Algorithms – Chapter 4 7

Depth-first searchDepth-first search

Explore graph always moving away from last visited Explore graph always moving away from last visited vertexvertex

Pseudocode for Depth-first-search of graph G=(V,E)Pseudocode for Depth-first-search of graph G=(V,E)

dfs(dfs(vv))count := count + 1count := count + 1mark mark vv with count with countfor each vertex for each vertex ww adjacent to adjacent to

vv dodoif if ww is marked with 0 is marked with 0

dfs(dfs(ww))

DFS(G)DFS(G)count :=0count :=0mark each vertex with 0 mark each vertex with 0

(unvisited)(unvisited)for each vertex vfor each vertex v V doV do

if v is marked with 0 if v is marked with 0 dfs(v)dfs(v)

Design and Analysis of Algorithms – Chapter 4 8

Example – undirected graphExample – undirected graph

Depth-first traversal:Depth-first traversal:

a b

e f

c d

g h

dfs(dfs(vv))count := count + 1count := count + 1mark mark vv with count with countfor each vertex for each vertex ww adjacent to adjacent to

vv dodoif if ww is marked with 0 is marked with 0

dfs(dfs(ww))

Design and Analysis of Algorithms – Chapter 4 9

QuestionQuestion

How to rewrite the procedure dfs(v), using a stack to How to rewrite the procedure dfs(v), using a stack to eliminate recursioneliminate recursion

dfs(dfs(vv))count := count + 1count := count + 1mark mark vv with count with countfor each vertex for each vertex ww adjacent to adjacent to vv dodo

if if ww is marked with 0 is marked with 0 dfs(dfs(ww))

Design and Analysis of Algorithms – Chapter 4 10

Non-recursive version of DFS Non-recursive version of DFS algorithmalgorithm

Algorithm dfs(v)s.createStack();s.push(v);count := count + 1mark v with countwhile (!s.isEmpty()) {

let x be the node on the top of the stack s;if (no unvisited nodes are adjacent to x)

s.pop(); // backtrackelse {

select an unvisited node u adjacent to x;s.push(u);count := count + 1mark u with count

}}

Design and Analysis of Algorithms – Chapter 4 11

Time efficiency analysisTime efficiency analysis

DFS can be implemented with graphs represented as:DFS can be implemented with graphs represented as:• Adjacency matrices:Adjacency matrices:

dfs(dfs(vv))count := count + 1count := count + 1mark mark vv with count with countfor each vertex for each vertex ww adjacent to adjacent to

vv dodoif if ww is marked with 0 is marked with 0

dfs(dfs(ww))

DFS(G)DFS(G)count :=0count :=0mark each vertex with 0 mark each vertex with 0

(unvisited)(unvisited)for each vertex vfor each vertex v V doV do

if v is marked with 0 if v is marked with 0 dfs(v)dfs(v)

Design and Analysis of Algorithms – Chapter 4 12

Time efficiency analysisTime efficiency analysis

DFS can be implemented with graphs represented as:DFS can be implemented with graphs represented as:• Adjacency linked lists:Adjacency linked lists:

dfs(dfs(vv))count := count + 1count := count + 1mark mark vv with count with countfor each vertex for each vertex ww adjacent to adjacent to

vv dodoif if ww is marked with 0 is marked with 0

dfs(dfs(ww))

DFS(G)DFS(G)count :=0count :=0mark each vertex with 0 mark each vertex with 0

(unvisited)(unvisited)for each vertex vfor each vertex v V doV do

if v is marked with 0 if v is marked with 0 dfs(v)dfs(v)

Design and Analysis of Algorithms – Chapter 4 13

Time efficiency analysisTime efficiency analysis

DFS can be implemented with graphs represented as:DFS can be implemented with graphs represented as:• Adjacency matrices: Adjacency matrices: ΘΘ((VV22))

• Adjacency linked lists: Adjacency linked lists: ΘΘ((VV+E) +E)

dfs(dfs(vv))count := count + 1count := count + 1mark mark vv with count with countfor each vertex for each vertex ww adjacent to adjacent to

vv dodoif if ww is marked with 0 is marked with 0

dfs(dfs(ww))

DFS(G)DFS(G)count :=0count :=0mark each vertex with 0 mark each vertex with 0

(unvisited)(unvisited)for each vertex vfor each vertex v V doV do

if v is marked with 0 if v is marked with 0 dfs(v)dfs(v)

Design and Analysis of Algorithms – Chapter 4 14

Application: checking graph Application: checking graph connectivity and finding connected connectivity and finding connected componentscomponents

dfs(dfs(vv))count := count + 1count := count + 1mark mark vv with count with countfor each vertex for each vertex ww adjacent to adjacent to

vv dodoif if ww is marked with 0 is marked with 0

dfs(dfs(ww))

DFS(G)DFS(G)count :=0count :=0mark each vertex with 0 mark each vertex with 0

(unvisited)(unvisited)for each vertex vfor each vertex v V doV do

if v is marked with 0 if v is marked with 0 dfs(v)dfs(v)

Design and Analysis of Algorithms – Chapter 4 15

Application: checking acyclicityApplication: checking acyclicity

dfs(dfs(vv))count := count + 1count := count + 1mark mark vv with count with countfor each vertex for each vertex ww adjacent to adjacent to

vv dodoif if ww is marked with 0 is marked with 0

dfs(dfs(ww))

DFS(G)DFS(G)count :=0count :=0mark each vertex with 0 mark each vertex with 0

(unvisited)(unvisited)for each vertex vfor each vertex v V doV do

if v is marked with 0 if v is marked with 0 dfs(v)dfs(v)

Design and Analysis of Algorithms – Chapter 4 16

Breadth-first searchBreadth-first search

Explore graph moving across to all the neighbors of last Explore graph moving across to all the neighbors of last visited vertexvisited vertex

Similar to level-by-level tree traversals Similar to level-by-level tree traversals

Applications: same as DFS, but can also find paths from a Applications: same as DFS, but can also find paths from a vertex to all other vertices with the smallest number of vertex to all other vertices with the smallest number of edgesedges

Design and Analysis of Algorithms – Chapter 4 17

Example – undirected graphExample – undirected graph

Breadth-first traversal:Breadth-first traversal:

a b

e f

c d

g h

Design and Analysis of Algorithms – Chapter 4 18

Example – undirected graphExample – undirected graph

Depth-first search could be implemented on a stackDepth-first search could be implemented on a stack How about breadth-first searchHow about breadth-first search

a b

e f

c d

g h

Design and Analysis of Algorithms – Chapter 4 19

Breadth-first search algorithmBreadth-first search algorithm

bfs(bfs(vv))

count := count + 1count := count + 1

mark mark vv with count with count

initialize queue with initialize queue with vv

while queue is not empty dowhile queue is not empty do

aa := front of queue := front of queue

for each vertex for each vertex ww adjacent to adjacent to aa do do

if if ww is marked with 0 is marked with 0

count := count + 1count := count + 1

mark mark ww with count with count

add add ww to the end of the queue to the end of the queue

remove remove a a from the front of the queuefrom the front of the queue

BFS(G)BFS(G)count :=0count :=0mark each vertex with 0mark each vertex with 0

for each vertex vfor each vertex v V V dodo

if v is marked if v is marked with 0 with 0

bfs(v)bfs(v)

Design and Analysis of Algorithms – Chapter 4 20

Breadth-first search: NotesBreadth-first search: Notes

BFS has same efficiency as DFS and can be BFS has same efficiency as DFS and can be implemented with graphs represented as:implemented with graphs represented as:• Adjacency matrices: Adjacency matrices: ΘΘ((VV22))• Adjacency linked lists: Adjacency linked lists: ΘΘ((VV+E)+E)

In-class exerciseIn-class exercise

Exercise 3.2.3 Gadget testingExercise 3.2.3 Gadget testing• A firm wants to determine the highest floor of A firm wants to determine the highest floor of

its n-story headquarters from which a gadget its n-story headquarters from which a gadget can fall with no impact on the gadgetcan fall with no impact on the gadget’’s s functionality. The firm has two identical gadgets functionality. The firm has two identical gadgets to experiment with. Design an algorithm in the to experiment with. Design an algorithm in the best efficiency class you can to solve this best efficiency class you can to solve this problem. problem.

Design and Analysis of Algorithms – Chapter 3 21

Design and Analysis of Algorithms – Chapter 3 22

Brute force polynomial Brute force polynomial evaluationevaluation

Problem: Find the value of polynomial Problem: Find the value of polynomial pp((xx) = ) = aannxxnn + + aann-1-1xxnn-1 -1 +… ++… + a a11xx1 1 + + aa0 0

at a point at a point xx = = xx00

Design and Analysis of Algorithms – Chapter 3 23

Brute force polynomial Brute force polynomial evaluationevaluation

Problem: Find the value of polynomial Problem: Find the value of polynomial pp((xx) = ) = aannxxnn + + aann-1-1xxnn-1 -1 +… ++… + a a11xx1 1 + + aa0 0

at a point at a point xx = = xx00

Algorithm: Algorithm: x := xx := x00

pp := 0.0 := 0.0for for ii := := nn down to 0 do down to 0 do power := 1power := 1 for for jj := 1 to := 1 to ii do do power := power *power := power * x x pp := := p p + + aa[[ii] * power ] * power return return pp

Efficiency:Efficiency:

Design and Analysis of Algorithms – Chapter 3 24

Brute force polynomial Brute force polynomial evaluationevaluation

Problem: Find the value of polynomial Problem: Find the value of polynomial pp((xx) = ) = aannxxnn + + aann-1-1xxnn-1 -1 +… ++… + a a11xx1 1 + + aa0 0

at a point at a point xx = = xx00

Algorithm: Algorithm: x := xx := x00

pp := 0.0 := 0.0for for ii := := nn down to 0 do down to 0 do power := 1power := 1 for for jj := 1 to := 1 to ii do do power := power *power := power * x x pp := := p p + + aa[[ii] * power ] * power return return pp

Efficiency: Efficiency: (n(n22))

Design and Analysis of Algorithms – Chapter 3 25

Brute force polynomial Brute force polynomial evaluationevaluation

Problem: Find the value of polynomial Problem: Find the value of polynomial pp((xx) = ) = aannxxnn + + aann-1-1xxnn-1 -1 +… ++… + a a11xx1 1 + + aa0 0

at a point at a point xx = = xx00

Algorithm: Algorithm: x := xx := x00

pp := 0.0 := 0.0for for ii := := nn down to 0 do down to 0 do power := 1power := 1 for for jj := 1 to := 1 to ii do do power := power *power := power * x x pp := := p p + + aa[[ii] * power ] * power return return pp

Efficiency: Efficiency: (n(n22)) Can we design a linear algorithm for this problemCan we design a linear algorithm for this problem

Design and Analysis of Algorithms – Chapter 3 26

Polynomial evaluation: Polynomial evaluation: improvementimprovement

We can do better by evaluating from right to left: We can do better by evaluating from right to left: pp((xx) = ) = aannxxnn + + aann-1-1xxnn-1 -1 +… +…

++ a a11xx1 1 + + aa0 0 at a point at a point xx = = xx00

Algorithm: Algorithm:

x := xx := x00

pp := := aa[0][0]power := 1power := 1for for ii := 1 to := 1 to nn do do

power := power *power := power * x x pp := := p p + + aa[[ii] * power ] * power

return return pp

Efficiency: Efficiency: (n)(n)

Design and Analysis of Algorithms – Chapter 3 27

Polynomial evaluation: Polynomial evaluation: improvementimprovement

We can do better by evaluating from right to left: We can do better by evaluating from right to left: pp((xx) = ) = aannxxnn + + aann-1-1xxnn-1 -1 +… +…

++ a a11xx1 1 + + aa0 0 at a point at a point xx = = xx00

Algorithm: Algorithm:

x := xx := x00

pp := := aa[0][0]power := 1power := 1for for ii := 1 to := 1 to nn do do

power := power *power := power * x x pp := := p p + + aa[[ii] * power ] * power

return return pp Efficiency: Efficiency: (n)(n) Can we design a better than linear algorithm for this Can we design a better than linear algorithm for this

problemproblem

Design and Analysis of Algorithms – Chapter 3 28

Brute force closest-pair Brute force closest-pair algorithmalgorithm

Closest pairClosest pair• ProblemProblem: find the closest pair among : find the closest pair among n n points in points in kk-dimensional -dimensional

spacespace

Design and Analysis of Algorithms – Chapter 3 29

Brute force closest-pair Brute force closest-pair algorithmalgorithm

Closest pairClosest pair

• ProblemProblem: find the closest pair among : find the closest pair among n n points in points in kk-dimensional -dimensional spacespace

• AlgorithmAlgorithm: Compute distance between each pair of points and : Compute distance between each pair of points and identify the pair resulting in the shortest distanceidentify the pair resulting in the shortest distance

• What is (or should be) the basic operation of the algorithm?What is (or should be) the basic operation of the algorithm?

Design and Analysis of Algorithms – Chapter 3 30

Brute force closest-pair Brute force closest-pair algorithmalgorithm

Closest pairClosest pair• ProblemProblem: find the closest pair among : find the closest pair among n n points in points in kk-dimensional -dimensional

spacespace• AlgorithmAlgorithm: Compute distance between each pair of points : Compute distance between each pair of points and and

identify the pair resulting in the shortest distanceidentify the pair resulting in the shortest distance

• Basic operation (squaring):Basic operation (squaring):

k

iii

k

iii

yxd

yxd

1

22

1

2

)(

)(

Design and Analysis of Algorithms – Chapter 3 31

Brute force closest-pair Brute force closest-pair algorithmalgorithm

Closest pairClosest pair• ProblemProblem: : find the closest pair among find the closest pair among n n points in points in kk-dimensional space-dimensional space

• AlgorithmAlgorithm: Compute distance between each pair of points : Compute distance between each pair of points and and identify the pair resulting in the shortest distanceidentify the pair resulting in the shortest distance

• Basic operation:Basic operation:

• How many different pairs of points?How many different pairs of points?

k

iii

k

iii

yxd

yxd

1

22

1

2

)(

)(

Principle of counting: Product Principle of counting: Product RuleRule

The Product RuleThe Product Rule• Suppose that a procedure can be broken down into a sequence of Suppose that a procedure can be broken down into a sequence of

two tasks. If there are ntwo tasks. If there are n11 ways to do the first task and for each of ways to do the first task and for each of

these ways of doing the first task, there are nthese ways of doing the first task, there are n22 ways to do the second ways to do the second

task, then there are ntask, then there are n11nn22 ways to do the procedure. ways to do the procedure.

Design and Analysis of Algorithms – Chapter 3 32

Principle of counting: Product Principle of counting: Product RuleRule

The Product RuleThe Product Rule• Suppose that a procedure can be broken down into a sequence of Suppose that a procedure can be broken down into a sequence of

two tasks. If there are ntwo tasks. If there are n11 ways to do the first task and for each of ways to do the first task and for each of

these ways of doing the first task, there are nthese ways of doing the first task, there are n22 ways to do the second ways to do the second

task, then there are ntask, then there are n11nn22 ways to do the procedure. ways to do the procedure.

CalculationCalculation• Applying the product rule, there are n*(n-1) pairs among n points Applying the product rule, there are n*(n-1) pairs among n points

• Considering (a, b) and (b, a) as the same, we divide the above Considering (a, b) and (b, a) as the same, we divide the above number by 2 and thus, there are n*(n-1)/2 different pairs of pointsnumber by 2 and thus, there are n*(n-1)/2 different pairs of points

Design and Analysis of Algorithms – Chapter 3 33

Design and Analysis of Algorithms – Chapter 3 34

Brute force closest-pair Brute force closest-pair algorithmalgorithm

Closest pairClosest pair• ProblemProblem: : find the closest pair among find the closest pair among n n points in points in kk-dimensional space-dimensional space

• AlgorithmAlgorithm: Compute distance between each pair of points : Compute distance between each pair of points and and identify the pair resulting in the shortest distanceidentify the pair resulting in the shortest distance

• Basic operation:Basic operation:

• EfficiencyEfficiency: :

k

iii

k

iii

yxd

yxd

1

22

1

2

)(

)(

(n(n22))

Design and Analysis of Algorithms – Chapter 3 35

In-class exerciseIn-class exercise

Can you design a faster algorithm than the one based on Can you design a faster algorithm than the one based on the brute-force strategy to solve the closest-pair problem the brute-force strategy to solve the closest-pair problem for n points xfor n points x11, x, x22, … , x, … , xnn on the real line? on the real line?

What is the time efficiency of your algorithm? What is the time efficiency of your algorithm?

Design and Analysis of Algorithms – Chapter 3 36

In-class exerciseIn-class exercise

Can you design a faster algorithm than the one based on Can you design a faster algorithm than the one based on the brute-force strategy to solve the closest-pair problem the brute-force strategy to solve the closest-pair problem for n points xfor n points x11, x, x22, … , x, … , xnn on the real line? on the real line?

Algorithm: Algorithm: • Step1: Sort the numbers in ascending order, O(nlogn)Step1: Sort the numbers in ascending order, O(nlogn)

• Step 2: Compute the differences between adjacent numbers in Step 2: Compute the differences between adjacent numbers in the sorted list, the sorted list, (n)(n)

• Step 3: Find the smallest such difference, Step 3: Find the smallest such difference, (n)(n)

Running time of the entire algorithm: Running time of the entire algorithm:

O(nlogn) + O(nlogn) + (n) + (n) + (n) = O(nlogn)(n) = O(nlogn)

Design and Analysis of Algorithms – Chapter 3 37

Convex hull problem Convex hull problem

Convex hullConvex hull• Problem: Problem:

Find Find smallest convex polygonsmallest convex polygon enclosing enclosing nn points on the plane points on the plane

• Convex:Convex:– A A geometric figure with no with no

indentations. indentations. – Formally, a geometric figure is Formally, a geometric figure is

convex if every convex if every line segment connecting connecting interior points is entirely is entirely contained within the figure's interior.contained within the figure's interior.

convex

Non-convex

Example: Convex HullExample: Convex Hull

p9

p2

p8

p6

p5

p4

p11

p1

p7p3

p10

Input: p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11

Output: p2,p9,p11,p4,p5,p6,p8,p2

Convex hull: application Convex hull: application domain*domain*

Computer visualization, ray tracingComputer visualization, ray tracing• (e.g. video games, replacement of bounding boxes)(e.g. video games, replacement of bounding boxes)

Path findingPath finding• (e.g. embedded AI of Mars mission rovers)(e.g. embedded AI of Mars mission rovers)

Geographical Information Systems (GIS)Geographical Information Systems (GIS)• (e.g. computing accessibility maps)(e.g. computing accessibility maps)

Visual pattern matchingVisual pattern matching• (e.g. detecting car license plates)(e.g. detecting car license plates)

Verification methodsVerification methods• (e.g. bounding of Number Decision Diagrams)(e.g. bounding of Number Decision Diagrams)

GeometryGeometry• (e.g. diameter computation)(e.g. diameter computation)

*slide refer to *slide refer to http://www.montefiore.ulg.ac.be/~briquet/algo3-chull-20070206.pdfhttp://www.montefiore.ulg.ac.be/~briquet/algo3-chull-20070206.pdf

Design and Analysis of Algorithms – Chapter 3 39

Design and Analysis of Algorithms – Chapter 3 40

Convex hull brute force Convex hull brute force algorithmalgorithm

Extreme pointsExtreme points of the of the convex polygonconvex polygon• Consider all the points in the Consider all the points in the

polygon as a set. An polygon as a set. An extreme extreme pointpoint is a point of the set that is a point of the set that is not a middle point of any is not a middle point of any line segment with end points line segment with end points in the set. in the set.

p9

p2

p8

p6

p5

p4

p11

p1

p7p3

p10

Design and Analysis of Algorithms – Chapter 3 41

Convex hull brute force Convex hull brute force algorithmalgorithm

Extreme pointsExtreme points of the of the convex polygonconvex polygon• Consider all the points in the Consider all the points in the

polygon as a set. An polygon as a set. An extreme extreme pointpoint is a point of the set that is a point of the set that is not a middle point of any is not a middle point of any line segment with end points line segment with end points in the set. in the set.

p9

p2

p8

p6

p5

p4

p11

p1

p7p3

p10

Which pairs of extreme points need to be connected Which pairs of extreme points need to be connected to form the boundary of the convex hull?to form the boundary of the convex hull?

Design and Analysis of Algorithms – Chapter 3 42

Convex hull brute force Convex hull brute force algorithmalgorithm

A line segment connecting two points PA line segment connecting two points Pii and P and Pj j of a of a

set of n points is a part of its convex hull’s set of n points is a part of its convex hull’s boundary if and only if all the other points of the boundary if and only if all the other points of the set lies on the same side of the straight line through set lies on the same side of the straight line through these two points.these two points.

Design and Analysis of Algorithms – Chapter 3 43

Convex hull brute force Convex hull brute force algorithmalgorithm

The straight line through two points (x1, y1), (x2, The straight line through two points (x1, y1), (x2, y2) in the coordinate plane can be defined by the y2) in the coordinate plane can be defined by the following equationfollowing equation• ax + by = cax + by = cwhere a = y2 – y1, b = x1 – x2, c = x1y2 - y1x2 where a = y2 – y1, b = x1 – x2, c = x1y2 - y1x2

Such a line divides the plane into two half-planes: Such a line divides the plane into two half-planes: for all the points in one of them: ax + by > c, while for all the points in one of them: ax + by > c, while for all the points in the other, ax + by < c. for all the points in the other, ax + by < c.

Design and Analysis of Algorithms – Chapter 3 44

Convex hull brute force Convex hull brute force algorithmalgorithm

AlgorithmAlgorithm: For each pair of points : For each pair of points pp11 and and pp22

determine whether all other points lie to the same determine whether all other points lie to the same side of the straight line through side of the straight line through pp11 and and pp2,2, i.e. whether i.e. whether

ax+by-c all have the same signax+by-c all have the same sign

EfficiencyEfficiency: :

Design and Analysis of Algorithms – Chapter 3 45

Convex hull brute force Convex hull brute force algorithmalgorithm

AlgorithmAlgorithm: For each pair of points : For each pair of points pp11 and and pp22

determine whether all other points lie to the same determine whether all other points lie to the same side of the straight line through side of the straight line through pp11 and and pp2,2, i.e. i.e.

whether ax+by-c all have the same signwhether ax+by-c all have the same sign

EfficiencyEfficiency: : (n(n33))

Design and Analysis of Algorithms – Chapter 3 46

Exhaustive search: definitionExhaustive search: definition

A brute force solution to a problem involving A brute force solution to a problem involving search for an element with a special property, search for an element with a special property, usually among combinatorial objects such as a usually among combinatorial objects such as a permutations, combinations, or subsets of a set.permutations, combinations, or subsets of a set.

Design and Analysis of Algorithms – Chapter 3 47

Exhaustive search: methodExhaustive search: method

Construct a way of listing all potential solutions to Construct a way of listing all potential solutions to the problem in a systematic mannerthe problem in a systematic manner• all solutions are eventually listedall solutions are eventually listed

• no solution is repeatedno solution is repeated Evaluate solutions one by one, perhaps Evaluate solutions one by one, perhaps

disqualifying infeasible ones and keeping track of disqualifying infeasible ones and keeping track of the best one found so farthe best one found so far

When search ends, announce the winnerWhen search ends, announce the winner

Design and Analysis of Algorithms – Chapter 3 48

Example 1: Traveling salesman Example 1: Traveling salesman problemproblem

Given Given nn cities with known distances between each cities with known distances between each pair, find the shortest tour that passes through all pair, find the shortest tour that passes through all the cities exactly once before returning to the the cities exactly once before returning to the starting city.starting city.

Alternatively: Find shortest Alternatively: Find shortest Hamiltonian circuitHamiltonian circuit in in a weighted connected graph. a weighted connected graph.

Example:Example:a b

c d

8

2

7

5 34

Design and Analysis of Algorithms – Chapter 3 49

Traveling salesman by exhaustive Traveling salesman by exhaustive searchsearch

Tour Cost . Tour Cost .

aa→b→c→d→a 2+3+7+5 = 17→b→c→d→a 2+3+7+5 = 17 aa→b→d→c→a 2+4+7+8 = 21→b→d→c→a 2+4+7+8 = 21 aa→c→b→d→a 8+3+4+5 = 20→c→b→d→a 8+3+4+5 = 20 aa→c→d→b→a 8+7+4+2 = 21→c→d→b→a 8+7+4+2 = 21 aa→d→b→c→a 5+4+3+8 = 20→d→b→c→a 5+4+3+8 = 20 aa→d→c→b→a 5+7+3+2 = 17→d→c→b→a 5+7+3+2 = 17

Efficiency:Efficiency:

Design and Analysis of Algorithms – Chapter 3 50

Traveling salesman by exhaustive Traveling salesman by exhaustive searchsearch

Tour Cost . Tour Cost .

aa→b→c→d→a 2+3+7+5 = 17→b→c→d→a 2+3+7+5 = 17 aa→b→d→c→a 2+4+7+8 = 21→b→d→c→a 2+4+7+8 = 21 aa→c→b→d→a 8+3+4+5 = 20→c→b→d→a 8+3+4+5 = 20 aa→c→d→b→a 8+7+4+2 = 21→c→d→b→a 8+7+4+2 = 21 aa→d→b→c→a 5+4+3+8 = 20→d→b→c→a 5+4+3+8 = 20 aa→d→c→b→a 5+7+3+2 = 17→d→c→b→a 5+7+3+2 = 17

Efficiency: (n-1)!/2Efficiency: (n-1)!/2

Design and Analysis of Algorithms – Chapter 3 51

Given a knapsack with maximum capacity Given a knapsack with maximum capacity WW, and , and a set a set SS consisting of consisting of nn items items

Each item Each item ii has some weight has some weight wwii and benefit value and benefit value vvii

ProblemProblem: How to pack the knapsack to achieve : How to pack the knapsack to achieve maximum total value of packed items?maximum total value of packed items?

0-1 Knapsack problem0-1 Knapsack problem

Design and Analysis of Algorithms – Chapter 3 52

W = 20

wi vi

109

85

54

43

32

Weight Benefit value

This is a knapsackMax weight: W = 20

Items

0-1 Knapsack problem: a 0-1 Knapsack problem: a picturepicture

Design and Analysis of Algorithms – Chapter 3 53

Problem, in other words, is to findProblem, in other words, is to find

Ti

iTi

i Wwv subject to max

0-1 Knapsack problem0-1 Knapsack problem

The problem is called a “The problem is called a “0-10-1” problem, ” problem, because each item must be entirely accepted or because each item must be entirely accepted or rejected.rejected.

In the “In the “Fractional Knapsack ProblemFractional Knapsack Problem,” we can ,” we can take fractions of items. take fractions of items.

Design and Analysis of Algorithms – Chapter 3 54

Let’s first solve this problem with a Let’s first solve this problem with a straightforward algorithmstraightforward algorithm

We go through all combinations (subsets) and We go through all combinations (subsets) and find the one with maximum value and with find the one with maximum value and with total weight less or equal to total weight less or equal to WW

0-1 Knapsack problem: brute-force 0-1 Knapsack problem: brute-force approachapproach

Design and Analysis of Algorithms – Chapter 3 55

Example 2: Knapsack ProblemExample 2: Knapsack Problem

Given Given nn items: items:• weights: weights: ww1 1 ww22 … w … wnn

• values: values: vv1 1 vv22 … v … vnn

• a knapsack of capacity a knapsack of capacity W W

Find the most valuable subset of the items that fit into the Find the most valuable subset of the items that fit into the knapsackknapsack

Example:Example:item weight value Knapsack capacity W=item weight value Knapsack capacity W=161611 2 $202 $2022 5 $305 $3033 10 $5010 $5044 5 $105 $10

Design and Analysis of Algorithms – Chapter 3 56

Knapsack by exhaustive searchKnapsack by exhaustive search

Subset Total weight Total valueSubset Total weight Total value 00 $0 $0

{1} 2 $20{1} 2 $20 {2} 5 $30{2} 5 $30 {3} 10 $50{3} 10 $50 {4} 5 $10{4} 5 $10 {1,2} 7 $50{1,2} 7 $50 {1,3} 12 $70{1,3} 12 $70 {1,4} 7 $30{1,4} 7 $30 {2,3} 15 $80{2,3} 15 $80 {2,4} 10 $40{2,4} 10 $40 {3,4} 15 $60{3,4} 15 $60 {1,2,3} 17 not feasible{1,2,3} 17 not feasible {1,2,4} 12 $60{1,2,4} 12 $60 {1,3,4} 17 not feasible{1,3,4} 17 not feasible {2,3,4} 20 not feasible{2,3,4} 20 not feasible{1,2,3,4} 22 not feasible{1,2,3,4} 22 not feasible

Most valuable subset?Most valuable subset?

Efficiency:Efficiency:

Design and Analysis of Algorithms – Chapter 3 57

Algorithm:Algorithm:• We go through all combinations and find the one We go through all combinations and find the one

with maximum value and with total weight less or with maximum value and with total weight less or equal to equal to WW

0-1 Knapsack problem: brute-force 0-1 Knapsack problem: brute-force approachapproach

Efficiency:Efficiency:• Since there are Since there are nn items, there are items, there are 22nn possible possible

combinations of items.combinations of items.

• Thus, the running time will be Thus, the running time will be O(2O(2nn))

Design and Analysis of Algorithms – Chapter 3 58

Brute force strengths and Brute force strengths and weaknessesweaknesses

Strengths:Strengths:• wide applicabilitywide applicability

• simplicitysimplicity

• yields reasonable algorithms for some important problemsyields reasonable algorithms for some important problems– sorting; matrix multiplication; closest-pair; convex-hullsorting; matrix multiplication; closest-pair; convex-hull

• yields standard algorithms for simple computational tasks yields standard algorithms for simple computational tasks and graph traversal problemsand graph traversal problems

Design and Analysis of Algorithms – Chapter 3 59

Brute force strengths and Brute force strengths and weaknessesweaknesses

Weaknesses:Weaknesses:• rarely yields efficient algorithmsrarely yields efficient algorithms

• some brute force algorithms unacceptably slow some brute force algorithms unacceptably slow – e.g., the recursive algorithm for computing Fibonacci numberse.g., the recursive algorithm for computing Fibonacci numbers

• not as constructive/creative as some other design techniquesnot as constructive/creative as some other design techniques


Recommended