Transform and Conquer

Design and Analysis of Algorithms - Chapter 6 1

Transform and ConquerTransform and ConquerSolve problem by transforming into:Solve problem by transforming into: a more convenient instance of the same problem (a more convenient instance of the same problem (instance implificationinstance implification))

• presortingpresorting• Gaussian eliminationGaussian elimination

a different representation of the same instance (a different representation of the same instance (representation changerepresentation change))• balanced search treesbalanced search trees• heaps and heapsortheaps and heapsort• polynomial evaluation by Horner’s rulepolynomial evaluation by Horner’s rule• Fast Fourier TransformFast Fourier Transform

a different problem altogether (a different problem altogether (problem reductionproblem reduction))• reductions to graph problemsreductions to graph problems• linear programming linear programming


Instance simplification - Instance simplification - PresortingPresortingSolve instance of problem by transforming into another Solve instance of problem by transforming into another

simpler/easier instance of the same problemsimpler/easier instance of the same problem

Presorting:Presorting:Many problems involving lists are easier when list is sorted.Many problems involving lists are easier when list is sorted. searching searching computing the median (selection problem)computing the median (selection problem) computing the modecomputing the mode finding repeated elementsfinding repeated elements


Selection ProblemSelection ProblemFind the kFind the kth th smallest element in A[1],…A[smallest element in A[1],…A[nn]. Special cases:]. Special cases:

• minimumminimum: k : k = 1= 1• maximummaximum: k : k = = nn• medianmedian: k : k = = nn/2/2

Presorting-based algorithm Presorting-based algorithm • sort listsort list• return A[return A[kk]]

Partition-based algorithm (Variable decrease & conquer):Partition-based algorithm (Variable decrease & conquer):• pivot/split at A[pivot/split at A[ss] using partitioning algorithm from quicksort] using partitioning algorithm from quicksort• if if ss==k k return A[ return A[ss] ] • else if else if s<k s<k repeat with sublist A[repeat with sublist A[ss+1],…A[+1],…A[nn]. ]. • else if s>k else if s>k repeat with sublist A[1],…A[repeat with sublist A[1],…A[ss-1]. -1].


Notes on Selection ProblemNotes on Selection Problem Presorting-based algorithm: Presorting-based algorithm: ΩΩ((nnlglgnn) + ) + ΘΘ(1) = (1) = ΩΩ((nnlglgnn) )

Partition-based algorithm (Variable decrease & conquer):Partition-based algorithm (Variable decrease & conquer):• worst case: T(worst case: T(nn) =T() =T(nn-1) + (-1) + (nn+1) -> +1) -> ΘΘ((nn22) ) • best case: best case: ΘΘ((nn) ) • average case: T(average case: T(nn) =T() =T(nn/2) + (/2) + (nn+1) -> +1) -> ΘΘ((nn) ) • Bonus:Bonus: also identifies the also identifies the kk smallest elements (not just the smallest elements (not just the kkthth))

Special cases max, min: better, simpler linear algorithm Special cases max, min: better, simpler linear algorithm (brute force)(brute force)

Conclusion:Conclusion: Presorting does Presorting does notnot help in this case. help in this case.


Finding repeated elementsFinding repeated elements Presorting-based algorithm: Presorting-based algorithm:

• use mergesort (optimal): use mergesort (optimal): ΘΘ((nnlglgnn) ) • scan array to find repeated scan array to find repeated adjacentadjacent elements: elements: ΘΘ((nn) )

Brute force algorithm: Brute force algorithm: ΘΘ((nn22) )

Conclusion:Conclusion: Presorting yields Presorting yields significantsignificant improvement improvement

Similar improvement for modeSimilar improvement for mode

What about searching?What about searching?

ΘΘ((nnlglgnn))


Taxonomy of Searching Taxonomy of Searching AlgorithmsAlgorithms Elementary searching algorithmsElementary searching algorithms

• sequential searchsequential search• binary searchbinary search• binary tree searchbinary tree search

Balanced tree searchingBalanced tree searching• AVL treesAVL trees• red-black treesred-black trees• multiway balanced trees (2-3 trees, 2-3-4 trees, B trees)multiway balanced trees (2-3 trees, 2-3-4 trees, B trees)

HashingHashing• separate chainingseparate chaining• open addressingopen addressing


Balanced trees: AVL treesBalanced trees: AVL trees For every node, difference in height between left and right For every node, difference in height between left and right

subtree is at most 1subtree is at most 1

AVL property is maintained through AVL property is maintained through rotationsrotations, each time , each time the tree becomes unbalancedthe tree becomes unbalanced

lg lg nn ≤≤ hh ≤ 1.4404 lg (≤ 1.4404 lg (n n + 2) - 1.3277 + 2) - 1.3277 average:average: 1.01 lg 1.01 lg n + n + 0.1 for large 0.1 for large nn

Disadvantage: needs extra storage for maintaining node Disadvantage: needs extra storage for maintaining node balancebalance

A similar idea: red-black trees (height of subtrees is A similar idea: red-black trees (height of subtrees is allowed to differ by up to a factor of 2)allowed to differ by up to a factor of 2)


AVL tree rotationsAVL tree rotations Small examples:Small examples:

• 1, 2, 31, 2, 3• 3, 2, 1 3, 2, 1 • 1, 3, 21, 3, 2• 3, 1, 23, 1, 2

Larger example: 4, 5, 7, 2, 1, 3, 6Larger example: 4, 5, 7, 2, 1, 3, 6

See figures 6.4, 6.5 for general cases of rotations; See figures 6.4, 6.5 for general cases of rotations;


General case: single R-rotationGeneral case: single R-rotation


Double LR-rotationDouble LR-rotation


Balance factorBalance factor Algorithm maintains Algorithm maintains balance factorbalance factor for each node. For example: for each node. For example:


HeapsortHeapsortDefinition:Definition:A A heapheap is a binary tree with the following conditions: is a binary tree with the following conditions: it is essentially complete:it is essentially complete:

The key at each node is The key at each node is ≥ keys at its children≥ keys at its children


Definition implies:Definition implies: Given Given n,n, there exists a unique binary tree with there exists a unique binary tree with nn nodes that nodes thatis essentially complete, with is essentially complete, with hh= lg = lg nn

The root has the largest keyThe root has the largest key

The subtree rooted at any node of a heap is also a heapThe subtree rooted at any node of a heap is also a heap


Heapsort Algorithm:Heapsort Algorithm:1.1. Build heapBuild heap

2.2. Remove root –exchange with last (rightmost) leafRemove root –exchange with last (rightmost) leaf

3.3. Fix up heap (excluding last leaf)Fix up heap (excluding last leaf)

Repeat 2, 3 until heap contains just one node.Repeat 2, 3 until heap contains just one node.


Heap constructionHeap construction Insert elements in the order given breadth-first in a binary Insert elements in the order given breadth-first in a binary

treetree

Starting with the last (rightmost) parental node, fix the Starting with the last (rightmost) parental node, fix the heap rooted at it, if it does not satisfy the heap condition:heap rooted at it, if it does not satisfy the heap condition:1.1. exchange it with its largest childexchange it with its largest child2.2. fix the subtree rooted at it (now in the child’s position)fix the subtree rooted at it (now in the child’s position)

Example: 2 3 6 7 5 9Example: 2 3 6 7 5 9


Root deletionRoot deletionThe root of a heap can be deleted and the heap fixed up as The root of a heap can be deleted and the heap fixed up as

follows:follows:

exchange the root with the last leaf exchange the root with the last leaf compare the new root (formerly the leaf) with each of its compare the new root (formerly the leaf) with each of its

children and, if one of them is larger than the root, children and, if one of them is larger than the root, exchange it with the larger of the two. exchange it with the larger of the two.

continue the comparison/exchange with the children of the continue the comparison/exchange with the children of the new root until it reaches a level of the tree where it is larger new root until it reaches a level of the tree where it is larger than both its children than both its children


RepresentationRepresentation Use an array to store breadth-first traversal of heap tree:Use an array to store breadth-first traversal of heap tree: Example:Example:

Left child of node Left child of node jj is at 2 is at 2jj Right child of node Right child of node jj is at 2 is at 2jj+1+1 Parent of node Parent of node jj is at is at j j /2/2 Parental nodes are represented in the first Parental nodes are represented in the first nn /2 locations /2 locations

9

1

5 3

4 2

1 2 3 4 5 6

9 5 3 1 4 2


Bottom-up heap construction Bottom-up heap construction algorithmalgorithm


Analysis of HeapsortAnalysis of HeapsortSee algorithm HeapBottomUp in section 6.4See algorithm HeapBottomUp in section 6.4

Fix heap with “problem” at height Fix heap with “problem” at height j: j: 2 2jj comparisons comparisons

For subtree rooted at level For subtree rooted at level ii it does 2( it does 2(hh--ii) comparisons) comparisons

Total for heap construction phase:Total for heap construction phase:

Σ 2(h-i) 2i = 2 ( n – lg (n + 1)) = Θ(n)i=0

h-1

# nodes at level i


Analysis of Heapsort Analysis of Heapsort (continued)(continued)Recall algorithm:Recall algorithm:

1.1. Build heapBuild heap

2.2. Remove root –exchange with last (rightmost) leafRemove root –exchange with last (rightmost) leaf

3.3. Fix up heap (excluding last leaf)Fix up heap (excluding last leaf)

Repeat 2, 3 until heap contains just one node.Repeat 2, 3 until heap contains just one node.

Θ(n)

Θ(log n)

n – 1 times

Total:Total: Θ(n) + Θ( n log n) = Θ(n log n)

• Note:Note: this is the worst case. Average case also Θ(n log n).


Priority queuesPriority queues A A priority queue priority queue is the ADT of an ordered set with the is the ADT of an ordered set with the

operations:operations:• find element with highest priority find element with highest priority • delete element with highest priority delete element with highest priority • insert element with assigned priority insert element with assigned priority

Heaps are very good for implementing priority queuesHeaps are very good for implementing priority queues


Insertion of a new elementInsertion of a new element Insert element at last position in heap. Insert element at last position in heap. Compare with its parent and if it violates heap conditionCompare with its parent and if it violates heap condition exchange themexchange them Continue comparing the new element with nodes up the Continue comparing the new element with nodes up the

tree until the heap condition is satisfiedtree until the heap condition is satisfied

Example: Example:

Efficiency:Efficiency:


Bottom-up vs. Top-down heap Bottom-up vs. Top-down heap constructionconstruction Top down:Top down: Heaps can be constructed by successively Heaps can be constructed by successively

inserting elements into an (initially) empty heapinserting elements into an (initially) empty heap

Bottom-up:Bottom-up: Put everything in and then fix it Put everything in and then fix it

Which one is better?Which one is better?

Date post:	08-Feb-2016
Category:	Documents
Upload:	shelby
View:	34 times
Download:	0 times

Transform and Conquer

Documents