Date post: | 29-Jan-2016 |
Category: |
Documents |
Upload: | ashley-martin |
View: | 215 times |
Download: | 0 times |
CPSC 311 Section 502Analysis of Algorithm
Fall 2002Department of Computer Science
Texas A&M University
CPSC 311 O. Burchan Bayazit – Fall 02
Announcements
•There will be a quiz on next Tuesday (Sep 17)
•A new homework will be put to the web site today ( due date next Thursday (Sep 19))
•Grades for the previous quiz is available
CPSC 311 O. Burchan Bayazit – Fall 02
Solving Recurrences: Master MethodThe master method provides a 'cookbook' method for solving recurrences of a certain form.
Master Theorem: Let a 1 and b > 1 be constants, let f(n) be a function, and let T(n) be defined on nonnegative integers as:
T(n) = aT(n/b) + f(n)
Then, T(n) can be bounded asymptotically as follows:
1. T(n) = (nlogba) if f(n) = O(nlogba-) for some constant > 0
2. T(n) = (nlogbalogn) if f(n) = (nlogba)
3. T(n) = (f(n)) if f(n) = (nlogba+) for some constant > 0and if af(n/b) cf(n) for some constant c < 1 and all sufficiently large n.
CPSC 311 O. Burchan Bayazit – Fall 02
Case 3 T(n) = (f(n)), if f(n) = (nlogba+) for some constant > 0 and if
af(n/b) cf(n) for some constant c < 1 and all sufficiently large n.
T(n) = aT(n/b) + f(n) T(n) =3T(n/4)+nlgn
a=3,b=4,f(n)=nlgn{PART I: Is f(n) = (nlogba+) for some >0?
nlogba+ = nlog43+=n0.79+
f(n)=nlgn can we find > 0 such that nlgn= (n0.79+ )
If we choose =1-0.79 the lower bound becomes (n )And nlgn has a lower bound n i.e., f(nlgn)= (n )So the first part holds.
CPSC 311 O. Burchan Bayazit – Fall 02
Case 3 T(n) = (f(n)), if f(n) = (nlogba+) for some constant > 0 and if
af(n/b) cf(n) for some constant c < 1 and all sufficiently large n.
T(n) = aT(n/b) + f(n) T(n) =3T(n/4)+nlgn
a=3,b=4,f(n)=nlgn{Part II: is there a constant <1 that satisfies af(n/b) cf(n) ?
af(n/b) = 3f(n/4) So we should find a c such3f(n/4)<=c. f(n) and c>13f(n/4)= 3(n/4)lg(n/4) and f(n)=nlgn (3/4)n lg(n)-(3/4)lg4 <=c.nlgn
If we select c=3/4 this inequality holds so T(n)= (nlgn)
CPSC 311 O. Burchan Bayazit – Fall 02
Sorting Algorithms Ch. 6 - 8
Slightly modified definition of the sorting problem:input: A collection of n data items <a1,a2,...,an> where each data
item has a key drawn from a linearly ordered set (e.g., ints, chars)
output: A permutation (reordering) <a'1,a'2,...,a'n> of the input sequence such that a'1 a'2 ... a'n
• In practice, one usually sorts 'records' according to their key (the non-key data is called satellite data.)
• If the records are large, we may sort an array of pointers.
CPSC 311 O. Burchan Bayazit – Fall 02
Sorting Algorithms• A sorting algorithm is in place if only a constant number of
elements of the input array are ever stored outside the array.
• A sorting algorithm is comparison based if the only operation we can perform on keys is to compare two keys.
CPSC 311 O. Burchan Bayazit – Fall 02
Merge Sort
A Divide-and-Conquer algorithm
•Divide: the n-element array to be sorted into two n/2-element subarrays
•Conquer: Sort the subarrays recursively using merge sort
•Combine: Merge the two sorted subarrays to find sorted array
CPSC 311 O. Burchan Bayazit – Fall 02
Merge Sort
5 2 4 6 1 3 8 7
Unsorted original array
CPSC 311 O. Burchan Bayazit – Fall 02
Merge SortDivide the problem to two subproblems
5 2 4 6 1 3 8 7
Call merge sort on subarrays
5 2 4 6 1 3 8 7
CPSC 311 O. Burchan Bayazit – Fall 02
Merge SortFurther divide subarrays
5 2 4 6 1 3 8 7
5 2 4 6 1 3 8 7
CPSC 311 O. Burchan Bayazit – Fall 02
Merge SortUntil the smallest array is reached
5 2 4 6 1 3 8 7
5 2 4 6 1 3 8 7
CPSC 311 O. Burchan Bayazit – Fall 02
Merge SortMerge results of two subarrays
2 5 4 6 1 3 7 8
5 2 4 6 1 3 8 7
5 2 4 6 1 3 8 7
CPSC 311 O. Burchan Bayazit – Fall 02
Merge SortFurther merge
2 5 4 6 1 3 7 8
5 2 4 6 1 3 8 7
CPSC 311 O. Burchan Bayazit – Fall 02
Merge SortFurther merge
2 4 5 6 1 3 7 8
2 5 4 6 1 3 7 8
5 2 4 6 1 3 8 7
CPSC 311 O. Burchan Bayazit – Fall 02
Merge SortFurther merge
2 4 5 6 1 3 7 8
5 2 4 6 1 3 8 7
CPSC 311 O. Burchan Bayazit – Fall 02
Merge Sort
1 2 3 4 5 6 7 8
Final merge
2 4 5 6 1 3 7 8
5 2 4 6 1 3 8 7
CPSC 311 O. Burchan Bayazit – Fall 02
Merge Sort
1 2 3 4 5 6 7 8
Result
5 2 4 6 1 3 8 7
CPSC 311 O. Burchan Bayazit – Fall 02
Merge Sort - Algorithm
Merge-Sort(A,start,end)
1. if start=end then {
2. return
3. }
4. half = (start+end)/2 5. Merge-Sort (A,start,half)
6. Merge-Sort (A,half+1,end)
7. Merge (A,start,half,end)
CPSC 311 O. Burchan Bayazit – Fall 02
Merging
2 4 5 6 1 3 7 8
CPSC 311 O. Burchan Bayazit – Fall 02
Merging
2 4 5 6 3 7 8
1
CPSC 311 O. Burchan Bayazit – Fall 02
Merging
4 5 6 7 8
1 2
3
CPSC 311 O. Burchan Bayazit – Fall 02
Merging
4 5 6 7 8
1 2 3
CPSC 311 O. Burchan Bayazit – Fall 02
Merging
5 6 7 8
1 2 3 4
CPSC 311 O. Burchan Bayazit – Fall 02
Merging
6 7 8
1 2 3 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Merging
7 8
1 2 3 4 5 6
CPSC 311 O. Burchan Bayazit – Fall 02
Merging
8
1 2 3 4 5 6 7
CPSC 311 O. Burchan Bayazit – Fall 02
Merging
1 2 3 4 5 6 7 8
CPSC 311 O. Burchan Bayazit – Fall 02
Merge AlgorithmMerge(A,start,half,end)
1. n1= half-start+1; n2 end-half;
2. for i= 1 to n1 {
3. Left[i]= A[start+i-1] // copy left subarray
4. }
5. for i= 1 to n2 {
6. R[i] A[half+i] // copy right subarray7. }
8. Left[n1+1] = Right[n2+1] =
9. i= j= 1
10. for k = start to end {
11. if Left[i] Right[j] then {12. A[k]=Left[i]
13. i= i+1 }
14. else { A[k]= Right[j]
15. j= j+1 }
16. }
Running Time = (n)
CPSC 311 O. Burchan Bayazit – Fall 02
Merge Sort - Analysis
Merge-Sort(A,start,end)
1. if start=end then {
2. return
3. }
4. half = (start+end)/2 5. Merge-Sort (A,start,half)
6. Merge-Sort (A,half+1,end)
7. Merge (A,start,half,end)
(1)
T (n/2)T (n/2)
(n)
T (n)=2T(n/2)+ (n) (nlgn)using master method
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort
heap concept• complete binary tree, except may be missing some rightmost
leaves on the bottom level.• each node contains a key• values in the nodes satisfy the heap property.
CPSC 311 O. Burchan Bayazit – Fall 02
Heap vs. Array
1
2 3
4
8 9
765
10
1 2 3 4 5 6 7 8 9 10
binary tree: an array implementation• root is A[1]• for element A[i]
- left child is in position A[2i]- right child is in position A[2i + 1]- parent is in A[i/2]
Left child=2i
i=2
Right child=2i+1
Parent =i/2
CPSC 311 O. Burchan Bayazit – Fall 02
Max-Heap
Max-heap property: A[Parent(i)] A[i]
In the array representation of a max-heap, the root of the treeis in A[1], and given the index i of a node, Parent(i) LeftChild(i) RightChild(i) return (i/2) return (2i) return (2i + 1)
20 18 15 11 10 9 6 2 4 5 3
1 2 3 4 5 6 7 8 9 10 11 A index
keys
20
18 15
11 10 9 6
2 4 5 3
1
2 3
4
8 9
765
1110
n = 11height = 3(# edges on longest leaf toroot path)
CPSC 311 O. Burchan Bayazit – Fall 02
Min-Heap
Min-heap property: A[Parent(i)] A[i]
Min-heaps are commonly used for priority queues in event-driven simulators.Parent(i) LeftChild(i) RightChild(i) return (i/2) return (2i) return (2i + 1)
2 3 5 4 6 9 11 10 15 20 18
1 2 3 4 5 6 7 8 9 10 11 A index
keys
2
3 5
4 6 9 11
10 15 20 18
1
2 3
4
8 9
765
1110
n = 11height = 3(# edges on longest leaf toroot path)
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
11 10
2 4 5
Assume we have a max-heap
12 3
4 56
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
11 10
2 4 5
The largest number is the root (max-heap property)
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
11 10
2 4 5
Take the root away
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
11 10
2 4
5
Put the last one to the root
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
11 10
2 4
5
Fix the max-heap
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 10
2 4
11
Fix the max-heap
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 10
2 4
11
Take the root away
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 10
2
4
11
Put the last one to the root
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 4
2
10
11
Fix the max-heap
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 4
2
10 11
Take the root away
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 4
2
10 11
Put the last one to the root
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
2 4
5
10 11
Fix the max-heap
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
2 4
5 10 11
Take the root away
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
2
4
5 10 11
Put the last one to the top
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
2
4 5 10 11
Take the top away
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
182 4 5 10 11
Put the last one
CPSC 311 O. Burchan Bayazit – Fall 02
Heap SortBasic Functions• HEAPIFY: maintain (fix) max-heap property • BUILD-HEAP: Create a max-heap from unsorted array
(requires HEAPIFY)• HEAPSORT: sorts an array (requires BUILD-HEAP and
HEAPIFY)How it works?
•Build a max-heap (the root is the largest number)
•Take the largest number away, put the last number in the heap to the root position and if max-heap property is not satisfied, fix it.
•Repeat until all the elements are processed.
CPSC 311 O. Burchan Bayazit – Fall 02
Heapify (fix max-heap)
Assume one node i does not satisfy MAX-HEAP property but its children are.
Heapify function corrects this by recursively fixing inconsistencies.
20
2 15
10 11 9 6
2 8 5 3
Not satisfy max-heap
CPSC 311 O. Burchan Bayazit – Fall 02
Assume one node i does not satisfy MAX-HEAP property but its children are.
Heapify function corrects this by recursively fixing inconsistencies.
20
11 15
10 2 9 6
2 8 5 3
Switch with the largest child
Heapify (fix max-heap)
CPSC 311 O. Burchan Bayazit – Fall 02
Assume one node i does not satisfy MAX-HEAP property but its children are.
Heapify function corrects this by recursively fixing inconsistencies.
20
11 15
10 2 9 6
2 8 5 3Not satisfy max-heap
Heapify (fix max-heap)
CPSC 311 O. Burchan Bayazit – Fall 02
Assume one node i does not satisfy MAX-HEAP property but its children are.
Heapify function corrects this by recursively fixing inconsistencies.
20
11 15
10 5 9 6
2 8 2 3Switch with the largest child
Heapify (fix max-heap)
CPSC 311 O. Burchan Bayazit – Fall 02
Heapify (fix max-heap)
•Find the largest child
•Switch the node i with its largest child
•Recursively HEAPIFY for the subtree where node i placed
CPSC 311 O. Burchan Bayazit – Fall 02
Heapify(A,i,n)/* A: Array, I: node to be fixed, n:heap-size */1. left 2i; right 2i + 1 /*indices of left & right children of A[i] */2. largest i;3. if left n and A[left] > A[i] then {4. largest left 5. }5. if right n and A[right] > A[largest] then {6. largest right 7. }7. if largest i then8. swap(A[i], A[largest])9. Heapify(A, largest,n)
T (n)=T(2n/3)+ (1) O (lgn)using master method
Heapify (fix max-heap)
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
Given unsorted array
Starting from largest non-leaf node down to the first node, call Heapify for each node
1
2 3
4 56
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
8
3 2
4 6 8 5
10 9 18 7
8 3 2 4 6 8 5 10 9 18 7
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
8
3 2
4 6 8 5
10 9 18 7
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
8
3 2
4 18 8 5
10 9 6 7
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
8
3 2
4 18 8 5
10 9 6 7
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
8
3 2
10 18 8 5
4 9 6 7
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
8
3 2
10 18 8 5
4 9 6 7
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
8
3 8
10 18 2 5
4 9 6 7
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
8
3 8
10 18 2 5
4 9 6 7
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
8
18 8
10 3 2 5
4 9 6 7
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
8
18 8
10 7 2 5
4 9 6 3
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
8
18 8
10 7 2 5
4 9 6 3
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
18
8 8
10 7 2 5
4 9 6 3
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
18
10 8
8 7 2 5
4 9 6 3
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
18
10 8
9 7 2 5
4 8 6 3
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
CPSC 311 O. Burchan Bayazit – Fall 02
Build-Heap
Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }
O(lgn) repeated (n/2 times)
T(n)=O(nlgn)
Can we do better?
Yes T(n)=O(n)
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort Algorithm
HeapSort(A,n)1. Build-Heap(A,n) /* put all elements in heap */2. for i = n downto 2 {3. swap A[1] A[i] /* puts max in ith array position */4. Heapify(A,1,i-1) /* restore heap property */5. }
Input: An n-element array A (unsorted).Output: An n-element array A in sorted order, smallest to largest.
Running time of HeapSort• 1 call to Build-Heap() O(n) time• n-1 calls to Heapify() each takes O(lgn) time O(nlgn) time
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
11 10
2 4 5
Assume we have a max-heap
12 3
4 56
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
11 10
2 4 5
The largest number is the root (max-heap property)
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
11 10
2 4 5
Take the root away
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
11 10
2 4
5
Put the last one to the root
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
11 10
2 4
5
Fix the max-heap
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 10
2 4
11
Fix the max-heap
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 10
2 4
11
Take the root away
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 10
2
4
11
Put the last one to the root
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 4
2
10
11
Fix the max-heap
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 4
2
10 11
Take the root away
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
5 4
2
10 11
Put the last one to the root
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
2 4
5
10 11
Fix the max-heap
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
2 4
5 10 11
Take the root away
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
2
4
5 10 11
Put the last one to the top
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
18
2
4 5 10 11
Take the top away
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Heap Sort – How it works
182 4 5 10 11
Put the last one
18 11 10 2 4 5
CPSC 311 O. Burchan Bayazit – Fall 02
Priority Queues: Application for Heaps
An application of max-priority queues is to schedule jobs on a shared processor. Need to be able to get the highest priority job Heap-Maximum(A,n) remove job from the queue Heap-Extract-Max(A,n) insert new jobs into queue Max-Heap-Insert(A, key,n) increase priority of jobs Heap-Increase-Key(A,i,key,n)
CPSC 311 O. Burchan Bayazit – Fall 02
Heap-Extract-Max
Heap-Extract-Max(A,n)1. max=A[1]2. A[1]=A[n]3. n=n-1 /* puts max in ith array position */4. Heapify(A,1,n) /* restore heap property */5. return max
T(n)=O(lgn)
CPSC 311 O. Burchan Bayazit – Fall 02
Heap-Increase-Key
Heap-Increase-Key(A,i,key,n)1. A[i]=key2. while I>1 and A[Parent[i]]<A[i] {3. swap A[Parent[i]] A[i] 4. i=Parent[i]
T(n)=O(lgn)
CPSC 311 O. Burchan Bayazit – Fall 02
Max-Heap-Insert
Max-Heap-Insert(A,i,key,n)1. n=n+1
2. A[i]=- 3. Heap-Increase-Key(A,n,key,n)
T(n)=O(lgn)