1

Issues with Matrix and VectorIssues with Matrix and Vector QuicksortQuicksort Determining Algorithm EfficiencyDetermining Algorithm Efficiency

Substitution Method Cost Tree Method Master Method

CSE 30331CSE 30331Lecture 7 – Exceptions, Algorithms IIILecture 7 – Exceptions, Algorithms III

2

Reading Reminder

More Algorithms Cormen: 4.1 – 4.3

QuickSort Cormen: Ch 7 (all) Ford & Topp: 15.1

3

Issues with Vector & Matrix

Your Vector<T> “is a” vector<T> So it does not need a private vector data member Can call existing vector<T> functions like this

vector<T>::at(i) vector<T>::operator[](i) vector<T>(r,c,val)

Your Matrix class ... must use your Vector<T> class NOT the STL vector

4

Quicksort Algorithm

Select pivot value from middle of vector

Partition vector into two sub-vectors One contains values smaller that pivot The other contains values larger than pivot Poor choice of pivot can lead to unbalanced pairs

of sub-vectors – leading to deeper recursion

Recursively sort each sub-vector

5

QuickSort() – sorts v[first,last)template <typename T>void quicksort(vector<T>& v, int first, int last) { int pivot Loc; T temp; if (last-first <= 1) return; //done else if (last–first == 2) { // only 2 values if (v[last] < v[first]) { temp = v[first]; v[first] = v[last]; v[last] = temp; } return; } else { // still sorting to be done pivotLoc = pivotIndex(v, first, last); quicksort(v, first, pivotLoc); quicksort(v, pivotLoc+1, last); }}

6

Pivot Index Algorithm

If only one element return its index

Else Select middle value as pivot and swap it with first value Scan up from left and down from right until we find leftmost

value larger than pivot and rightmost value less than pivot Swap these two values and repeat scan When scanUp and scanDown pass each other

Copy value from scanDown position to first position Copy pivot into scanDown position Return scanDown position

7

pivotIndex()template <typename T>int pivotIndex(vector<T>& v, int first, int last) { int mid, scanUp, scanDown; T pivot, temp; if (first == last) return last; // empty partition else if (first == last-1) return first; // one element partition else { mid = (last + first) / 2; pivot = v[mid]; v[mid] = v[first]; v[first] = pivot; scanUp = first+1; scanDown = last-1; // continued ...

8

pivotIndex() continued for (;;) { while (scanUp <= scanDown && v[scanUp] < pivot) scanUp++; // find leftmost large value while (pivot < v[scanDown]) scanDown--; // find rightmost small value if (scanUp >= scanDown) break; temp = v[scanUp]; // swap two values v[scanUp] = v[scanDown]; v[scanDown] = temp; scanUp++; scanDown--; // reset for next scan } v[first] = v[scanDown]; // move rightmost small value v[scanDown] = pivot; // put pivot there return scanDown; // return position of pivot }}

9

Quicksort Example

1 5 0 3 0 0 6 5 0 5 5 0 8 0 0 4 0 0 3 5 0 4 5 0

s can U p s can D o w n

v [0 ] v [9 ]v [8 ]v [7 ]v [6 ]v [5 ]v [4 ]v [3 ]v [2 ]v [1 ]

p iv o t

5 0 0 9 0 0

Quicksort recursive calls to partition a list into smaller and smaller sublists about a value called the pivot.

Example: Let v be a vector containing 10 integer values:

v = {800,150,300,650,550,500,400,350,450,900}

10

Quicksort Example (Cont…)B efo re t h e exch an ge

A ft er t h e exch an ge an d u p d at es t o s can U p an d s can D o w n

1 5 0 3 0 0 6 5 0 5 5 0 8 0 0 4 0 0 3 5 0 4 5 0

s can U p s can D o w n

v [0 ] v [9 ]v [8 ]v [7 ]v [6 ]v [5 ]v [4 ]v [3 ]v [2 ]v [1 ]

p iv o t

5 0 0 9 0 0

1 5 0 3 0 0 4 5 0 5 5 0 8 0 0 4 0 0 3 5 0 6 5 0

s can U p s can D o w n

v [0 ] v [9 ]v [8 ]v [7 ]v [6 ]v [5 ]v [4 ]v [3 ]v [2 ]v [1 ]

p iv o t

5 0 0 9 0 0

11

Quicksort Example (Cont…)B efo re t h e exch an ge

A ft er t h e exch an ge an d u p d at es t o s can U p an d s can D o w n

1 5 0 3 0 0 4 5 0 5 5 0 8 0 0 4 0 0 3 5 0 6 5 0

s can U p s can D o w n

v [0 ] v [9 ]v [8 ]v [7 ]v [6 ]v [5 ]v [4 ]v [3 ]v [2 ]v [1 ]

p iv o t

5 0 0 9 0 0

1 5 0 3 0 0 4 5 0 3 5 0 8 0 0 4 0 0 5 5 0 6 5 0

s can U p s can D o w n

v [0 ] v [9 ]v [8 ]v [7 ]v [6 ]v [5 ]v [4 ]v [3 ]v [2 ]v [1 ]

p iv o t

5 0 0 9 0 0

12

Quicksort Example (Cont…)B efo re t h e exch an ge

A ft er t h e exch an ge an d u p d at es t o s can U p an d s can D o w n

1 5 0 3 0 0 4 5 0 3 5 0 8 0 0 4 0 0 5 5 0 6 5 0

s can U p s can D o w n

v [0 ] v [9 ]v [8 ]v [7 ]v [6 ]v [5 ]v [4 ]v [3 ]v [2 ]v [1 ]

p iv o t

5 0 0 9 0 0

1 5 0 3 0 0 4 5 0 3 5 0 4 0 0 8 0 0 5 5 0 6 5 0

s can U ps can D o w n

v [0 ] v [9 ]v [8 ]v [7 ]v [6 ]v [5 ]v [4 ]v [3 ]v [2 ]v [1 ]

p iv o t

5 0 0 9 0 0

13

Quicksort Example (Cont…)

4 0 0 1 5 0 3 0 0 4 5 0 3 5 0 5 0 0 8 0 0 5 5 0 6 5 0

v [0 ] v [9 ]v [8 ]v [7 ]v [6 ]v [5 ]v [4 ]v [3 ]v [2 ]v [1 ]

9 0 0

P iv o t in it s fin al p o s it io n

4 0 0 1 5 0 3 0 0 4 5 0 3 5 0 5 0 0 8 0 0 5 5 0 6 5 0

v [0 ] v [9 ]v [8 ]v [7 ]v [6 ]v [5 ]v [4 ]v [3 ]v [2 ]v [1 ]

v [0 ] - v [4 ] v [6 ] - v [9 ]

9 0 0

14

Quicksort Example (Cont…)

p iv o t

3 0 0 1 5 0 4 0 0 4 5 0 3 5 0

s can U p

v [0 ] v [4 ]v [3 ]v [2 ]v [1 ]

In it ial Valu es

s can D o w n

p iv o t

3 0 0 1 5 0 4 0 0 4 5 0 3 5 0

v [0 ] v [4 ]v [3 ]v [2 ]v [1 ]

s can U p

A ft er Scan s St o p

s can D o w n

1 5 0 3 0 0 4 0 0 4 5 0 3 5 0

v [0 ] v [4 ]v [3 ]v [2 ]v [1 ]

15

Quicksort Example (Cont…)

p iv o t

6 5 0 5 5 0 8 0 0 9 0 0

s can U p

v [6 ] v [9 ]v [8 ]v [7 ]

In it ial Valu es

s can D o w n

p iv o t

6 5 0 5 5 0 8 0 0 9 0 0

v [6 ] v [9 ]v [8 ]v [7 ]

s can U p

A ft er St o p s

s can D o w n

5 5 0 6 5 0 8 0 0 9 0 0

v [6 ] v [9 ]v [8 ]v [7 ]

16

Quicksort Example (Cont…)

v [0 ] v [4 ]v [3 ]v [2 ]v [1 ] v [6 ] v [9 ]v [8 ]v [7 ]v [5 ]

1 5 0 9 0 08 0 06 5 05 5 05 0 03 5 04 5 04 0 03 0 0

4 0 0 4 5 0 3 5 0

v [4 ]v [3 ]v [2 ]

B efo re P art it io n in g

3 5 0 4 0 0 4 5 0

v [4 ]v [3 ]v [2 ]

A ft er P art it io n in g

1 5 0 3 0 0 3 5 0 4 0 0 4 5 0 5 0 0

v [0 ] v [4 ]v [3 ]v [2 ]v [1 ]

5 5 0 6 5 0 8 0 0 9 0 0

v [6 ] v [9 ]v [8 ]v [7 ]v [5 ]

17

Quicksort Efficiency (Average)

Levels in call tree (log n) Elements at each level (average case)

Level 0 (1 vector of n elements) Level 1 (2 vectors of approx. n/2 elements) Level 2 (4 vectors of approx. n/4 elements) …. Level k (2k vectors of approx. n/2k elements) Each level has n elements & Θ(n) effort required to find all

pivotIndexes & partitions at each level There are approx. k = log n levels

Quicksort is Θ(n log n) in best and average cases

18

Quicksort (Worst Case)

If pivot is largest or smallest value One sub-vector will be empty The other will contain n-1 values

If this happens at every level of recursion The call tree has n-1 rather than log n levels The effort to find the pivot Index is then

(n-1) + (n-2) + … + 2 + 1 = n(n-1)/2

So quicksort is Θ(n2) in the worst case It is easy to see how this could happen if the first value is

chosen as the pivot and the array is already sorted.

19

Determining Algorithm Efficiency

First we must determine a recurrence relation for the cost of the recursive algorithm

Then there are three primary methods we can use to determine the efficiency Θ(?) The Substitution Method The Cost Tree Method The Master Method

20

Substitution

Step 1 – guess form of solution Step 2 – use induction to find the constants

that show solution works Example:

Cormen shows (on pages 63-64) that T(n) = 2T(n/2) + n = O(n lg n), for c ≥ 1

Can we prove T(n) = 2T(n/2)+n = Ω(n lg n)

21

SubstitutionInductive proof (general case)

Prove T(n) = 2T(n/2)+n = Ω(n lg n)

Assume T(n/2) = 2T(n/4) + n/2 ≥ cn/2 lg n/2

Show T(n) ≥ cn lg n T(n) = 2T(n/2) + n ≥ 2(cn/2 lg n/2) + n = cn lg n – cn lg 2 + n = cn lg n – cn + n = cn lg n + (1-c)n = cn lg n, for c = 1

22

SubstitutionInductive proof (base case)

Finding the base case for the inductive proof can be difficult T(1) = 1 is true given our recurrence, but ... T(1) <= cn lg n = c lg 1 = 0 is false So some other base case is needed ... T(2) = 2T(2/2) + n = 2(1) + 2 = 4

<= cn lg n = 3c lg 2 = 2c for c >= 2 T(3) = 2T(3/2) + n = 2(1) + 3 = 5

<= cn lg n = 3c lg 3 = 3c for c >= 2

23

Substitution - difficulties

No good strategy to guessing correct solution Math may not work out unless …

Subtract low order term rather than add Change form of variables, work induction, and

substitute original form at end May need to prove base case for value of n0 other

than base case value for recurrence relation Remember to solve induction for exact form

of hypothesis, else you didn’t really get a proof

24

Cost Tree Analysis

Expand recurrence relation, level by level Number of sub-problems and size of each

determines number and nature of children of each node

Eventually, each node has a cost based on dividing and reassembling partial results

Leaves have costs based on base case of the recurrence relation

25

Example – Merge sort Cost

Rewriting the recurrence as …

... and assuming n = 2k

We can create a cost tree by expanding the recurrence at each level

otherwisecnnT

cnifcnT

)2/(2)(

T(n/2)

T(n) cncn

T(n/4)

cn/2T(n/2) cn/2

T(n/4) T(n/4) T(n/4)

26

Merge sort Cost

Total cost at each level of the tree is … 2c(n/2), 4c(n/4), 8c(n/8), … or in general … 2ic(n/2i) = cn

At the bottom level there are n subarrays of 1 element each and the cost there is … n*T(1) = cn

Depth of tree for sort of n = 2k elements is … k = lg n, so the tree has … 1 + k = 1 + lg n levels, but only n lg n merges

Therefore, the total cost of the algorithm is … cn lg n + cn, which is logarithmic Θ(n lg n)

27

Master Method

This is the “cookbook” approach to solving recurrences with the form

Where a ≥ 1, b > 1, and f(n) is an asymptotically positive function

a is the number of sub-problems, n/b is the size of each sub-problem, f(n) is the cost of dividing and reassembling

There are three distinct classes of solutions

)()/()( nfbnaTnT

28

Master Method

Case 1:

Case 2:

Case 3:

)()( log abnnTthen

)()( log abnnfif

01),()/( nnallandcsomeforncfbnafifand

))(()( nfnTthen

0),()( log somefornnfif ab

0),()( log somefornOnfif ab

)lg()( log nnnTthen ab

29

Master Method (Case 1)

Example:

)()( log abnnTthen

0),()( log somefornOnfif ab

nnTnT 200)2/(4)( 24loglog,2,4 2 aandbaso b

?)(200)( log abnOnnfisnow

)(200)( nOnnfclearly

)()()()(, 24loglog 2 nnnnTso ab

nnnnthenlet ab 1214loglog 2,1

30

Master Method (Case 2)

Example:

)()( log abnnfif

)(20)( nnnfand

nnTnT 20)2/(2)(

)lg()lg()lg()(, 2loglog 2 nnnnnnnTso ab

)lg()( log nnnTthen ab

12loglog,2,2 2 aandbaso b

?)(20)( log abnnnfisnow

nnnnclearly ab 12loglog 2,

31

Master Method (Case 3)

Example:

01),()/( nnallandcsomeforncfbnafifand

))(()( nfnTthen

0),()( log somefornnfif ab

2)2/(2)( nnTnT 12loglog,2,2 2 aandbaso b

?)()( log2 abnnnfisnow21112loglog 2,1 nnnnthenlet ab

)()( 22 nnnfclearly

)())(()(, 2nnfnTso 2

1?)(

2)

2(2),()/( 2

22 cforyesnc

nnorncfbnafis

32

Master Method Examples

Thanks to … wikipedia.org For excellent and clear examples of the Master

Method and other cool stuff

33

Summary

Quicksort algorithm uses a partitioning strategy that finds the final location of a

pivot element within an interval [first,last). The pivot splits the interval into two parts, [first, pivotIndex),

[pivotIndex, last). All elements in the lower interval have values pivot and all elements in the upper interval have values pivot.

running time: Θ(n lg n) worst case: of Θ(n2), unlikely to occur!??

34

Summary

Determining Recursive Algorithm Efficiency

First find a recurrence relation for the cost of the recursive algorithm

Then use one of the three primary methods to solve the recurrence for its Θ(?) The Substitution Method The Cost Tree Method The Master Method