04 Iterative Algorithms

7/27/2019 04 Iterative Algorithms

1/177

Central Algorithmic Techniques

Iterative Algorithms


2/177

COSC 3101, PROF. J. ELDER 2

CodeRepresentation of an Algorithm

class InsertionSortAlgorithm extends SortAlgorithm {

void sort(int a[]) throws Exception {

for (int i = 1; i < a.length; i++) {int j = i;

int B = a[i];

while ((j > 0) && (a[j-1] > B)) {

a[j] = a[j-1];

j--; }

a[j] = B;

}}

Pros and Cons?


3/177



Runs on computers

Precise and succinct

I am not a computer

I need a higher level of intuition.

Prone to bugs

Language dependent

Pros: Cons:


4/177


Two Key Types of Algorithms


Recursive Algorithms


5/177



Take one step at a time

towards the final destination

loop (done)

take step

end loop


6/177


Loop Invariants

A good way to structure many programs:

Store the key information you currently know in somedata representation.

In the main loop,

take a step forward towards destination

by making a simple change to this data.


7/177


The Getting to School Problem


8/177


Problem Specification Pre condition: location of home and school

Post condition: Traveled from home to school


9/177


General Principle Do not worry about the entire computation.

Take one step at a time!


10/177


A Measure of Progress

79 km

to school

75 kmto school


11/177


Algorithm specifies from which locations

it knows how to step.

Safe Locations


12/177


The computation is presently in a safe location.

May or may not be true.

Loop Invariant


13/177


Defining Algorithm From every safe location,

define one step towards school.


14/177


Take a step What is required of this step?


15/177


Can we be assured that the computationwill always be in a safe location?

If the computation is in a safe location,it does not step into an unsafe one.

Maintain Loop Invariant

No. What if it is not initially true?


16/177


From the Pre-Conditions on the input instancewe must establish the loop invariant.

Establishing Loop Invariant


17/177



Can we be assured that the

computation will always bein a safe location?

By what principle?


18/177


Maintain Loop Invariant By Induction the computation will

always be in a safe location.

(0)

, ( )

, ( ) ( 1)

S

i S i

i S i S i


19/177


Ending The Algorithm Define Exit Condition

Termination: With sufficient progress,

the exit condition will be met.

When we exit, we know

exit condition is true

loop invariant is true

from these we must establish

the post conditions.

Exit

Exit

0 km Exit


20/177


Lets Recap


21/177


Designing an AlgorithmDefine Problem Define Loop Invariants Define Measure of

Progress

Define Step Define Exit Condition Maintain Loop Inv

Make Progress Initial Conditions Ending

km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit


22/177

Simple Example

Insertion Sort Algorithm


23/177



class InsertionSortAlgorithm extends SortAlgorithm {

void sort(int a[]) throws Exception {

for (int i = 1; i < a.length; i++) {

int j = i;

int B = a[i];

while ((j > 0) && (a[j-1] > B)) {

a[j] = a[j-1];

j--; }

a[j] = B;

}}


24/177


Higher Level Abstract ViewRepresentation of an Algorithm

9 km

5 km


25/177


Designing an AlgorithmDefine Problem Define Loop Invariants Define Measure of

Progress



km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit


26/177


Problem Specification Precondition: The input is a list of n values

with the same value possibly repeated. Post condition: The output is a list consistingof the same n values in non-decreasing order.

88 149825

62

52

79

3023

31 14,23,25,30,31,52,62,79,88,98


27/177


88 14

9825 62

52

79

3023

31 14,23,25,30,31,52,62,79,88,98

14

9825 62

79

3023,31,52,88

Some subset of the elements are sorted

The remaining elements are off to the side.

Define Loop Invariant


28/177


Defining Measure of Progress

14

9825 62

79

3023,31,52,88

6 elements

to school


29/177


Define Step

Select arbitrary element from side.

Insert it where it belongs.

9825 62

79

23,31,52,88

14

9825

79

3023,31,52, 62 ,88

1430


30/177


Making progress while Maintaining theloop invariant

9825 62

79

23,31,52,88

14

98

25

79

3023,31,52, 62 ,88

1430

79 km 75 km

5 elements

to school

6 elements

to school

Exit

Sorted sublist

B i i &


31/177


88 14

9825 62

52

79

3023

31

88 14

98

25 62

52

79

3023

31 n elementsto school

14,23,25,30,31,52,62,79,88,98

14,23,25,30,31,52,62,79,88,980 elements

to school

Beginning &Ending

km Exit0 km Exit


32/177


Running Time

Inserting an element into a list of size itakes (i) time.

Total = 1+2+3++n = (n 2)

9825 62

79

23,31,52,88

14

9825

79

3023,31,52, 62 ,88

1430


33/177


OkI know you knew Insertion Sort

But hopefully you are beginning to appreciate

Loop Invariants

for describing algorithms


34/177

Assertions

in Algorithms


35/177


Purpose of Assertions

Useful for

thinking about algorithms developing

describing proving correctness


36/177


Definition of Assertions

An assertion is a statement about thecurrent state of the data structure that is

either true or false.

eg. the amount in your bank account is notnegative.


37/177



It is made at some particular point during theexecution of an algorithm.

If it is false, then something has gone wrong in thelogic of the algorithm.


38/177



An assertion is not a task for the algorithm toperform.

It is only a comment that is added for thebenefit of the reader.


39/177


Specifying a Computational Problem

Example of Assertions

Preconditions: Any assumptions that must betrue about the input instance.

Postconditions: The statement of what must

be true when the algorithm/program returns..


40/177


Definition of Correctness

&

If the input meets the preconditions,

then the output must meet the postconditions.

If the input does not meet the preconditions, thennothing is required.


41/177


An Example: A Sequence of Assertions

if( ) then

code else

code end if

if( ) then

code else

code end if

any code

How is this proved?


Must check 2 r differentsettings of paths through the code.

Is there a faster way?


42/177



if( ) then

code else

code end if

if( ) then

code else

code end if

Step 1

code

code


43/177



if( ) then

code else

code end if

if( ) then

code else

code end if

Step 2

code

code


44/177



if( ) then

code else

code end if

if( ) then

code else

code end if

Step r

code

code

A S f A i


45/177


A Sequence of Assertions

if( ) then

code else

code end if

if( ) then

code else

code end if

Step r

code

code


46/177


47/177


codeA loop

exit when codeBendloop codeC

Definition of Correctness?

Iterative AlgorithmsLoop Invariants


48/177


codeA loop


Definition of Correctness any code

How is this proved?



49/177


codeA loop

exit when

codeBendloop codeC

The computation may go aroundthe loop an arbitrary number of times.

any code

Is there a faster way?




50/177


codeA loop


Step 0codeA


51/177


codeA loop


Step 1 codeB


52/177


codeA loop


Step 2 codeB


53/177


codeA loop


Step 3 codeB


54/177


codeA loop


Step i codeB


55/177


codeA loop

exit when codeBendloopcodeC

Last Step codeC


Partial Correctness


56/177


Partial Correctness

Proves that IF the program terminates then it works

&

codeC

Clean up loose ends

codeB

Maintaining Loop Invariant

codeA


Exit

Exit


57/177


Algorithm Termination

km

79 km 75 km

Measure of progress

0 km

Exit


58/177


Algorithm Correctness

Partial Correctness+ Termination Correctness


59/177


60/177


Don t start coding

You must design a working algorithm first.


61/177


62/177


Start with Small Steps

What basic steps might you follow to make some kind of progress towards the answer?

Describe or draw a picture of what the data structure mightlook like after a number of these steps.


63/177


Picture from the Middle

Leap into the middle of the algorithm.

What would you like your data structure to look likewhen you are half done?


64/177


Ask for 100%

Pretend that a genie has granted your wish. You are now in the middle of your computation and your

dream loop invariant is true.


65/177


Ask for 100%

Maintain the Loop Invariant: From here, are you able to take some computational steps that

will make progress while maintaining the loop invariant?


66/177


Ask for 100%

If you can maintain the loop invariant, great. If not,

Too Weak: If your loop invariant is too weak, then the genie hasnot provided you with everything you need to move on.

Too Strong: If your loop invariant is too strong, then you will notbe able to establish it initially or maintain it.


67/177


Differentiating between Iterations

x=x+2 Meaningful as code

False as a mathematical statement

x = x i = value at the beginning of the iterationx = x i+1 = new value after going around the loop one more time.

x'' = x'+2

Meaningful as a mathematical statement


68/177

Loop Invariantsfor


Three

Search Examples


69/177


Define Problem: Binary Search

PreConditions Key 25

Sorted List

PostConditions Find key in list (if there).

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95


70/177


Define Loop Invariant

Maintain a sublist.

If the key is contained in the original list, then the key iscontained in the sublist.

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95


71/177


Define Step

Make Progress


key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95


72/177


Define Step

Cut sublist in half.

Determine which half the key would be in.

Keep that half.

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If key mid ,then key is inleft half.

If key > mid ,then key is inright half.

mid


73/177


Define Step

It is faster not to check if the middle element is the key.

Simply continue.

key 43

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95




74/177


Make Progress

The size of the list becomes smaller.

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

79 km

75 km

79 km 75 km

Exit


75/177


Initial Conditionskm

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If the key is contained in theoriginal list,

then the key is contained in thesublist.

The sublist is theentire original list.

n km


76/177


Ending Algorithm



Sublist contains one element.

Exit

Exit

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

0 km

If the key iscontained in theoriginal list,then the key is atthis location.

key 25


77/177


If key not in original list



Loop invariant true,even if the key is notin the list.

If the key is contained inthe original list, then thekey is at this location.

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

key 24

Conclusion still solves the problem.Simply check this one locationfor the key.


78/177

BinarySea rch(A[1..n], )e k y


79/177


: A[1..n] is sorted in non-decreasing order: If is in A[1..n], algorithm returns

1,its location

loop-invariant>: If is inwhile

p q key

key q p

n

if [ ]

els

2

1

return( )

return("Key n

A[1..n], then

e

endendif [ ]

end

is in A[p..

ot in list")

q] p q mid

q mid

p mi

key A m

key

id

key A p

e

d

p lse

Algorithm Definition Completed


80/177


g pDefine Problem Define Loop Invariants Define Measure of

Progress



km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit

BinarySea rch(A[1..n], )e k y


81/177



1,its location

loop-invariant>: If is inwhile

p q key

key q p

n

if [ ]

els

2

1

return( )

return("Key n

A[1..n], then

e

endendif [ ]

end

is in A[p..

ot in list")

q] p q mid

q mid

p mi

key A m

key

id

key A p

e

d

p lse


82/177


Simple, right?

Although the concept is simple, binary search isnotoriously easy to get wrong.

Why is this?


83/177


The Devil in the Details

The basic idea behind binary search is easy to grasp.

It is then easy to write pseudocode that works for atypical case.

Unfortunately, it is equally easy to write pseudocode thatfails on the boundary conditions .


84/177


1

if [ ]

else

end

q mid

p

key A mid

mid


1

if [ ]

else

end

q mid

p

key A mid

mid or

What condition will break the loop invariant?


85/177



key 36

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

mid

sC eod lek cey tA[m rige hid] t lf: ha

Bug!!


86/177


1

if [ ]

else

end

q mid

p

key A mid

mid


1

if [ ]

else

end

q mid

p

key A mid

mid

if < [ ]

else

end

1q mid

p

key A mid

mid

OK OK Not OK!!


87/177


key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95


mid2

p q

mid2

p q

or

Shouldnt matter, right? Select mid2

p q


88/177


6 74


key 25

9591888372605351494336252121181353



mid


89/177

h l h l

Exit


90/177


2513 74


key 25

9591888372605351494336212118653



Another bug! No progress

toward goal:Loops Forever!mid

79 km 75 km

Exit

h l h l


91/177


if [

mid

]2

1else

end

key A mi

p q

q mid

p mid

d


if [

mid

]2

1else

end

key A mi

p q

q mid

p mid

d

if < [

mid2

1]

else

end

key A mid

p q

q mid

p mid

OK OK Not OK!!

H M P ibl Al i h ?


92/177


if [

mid

]2

1else

end

key A mi

p q

q mid

p mid

d

How Many Possible Algorithms?

midr2

o ? p q

if < [or ?]key A mid

else

o

end

1r q mid

p mid

Alternative Algorithm: Less Efficient but More Clear


93/177



1,its location

loop-invariant>: If is

BinarySea

in

rch(A[1..n],

whil

)

e p q

key

key q p

e

n

k y

2

retur

A[1..n

n(mid)

], then

if = [ ]

elseif root ,then key isin right half.

If key = root ,then key isfound

Algorithm Definition Completed


100/177


Define Problem Define Loop Invariants Define Measure of Progress



km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit


101/177

End of Lecture 4

Sept 20, 2007

Card Trick


102/177


A volunteer, please.

Card Trick


103/177

Loop Invariantsfor


A Third

Search Example:

A Card Trick


104/177


Pick a Card

Done

Loop Invariant:


105/177


The selected card is one of these.


106/177


Which column?

left

Loop Invariant:


107/177



Selected column is placed


108/177


in the middle

I will rearrange the cards


109/177

COSC 3101, PROF. J. ELDER109


110/177


111/177


Which column?

right

Loop Invariant:


112/177



Selected column is placed


113/177


in the middle

I will rearrange the cards


114/177



115/177


Which column?

left

Loop Invariant:


116/177



Selected column is placed h ddl


117/177


in the middle


118/177

Algorithm Definition CompletedDefine Problem Define Loop Invariants Define Measure of


119/177


Define Problem Define Loop Invariants Define Measure of Progress



km

79 kmto school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit

Ternary Search


120/177


Ternary Search

How many iterations are required to guaranteesuccess?

Loop Invariant: selected card in central subset of cards

1Size of subset = / 3

wheretotal number of cards

iteration index

i n

n

i


121/177

Loop Invariantsfor


A Fourth Example:

Partitioning

(Not a search problem:

can be used for sorting, e.g., Quicksort)

The Partitioning Problem


122/177


g

88 149825

62

52

79

3023

31

Input:

14

2530

233188 98

6279

52

Output:x=52

Problem: Partition a list into a set of small values and a set of large values.

Precise Specification


123/177


p

[ ... ] is an arbitrary list of values. [ ] is the pivoPrecondit .i ton: A p r x A r

p r

is rearranged such that [ ... 1] [ ] [ 1... ]for some q.Postcondition: A A p q A q x A q r

p r q

Loop Invariant


124/177


3 subsets are maintained

One containing values lessthan or equal to the pivot

One containing valuesgreater than the pivot

One containing values yetto be processed

p



125/177


g p

Consider element at location j

If greater than pivot, incorporate into> set by incrementing j.

If less than or equal to pivot,incorporate into set by swapping

with element at location i+1 andincrementing both i and j.

Measure of progress: size of unprocessed set.



126/177


g p



127/177


g p

Establishing Postcondition


128/177


g

on exit j

Exhaustive on exit

Establishing Postcondition


129/177


g

An Example


130/177


Running Time


131/177


Each iteration takes (1) time Total = (n)

or

Algorithm Definition CompletedDefine Problem Define Loop Invariants Define Measure of


132/177


Progress



km

79 kmto school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit


133/177

More Examples of Iterative Algorithms

Using Constraints on Input to Achieve Linear-Time Sorting

Recall: InsertionSort


134/177


2

2

( 1)Worst case (reverse order): : 1 ( ) ( )

2

n

j j

n nt j j T n n

Recall: MergeSort


135/177


( ) ( log )T n n n

Comparison Sorts


136/177


InsertionSort and MergeSort are examples of (stable)Comparison Sort algorithms.

QuickSort is another example we will study shortly.

Comparison Sort algorithms sort the input by successivecomparison of pairs of input elements.

Comparison Sort algorithms are very general: theymake no assumptions about the values of the input

elements.

Comparison Sorts


137/177


2InsertionSort is ( ).n

MergeSort is ( log ).n n

Can we do better?

Comparison Sort: Decision Trees


138/177


Example: Sorting a 3-element array A[1..3]

Comparison Sort


139/177


Worst-case time is equal to the height of the binarydecision tree.

The height of the tree is the log of the number of leaves.

The leaves of the tree represent all possible

permutations of the input. How many are there?

log ! ( log )n n nThus MergeSort is asymptotically optimal.

Linear Sorts?


140/177


Comparison sorts are very general, but are ( log ) n n

Faster sorting may be possible if we can constrain the nature of the input.

Example 1. Counting Sort


141/177


Counting Sort applies when the elements to be sortedcome from a finite (and preferably small) set .

For example, the elements to be sorted are integers inthe range [0k -1], for some fixed integer k.

We can then create an array V[0k -1] and use it tocount the number of elements with each value [0k -1].

Then each input element can be placed in exactly the

right place in the output array in constant time .

Counting Sort


142/177


Input: N records with integer keys between [0k -1].

Output: Stable sorted keys.

Algorithm: Count frequency of each key value to determine transition

locations Go through the records in order putting them where they go.

Input:Output: 0 0 0 0 0 1 1 1 1 1 1 1 1 1 2 2 3 3 3

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0

CountingSort

I 0 0 0 0 0


143/177


Stable sort: If two keys are the same, their order does not change.

Input:

Output:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0

Thus the 4 th record in input with digit 1 must be

the 4 th record in output with digit 1.

It belongs at output index 8, because 8 records go before itie, 5 records with a smaller digit & 3 records with the same digit

0 0 0 0 0 1 1 1 1 1 1 1 1 1 2 2 2 33

Count These!

Index: 11109876543210 12 13 14 15 16 17 18

CountingSort

I 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


144/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

# of records with digit v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0

32102395

N records. Time to count? (N)

CountingSort

I 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


145/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:# of records with digit v:

# of records with digit < v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0

32103395171450

N records, k different values. Time to count? (k)

CountingSort

I t 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


146/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:# of records with digit < v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0

3210171450

= location of first record with digit v.

0 0 0 0 0 1 1 1 1 1 1 1 1 1 2 2 2 33

CountingSort

I t 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


147/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0

3210171450Location of first record

with digit v.

Algorithm: Go through the records in order putting them where they go.

10 ?

Loop Invariant


148/177


The first i-1 keys have been placed in the correctlocations in the output array

The auxiliary data structure v indicates the location atwhich to place the i th key for each possible key value

from [1..k-1] .


149/177

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


150/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0

3210171460Location of next record

with digit v.

0


1

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


151/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


with digit v.

0 0


1

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


152/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


with digit v.

0 1


10

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


153/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


with digit v.

0 1


10 3

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


154/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


with digit v.

0 1


10 1 3

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


155/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


with digit v.

0 1


10 1 31

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


156/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


with digit v.

0 1


10 331 1

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


157/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


with digit v.

0 1


10 1 31 1 3

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


158/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


with digit v.

0 1


10 0 1 1 1 3 3

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


159/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


with digit v.

0 1


10 21 1 1 3 30

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


160/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


with digit v.

0 1


10 11 1 1 3 30 2


161/177

CountingSort

Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


162/177


Input:

Output:Index: 11109876543210 12 13 14 15 16 17 18

Value v:

1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0


with digit v.

0 110 11 1 1 3 30 20 0 1 1 1 2 2

(N)Time = (N+k)Total =


163/177

End of Lecture 5

Sept 25, 2007

Input:

Example 2. RadixSort 344125

125224


164/177


Input: A of stack of N punch cards. Each card contains d digits. Each digit between [0k -1]

Output: Sorted cards.

125333134224334

143225325243

Digit Sort: Select one digit Separate cards into k piles

based on selected digit (e.g., Counting Sort).

225

325

333134

334

344143

243Stable sort: If two cards are the same for that digit,their order does not change.

RadixSort

344 125 125


165/177


344

125333134224

334143225325243

Sort wrt whichdigit first?

The mostsignificant.

125

134 143 224225

243 344 333334325

Sort wrt whichdigit Second?

The next mostsignificant.

125

224225 325 134

333334 143 243 344

All meaning in first sort lost.

RadixSort


166/177


344125333134

224334143225325243

Sort wrt whichdigit first?

Sort wrt whichdigit Second?

The leastsignificant.

333 143243 344

134224334 125225325

The next leastsignificant.

224125225325

333 134334 143243 344


167/177

RadixSort


168/177


Sort wrt i+1stdigit.

2 241 252 253 25

3 33 1 343 34 1 432 43 3 44

Is sorted wrtfirst i digits.

1 25 1 34 1 43

2 242 252 43

3 25 3 33 3 34 3 44

Is sorted wrtfirst i+1 digits.

i+1

These are in thecorrect order

because sortedwrt high order digit

RadixSort


169/177


Sort wrt i+1stdigit.

2 241 252 253 25

3 33 1 343 34 1 432 43 3 44

Is sorted wrtfirst i digits.

1 25 1 34 1 43

2 242 252 43

3 25 3 33 3 34 3 44

i+1

These are in thecorrect order

because was sorted &stable sort left sorted

Is sorted wrtfirst i+1 digits.

Loop Invariant


170/177


The keys have been correctly stable-sorted with respectto the i-1 least-significant digits.

Running Time


171/177


Running time is ( ( ))Where

# of digits in each number# of elements to be sorted# of possible values for each digit

d n k

d n k

Example 3. Bucket Sort


172/177


Applicable if input is constrained to finite interval, e.g.,[01).

If input is random and uniformly distributed, expected run time is (n).

Bucket Sort


173/177


Loop Invariants


174/177


Loop 1 The first i-1 keys have been correctly placed into buckets of

width 1/n .

Loop 2

The keys within each of the first i-1 buckets have been correctlystable-sorted.

PseudoCode


175/177


(1)

(1)( )n

n

( )n

Examples of Iterative Algorithms


176/177


Binary Search Partitioning

Insertion Sort

Counting Sort Radix Sort

Bucket Sort

Which can be made stable?

Which sort in place?

How about MergeSort?


177/177

End of Iterative Algorithms

Date post:	14-Apr-2018
Category:	Documents
Upload:	amdileeyas
View:	218 times
Download:	0 times

04 Iterative Algorithms

Documents