01 Iterative methods

7/25/2019 01 Iterative methods

1/356

1

Iterative Algorithms

& Loop Invariants

Jeff Edmonds

York University COSC 3101LectureLecture11

Understanding Algorithms Abstractly

Home to School Problem

Steps in an Iterative Algorithm

Assertions

Insertion SortDesigning an Algorithm

Finding Errors

ypical Loop Invariants

!inary Search Li"e E#ampleshe Partition Problem

$reatest %ommon Divisor

!c"et '(ic") Sort *or Hmans

+adi#,%onting Sort

Lo-er !ond *or Sort


2/356

.

he $oal is to

U/DE+SA/D

Algorithms

Fndamentally to their core


3/356

0

+epresentation

Understand the relationship bet-een

di**erent representations o* the same

in*ormation or idea

1

2

3+dich ---2discretemath2com


4/356

3

Di**erent +epresentations

o* Algorithms4 %ode

4 +nning e#ample

4 DFA and ring 5achines

4 Higher level abstract vie-


5/356

6

%ode

+epresentation o* an Algorithm class InsertionSortAlgorithm e#tends SortAlgorithm 7

void sort'int a89) thro-s E#ception 7

*or 'int i : 1; i < a2length; i==) 7

int > : i;

int ! : a8i9;

-hile ''> ? @) && 'a8>19 ? !)) 7

a8>9 : a8>19;

>; B

a8>9 : !;

BB Pros and %onsC


6/356

%ode

+epresentation o* an Algorithm

4 +ns on compters

4 Precise and sccinct

4 Perception that being

able to code is the

only thing needed toget a >ob2 '*alse)

4 I am not a compter

4 I need a higher level o*

intition2

4 Prone to bgs

4 Langage dependent

Pros %ons


7/356

+nning E#ample


ry ot a problem or

soltion on small e#amples2


8/356

+nning E#ample


13

G.6

.

6.

G

0@

.0

01

13

G


9/356G

+nning E#ample


[email protected]

Pros and %onsC


10/3561@

+nning E#ample


4 %oncrete

4 Dynamic

4 isal

4 +elies on yo to *ind the

pattern2

4 Does not e#plain -hy it

-or"s2

4 Demonstrates *or onlyone o* many inpts2

Pros %ons


11/35611

DFA and ring 5achines


Pros and %onsC


12/3561.

DFA and ring 5achines


4 $ood *or theorems

abot algorithms

4 /ot good *or designing

algorithms

Pros %ons


13/35610

Higher Level Abstract ie-


i1i

i

i

@=

1


14/35613

Levels o* Abstraction

It is hard to thin" o* love in terms o* the *iring o*

nerons2

vs

So*t-are developers vie- sbsystems as entities -ithseparate personalities roles and interactions

not details o* code2

vs


15/35616



4 Intitive *or hmans

4 Use*l *or

J thin"ing abot

J designing

J describing algorithms

4 ie- *rom -hich

correctness is sel*

evident2

4 5athematical mmbo

>mbo

4 oo abstract

4 Stdents resist it

Pros %ons


16/3561

Abstract a-ay the inessential *eatres o* a problem

=

ale Simplicityale Simplicity

$oal Understand and thin" abot comple#algorithm in simple -ays2

DonKt tne ot2 here are deep ideas -ithin

the simplicity2


17/3561


4 $iven a concrete algorithm eg2 5ergeSortand a concrete inpt eg2


18/3561


4 $iven a ne- comptational problem on e#amdesign a ne- concrete algorithm that solves it

*ollo-ing the M5eta AlgorithmN steps2

4 I* yo do not get all the details o* the ne-algorithm correct it -ont be a disaster2

4 I* yo do not get the steps o* the M5eta

Algorithm correct it -ill be a disaster24 here are only a *e- M5eta AlgorithmsN

Learn themO


19/3561G


4 M5eta AlgorithmsNJIterative Algorithms

J+ecrsive Algorithms

J$reedy AlgorithmsJ+ecrsive !ac" rac"ing

JDynamic Programming2

J/P%ompleteness


20/356.@


4 M5eta 5athNJE#istential and Universal (anti*iers

JAsymptotic $ro-th

JAdding 5ade Easy Appro#imationsJ+ecrrence +elations


21/356.1

ale %orrectness

4 DonKt tac" on a *ormal proo* o* correctness

a*ter coding to ma"e the pro*essor happy2

4 It need not be mathematical mmbo >mbo2

4 $oal o thin" abot algorithms in sch -aythat their correctness is transparent2


22/356..

E#plaining an Algorithm

4 $oal o provide a nmber o*

di**erent notations analogies and

abstractions -ithin -hich todevelop thin" abot and

describe algorithms2


23/356

.0

-o ey ypes o* Algorithms

4 Iterative Algorithms

4 +ecrsive Algorithms

i l i h


24/356

.3

4 A good -ay to strctremany programs

4 Store the "ey in*ormationyo crrently "no- in somedata strctre2

4 In the main loop ta"e a step*or-ard to-ards destination

4 by ma"ing a simple changeto this data2

Iterative Algorithmsloop 'ntil done)

ta"e step

end loop


25/356

.6

I li"e nderstanding thingssing a story2

I -ill no- tell one2


26/356

.

he $etting to School Problem


27/356

.

Problem Speci*ication4Pre condition location o* home and school

4Post condition traveled *rom home to school


28/356

.

Algorithm4Qhat is an AlgorithmC


29/356

.G

Algorithm4he algorithm de*ines the comptation rote2


30/356

0@

Algorithm4Is this reasonableC


31/356

01

%omple#ity4here are an in*inite nmber o* inpt instances

4Algorithm mst -or" *or each2


32/356

0.

%omple#ity4Di**iclt to predict -here comptation might be

in the middle o* the comptation2


33/356

00

4he crrent MStateN o* comptation is

determined by vales o* all variables2

Location o* %omptation


34/356

03

4Sppose the comptation ends p hereC



35/356

06

DonKt Panic4Qhere ever the comptation might be

ta"e best step to-ards school2

$ l i i l


36/356

0

$eneral Principle4Do not -orry abot the entire comptation2

4a"e one step at a timeO

D *i i Al i h


37/356

0

De*ining Algorithm4Qherever the comptation might be


"


38/356

0

a"e a step4Qhat is reRired o* this stepC

A 5 * P


39/356

0G

A 5easre o* Progress

G "m

to school

6 "m

to school

D *i i Al i h


40/356

3@

G "m 6 "m

De*ining Algorithm4Is this s**icient to de*ine a -or"ing algorithmC

Q "i Al i h


41/356

31

Qor"ing Algorithm4%omptation steadily approaches goal

G "m

to school

6 "m

to school

D *i i Al ith


42/356

3.

De*ining Algorithm4E#tra reRirements

2GGG "m

G "m

@ "mkm

G "m 6 "m


43/356

30

4Qherever the comptation might be


De*ine a Step


44/356

33

4Some location too con*sing *or algorithm

4Algorithm too lay to de*ine step *or every

location2

%omptation Stc"


45/356

36

4Algorithm speci*ies *rom -hich locations

it "no-s ho- to step2

Sa*e Locations

i


46/356

3

4Mhe comptation is presently in a sa*e location2N

45aybe tre and maybe not2 I* not something has gone -rong2

Loop Invariant

D *i i Al ith


47/356

3

De*ining Algorithm4From every sa*e location

de*ine one step to-ards school2

" t


48/356

3

a"e a step4Qhat is reRired o* this stepC

5 i i L I i


49/356

3G

4I* the comptation is in a sa*e location

it does not step into an nsa*e one2

5aintain Loop Invariant

5 i i L I i


50/356

6@

4%an -e be assred that the comptation

-ill al-ays be in a sa*e locationC

4I* the comptation is in a sa*e locationit does not step into an nsa*e one2


5 i i L I i


51/356

61

4%an -e be assred that the comptation

-ill al-ays be in a sa*e locationC

4I* the comptation is in a sa*e locationit does not step into an nsa*e one2


/o2 Qhat i* it is not initially treC

E t bli hi L I i t


52/356

6.

From the Pre%onditions on the inpt instance

-e mst establish the loop invariant2

Establishing Loop Invariant

5 i t i L I i t


53/356

60


4%an -e be assred that the

comptation -ill al-ays bein a sa*e locationC

4!y -hat principleC

5 i t i L I i t


54/356

63

5aintain Loop Invariant4!y Indction the comptation -ill

al-ays be in a sa*e location2

+

S

i S i S i

i S i

' )

' ) ' )

' )

@

1


55/356

66

Ending he Algorithm

E#itDe*ine E#it %ondition

ermination@ "m E#it

Qith s**icient progress

the e#it condition -ill be met2

E#it

*rom these -e mst establish

the post conditions2

E#it

Qhen -e e#it -e "no-4e#it condition is tre4loop invariant is tre

LetKs +ecap


56/356

6

Let s +ecap

Designing an Algorithm

Is this s**icientC


57/356

6

Designing an AlgorithmDe*ine Problem De*ine Loop

Invariants

De*ine 5easre o*Progress

De*ine Step De*ine E#it %ondition 5aintain Loop Inv

5a"e Progress Initial %onditions Ending

km

G "m

to school

E#it

E#it

G "m 6 "m

E#it

E#it

@ "m E#it

Is this s**icientC

%onsider an Algorithm


58/356

6


km E#it@ "m E#it

E#it

G "m 6 "m

E#it

Loop Invariant 5aintained


59/356

6G

Loop Invariant 5aintained

E#it

%omptation can al-ays proceed


60/356

@

%omptation can al-ays proceed

%omptation al-ays ma"es progress


61/356

1

%omptation al-ays ma"es progress

G "m 6 "m

G "m 6 "m

E#it

%omptation erminates


62/356

.


km

G"m 6 "m

@ "m

@ "m

@ "m E#it

E#it



63/356

0


E#it

E#it



64/356

3


his is s**icientO

km E#it@ "m E#it

E#it

G "m 6 "m

E#it


65/356

6

his completesTne Higher Level Abstract ie-

o* Iterative Algorithms


66/356

Assertions

Qe -ill no- redo

the previos ideasa little more *ormally

Paradigm Shi*t


67/356

Paradigm Shi*t

Is theblac"the *orm and the greenthe bac"grondC

Is the greenthe *orm and theblac"the bac"grondC

It is help to have di**erent -ays o* loo"ing at it2

A SeRence o* Actions


68/356

R

5a#' abc )

i*' b?m )

m : b

endi*

m : a

i*' c?m )

m : c

endi*

retrn'm)

I* yo are describing an alg to a

*riend is this -hat yo sayC

Do yo thin" abot an algorithm asa seRence o* actionsC

Do yo e#plain it by saying

MDo this2 Do that2 Do thisNC

Do yo get lost abot

-here the comptation is

and -here it is goingC

Qhat i* there are many pathsthrogh many i*s and loopsC

Ho- do yo "no- it -or"s

*or every pathon every inptC



69/356

G

R

5a#' abc )

i*' b?m )

m : b

endi*

m : a

i*' c?m )

m : c

endi*

retrn'm)

At least tell me -hat the algorithm

is spposed to do2

4 PreconditionsAny assmptions that mst be treabot the inpt instance2

4 Postconditions

he statement o* -hat mst be tre-hen the algorithm,programretrns2

Mpre%ond Inpt has 0 nmbers2N

Mpost%ond

retrn ma# in 7abcBN



70/356

@

R

5a#' abc )

i*' b?m )

m : b

endi*

m : a

i*' c?m )

m : c

endi*

retrn'm)

Ho- can yo possibly nderstand

this algorithm -ithot "no-ing

-hat is tre -hen the comptationis hereC


Mpost%ond

retrn ma# in 7abcBN



71/356

1

R

5a#' abc )

i*' b?m )

m : b

endi*

m : a

i*' c?m )

m : c

endi*

retrn'm)

Ho- can yo possibly nderstand

this algorithm -ithot "no-ing

-hat is tre -hen the comptationis hereC

Tell me!

Massert m is ma# in 7abBN


Mpost%ond

retrn ma# in 7abcBN


72/356

.

ale %orrectness

4 DonKt tac" on a *ormal proo* o* correctness

a*ter coding to ma"e the pro*essor happy2

4 It need not be mathematical mmbo >mbo24 $oal o thin" abot algorithms in sch -ay

that their correctness is transparent2

A SeRence o* Actionss


73/356

0

5a#' abc )


i*' b?m )

m : b

endi*

Massert m is ma# in 7aBN

m : a

i*' c?m )

m : c

endi*


Massert m is ma# in 7abcBN

retrn'm)

Mpost%ond

retrn ma# in 7abcBN

vs

A SeRence o* Assertions

It is help to havedi**erent -ays o*

loo"ing at it2


74/356

3

Prpose o* Assertions

Use*l *or

Jthin"ing abot algorithmsJdeveloping

Jdescribing

Jproving correctness


75/356

6

De*inition o* Assertions

An assertion is a statement

abot the crrent state o* the data

strctre that is either tre or *alse2

eg2 the amont in yor ban" accont

is not negative2


76/356


It is made at some particlar point dringthe e#ection o* an algorithm2

It shold be tre independent o*the path *ollo-ed throgh the code

and independent o* the inpt2

I* it is *alse then something has gone-rong in the logic o* the algorithm2


77/356


An assertion is not a tas" *or the

algorithm to per*orm2

It is only a comment that is added

*or the bene*it o* the reader2


78/356

Dont !e Frightened

An assertion need not consist o* *ormal

mathematical mmble >mble

Use an in*ormal description

and a pictre

5a#' A )Speci*ying a %omptational


79/356

G

5a#' A )

Mpre%ond

Inpt is array A81n9

o* n vales2N

i : 1

m : A819

loope#it -hen 'i:n)

i : i = 1

m : ma#'mA8i9)

endloopretrn'm)

Mpost%ond

m is ma# in A81n9N

4 PreconditionsAny assmptions that mst be

tre abot the inpt instance2

4Postconditionshe statement o* -hat mst

be tre -hen the

algorithm,program retrns2

Problem


80/356

@

De*inition o* %orrectness


81/356

1


82/356

.


83/356

0


84/356

3


85/356

6


86/356

' )

Mpre%ond

Inpt has 0 nmbers2N

m : aMassert m is ma# in 7aBN

i*' b?m)

m : b

endi*


i*' c?m)

m : c

endi*

Massert m is ma# in 7abcBNretrn'm)

Mpost%ond

retrn ma# in 7abcBN

E#ample


87/356

E#ample


88/356

E#ample


89/356

G

Mpre%ond

Inpt has 0 nmbers2N

m : aMassert m is ma# in 7aBN

i*' b?m)

m : b

endi*


i*' c?m)

m : c

endi*

Massert m is ma# in 7abcBNretrn'm)

Mpost%ond

retrn ma# in 7abcBN

E#ample


90/356

G@

Loop Invariants pre%ond Inpt is array A81n9

o* n vales2N

i : 1

m : A819

loop

e#it -hen 'i:n)

i : i = 1

m : ma#'mA8i9)endloop

retrn'm)

Mpost%ond

m is ma# in A 1n N

Any assmptions that

mst be tre abot the

state o* comptation

every time it is at the

top o* the loop2

Loop Invariants5a#' A )Mpre%ond


91/356

G1


o* n vales2N

i : 1

m : A819

loop

Mloopinvariant m is ma# in A81i9N

e#it -hen 'i:n)

i : i = 1


retrn'm)

Mpost%ond

m is ma# in A 1n N

Any assmptions that

mst be tre abot the

state o* comptation

every time it is at the

top o* the loop2

Loop Invariants5a#' A )Mpre%ond


92/356

G.

4 /ot >st to please the pro*2

4 /ot >st to prove correctness24 Use them to thin" abot yor

algorithmO

4 !e*ore yo start coding2

4 Qhat do yo -ant to betre in the middle o* yorcomptationCJ Wo needto "no-2

J Let yor reader "no-2


o* n vales2N

i : 1

m : A819

loop


e#it -hen 'i:n)

i : i = 1


retrn'm)

Mpost%ond

m is ma# in A 1n N



93/356

G0


94/356

G3


95/356

G6

g

Loop Invariantspre%ond

Inpt is array A81n9

o* n vales2N

i : 1

m : A819

loop


e#it -hen 'i:n)

i : i = 1


retrn'm)

Mpost%ond

m is ma# in A 1n N

Step @


96/356

G

Step 1


97/356

G

Last Step


98/356

G

Proves that IFthe program terminates then it -or"s


99/356

GG

g

Wo need to de*ine some

5easre o* progress

to prove that yor algorithmeventally terminates2


100/356

1@@

Simple E#ample

Insertion Sort Algorithm

+econsidering Simple E#amples


101/356

1@1

+econsidering Simple E#amples

A good martial arts stdent -ill attentively repeat eachA good martial arts stdent -ill attentively repeat each

*ndamental techniRe many times2*ndamental techniRe many times2

In contrast many college stdents tne ot -hen a conceptIn contrast many college stdents tne ot -hen a concept'e2g2 depth *irst search) is taght more than once2'e2g2 depth *irst search) is taght more than once2

he better stdents listen care*lly in order to re*ine andhe better stdents listen care*lly in order to re*ine and

develop their nderstanding2develop their nderstanding2

Repetition s !n Opportunity+dich ---2discretemath2com

%ode


102/356

1@.

%ode

+epresentation o* an Algorithm class InsertionSortAlgorithm e#tends SortAlgorithm 7 void sort'int a89) thro-s E#ception 7

*or 'int i : 1; i < a2length; i==) 7

int > : i;

int ! : a8i9;

-hile ''> ? @) && 'a8>19 ? !)) 7

a8>9 : a8>19;

>; B

a8>9 : !;

BB


103/356

1@0



G "m

6 "m

Designing an Algorithm*i bl *i *i *


104/356

1@3

De*ine Problem De*ine LoopInvariants




km

G "m

to school

E#it

E#it

G "m 6 "m

E#it

E#it

@ "m E#it

Problem Speci*ication


105/356

1@6

4Precondition he inpt is a list o* nvales

-ith the same vale possibly repeated2

4Post condition he otpt is a list consisting

o* the same nvales in nondecreasing order2

13

G.6.

6.

G

[email protected]

01 [email protected]



106/356

1@

4MStateN o* comptation determined

by vales o* all variables2



107/356

1@

13

G.6 .

6.

G

[email protected]


4MStateN o* comptation determined

by vales o* all variables2

13


108/356

1@

13

G.6 .

6.

G

[email protected]


.0

4Location too con*sing *or algorithm

4Algorithm too lay to de*ine step *orevery location2

13


109/356

1@G

4Algorithm speci*ies *rom -hich locations

it "no-s ho- to step2

13

G.6 .

6.

G

[email protected]


De*ine Loop Invariant


110/356

11@

13

G.6 .

6.

G

[email protected]


13

G.6 .

G

[email protected].

4Some sbset o* the elements are sorted

4he remaining elements are o** to the side2

Location /ot Determined


111/356

111

13

G.6 .

G

[email protected].

.0

G.6

01

[email protected]

Qhich sbset o* the elements are sorted is

not speci*ied2


112/356

11.

De*ining 5easre o* Progress

13

G

.6 .

G

[email protected].

elements

to school


113/356

110

De*ine Step

G.6 .

G

.0016.130@


114/356

113

De*ine Step

4 Select arbitrary element *rom side24 Insert it -here it belongs2

G.6 .

G

.0016.

13

G.6

G

[email protected]..

130@

5a"ing progress -hile

E#it


115/356

116

5aintaining the loop invariant

G

.6 .

G

.0016.

13

G.6

G

0@

.0016..

130@

G "m 6 "m

6 elements

to school

elements

to school

Sorted sblist

!eginning &Ending

km E#it@ "m E#it


116/356

11

88 14

98256

2

52

79

3023

31

8814

98256

2

52

79

3023

31nelements

to school

14,23,25,30,31,52,62,79,88,

14,23,25,30,31,52,62,79,88,98

@ elements

to school

Ending


117/356

11

+nning imeInserting an element into a list o* sie i

ta"es 'i) time2

otal : 1=.=0==n

G.6 .

G

.0016.

13

G.6

G

0@

.0016..

130@

: C


118/356

Adding 5ade Easy


119/356

11G

1 & 2 & 3 & " " " & n'1 & n = S

n & n'1 & n'2 & " " " & 2 & 1 = S

1 2 " " " " " " " " n

= num#er of ye((o$ dots

n " " " " " " " 2 1

= num#er of $%ite dots

Adding 5ade Easy


120/356

1.@

1 & 2 & 3 & " " " & n'1 & n = S

n & n'1 & n'2 & " " " & 2 & 1 = S

= num#er of $%ite dots

= num#er of ye((o$ dots

n&1 n&1 n&1 n&1 n&1

n

n

n

n

n

n

= n)n&1* dots in t%e

+rid

2Sdots

S:n'n=1)

.

Adding 5ade Easy


121/356

1.1

Adding 5ade Easy


122/356

1..

+ i i


123/356

1.0

+nning ime

Inserting an element into a list o* sie ita"es 'i) time2

otal : 1=.=0==n

G.6 .

G

.0016.

13

G.6

G

0@

.0016..

130@

: 'n.)

Algorithm De*inition %ompletedDe*ine Problem De*ine Loop De*ine 5easre o*


124/356

1.3

De*ine Problem De*ine LoopInvariants




km

G "m

to school

E#it

E#it

G "m 6 "m

E#it

E#it

@ "m E#it

Wet Another Abstract ie-


125/356

1.6

Wet Another Abstract ie-

4Wor >ob is only to

4rst -ho passes yo the baton

4$o arond once

4Pass on the baton

i1 i

ii

@=

1

A +elay +ace

Person @ %arries baton region to sa*e region

E t bli hi L I i t


126/356

1.


127/356

1.

i1 i


128/356

1.

=1


129/356

1.G

as a +elay +ace

13

G.6 .

6.

G

[email protected]

01

Establish Loop Invariant


130/356

10@

p

*rom preconditions

@

.0

13

G.6 .

6.

G

0@01

5 i t i i L I i t


131/356

101

5aintaining Loop Invariant

1

13

G.6 .

6.

G

[email protected]

01

5 i t i i L I i t


132/356

10.


i1 i

13

G.6 .

6.

G

[email protected]

01

5 i t i i L I i t


133/356

100


i

13

G.6 .

6.

G

[email protected]

01

5 i t i i L I i t


134/356

103


i1 i

13

G.6 .

6.

G

[email protected]

01

5aintaining Loop In ariant


135/356

106


13

G.6 .

6.

G

[email protected]

01

i



136/356

10


i1 i

13

G.

6.

G

[email protected]

.601



137/356

10


i

13

G.

6.

G

[email protected]

.601



138/356

10


i1 i

13

G.

G

[email protected]

.6016.



139/356

10G


[email protected]

%lean p loose ends


140/356

13@

%lean p loose ends

=1

[email protected]

T"


141/356

131

I "no- yo "ne- Insertion Sort

!t hope*lly yo are beginning to appreciate

Loop Invariants*or describing algorithms

E#plaining Insertion Sort


142/356

13.

Qe maintain a sbset o* elements sorted-ithin a list2 he remaining elements areo** to the side some-here2 Initiallythin" o* the *irst element in the array as

a sorted list o* length one2 Tne at a time-e ta"e one o* the elements that is o** tothe side and -e insert it into the sortedlist -here it belongs2 his gives a sorted

list that is one element longer than it -asbe*ore2 Qhen the last element has beeninserted the array is completely sorted2

E#plaining Selection Sort Qe maintain that the ksmallest o* the elements


143/356

130

are sorted in a list2 he larger elements are in a set

on the side2 Initially -ith k:@ all the elementsare in this set2 Progress is made by *inding thesmallest element in the remaining set o* largeelements and adding this selected element at theend o* the sorted list o* elements2 his increases kby one2 Stop -ith k:n2 At this point all theelements have been selected and the list is sorted2

13.0.60@01 ects


185/356

16

E#tra

p y >

Qe have read in the *irst iob>ects2

Qe -ill pretend that this pre*i#

is the entire inpt2Qe have a soltion *or this pre*i#

pls some additional in*ormation

Soltion

Loop Invariants andDeterministic Finite AtomatonL : 7 7@1B] ^ has length at most three and the

nmber o* 1s is odd B2


186/356

1

B

5ore o* the InptLoop Invariant

he inpt consistso* an array o* ob>ects


187/356

1

E#tra

p y >

Qe read in the i=1stob>ect2

Qe -ill pretend that this larger pre*i#

is the entire inpt2Qe e#tend the soltion -e have

to one *or this larger *or pre*i#2

' l dditi l i * ti )

Soltion

E#traSoltion




188/356

1

E#tra

p y >

Soltion

E#traSoltion

G "m 6 "m

E#it

ito i=1




189/356

1G

p y >

End the end -e have read in the entire inpt2

he LI gives s that -e have a soltion

*or this entire inpt2

Soltion

E#it

Insertion Sort

he inpt consistso* an array o* integers


190/356

1G@

p y g


Qe -ill pretend that this pre*i#

is the entire inpt2Qe have a soltion *or this pre*i#

Soltion

[email protected]

.0016.

Insertion Sort

he inpt consistso* an array o* integers


191/356

1G1

[email protected]

.0016.

Qe read in the i=1stob>ect2

Qe -ill pretend that this larger pre*i#

is the entire inpt2Qe e#tend the soltion -e have

to one *or this larger *or pre*i#2

.0.6016.




192/356

1G.

A stdent midterm ans-er

Each iteration considers the ne#t inpt item2

Loop Invariants are pictres

o* crrent state2/ot actionsO/ot algorithmsO




193/356

1G0


Qe have considered the ithinpt element2

Qhat abot elements 81222i9C




194/356

1G3


Qe have considered inpt elements 8122i92

SoC




195/356

1G6


Qe have a soltion *or each2

Soltion


Each ob>ect does not need a separate soltion

he inpt as a -hole needs a soltion2


196/356

Longest !loc" o* Tnes


197/356

1G

Alg reads the digits one at a time and remembers

enogh abot -hat read so that it does not need

to reread anything2 'i2e2 a DFA)

@@1@1111@@11@@@11111@@@@11@@1



198/356

1G

@@1@1111@@11@@@11

Qhen it has read this mch

-hat does it rememberC

4Largest bloc" so *ar2

111

+ead the ne#t character &redetermine the largest bloc" so *ar2



199/356

1GG

@@1@1111@@11@@@11

Qhen it has read this mch

-hat does it rememberC

4Largest bloc" so *ar2

111

+ead the ne#t character &redetermine the largest bloc" so *ar

& crrent largest2

4Sie o* crrent bloc"2

ypical ypes o* Loop Invariants


200/356

.@@

4 5ore o* the inpt

4 5ore o* the otpt

4 /arro-ed the search space4 %ase Analysis

4 Qor" Done

5ore o* the TtptLoop Invariant

he otpt consistso* an array o* ob>ects


201/356

.@1

I have prodced the *irst iob>ects2

Prodce the i=1stotpt ob>ect2

E#itG "m 6 "m

E#it

ito i=1 Done -hen

otpt nob>ects2

Selection Sort

he otpt consists o* an array o* [email protected]

Inpt


202/356

.@.

he otpt consistso* an array o* [email protected]

I have prodced the *irst iob>ects2

Prodce the i=1stotpt ob>ect2

E#it

Done -hen

otpt nob>ects2

ypical ypes o* Loop Invariants


203/356

.@0

4 5ore o* the inpt

4 5ore o* the otpt

4 /arro-ed the search space4 %ase Analysis

4 Qor" Donehree

!inary Search li"e

e#amples

De*ine Problem !inary Search


204/356

.@3

4 Pre%onditions

Jey .6

JSorted List

4 Post%onditions

JFind "ey in list 'i* there)2

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6



205/356

.@6

4 5aintain a sblist

4 Sch that

"ey .6

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6



206/356

.@

4 5aintain a sblist2

4 I* the "ey is contained in the original list

then the "ey is contained in the sblist2

"ey .6

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

De*ine Step


207/356

.@

4 5a"e Progress

4 5aintain Loop Invariant

"ey .6

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

De*ine Step


208/356

.@

4 %t sblist in hal*2

4 Determine -hich hal* "ey -old be in2

4 eep that hal*2"ey .6

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

De*ine Step


209/356

.@G


4 Determine -hich hal* the "ey -old be in2


0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

I* "ey \ mid

then "ey is in

le*t hal*2

I* "ey ? mid

then "ey is in

right hal*2

De*ine Step


210/356

.1@

4 It is *aster not to chec" i* the middle

element is the "ey2

4 Simply contine2

"ey 30

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

I* "ey \ mid

then "ey is in

le*t hal*2

I* "ey ? mid

then "ey is in

right hal*2

5a"e Progress G "m 6 "mE#it


211/356

.11

4 he sie o* the list becomes smaller2

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

G "m

6 "m

Initial %onditionskm


212/356

.1.

"ey .6

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

4 I* the "ey is contained inthe original list

then the "ey is containedin the sblist2

4 he sblist is theentire original list2

n "m

Ending AlgorithmE#it

"ey .6


213/356

.10

4 I* the "ey is contained in theoriginal list

then the "ey is contained in thesblist2

4 Sblist contains one element2

E#it

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

@ "m

4 I* the "ey iscontained in theoriginal list

then the "ey is atthis location2

"ey .6

I* "ey not in original list


214/356

.13

4 I* the "ey is contained inthe original list

then the "ey is contained

in the sblist2

4 Loop invariant treeven i* the "ey is notin the list2

4 I* the "ey iscontained in theoriginal list

then the "ey is atthis location2

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

"ey .3

4 %onclsion still solvesthe problem2

Simply chec" this onelocation *or the "ey2

+nning imehe sblist is o* sie n n,. n,3 n,1


215/356

.16

he sb list is o* sie n ,. ,3 ,1

Each step '1) time2

otal :'log n)

"ey .6

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

I* "ey \ mid

then "ey is in

le*t hal*2

I* "ey ? mid

then "ey is in

right hal*2

%ode


216/356

.1

Algorithm De*inition %ompletedDe*ine Problem De*ine Loop

Invariants


G "m


217/356

.1



km

to school

E#it

E#it

G "m 6 "m

E#it

E#it

@ "m E#it

Stdy


218/356

.1

4 5any e#perienced programmers -ere as"ed

to code p binary search2

@X got it -rong

$ood thing is -as not *or a

nclear po-er plant2

5a"e Precise De*initions


219/356

.1G

4 5aintain a sblist -ith end points i&j2

"ey .6

i j

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6



220/356

..@

4 5aintain a sblist -ith end points i&j2

"ey .6

i j

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

Does not matter -hich

bt yo need to be consistent2



221/356

..1

4 I* the sblist has even length

-hich element is ta"en to be midC

"ey .6

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

mid or mid C. .

i j i j+ + = =



222/356

...

4 I* the sblist has even length

-hich element is ta"en to be midC

"ey .6

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

Shold not matter

%hoose right2mid

.

i j+ =

mid

Common ,u+s


223/356

..0


4 Determine -hich hal* the "ey -old be in2


0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

I* "ey \ mid

then "ey is in

le*t hal*2

I* "ey ? mid

then "ey is in

right hal*2

Common ,u+s


224/356

..3

4 I* the middle element is the "ey

it can be s"ipped over2

"ey 30

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

I* "ey \ mid

then "ey is in

le*t hal*2

I* "ey ? mid

then "ey is in

right hal*2

E#it

Common ,u+s4


225/356

..6

Fi# the bg2

"ey 30

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

I* "ey \ mid

then "ey is in

le*t hal*2

I* "ey ? mid

then "ey is in

right hal*2

Common ,u+s4


226/356

..

Second *i#by ma"ing the le*t hal* slightly bigger2

"ey 30

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

I* "ey \ mid

then "ey is in

le*t hal*2

I* "ey ? mid

then "ey is in

right hal*2

4/e- bgC

Common ,u+s4

G "m 6 "m

E#it


227/356

..

Second *i#by ma"ing the le*t hal* slightly bigger2

"ey 30

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

4/e- bgC

0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6

/o progress is made2

Qhy is !inary Search so Easy to $et QrongC


228/356

..

C+.

T i j+

T+?C

mid.

i j+ =

1

i* "ey L'mid).

4 Ho- many

possible

algorithmsC

4Ho- manycorrect

algorithmsC

4 Probability o*

sccess bygessingC

0i: mid

else

j: mid 1

T+

i: mid = 1

else

j: mid

5oral


229/356

..G

4 Use the loop invariant method to thin"

abot algorithms2

4 !e care*l -ith yor de*initions2

4 !e sre that the loop invariant is al-ays

maintained2

4!e sre progress is al-ays made2


230/356

.0@

Loop Invariants*or


A second

!inary Search li"e

e#ample

De*ine Problem !inary Search ree4


231/356

.01

0

.6

1

3 .1

01

. 06

61

3.

3@ 3G

0

66 1

JA binary search tree2

J


232/356

.0.

0

.6

1

3 .1

01

. 06

61

3.

3@ 3G

0

66 1

\ \

De*ine Loop Invariant4 5aintain a sbtree2


233/356

.00

4 I* the "ey is contained in the original treethen the "ey is contained in the sbtree2

"ey 1

0

.6

1

3 .1

01

. 06

61

3.

3@ 3G

0

66 1

De*ine Step4 %t sbtree in hal*2

4


234/356

.03

Determine -hich hal* the "ey -old be in24 eep that hal*2

"ey 1

0

.6

1

3 .1

01

. 06

61

3.

3@ 3G

0

66 1

I* "ey < root

then "ey is

in le*t hal*2

I* "ey ? root

then "ey is

in right hal*2

I* "ey : root

then "ey is

*ond


Invariants


G "m


235/356

.06



km

to school

E#it

E#it

G "m 6 "m

E#it

E#it

@ "m E#it


236/356

.0

Loop Invariants*or


A third

!inary Search li"e

e#ample

%ard ric"


237/356

.0

4 A volnteer please2

Pic" a %ard


238/356

.0

Done

Loop Invariant

he selected card is one o*

these2


239/356

.0G

Qhich colmnC


240/356

.3@

le*t


241/356

Selected colmn is placed

in the middle2


242/356

.3.

I -ill rearrange the cards2


243/356

.30

+ela# Loop Invariant

I -ill remember the same

abot each colmn2


244/356

.33

Qhich colmnC


245/356

.36

right

Loop Invariant


these2


246/356

.3


in the middle2


247/356

.3

I -ill rearrange the cards2


248/356

.3

Qhich colmnC


249/356

.3G

le*t

Loop Invariant


these2


250/356

.6@


in the middle2


251/356

.61


252/356

he MPartitioningN Problem'sed in Ric"sort)


253/356

.60

13G.6.

6.

G

[email protected]

01

p:6.

13

.60@

.001

G

.G

\ 6. \

Inpt Ttpt


p 6.


254/356

.63

or

01 .0 .6 G 0@ 13 . G

p:6.

\ p p \

01 .0 1 .6 G 0@ . G

p:6.

\ p p \

De*ining 5easre o* Progress

p:6.


255/356

.66

3 elements

not

loo"ed at

01 .0 .6 G 0@ 13 . G

p:6.

De*ine Step

45 " P


256/356

.6

5a"e Progress4 5aintain Loop Invariant

01 .0 .6 G 0@ 13 . G

p:6.

\ p p \

De*ine Step

45 " P


257/356

.6

5a"e Progress4 5aintain Loop Invariant

01 .0 .6 G 0@ 13 . G

p:6.

\ p p \

01 .0 13 .6 G 0@ . G

p:6.

\ p p \

De*ine Step

01 .0 .6 G 0@ 13 . G

p:6.

For cases

01 .0 13 .6 G 0@ . G

p:6.


258/356

.6

01 .0 .6 G 0@ 13 . G

01 .0 13 .6 G 0@ . G

01 .0 13 .6 G 0@ . G

01 .0 13 .6 G 0@ . G

01 .0 .6 G 0@ 1 . G

p:6.

01 .0 .6 G 0@ 1 . G

01 .0 1 .6 G 0@ . G

p:6.

01 .0 .6 G 0@ 1 . G

13

.6 .

6.

0@01

!eginning &Ending

km E#it@ "m E#it

p:6.


259/356

.6G

G.6 .

G

[email protected]

01p:6.

.6 01 G . 13 0@ G .0

.0 0@ .6 01 13 . G G

p:6.

.6

0@01

13.0

.G

G

\ 6. \

n1 elements

not

loo"ed at

@ elements

not

loo"ed at

+nning imeEach iteration ta"es '1) time2


260/356

.@

otal : 'n)

01 .0 .6 G 0@ 13 . G

p:6.

\ p p \

01 .0 13 .6 G 0@ . G

p:6.

\ p p \


Invariants


G "m


261/356

.1



km

to school

E#it

E#it

G "m 6 "m

E#it

E#it

@ "m E#it

[email protected]

p:6. [email protected]

[email protected]

[email protected]


262/356

..

[email protected]

[email protected]

[email protected]

[email protected]

[email protected]

[email protected]

[email protected]

[email protected]

racing an

E#ample


263/356

$%D'ab)

Inpt


264/356

.3

: 3Ttpt $%D'ab)

5aintain vales


265/356

.6

:


266/356

.


267/356

.

Speci*ies ho- the rnning time depends on

the sie o* the inpt2

A *nction mapping

MsieN o* inpt

MtimeN 'n) e#ected 2

De*inition o* ime


268/356

.

4 _ o* seconds 'machine dependent)2

4 _ lines o* code e#ected2

4 _ o* times a speci*ic operation is per*ormed'e2g2 addition)2

QhichC

De*inition o* ime


269/356

.G

4 _ o* seconds 'machine dependent)2

4 _ lines o* code e#ected2

4 _ o* times a speci*ic operation is per*ormed'e2g2 addition)2

hese are all reasonable de*initions o* timebecase they are -ithin a constant *actor o*

each other2

Sie o* Inpt InstanceC

0G.@


270/356

.@

0G.@

Sie o* Inpt Instance

0G.@

6

1KK


271/356

.1

4 Sie o* paper4 _ o* bits

4 _ o* digits

4 ale

n : . in.

n : 1 bits

n : 6 digits

n : 0G.@

.KK

0G.@ 1

Qhich are reasonableC


0G.@

6

1KK


272/356

..


4 _ o* digits

4 ale

n : . in.

n : 1 bits

n : 6 digits

n : 0G.@

4 Intitive

.KK

0G.@ 1


0G.@

6

1KK


273/356

.0


4 _ o* digits

4 ale

n : . in.

n : 1 bits

n : 6 digits

n : 0G.@

4 Intitive4 Formal

.KK

0G.@ 1


0G.@

6

1KK


274/356

.3


4 _ o* digits

4 ale

n : . in.

n : 1 bits

n : 6 digits

n : 0G.@

4 Intitive4 Formal

4 +easonable

_ o* bits : 020. ] _ o* digits

.KK

0G.@ 1


0G.@

6

1KK


275/356

.6


4 _ o* digits

4 ale

n : . in.

n : 1 bits

n : 6 digits

n : 0G.@

4 Intitive4 Formal

4 +easonable

4 Unreasonable

_ o* bits : log.'ale)

ale : ._ o* bits

.KK

0G.@ 1

$%D'ab)

Inpt


276/356

.


277/356

.


278/356

.


279/356

.G

:


280/356

.@


Inpt A pile o* things to sort2

'E# E#ams)

Ttpt Sort them


281/356

.1

Ttpt Sort them+eRirements

4Easy to e#ecte by hmans4Tnly can have piles on des"4%an move one e#am 'or pile)

at a time24Fast J T'nlog'n)) time2

Li"ely the only algorithm

in this corse -hich yo -ill

e#ecte by hand yorsel*2


8A`9Inpt


282/356

..

Denotes an nsorted pile o* e#ams

-ith last names starting in the range 8A`9


8A`9


283/356

.0

Psychology stdy

Hmans can only thin" abot6 things at once2

Is the *irst letter

o* the nameon the *irst e#am

8AE9

8F98LT98P98U`9

in


8A`9


284/356

.3

8AE9 8F9 8LT9 8P9 8U`9

!c"et Sort4Pt e#ams one at a time

in the correct bc"et2


8AE98F9

8LT98P 9

89

t d


285/356

.6

8 98P98U`9

A stac" o* piles4each pile nsorted4all in one ile be*ore all in ne#t

A sorted pile

4be*ore all the rest

'pside do-n)

sorted

8AE98F9

8LT98P 9

89

t d



286/356

.

8 98P98U`9

sorted

8A9 8!9 8%9 8D9 8E9

!c"et Sort4Pt e#ams one at a time

in the correct bc"et2

89

sorted

8A98!98%98D9



287/356

.


A sorted pile


'pside do-n)

sorted

8F98LT9

8P98U`9

8 98E9

89

sorted

8A98!98%98D9



288/356

.

sorted8F98LT9

8P98U`9

8 98E9

8AAAE98AFA98ALAT98APA98AUA`9

89

sorted8!9


L

8AAAE9

8AUA`9

L


289/356

.G

sorted8F9

8U`9

8E9L

L


A sorted pile


'pside do-n)

89

sorted8!9


L

8AAAE9

8AUA`9

L


290/356

.G@

sorted8F9

8U`9

8E9L

L

8AAAE9

89

sorted8!9


L

8AFA9

8AUA`9

L


291/356

.G1

sorted8F9

8U`9

8E9L

L

8AAAE9

sorted

Qhen s**iciently small

sort by hand

8AAAE9

sorted8!9


L

8AFA9

8AUA`9

L


292/356

.G.

sorted8F9

8U`9

8E9L

L


A sorted pile


'pside do-n)

8AAAE9

sorted8!9


L

8AFA9

8AUA`9

L


293/356

.G0

sorted8F9

8U`9

8E9L

L

8AFA9

sorted

8AAA9

sorted8!9


L

8ALAT9

8AUA`9

L


294/356

.G3

sorted8F9

8U`9

8E9L

L


A sorted pile


'pside do-n)

8AAA9

sorted8!9


L

8ALAT9

8AUA`9

L


295/356

.G6

sorted8F9

8U`9

8E9L

L

8ALAT98APA9

8AUA`9sorted

sortedsorted

8F9

8!9

8E9


L

L8AAA`9

sorted


296/356

.G

8U`9

Lsorted

8F9

8!9

8E9


L

L

8A9

sorted


297/356

.G

8U`9

L


A sorted pile


'pside do-n)

sorted

8A9

8F9

8!9

8E9


L

L

sorted


298/356

.G

8U`9

L

8!A!E98!F!98!L!T9 8!P!98!U!`9

sorted

8A9

8%9


L

8!A!E9

8!U!`9

L

sorted


299/356

.GG

8F9

8U`9

8E9L

L


A sorted pile


'pside do-n)

sorted

8A`9


8`U``9

sorted


300/356

0@@

8`U``9

sorted

sorted

8A`9


sorted


301/356

0@1

E#it

sorted

Inpt A o* stac" o*Npnch cards2 Each card contains ddigits2

Each digit bet-een @&'k1)

+adi#Sort 0331.6

000

103..3

1.6..3

..6

0.6

000


302/356

0@.

Ttpt Sort the cards2 ..3

003

130

..6

0.6

.30

!c"et Sort 5achineSelect one digit

Separates cards into kpiles2

-rt selected digit2

000

103

003

033

130

.30

Stable sort I* t-o cards are the same *or that digit

their order does not change2

+adi#Sort

033

1.6000 Sort -rt -hich

1.6

103130 Sort -rt -hich

1.6

..3

..6


303/356

0@0

000

103

..3

003

130

..6

0.6.30

digit *irstC

he most

signi*icant2

130

..3

..6

.30

033

000

0030.6

digit SecondC

he ne#t most

signi*icant2

..6

0.6

103

000

003

130

.30033

All meaning in *irst sort lost2

+adi#Sort

033

1.6000 Sort -rt -hich Sort -rt -hich

000

130.30

..3

1.6..6


304/356

0@3

000

103

..3

003

130

..6

0.6.30

digit *irstC digit SecondC

he least

signi*icant2

.30

033

103

..3

003

1.6

..60.6

he ne#t least

signi*icant2

..6

0.6

000

103

003

130

.30033

+adi#Sort

033

1.6000 Sort -rt -hich Sort -rt -hich

000

130.30

. .3

1 .6. .6


305/356

0@6

000

103

..3

003

130

..6

0.6.30

digit *irstC digit SecondC

he least

signi*icant2

.30

033

103

..3

003

1.6

..60.6

he ne#t least

signi*icant2

. .6

0 .6

0 00

1 03

0 03

1 30

. 300 33

Is sorted -rt *irst idigits2

. .3

1 .6. .6 Is sorted -rt

1 .6

1 031 30 Is sorted -rt

+adi#Sort


306/356

0@

Sort -rt i=1stdigit2

. .6

0 .6

0 00

1 03

0 03

1 30

. 300 33

Is sorted -rt

*irst idigits2

1 30

. .3

. .6

. 30

0 .6

0 000 03

0 33

Is sorted -rt

*irst i=1 digits2

i=1

hese are in the

correct order

becase sorted

-rt high order digit2

. .3

1 .6. .6 Is sorted -rt

1 .6

1 031 30 Is sorted -rt

+adi#Sort


307/356

0@

Sort -rt i=1stdigit2

. .6

0 .6

0 00

1 03

0 03

1 30

. 300 33

Is sorted -rt

*irst idigits2

1 30

. .3

. .6

. 30

0 .6

0 000 03

0 33i=1

hese are in the

correct order

becase -as sorted &

stable sort le*t sorted2

Is sorted -rt

*irst i=1 digits2

%ontingSortInpt

Ttpt @ @ @ @ @ 1 1 1 1 1 1 1 1 1 . . 0 00

1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @


308/356

0@

InptNrecords each labeled -ith a digit

bet-een @&'k1)2

Ttpt Stable sort the nmbers2

Algorithm %ont to determine -here records go2

$o throgh the records in order ptting them -here they go2

%ontingSortInpt

Ttpt

1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ @ @ @ @ 1 1 1 1 1 1 1 1 1 . . . 00

Inde# [email protected]@ 1. 10 13 16 1 1 1


309/356

0@G

Stable sort I* t-o records are the same *or that digit

their order does not change2here*ore the 3threcord in inpt -ith digit 1 mst be

the 3th record in otpt -ith digit 12

It belongs in otpt inde# becase records go be*ore itie 6 records -ith a smaller digit &

0 records -ith the same digit

%ont

hese

%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @


310/356

01@

ale v_ o* records -ith digit v 0.1@ .0G6

Nrecords kdi**erent vales2 ime to contC'/)

'/ ")

Qe have conted _ o* each vale

in the *irst ivales2

%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @


311/356

011

ale v_ o* records -ith digit v

_ o* records -ith digit < v

[email protected]

1136@

Nrecords kdi**erent vales2 ime to contC 'k.)

oo mch

%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @


312/356

01.

ale v_ o* records -ith digit v

_ o* records -ith digit < v

[email protected]

1136@

%ompted

Nrecords kdi**erent vales2 ime to contC 'k)

%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ @ @ @ @ 1 1 1 1 1 1 1 1 1 . . . 00


313/356

010

ale v_ o* records -ith digit < v 0.1@ 1136@Location o* *irst record

-ith digit v2

%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

1@ C


314/356

013

ale v 0.1@1136@Location o* *irst record

-ith digit v2

Algorithm $o throgh the records in order ptting them -here they go2

%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

1


315/356

016

ale v 0.1@1136@Location o* ne#trecord

-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 1


316/356

01

ale v 0.1@113@Location o* ne#trecord

-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ @ 1


317/356

01

ale v0.1@1131Location o* ne#trecord

-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 11@


318/356

01

ale v0.1@ 113.Location o* ne#trecord

-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 11@ 0


319/356

01G


-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 11@ 1 0


320/356

0.@


-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 11@ 1 01


321/356

0.1


-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 11@ 001 1


322/356

0..

ale v0.1@ 113G.Location o* ne#trecord

-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 11@ 1 01 1 0


323/356

0.0

ale v0.1@ 1G13G.Location o* ne#trecord

-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 11@ @ 1 1 1 0 0


324/356

0.3

ale v0.1@ [email protected] o* ne#trecord

-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 11@ .1 1 1 0 0@


325/356

0.6

ale v 0.1@ 1G131@0Location o* ne#trecord

-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 11@ 11 1 1 0 0@ .


326/356

0.


-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 11@ 11 1 1 0 0@ .


327/356

0.


-ith digit v2


%ontingSortInpt

Ttpt


1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @

@ 11@ 11 1 1 0 0@ .@ @ 1 1 1 . .


328/356

0.

ale v 0.1@ 1G1136Location o* ne#trecord

-ith digit v2


'/)ime : '%onting Tps)

'log / = log ") 'bit ops)'/=")otal :

+adi#,%onting SortInpt Nnmbers eachLbits long2

Ttpt Sort the nmbers2

1111@111@1@@1@1@@@1@1

N 1@1@@1@1@1@@@1@11@111


329/356

0.G

L

N 1@1@@1@1@1@@@1@11@11111@1@@@11@1@@1111@1@1

111 1@1 11@ 1@@ 1@1 @@@ 1@11@1 @@1 @1@ 1@@ @1@ 11@ 111

11@ 1@@ @11 @1@ @11 11@ 1@1

d:L , lo k

N

: log k 'k-ill be set toN)

+adi#,%onting SortInpt Nnmbers eachLbits long2

Each card contains ddigits2

Each digit bet-een @&'k1)Ttpt Sort the nmbers2


330/356

00@

111 1@1 11@ 1@@ 1@1 @@@ 1@11@1 @@1 @1@ 1@@ @1@ 11@ 11111@ 1@@ @11 @1@ @11 11@ 1@1

d:L , log k

N

: log k

digit bet-een @&'k1)

sing %onting Sort2

p

Use +adi# Sort Sort -rt each digit

+adi#,%onting Sort


331/356

001

+adi#,%onting Sort

ime :: '_ o* digits) 'ime o* %onting Sort)

'ime o* +adi# Sort)


332/356

00.

' g ) ' g )

: L,log k 'N=k) 'logN= log k) bit ops

Set kto minimie time2

Qants kbig2 +eally -ants ksmall bt does not

care as long as kN2

Set k:N

+adi#,%onting Sort

ime :: '_ o* digits) 'ime o* %onting Sort)

'ime o* +adi# Sort)


333/356

000

' g ) ' g )

: L,log k 'N=k) 'logN= log k) bit ops

: L,logN N logN bit ops

: 'LN) bit ops

Sie : Sie o* Nnmbers eachLbits long2

: 'LN) : n

: 'n) bit ops

MLinearN ime,ut sortin+ s%ou(d t/ke )N(o+ N* time...

+adi#,%onting Sortime : 'LN) bit ops

Sie : Sie o* Nnmbers eachLbits long2: 'LN)

'NlogN) bit ops


334/356

003

5erge or (ic" Sortime : 'NlogN) comparisons

Sie : Sie o* Nnmbers each arbitrarily long2: 'NlogN) bits2

L:? logN i* yo -antNdistinct nmbers2

: 'n)

5erge (ic" and Heap Sort can sort/nmbers

sing T'/ log /)comparisons bet-een the vales2

A Lo-er !ond on Sorting


335/356

006

heorem /o algorithm can sort *aster2

A I 8 A'I) =P'I) or ime'AI) lo-er'^I^)9

I have an algorithm Athat Iclaim -or"s and is *ast2



336/356

00

I -in i* Aon inpt Igives4the -rong otpt or

4rns slo-2

claim -or"s and is *ast2

Th yeah I have an inpt I

*or -hich it does not 2

4 Sorting

J Inpt

J Ttpt

630.1

11100.103.0ales

Inde#es

3.013303.0.110110ales

Inde#es



337/356

00

4 Lo-er !ond

J Inpt An Algorithm *or Sorting A

J Ttpt An instance I

on -hich alg Aeither

4 a"es too mch time

4 Tr give the -rong ans-er

630.1

11100.103.0ales

Inde#es

3.013

303.0.110011ales

Inde#es

Algorithm De*inition %ompletedDe*ine Problem De*ine LoopInvariants


D *i St D *i E it % diti 5 i t i L I

G "m

to school


338/356

00



km E#it

E#it

G "m 6 "m

E#it

E#it

@ "m E#it

I give yo algorithm A

I claim it sorts2

I mst otpt an instance

630.1

11100.103.0alesInde#es



339/356

00G

on -hich alg Aeither4a"es too mch time

4Tr give the -rongans-er

It might as -ell be a

permtation o* 122/

'inde#es not sho-n)01.63


I claim it sorts2A Lo-er !ond on Sorting

/eed to "no-

-hat the algorithm does

be*ore -e can "no-

-hat inpt to give it


340/356

03@

-hat inpt to give it2

/eed to give the

algorithm an inpt

be*ore -e can "no-

-hat the algorithm does inpt2

!rea" this cycleone iteration at a time2


+estrict the search space


341/356

031

1 . 0 3 6

I maintain a set o* instances

'permtations o* 122n)



342/356

03.

1 . 0 3 6

0 . 6 1 3

3 6 . 1 0

6 3 . 0 1

Th dearO

he *irst t time steps o* my

algorithm Aare the same given

any o* these instancesOOO

Initially I consider all /O

permtations o* 122/


1 . 0 3 6


343/356

030

Th dearO

he *irst @ time steps o* my

algorithm Aare the same given


1 . 0 3 6

0 . 6 1 3

3 6 . 1 0

6 3 . 0 1

Initially I consider all /O

permtations o* 122/


1 . 0 3 6


344/356

033

he measre o* progressis the nmber o* instances2

Initially there are /O o* them2

G "m km

1 . 0 3 6

0 . 6 1 3

3 6 . 1 0

6 3 . 0 1

1 . 0 3 6

5yt=1sttime step

a< a.

6th bit o* a = a.



345/356

036

1 . 0 3 6

0 . 6 1 3

3 6 . 1 0

6 3 . 0 1

!ecase the *irst t time steps

are the same -hat the alg

"no-s is the same2

Hence its t=1ststep is the same2

6 bit o* a= a.Any yes,no

Restion

1 . 0 3 6

I partition my instances based

on -hat they do on this t=1st

step25yt=1sttime step

a< a.

6th bit o* a = a.



346/356

03

1 . 0 3 6

0 . 6 1 3

3 6 . 1 0

6 3 . 0 1

!ecase the *irst t time steps

are the same -hat the alg

"no-s is the same2

Hence its t=1ststep is the same2

6 bit o* a= a.Any yes,no

Restion

1 . 0 3 6

I "eep the larger hal*2



347/356

03

1 . 0 3 6

0 . 6 1 3

3 6 . 1 0

6 3 . 0 1

1 . 0 3 6




348/356

03

1 . 0 3 6

0 . 6 1 3

Th dearO

he *irst t=1 time steps o* myalgorithm Aare the same given


1 . 0 3 6




349/356

03G

0 6

0 . 6 1 3

G "m

Initially I have nO Permtations2

A*ter t time steps I have /O,.t

1 . 0 3 6




350/356

06@

0 . 6 1 3

Qe e#it -hen there are t-o

instances2E#it

Initially I have nO Permtations2

A*ter t time steps I have /O,.t

: log'/O)


N/ *actors each at least N/


351/356

061


instances2E#it : log'/O)

/O : 1 . 0 N/2 N

/ *actors each at most /2

/2*actors each at least /

22

! //

: '/ log'/))2


I claim it sorts2A Lo-er !ond on Sorting

E#it



630.1

11100.103.0ales

Inde#es


352/356

06.

a"es too mch time

4Trgive the -rongans-er

A Lo-er !ond on SortingE#it



630.1

11100.103.0ales

Inde#es

TopsO


353/356

060

%ase 1 he algorithm does notstop at time on these t-o

instances2

E#it : log'/O) : '/ log'/))2

a"es too mch time


TopsO

I mst give

the -rong

ans-er on oneo* theseO



630.1

11100.103.0ales

Inde#es

A Lo-er !ond on SortingE#it


354/356

063

a"es too mch time


%ase . he algorithm stops at time and gives an ans-er2


instancesE#itand these need di**erent ans-ers2

he *irst time steps o* alg Aare

the same on these t-o instances

and hence the same ans-er is

Theorem: For every sorting algorithmA

on the orst "ase in#$t instan"eI%Nlog&N'"om#arisons %or other bit o#erations'

d b d N b



355/356

066

need to be exe"$ted to sortNobje"ts(

A I A%I' =)%I' or Time%AI' N log&N

)roo*: )rover+Adversary ,ame

End Iterative Algorithms


356/356

End Iterative Algorithms

5ath +evie-
http://01.5-short_math.ppt/http://01.5-short_math.ppt/

Date post:	26-Feb-2018
Category:	Documents
Upload:	katerina-dimitrakopoulou
View:	222 times
Download:	0 times

01 Iterative methods

Documents