Complexity and Big-O Notation
Example N! = # ways to put N objects in linear order.
Generating all such orderings (permutations) on N objects takes at least N! time.
Example Adding two n-bit integers at most 2n bit operations.
Time estimate for two methods
sec 5 yrs 103 1000
sec 105 yrs 103 100
sec 105 sec 10 10
GCDEuclidean GCD Naïven
opsbit 5n :sEuclid'
opsbit 10least at : GCD Naïve
10ba,
digits#n
sec 10 takesopbit one Assume
983
3-83
6-
3
n
n
-9
Time complexity
n} size ofinput an is I :max{T(I)t(n)
case-worst
n size of inputs allon timeave. :complexity timecase-average
n size of inputs allon max time :complexity timecase-worst
n size of inputs allon min time :complxity timecase-best
I oflength : |I|
I.input on executed (ops) nsinstructio# :T(I)
A input toan :I
algorithman :A
Order of a function
factors)cosntant (Ignoringorder of is
10 like grows )(
10 71010 10
10 100710 10
10 10710 100
10 1710 10
10
10 :large is when ermDominant t
710)(
2
2
201100201100
773
55
33
2
2
2
nt(n)
nnt
n t(n) n
nn
nnnt
Definitions
))(()( and ))(()( if
)( ))(()(
.many finitely but allfor |)(||)(|
s.t constant some exists thereif
)( ))(()(
.many finitely but allfor |)(||)(|
s.t. constant some exists thereif
)( ))(()(
}{0,1,2,... numbers naturalon function a :)(
ngnfngOnf
n) order g(f(n) is ofngnf
nngcnf
c
least g(n) order at f(n) is ofngnf
nngcnf
c
most g(n) order at f(n) is ofngOnf
nf
Example
)(376
0 if )1(376
)(376
1 if 21 )(7
)1(7376
)(376
33
333
33
3333
33
33
nnn
nnnnnn
nnn
nnnnn
nnnn
nOnn
Example
induction.by Prove
)2(
. if )( general,In
. allalmost for 0)(
0,constantany for :)(23
3
n
ji
On
jinOn
ncnncnn
cnOn
Theorem
)()(Then
.0 where
),0( degree of polynomial a be
...)(
Let
01
1
k
i
k
kk
kk
nnf
a
ak
anananf
Proof
)()( Hence,
....
thatobserve ),()( prove To
).()( So
)...(
......
,1Then when ....Let
01
1
0
0101
1
0
k
kk
kk
kk
k
k
kkk
kkk
kk
kk
kk
k
nnf
naanana
nnf
nOnf
cnnaa
nananaanana
naac
Example
.log
that,sidesboth on log by taking followsit
,increasing are functionsboth that and 2 From
)(log
2 nn
n
nOn
n
Big-O is transitive
e. transitivare (.) and (.) Similarly,
allalmost for
thatfollowsIt
allalmost for |
andallalmost for |
thatsoandconstantsexist Then there
andSuppose
n.cd|h(n)||f(n)|
n.d|h(n)| |g(n)
n, c|g(n)| |f(n)
d c
O(h(n))g(n)O(g(n)) f(n)
Corollary
. sinceion contradictA
. ivity thatby transit followsIt
. and
)),(( ),( Now
. and )( sinceBut
)).(( Otherwise :Pr
)).((Then .with and
degree of spolynomial are andSuppose
md
)O(nn
)O(nf(n)
nfOg(n)g(n)On
)O(nf(n)g(n)On
nfOg(n)oof
nfOg(n)dmd m
g(n) f(n)
md
m
d
md
Some common functions
n
nn
log
sPolynomial
2
!
Decreasing order
Example
).log(!log Hence
.1 if ,loglog..loglog
1log2log...)1log(log!log
:Pr
)log(!log
nnOn
nnnnnn
nnn
oof
nnn
).log(!log Therefore,
log2
1
)1)(log2/(2/log2/
1log2log...)1log(log!log
Hence
.2/log...)1log(log
hand,other On the
nnn
nn
nnnn
nnn
nnn
Example
).(3!log Therefore,
).(3!log Therefore,
.)2(3
3log3!log
that followsit ,log Since
. allalmost for log!log
,constant a is therehence ),log(!log
).(3!log so ,3!log
Pr
)(3!log
22
22
222
22
2222
22
nnn
nOnn
nccnn
ncnnnn
nn
nncnn
cnnOn
nnnnnn
oof
nnn
Example
)(
......21executed is 3 line times#
.1: 3.
do to1:for 2.
do to1:for .1
2n
ni
xx
ij
ni
Example
end .7
;2
: 6.
;1: 5.
do to1:for 4.
begin 3.
do 1 while.2
: .1
jj
xx
ji
j
nj
)()(
)()(
2...)4
1
2
11(
1...42
)(
)()()(
executed is 5 line times#)(Let
nnt
nOnt
nn
nnnnt
nntnnt
nt
Select a theta notation
)log(
6loglog)18.(3
)(
90)12.(2
)(
1000672.1
2
2
3
3
3
3
nn
nnnn
n
n
n
nn
Fibonacci sequence as the worst case for gcd
01.22
123
235
358
5813
13 8, 5, 3, 2, 1, 1,
3,,1
divisions? ofnumber given a takespair that Smallest
2121
nfffff nnn
Comparing a gcd computation with Fibonacci seq.
62=2.23+16
23=16+7
16=2.7+2
7=3.2+1
2=2.1+0
13=8+5
8=5+3
5=3+2
3=2+1
2=2.1+0
.,
then divisons requires ),Gcd( If
1 nn fbfa
nba
Theorem
. So,
. , induction,By
divisions. 1 requires ),Gcd( So
.0 with ,Then
.1 suppose :step Inductive
.1,1:1 Basis
.on induction by :Pr
., then divisons requires ),Gcd( If
11
11)1(
121
1
nnn
nnn
nn
nn
fffrbrqba
frffb
nrb
brrqba
n
bffaffn
noof
fbfanba
.
)1(
Then
.that induction by Suppose
3:2
.2:1
on induction by Pr
.1 if ,Then
) 2
51( 0.1-x-
equation theroot to positive thebe Let
121
11
1232)1(
2
241
31
2
2
nn
nnn
nnnn
nn
n
n
nn
ffff
f
ffn
ffn
noof
nf
x
Corollary
ba
a
an
fa
fbfanban
n
nn
,in digits5#divisons#
208.0log,2
51
log1 So,
., then divisons requires ),Gcd( If1
1
1
