8/9/2019 Amortized 4up
1/12
Princeton University COS 423 Theory of Algorithms Spring 2001 Kevin Wayne
Amortized Analysis
2
Beyond Worst Case Analysis
Worst-case analysis.
s Analyze running time as function of worst input of a given size.
Average case analysis.
s Analyze average running time over some distribution of inputs.
s Ex: quicksort.
Amortized analysis.
s Worst-case bound on sequence of operations.
s Ex: splay trees, union-find.
Competitive analysis.s Make quantitative statements about online algorithms.
s Ex: paging, load balancing.
3
Amortized Analysis
Amortized analysis.
s Worst-case bound on sequence of operations.
no probability involved
s Ex: union-find.
sequence of m union and find operations starting with n
singleton sets takes O((m+n) (n)) time.single union or find operation might be expensive, but only (n)
on average
4
Dynamic Table
Dynamic tables.
s Store items in a table (e.g., for open-address hash table, heap).
s Items are inserted and deleted.
too many items inserted copy all items to larger tabletoo many items deleted copy all items to smaller table
Amortized analysis.
s Any sequence of n insert / delete operations take O(n) time.
s Space used is proportional to space required.
s Note: actual cost of a single insert / delete can be proportional to n
if it triggers a table expansion or contraction.
Bottleneck operation.
s We count insertions (or re-insertions) and deletions.
s Overhead of memory management is dominated by (or
proportional to) cost of transferring items.
8/9/2019 Amortized 4up
2/12
5
Aggregate method.
s Sequence of n insert ops takes O(n) time.
s Let ci = cost of ith insert.
Initialize table size m = 1.
INSERT(x)
IF (number of elements in table = m)Generate new table of size 2m.
Re-insert m old elements into new table.
m 2m
Insert x into table.
Dynamic Table Insert
Dynamic Table: Insert
=otherwise1
2ofpowerexactanis1if -iici n
nn
ncn
j
jn
ii
3
)12(
22
log
01
x} .
Implementing Find(x, S).s Call Splay(x, S).
s If x is root, then return x; otherwise return NO.
18
Splay
Implementing Join(S, S).
s Call Splay(+, S) so that largest element of S is at root and allother elements are in left subtree.
s Make S the right subtree of the root of S.
Implementing Delete(x, S).
s Call Splay(x, S) to bring x to the root if it is there.
s Remove x: let S and S be the resulting subtrees.
s Call Join(S, S).
Implementing Insert(x, S).
s
Call Splay(x, S) and break tree at root to form S and S.s Call Join(Join(S, {x}), S).
19
Implementing Splay(x, S)
Splay(x, S): do following operations until x is root.
s ZIG: If x has a parent but no grandparent, then rotate(x).
s ZIG-ZIG: If x has a parent y and a grandparent, and if both x and y
are either both left children or both right children.
s ZIG-ZAG: If x has a parent y and a grandparent, and if one of x, y
is a left child and the other is a right child.
A B
xC
y
CB
y
A
x
ZIG(x)
ZAG(y)
root
20
Implementing Splay(x, S)
Splay(x, S): do following operations until x is root.
s ZIG: If x has a parent but no grandparent.
s ZIG-ZIG: If x has a parent y and a grandparent, and if both x and y
are either both left children or both right children.
s ZIG-ZAG: If x has a parent y and a grandparent, and if one of x, y
is a left child and the other is a right child.
ZIG-ZIG
A B
x
C
y
D
z
DC
z
B
y
A
x
8/9/2019 Amortized 4up
6/12
21
Implementing Splay(x, S)
Splay(x, S): do following operations until x is root.
s ZIG: If x has a parent but no grandparent.
s ZIG-ZIG: If x has a parent y and a grandparent, and if both x and y
are either both left children or both right children.
s ZIG-ZAG: If x has a parent y and a grandparent, and if one of x, y
is a left child and the other is a right child.
ZIG-ZAG
B C
x
D
y
z
DC
y
x
A
BA
z
22
Splay Example
Apply Splay(1, S) to tree S:10
9
8
7
6
5
4
3
2
1
ZIG-ZIG
23
Splay Example
Apply Splay(1, S) to tree S:
ZIG-ZIG
10
9
8
7
6
5
4
1
2
3
24
Splay Example
Apply Splay(1, S) to tree S:
ZIG-ZIG
10
9
8
7
6
1
2
3
4
5
8/9/2019 Amortized 4up
7/12
8/9/2019 Amortized 4up
8/12
29
Splay Tree Analysis
Definitions.
s Let S(x) denote subtree of S rooted at x.
s |S| = number of nodes in tree S.
s (S) = rank = log |S| .
s (x) = (S(x)). 2
8
4
63
10
1
9
5 7
|S| = 10(2) = 3(8) = 3(4) = 2
(6) = 1(5) = 0
S(8)
30
Splay Tree Analysis
Splay invariant: node x always has at least (x) credits on deposit.
Splay lemma: each splay(x, S) operation requires 3((S) - (x)) + 1credits to perform the splay operation and maintain the invariant.
Theorem: A sequence of m operations involving n inserts takesO(m log n) time.
Proof:
s (x) log n at most 3 log n + 1 credits are needed foreach splay operation.
s Find, insert, delete, join all take constant number of splays plus
low-level operations (pointer manipulations, comparisons).
s
Inserting x requires log n credits to be deposited to maintaininvariant for new node x.
s Joining two trees requires log n credits to be deposited tomaintain invariant for new root.
31
Splay Tree Analysis
Splay invariant: node x always has at least (x) credits on deposit.
Splay lemma: each splay(x, S) operation requires 3((S) - (x)) + 1credits to perform the splay operation and maintain the invariant.
Proof of splay lemma: Let (x) and (x) be rank before and singleZIG, ZIG-ZIG, or ZIG-ZAG operation on tree S.
s We show invariant is maintained (after paying for low-level
operations) using at most:
3((S) - (x)) + 1 credits for each ZIG operation. 3((x) - (x)) credits for each ZIG-ZIG operation. 3((x) - (x)) credits for each ZIG-ZAG operation.
s Thus, if a sequence of of these are done to move x up the tree, we
get a telescoping sum total credits 3((S) - (x)) + 1.
32
Splay Tree Analysis
Proof of splay lemma (ZIG): It takes 3((S) - (x)) + 1 credits toperform a ZIG operation and maintain the splay invariant.
s In order to maintain invariant, we must pay:
s Use extra credit to pay for
low-level operations.))()((3
))()((3
)()(
)()()()()()(
xS
xx
xx
xyyxyx
=
=+
A B
x
C
y
CB
y
A
xS S
(y) = (x)
ZIG
root
(x) = (S)
8/9/2019 Amortized 4up
9/12
33
Splay Tree Analysis
Proof of splay lemma (ZIG-ZIG): It takes 3((x) - (x)) credits toperform a ZIG-ZIG operation and maintain the splay invariant.
s If (x) > (x), then can afford topay for constant number of low-level
operations and maintain invariant using 3((x) - (x)) credits.
))()((2
))()(())()((
))()(())()((
)()()()()()()()()()(
xx
xxxx
yzxy
yxzyzyxzyx
=
+
+=
+=++
S
A B
xC
yD
z
DC
zB
yA
x S
ZIG-ZIG
34
Splay Tree Analysis
Proof of splay lemma (ZIG-ZIG): It takes 3((x) - (x)) credits toperform a ZIG-ZIG operation and maintain the splay invariant.
s Nasty case: (x) = (x).
s We show in this case (x) + (y) + (z) < (x) + (y) + (z).dont need any credit to pay for invariant
1 credit left to pay for low-level operations
so, for contradiction, suppose (x) + (y) + (z) (x) + (y) + (z).
s Since (x) = (x) = (z), by monotonicity (x) = (y) = (z).
s After some algebra, it follows that (x) = (z) = (z).
s Let a = 1 + |A| + |B|, b = 1 + |C| + |D|, then
log a = log b = log (a+b+1)
s WLOG assume b a.S
A B
xC
yD
z
DC
zB
yA
xS
a
a
aba
log
log1
)2log()1log(
>
+=
++
ZIG-ZIG
35
Splay Tree Analysis
Proof of splay lemma (ZIG-ZAG): It takes 3((x) - (x)) credits toperform a ZIG-ZAG operation and maintain the splay invariant.
s Argument similar to ZIG-ZIG.
ZIG-ZAG
B C
xD
y
z
DC
y
x
A
BA
z
Princeton University COS 423 Theory of Algorithms Spring 2001 Kevin Wayne
Augmented Search Trees
8/9/2019 Amortized 4up
10/12
37
Support following operations.
Interval-Insert(i, S): Insert interval i = (li, ri ) into tree S.
Interval-Delete(i, S): Delete interval i = (li, ri ) from tree S.
Interval-Find(i, S): Return an interval x that overlaps i, or
report that no such interval exists.
Interval Trees
(7, 10)
(5, 11)
(4, 8) (21, 23)
(15, 18)
(17, 19)
38
(4, 8)
Key ideas:
s Tree nodes contain interval.
s BST keyed on left endpoint.
Interval Trees
(7, 10)
(5, 11)
(4, 8) (21, 23)
(15, 18)
(17, 19)
(17, 19)
Key Interval
(5, 11) (21, 23)
(15, 18)
(7, 10)
39
(4, 8)
Key ideas:
s Tree nodes contain interval.
s BST keyed on left endpoint.
s Additional info: store max
endpoint in subtree rooted
at node.
Interval Trees
(7, 10)
(5, 11)
(4, 8) (21, 23)
(15, 18)
(17, 19)
(17, 19)
max insubtree
(5, 11) 18 (21, 23) 23
8 (15, 18) 18
(7, 10) 10
23
40
x root(S)
WHILE (x != NULL)
IF (x overlaps i)
RETURN t
IF (left[x] = NULL OR
max[left[x]] < li)
x right[x]ELSE
x left[x]RETURNNO
Splay last node on path
traversed.
Interval-Find (i, S)
Finding an Overlapping Interval
Interval-Find(i, S): return an interval x that overlaps i = (li, ri ), orreport that no such interval exists.
(4, 8)
(17, 19)
(5, 11) 18 (21, 23) 23
8 (15, 18) 18
(7, 10) 10
23
8/9/2019 Amortized 4up
11/12
41
Finding an Overlapping Interval
Interval-Find(i, S): return an interval x that overlaps i = (li, ri ), orreport that no such interval exists.
Case 1 (right). If search goes right,
then there exists an overlap in rightsubtree or no overlap in either.
Proof. Suppose no overlap in right.
s left[x] = NULL no overlap in left.
smax[left[x]] < li no overlap in left.
x root(S)
WHILE (x != NULL)
IF (x overlaps i)
RETURN t
IF (left[x] = NULL OR
max[left[x]] < li)
x right[x]ELSE
x left[x]RETURNNO
Splay last node on path
traversed.
Interval-Find (i, S)
i = (li, ri )left[x]
max
42
Interval-Find(i, S): return an interval x that overlaps i = (li, ri ), orreport that no such interval exists.
Case 2 (left). If search goes left,
then there exists an overlap in leftsubtree or no overlap in either.
Proof. Suppose no overlap in left.
s li max[left[x]] = rj forsome interval j in left subtree.
s Since i and j dont overlap, we have
li ri lj rj.
s
Tree sorted byl
for any intervalk in right subtree: ri lj lk no overlap in right subtree.
Finding an Overlapping Interval
x root(S)
WHILE (x != NULL)
IF (x overlaps i)
RETURN x
IF (left[x] = NULL OR
max[left[x]] < li)
x right[x]ELSE
x left[x]RETURNNO
Splay last node on path
traversed.
Interval-Find (i, S)
i = (li, ri ) j = (lj, rj)
k = (lk, rk)
43
Interval Trees: Running Time
Need to maintain augmented data structure during tree-modifying ops.
s Rotate: can fix sizes in O(1) time by looking at children:
A14
B19
C
30 ZIG
ZAG
(11, 35) 35
(6, 20) 20
C30
B19
A
14
(6, 20) ?
(11, 35) ?
35
35
=
xr
xright
xleft
x ]][max[
]][max[
max]max[
44
VLSI Database Problem
VLSI database problem.
s Input: integrated circuit represented as a list of rectangles.
s Goal: decide whether any two rectangles overlap.
Algorithm idea.
s Move a vertical "sweep line" from left to right.
s Store set of rectangles that intersect the sweep line in an interval
search tree (using y interval of rectangle).
8/9/2019 Amortized 4up
12/12
45
Sort rectangle by x coordinate (keep two copies of
rectangle, one for left endpoint and one for right).
FORi = 1 to 2N
IF (ri is "left" copy of rectangle)
IF (Interval-Find(ri, S))
RETURN YES
ELSE
Interval-Insert(ri, S)
ELSE (ri is "right" copy of rectangle)
Interval-Delete(ri
, S)
VLSI (r1, r2 ,..., rN)
VLSI Database Problem
46
Order Statistic Trees
Add following two operations to BST.
Select(i, S): Return ith smallest key in tree S.
Rank(i, S): Return rank of x in linear order of tree S.
Key idea: store size of subtrees in nodes.
m 8
c 5 P 2
b 1 f 2
d 1 h 1
q 1
Key Subtree size
47
Order Statistic Trees
Need to ensure augmented data structure can be maintained duringtree-modifying ops.
s Rotate: can fix sizes in O(1) time by looking at children.
V6
X4
Z17
ZIG
ZAG
y 29
w 11
Z17
X4
V6
w29
y22