1
Abstract Data TypesStack, Queue
Amortized analysis
2
ADT is an interface
• It defines– the type of the data stored– operations, what each operation does
(not how)– parameters of each operation
3
ADT
Application
Implementation of theData structure
חוזה בין מתכנת האפליקציה
ומיישם מבנה הנתונים
ממשק
4
Example: Stacks
• Push(x,S) : Insert element x into S• Pop(S) : Delete the last element inserted into
S• Empty?(S): Return yes if S is empty• Top(S): Return the last element inserted into
S• Size(S)• Make-stack()
5
The Stack Data Abstraction
push
push
push
6
The Stack Data Abstraction
push
push
push
pop
push
Last in,First out.
7
• Evaluate an expression in postfix or Reverse Polish Notation
Infix Postfix
(2+ 3) * 5 2 3 + 5 *
( (5 * (7 / 3) ) – (2 * 7) ) 5 7 3 / * 2 7 *-
A stack application
8
2 3 + 5 *
A stack application
23
9
2 3 + 5 *
A stack application
55
10
2 3 + 5 *
A stack application
25
11
S ← make-stack()while ( not eof ) do B ← read the next data;
if B is an operand then push(B,S) else X ← pop(S)
Y ← pop(S) Z ← Apply the operation B on X and Y push(Z,S)return(top(S))
Pseudo-code
12
Implementation
• We will be interested in algorithms to implement the ADT..
• And their efficiency..
13
Using an array
12 1 3A
A[0] A[1]
t
The stack is represented by the array A and variable t
A[2] A[N-1]
1213
14
Using an array
12 1 3A
A[0] A[1]
t
make-stack(): Allocates the array A, which is of some fixed size N, sets t ← -1
The stack is represented by the array A and variable t
A[2] A[N-1]
15
Operations
12 1 3A
t
size(S): return (t+1)
empty?(S): return (t < 0)
top(S): if empty?(S) then error else return A[t]
A[N-1]A[0] A[1]
A[2]
16
Pop
12 1 3A
t
pop(S): if empty?(S) then error else e ←A[t] t ← t – 1 return (e)
A[N-1]A[0] A[1]
A[2]
pop(S)
17
Pop
12 1 3A
t
A[N-1]A[0] A[1]
A[2]
pop(S): if empty?(S) then error else e ←A[t] t ← t – 1 return (e)
pop(S)
18
Push
12 1 3A
t
push(x,S): if size(S) = N then error else t ←t+1 A[t] ← x
A[N-1]A[0] A[1]
A[2]
push(5,S)
19
Push
12 1 5A
t
push(x,S): if size(S) = N then error else t ←t+1 A[t] ← x
A[N-1]A[0] A[1]
A[2]
push(5,S)
20
Implementation with lists
12 1 5
top
size=3
x.element
x.nextx
21
Implementation with lists
12 1 5
top
make-stack(): top ← null size ← 0
size=3
22
Operations
12 1 5
top
size(S): return (size)
empty?(S): return (top = null)
top(S): if empty?(S) then error else return top.element
size=3
23
Pop
12 1 5
top
pop(S): if empty?(S) then error else e ←top.element top ← top.next size ← size-1 return (e)
pop(S)
size=3
24
Pop
12 1 5
top
pop(S): if empty?(S) then error else e ←top.element top ← top.next size ← size-1 return (e)
pop(S)
size=2
25
Garbage collection
1 5
top
pop(S): if empty?(S) then error else e ←top.element top ← top.next size ← size-1 return (e)
pop(S)
size=2
26
Push
1 5
topsize=2
push(x,S): n = new node n.element ←x n.next ← top top ← n size ← size + 1 push(5,S)
27
Push
1 5
topsize=2
push(x,S): n = new node n.element ←x n.next ← top top ← n size ← size + 1 push(5,S)
5
28
Push
1 5
topsize=2
push(x,S): n = new node n.element ←x n.next ← top top ← n size ← size + 1 push(5,S)
5
29
Push
1 5
topsize=3
push(x,S): n = new node n.element ←x n.next ← top top ← n size ← size + 1 push(5,S)
5
30
Analysis
• Bound the running time of an operation on the worst-case
• As a function of the “size”, n, of the data structure
• T(n) < 4n+7• Too detailed, we are just interested
in the order of growth
31
Big-O
) ו- כך ש:c - קיים ) ( ( ))T n O f n0n
0)()( nnnfcnT
דוגמא:2
( ) ( 1)T n n
)(4)1(222nOnn
32
Big-O
))(()( ngOnf
cg(n)
f(n)
n0
33
• 4n O(n2)
• 4n2 O(n2)
• 2n O(n10)
• 10 O(1)
• 100n3+10n O(n3)
• log2(n) O(log10(n))
More examples
34
The running time of our stack and queue operations
• Each operation takes O(1) time
35
Stacks via extendable arrays
• We do not want our implementation using arrays to be limited to only N elements
• When the array is full we will double its size
36
Push
push(x,S): if size(S) = N then allocate a new array of size 2N copy the old array to the new one t ←t+1 A[t] ← x
12 1A
t
A[N-1]A[0] A[1]
37
Push
push(x,S): if size(S) = N then allocate a new array of size 2N copy the old array to the new one t ←t+1 A[t] ← x
12 1 3 3A 4 5 7 3 2 8 1
A[N-1]A[0] A[1]
t
push(5,S)
38
Push
push(x,S): if size(S) = N then allocate a new array of size 2N copy the old array to the new one t ←t+1 A[t] ← x
push(5,S)
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
39
Push
push(x,S): if size(S) = N then allocate a new array of size 2N copy the old array to the new one t ←t+1 A[t] ← x push(5,S)
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
12 1 3 3 4 5 7 3 2 8 1
t
40
Push
push(x,S): if size(S) = N then allocate a new array of size 2N copy the old array to the new one t ←t+1 A[t] ← x
t
A[2N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
push(5,S)
41
Push
push(x,S): if size(S) = N then allocate a new array of size 2N copy the old array to the new one t ←t+1 A[t] ← x
5
A[2N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
push(5,S)
42
Analysis
• An operation may take O(n) worst case time !
• But that cannot happen often..
43
Amortized Analysis
• How long it takes to do m operations ?
• Well, O(nm)
• Yes, but can it really take that long ?
44
45
x
46
x x
47
x x
x x x
48
x x
x x x x
49
x x
x x x x
x x x x x
50
x x
x x x x
x x x x x x
51
x x
x x x x
x x x x x x x
52
x x
x x x x
x x x x x x x x
53
x x
x x x x
x x x x x x x x
x x x x x x x x x
54
x x
x x x x
x x x x x x x x
x x x x x x x x x x
55
x x
x x x x
x x x x x x x x
x x x x x x x x x x x
56
x x
x x x x
x x x x x x x x
x x x x x x x x x x x
Only after z-1 pushes that cost 1 we will have a push that costs 2z+1
Let z be the size of the array we just copied
57
x x
x x x x
x x x x x x x x
x x x x x x x x x x x
3+(2∙4+1)=3∙4
7+(2∙8+1)= 3∙8
Total cost O(m) !!
Start from the second row: z=2
1+(2∙2+1)=3∙2
z=4 z=8
58
Theorem: A sequence of m operations on a stack takes O(m) time
proof.
59
Amortized Analysis (The bank’s view)
• We will have a bank
• An operation can either pay a token for a unit of work by itself, or take a token from the bank to pay for it
• An operation can put tokens in the bank
• If the bank never has a negative balance then the total # of tokens spent by the operations bounds the total work
60
For a proof:
• Define exactly how each operation pays for its work, and how many tokens it puts in the bank
• Prove that the balance in the bank is always non-negative
• Count the total # of tokens – that’s your bound
61
“Easy” push
• when we push an item, we also put ≤ two tokens, one on the item which we insert, and one on another item if there is some item without a token
A[N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
62
5
A[N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
“Easy” push
• when we push an item, we also put ≤ two tokens, one on the item which we insert, and one on another item if there is some item without a token
63
5 6
A[N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
64
5 6 6 7 1 10 4 67 2 5 7
A[N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
65
“hard” push
• Tokens from the bank “pay” for copying the array into the larger array
5 6 6 7 1 10 4 67 2 5 7
A[N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
66
• Then you pay a token and put two in the bank as an “easy” push
5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 A
t
5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 4
t
“hard” push
67
Need to prove
• The balance is never negative:
• When we get to an expensive push there are enough tokens to pay for copying
• By induction: prove that after the i-th push following a copying there are 2i tokens in the bank
68
How many tokens we spent ?
• Each operation spent 3 tokens
69
Summary
• So the total # of tokens is 3mTHM: A sequence of m operations takes
O(m) time !! (we finished the proof)
• Each operation takes O(1) amortized time
70
Theorem: A sequence of m operations on a stack takes O(m) time
proof (2) .
• We formalize the bank as a potential function
71
Amortized(op) = actual(op) +
Define
Define: a potential function
72
Amortized(op1) = actual(op1) + 1- 0
Amortized(op2) = actual(op2) + 2- 1
Amortized(opn) = actual(opn) + n- (n-1)
……
+
iAmortized(opi) = iactual(opi) + n- 0
iAmortized(opi) iactual(opi) if n- 0 0
73
Example: Our extendable arrays
Define a potential of the stack
2( 1) 2( 1) 0( )
0
t N if t NS
Otherwise
74
Amortized(push) = actual(push) + =
1 + 2(t+2) – N - (2(t+1) – N) = 3
Amortized(push) = N + 1 + =
N + 1 + (2 - N) = 3
Amortized(pop) = 1 + =
1 + 2(t-1) – N - (2t – N) = -1
With copying:
Without copying:
75
Amortized(op1) = actual(op1) + 1- 0
Amortized(op2) = actual(op2) + 2- 1
Amortized(opn) = actual(opn) + n- (n-1)
……
+
iAmortized(opi) = iactual(opi) + n- 0
iAmortized(opi) iactual(opi) if n- 0 03m ≥
76
Summary
• So the total # of tokens is 3m THM: A sequence of m operations
starting from an empty stack takes O(m) time !!
• Each operations take O(1) amortized time
77
Queue
• Inject(x,Q) : Insert last element x into Q• Pop(Q) : Delete the first element in Q• Empty?(Q): Return yes if Q is empty• Front(Q): Return the first element in Q• Size(Q)• Make-queue()
78
The Queue Data Abstraction
inject
inject
79
The Queue Data Abstraction
inject
inject
inject
inject
popFirst in,First out (FIFO).
80
Using an array
12 1 4 2A 5
A[0] A[1]
t
pop(Q)
A[2] A[N-1]
81
Using an array
1 4 2 5A
A[0] A[1]
t
pop(Q)
A[2] A[N-1]
82
Using an array
1 4 2 5A
A[0] A[1]
t
A[2] A[N-1]
This would be inefficient if we insist that elements span a prefix of the array
83
Using an array
12 1 4 2A 5
A[0] A[1]
r
A[2] A[N-1]
f
A
A[0] A[1]
r
A[2]
f
Empty queue f=r
84
Using an array
12 1 4 2A 5
A[0] A[1]
r
pop(Q)
A[2] A[N-1]
f
85
Using an array
1 4 2A 5
A[0] A[1]
A[2] A[N-1]
f r
pop(Q)inject(5,Q)
86
Using an array
1 4 2A 5 5
A[0] A[1]
A[2] A[N-1]
f r
pop(Q)inject(5,Q)inject(5,Q)
87
Using an array
1 4 2A 5 5 5
A[0] A[1]
A[2] A[N-1]
f r
pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)
88
Using an array
2A 5 5 5
A[0] A[1]
A[2] A[N-1]
f r
pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….
89
Using an array
A 5 5 5 5
A[0] A[1]
r
A[2] A[N-1]
f
pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….
90
Make the array “circular”
5 5 A 5 5
A[0] A[1]
r
A[2] A[N-1]
f
Pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….
91
Operations
empty?(Q): return (f = r)
top(Q): if empty?(Q) then error else return A[f]
1 4 2A 5
A[0] A[1]
A[2] A[N-1]
f r
92
Operations
size(Q): if (r >= f) then return (r-f) else return N-(f-r)
1 4 2A 5
A[0] A[1]
A[2] A[N-1]
f r
93
Operations
size(Q): if (r >= f) then return (r-f) else return N-(f-r)
5 5 A 5 5
A[0] A[1]
r
A[2] A[N-1]
f
94
Pop
pop(Q)
1 4 2A 5
A[0] A[1]
A[2]
f r
pop(Q): if empty?(Q) then error else e ←A[f] f ← (f + 1) mod N return (e)
95
Pop
pop(Q): if empty?(Q) then error else e ←A[f] f ← (f + 1) mod N return (e)
pop(Q)
1 4 2A 5
A[0] A[1]
A[2]
f r
96
Inject
inject(x,Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N
inject(5,Q)
4 2A 5
A[0] A[1]
A[2]
f r
97
Inject
inject(x,Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N
inject(5,Q)
4 2A 5 5
A[0] A[1]
A[2]
f r
98
Implementation with lists
12 1 5
headsize=3
tail
inject(4,Q)
99
Implementation with lists
12 1 5
headsize=3
tail
inject(4,Q)
4
100
Implementation with lists
12 1 5
headsize=3
tail
inject(4,Q)
4
Complete the details by yourself
101
Double ended queue (deque)
• Push(x,D) : Insert x as the first in D• Pop(D) : Delete the first element of D• Inject(x,D): Insert x as the last in D• Eject(D): Delete the last element of D• Size(D)• Empty?(D)• Make-deque()
102
Implementation with doubly linked lists
5
head
size=2
tail
13
x.next
x.element
x.prev
x
103
Empty list
head
size=0
tail
We use two sentinels here to make the code simpler
104
Push
push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1
5
head
size=1
tail
105
push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1
5
head
size=1
tail
push(4,D)
4
106
push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1
5
head
size=1
tail
push(4,D)
4
107
push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1
5
head
size=1
tail
push(4,D)
4
108
push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1
5
head
size=2
tail
push(4,D)
4
109
Implementations of the other operations are similar
• Try by yourself