CSC401 – Analysis of Algorithms CSC401 – Analysis of Algorithms Lecture Notes 6 Lecture Notes 6 Dictionaries and Search Trees Dictionaries and Search Trees Objectives: Objectives: Introduce dictionaries and its diverse Introduce dictionaries and its diverse implementations implementations Introduce binary search trees and present Introduce binary search trees and present operations on binary search trees operations on binary search trees Analyze the performance of binary search Analyze the performance of binary search tree operations tree operations Introduce balanced binary search trees: Introduce balanced binary search trees: AVL tree and Red-Black tree AVL tree and Red-Black tree Introduce a design pattern: Locator Introduce a design pattern: Locator
Transcript
CSC401 – Analysis of AlgorithmsCSC401 – Analysis of Algorithms Lecture Notes 6 Lecture Notes 6
Dictionaries and Search TreesDictionaries and Search Trees
Objectives:Objectives:
Introduce dictionaries and its diverse Introduce dictionaries and its diverse implementationsimplementationsIntroduce binary search trees and present Introduce binary search trees and present operations on binary search treesoperations on binary search treesAnalyze the performance of binary search Analyze the performance of binary search tree operationstree operationsIntroduce balanced binary search trees: AVL Introduce balanced binary search trees: AVL tree and Red-Black treetree and Red-Black treeIntroduce a design pattern: LocatorIntroduce a design pattern: Locator
22
Dictionary ADTDictionary ADTThe dictionary ADT The dictionary ADT models a searchable models a searchable collection of key-collection of key-element itemselement itemsThe main operations of The main operations of a dictionary are a dictionary are searching, inserting, searching, inserting, and deleting itemsand deleting itemsMultiple items with the Multiple items with the same key are allowedsame key are allowedApplications:Applications:– address bookaddress book– credit card authorizationcredit card authorization– mapping host names mapping host names
(e.g., cs16.net) to (e.g., cs16.net) to internet addresses (e.g., internet addresses (e.g., 128.148.34.101)128.148.34.101)
Dictionary ADT methods:Dictionary ADT methods:– findElementfindElement(k): if the (k): if the
dictionary has an item with dictionary has an item with key k, returns its element, key k, returns its element, else, returns the special else, returns the special element NO_SUCH_KEY element NO_SUCH_KEY
– insertIteminsertItem(k, o): inserts (k, o): inserts item (k, o) into the item (k, o) into the dictionarydictionary
– removeElementremoveElement(k): if the (k): if the dictionary has an item with dictionary has an item with key k, removes it from the key k, removes it from the dictionary and returns its dictionary and returns its element, else returns the element, else returns the special element special element NO_SUCH_KEY NO_SUCH_KEY
Log FileLog FileA log file is a dictionary implemented by means of A log file is a dictionary implemented by means of an unsorted sequencean unsorted sequence– We store the items of the dictionary in a sequence (based We store the items of the dictionary in a sequence (based
on a doubly-linked lists or a circular array), in arbitrary on a doubly-linked lists or a circular array), in arbitrary orderorder
Performance:Performance:– insertIteminsertItem takes takes OO(1)(1) time since we can insert the new time since we can insert the new
item at the beginning or at the end of the sequenceitem at the beginning or at the end of the sequence– findElementfindElement and and removeElement removeElement take take OO((nn)) time since in time since in
the worst case (the item is not found) we traverse the the worst case (the item is not found) we traverse the entire sequence to look for an item with the given keyentire sequence to look for an item with the given key
The log file is effective only for dictionaries of The log file is effective only for dictionaries of small size or for dictionaries on which insertions small size or for dictionaries on which insertions are the most common operations, while searches are the most common operations, while searches and removals are rarely performed (e.g., historical and removals are rarely performed (e.g., historical record of logins to a workstation)record of logins to a workstation)
44
Binary SearchBinary SearchBinary search performs operation Binary search performs operation findElementfindElement(k) (k) on a dictionary implemented by means of an on a dictionary implemented by means of an array-based sequence, sorted by keyarray-based sequence, sorted by key– similar to the high-low gamesimilar to the high-low game– at each step, the number of candidate items is halvedat each step, the number of candidate items is halved– terminates after a logarithmic number of stepsterminates after a logarithmic number of steps
Example: Example: findElementfindElement(7)(7)
1 3 4 5 7 8 9 11 14 16 18 19
1 3 4 5 7 8 9 11 14 16 18 19
1 3 4 5 7 8 9 11 14 16 18 19
1 3 4 5 7 8 9 11 14 16 18 19
0
0
0
0
ml h
ml h
ml h
lm h
55
Lookup TableLookup TableA lookup table is a dictionary implemented by means of A lookup table is a dictionary implemented by means of a sorted sequencea sorted sequence– We store the items of the dictionary in an array-based We store the items of the dictionary in an array-based
sequence, sorted by keysequence, sorted by key– We use an external comparator for the keysWe use an external comparator for the keys
Performance:Performance:– findElementfindElement takes takes OO(log (log nn)) time, using binary search time, using binary search– insertIteminsertItem takes takes OO((nn)) time since in the worst case we have to time since in the worst case we have to
shift shift nn22 items to make room for the new item items to make room for the new item– removeElement removeElement take take OO((nn)) time since in the worst case we have time since in the worst case we have
to shift to shift nn22 items to compact the items after the removal items to compact the items after the removal
The lookup table is effective only for dictionaries of The lookup table is effective only for dictionaries of small size or for dictionaries on which searches are the small size or for dictionaries on which searches are the most common operations, while insertions and most common operations, while insertions and removals are rarely performed (e.g., credit card removals are rarely performed (e.g., credit card authorizations)authorizations)
66
Binary Search TreeBinary Search TreeA binary search tree is A binary search tree is a binary tree storing a binary tree storing keys (or key-element keys (or key-element pairs) at its internal pairs) at its internal nodes and satisfying nodes and satisfying the following property:the following property:– Let Let uu, , vv, and , and ww be three be three
nodes such that nodes such that uu is in is in the left subtree of the left subtree of vv and and ww is in the right subtree is in the right subtree of of vv. We have . We have keykey((uu)) keykey((vv) ) keykey((ww))
External nodes do not External nodes do not store itemsstore items
An inorder traversal of An inorder traversal of a binary search trees a binary search trees visits the keys in visits the keys in increasing orderincreasing order
6
92
41 8
77
SearchSearchTo search for a key To search for a key kk, , we trace a downward we trace a downward path starting at the rootpath starting at the root
The next node visited The next node visited depends on the depends on the outcome of the outcome of the comparison of comparison of kk with with the key of the current the key of the current nodenode
If we reach a leaf, the If we reach a leaf, the key is not found and we key is not found and we return NO_SUCH_KEYreturn NO_SUCH_KEY
Example: Example: findElementfindElement(4)(4)
Algorithm findElement(k, v)if T.isExternal (v)
return NO_SUCH_KEYif k key(v)
return findElement(k, T.leftChild(v))else if k key(v)
return element(v)else { k key(v) }
return findElement(k, T.rightChild(v))
6
92
41 8
88
InsertionInsertionTo perform operation To perform operation insertIteminsertItem(k, o), we (k, o), we search for key ksearch for key k
Assume k is not Assume k is not already in the tree, already in the tree, and let let w be the and let let w be the leaf reached by the leaf reached by the searchsearch
We insert k at node w We insert k at node w and expand w into an and expand w into an internal nodeinternal node
Example: insert 5Example: insert 5
6
92
41 8
w
6
92
41 8
5w
99
DeletionDeletionTo perform operation To perform operation removeElementremoveElement((kk), we ), we search for key search for key kk
Assume key Assume key kk is in the is in the tree, and let let tree, and let let vv be the be the node storing node storing kk
If node If node vv has a leaf child has a leaf child ww, we remove , we remove vv and and ww from the tree with from the tree with operation operation removeAboveExternalremoveAboveExternal((ww))
Example: remove 4Example: remove 4
6
92
51 8
6
92
41 8
5
vw
1010
Deletion (cont.)Deletion (cont.)We consider the case We consider the case where the key where the key kk to be to be removed is stored at a removed is stored at a node node vv whose children whose children are both internalare both internal– we find the internal node we find the internal node
w w that follows that follows vv in an in an inorder traversalinorder traversal
– we copy we copy keykey((ww)) into node into node vv– we remove node we remove node w w and its and its
left child left child z z (which must be (which must be a leaf) by means of a leaf) by means of operation operation removeAboveExternalremoveAboveExternal((zz))
Example: remove 3Example: remove 3
3
1
8
6 9
5
v
w
z
2
5
1
8
6 9
v
2
1111
PerformancePerformanceConsider a dictionary Consider a dictionary with with nn items items implemented by means implemented by means of a binary search tree of a binary search tree of height of height hh– the space used is the space used is OO((nn))– methods methods findElement findElement , ,
insertIteminsertItem and and removeElementremoveElement take take OO((hh) ) timetime
The height The height hh is is OO((nn) ) in in the worst case and the worst case and OO(log (log nn)) in the best case in the best case
1212
AVL Tree DefinitionAVL Tree Definition
AVL trees are AVL trees are balanced.balanced.
An AVL Tree is a An AVL Tree is a binary search treebinary search tree such that for every such that for every internal node v of T, internal node v of T, the the heights of the heights of the children of v can differ children of v can differ by at most 1by at most 1..
88
44
17 78
32 50
48 62
2
4
1
1
2
3
1
1
An example of an AVL tree where the heights are shown next to the nodes:
1313
Height of an AVL TreeHeight of an AVL TreeFactFact: The : The heightheight of an AVL tree of an AVL tree
storing n keys is O(log n).storing n keys is O(log n).ProofProof: Let us bound : Let us bound n(h):n(h): the minimum number of the minimum number of internal nodes of an AVL tree of height h.internal nodes of an AVL tree of height h.We easily see that n(1) = 1 and n(2) = 2We easily see that n(1) = 1 and n(2) = 2For n > 2, an AVL tree of height h contains the root For n > 2, an AVL tree of height h contains the root node, one AVL subtree of height n-1 and another of node, one AVL subtree of height n-1 and another of height n-2.height n-2.That is, n(h) = 1 + n(h-1) + n(h-2)That is, n(h) = 1 + n(h-1) + n(h-2)Knowing n(h-1) > n(h-2), we get n(h) > 2n(h-2). SoKnowing n(h-1) > n(h-2), we get n(h) > 2n(h-2). Son(h) > 2n(h-2), n(h) > 4n(h-4), n(h) > 8n(n-6), … (by n(h) > 2n(h-2), n(h) > 4n(h-4), n(h) > 8n(n-6), … (by
Solving the base case we get: n(h) > 2 Solving the base case we get: n(h) > 2 h/2-1h/2-1
Taking logarithms: h < 2log n(h) +2Taking logarithms: h < 2log n(h) +2Thus the height of an AVL tree is O(log n)Thus the height of an AVL tree is O(log n)
3
4 n(1)
n(2)
1414
Insertion in an AVL TreeInsertion in an AVL TreeInsertion is as in a binary search treeInsertion is as in a binary search treeAlways done by expanding an external node.Always done by expanding an external node.Example:Example: 44
17 78
32 50 88
48 62
54
w
b=x
a=y
c=z
44
17 78
32 50 88
48 62
before insertion after insertion
1515
Trinode RestructuringTrinode Restructuringlet (let (aa,,bb,,cc) be an inorder listing of ) be an inorder listing of xx, , yy, , zzperform the rotations needed to make perform the rotations needed to make bb the the topmost node of the threetopmost node of the three
b=y
a=z
c=x
T0
T1
T2 T3
b=y
a=z c=x
T0 T1 T2 T3
c=y
b=x
a=z
T0
T1 T2
T3 b=x
c=ya=z
T0 T1 T2 T3
case 1: single rotation(a left rotation about a)
case 2: double rotation(a right rotation about c, then a left rotation about a)
(other two cases are symmetrical)
1616
Insertion Example, continuedInsertion Example, continued
Removal in an AVL TreeRemoval in an AVL TreeRemoval begins as in a binary search tree, Removal begins as in a binary search tree, which means the node removed will become an which means the node removed will become an empty external node. Its parent, w, may cause empty external node. Its parent, w, may cause an imbalance.an imbalance.Example: Example:
44
17
7832 50
8848
62
54
44
17
7850
8848
62
54
before deletion of 32 after deletion
2020
Rebalancing after a RemovalRebalancing after a RemovalLet Let zz be the be the first unbalancedfirst unbalanced node encountered node encountered while travelling up the tree from w. Also, let y be while travelling up the tree from w. Also, let y be the child of z with the larger height, and let x be the child of z with the larger height, and let x be the child of y with the larger height.the child of y with the larger height.We perform We perform restructurerestructure(x) to restore balance at z.(x) to restore balance at z.As this restructuring may upset the balance of As this restructuring may upset the balance of another node higher in the tree, we must continue another node higher in the tree, we must continue checking for balance until the root of T is reachedchecking for balance until the root of T is reached
44
17
7850
8848
62
54
w
c=x
b=y
a=z
44
17
78
50 88
48
62
54
2121
Running Times for AVL TreesRunning Times for AVL Trees
a single restructure is O(1)a single restructure is O(1)– using a linked-structure binary treeusing a linked-structure binary tree
find is O(log n)find is O(log n)– height of tree is O(log n), no restructures neededheight of tree is O(log n), no restructures needed
insert is O(log n)insert is O(log n)– initial find is O(log n)initial find is O(log n)– Restructuring up the tree, maintaining heights is O(log n)Restructuring up the tree, maintaining heights is O(log n)
remove is O(log n)remove is O(log n)– initial find is O(log n)initial find is O(log n)– Restructuring up the tree, maintaining heights is O(log n)Restructuring up the tree, maintaining heights is O(log n)
2222
Red-Black TreeRed-Black TreeA red-black tree can also be defined as a binary A red-black tree can also be defined as a binary search tree that satisfies the following properties:search tree that satisfies the following properties:– Root PropertyRoot Property: the root is black: the root is black– External PropertyExternal Property: every leaf is black: every leaf is black– Internal PropertyInternal Property: the children of a red node are black: the children of a red node are black– Depth PropertyDepth Property: all the leaves have the same black depth: all the leaves have the same black depth
9
154
62 12
7
21
Theorem:Theorem: A red-black A red-black tree storing tree storing nn items items has height has height OO(log (log nn))– The height of a red-The height of a red-
black tree is at most black tree is at most twice the height of its twice the height of its associated (2,4) tree, associated (2,4) tree, which is which is OO(log (log nn))
The search algorithm for a binary search tree is the The search algorithm for a binary search tree is the same as that for a binary search treesame as that for a binary search treeBy the above theorem, searching in a red-black tree By the above theorem, searching in a red-black tree takes takes OO(log (log nn)) time time
2323
InsertionInsertionTo perform operation To perform operation insertIteminsertItem((kk, , oo)), we execute the , we execute the insertion algorithm for binary search trees and color insertion algorithm for binary search trees and color redred the the newly inserted node newly inserted node z z unless it is the rootunless it is the root– We preserve the root, external, and depth propertiesWe preserve the root, external, and depth properties– If the parent If the parent vv of of zz is black, we also preserve the internal is black, we also preserve the internal
property and we are done property and we are done – Else (Else (vv is red ) we have a is red ) we have a double reddouble red (i.e., a violation of (i.e., a violation of
the internal property), which requires a reorganization of the internal property), which requires a reorganization of the treethe tree
Example where the insertion of 4 causes a double red:Example where the insertion of 4 causes a double red:
6
3 8
6
3 8
4z
v v
z
2424
Remedying a Double RedRemedying a Double RedConsider a double red with child Consider a double red with child z z and parent and parent vv, and let , and let ww be the sibling of be the sibling of vv
4
6
7z
vw2
4 6 7
.. 2 ..
Case 1Case 1: : ww is black is black– The double red is an The double red is an
incorrect replacement incorrect replacement of a 4-nodeof a 4-node
– RestructuringRestructuring: we : we change the 4-node change the 4-node replacementreplacement
Case 2Case 2: : ww is red is red– The double red The double red
corresponds to an corresponds to an overflowoverflow
– RecoloringRecoloring: we : we perform the perform the equivalent of a equivalent of a splitsplit
4
6
7z
v
2 4 6 7
2w
2525
RestructuringRestructuringA restructuring remedies a child-parent double red when A restructuring remedies a child-parent double red when the parent red node has a black siblingthe parent red node has a black sibling
It is equivalent to restoring the correct replacement of a 4-It is equivalent to restoring the correct replacement of a 4-nodenode
The internal property is restored and the other properties The internal property is restored and the other properties are preservedare preserved
4
6
7z
vw2
4 6 7
.. 2 ..
4
6
7
z
v
w2
4 6 7
.. 2 ..
2626
Restructuring (cont.)Restructuring (cont.)There are four restructuring configurations There are four restructuring configurations depending on whether the double red nodes depending on whether the double red nodes are left or right childrenare left or right children
2
4
6
6
2
4
6
4
2
2
6
4
2 6
4
2727
RecoloringRecoloringA recoloring remedies a child-parent double red when the A recoloring remedies a child-parent double red when the parent red node has a red siblingparent red node has a red siblingThe parent The parent vv and its sibling and its sibling ww become black and the become black and the grandparent grandparent uu becomes red, unless it is the root becomes red, unless it is the rootIt is equivalent to performing a split on a 5-nodeIt is equivalent to performing a split on a 5-nodeThe double red violation may propagate to the The double red violation may propagate to the grandparent grandparent uu
4
6
7z
v
2 4 6 7
2w
4
6
7z
v
6 7
2w
… 4 …
2
2828
Analysis of InsertionAnalysis of InsertionRecall that a red-black Recall that a red-black tree has tree has OO(log (log nn)) height height
Step 1 takes Step 1 takes OO(log (log nn)) time time because we visit because we visit OO(log (log nn)) nodesnodes
Step 2 takes Step 2 takes OO(1)(1) time time
Step 3 takes Step 3 takes OO(log (log nn)) time time because we performbecause we perform– OO(log (log nn) ) recolorings, each recolorings, each
taking taking OO(1)(1) time, and time, and– at most one restructuring at most one restructuring
taking taking OO(1)(1) time time
Thus, an insertion in a Thus, an insertion in a red-black tree takes red-black tree takes OO(log (log nn)) time time
1.1. We search for key We search for key kk to locate to locate the insertion node the insertion node zz
2.2. We add the new item (We add the new item (kk, , oo) at ) at node node z z and color and color zz red red
3. 3. whilewhile doubleReddoubleRed((zz))if if isBlackisBlack((siblingsibling((parentparent((zz))))))
z z restructure restructure((zz))returnreturn
else else {{ siblingsibling((parentparent((zz) is red }) is red } z z recolor recolor((zz))
2929
DeletionDeletionTo perform operation To perform operation removeremove((kk)), we first execute the , we first execute the deletion algorithm for binary search treesdeletion algorithm for binary search trees
Let Let vv be the internal node removed, be the internal node removed, ww the external node the external node removed, and removed, and rr the sibling of the sibling of ww– If either If either vv of of rr was red, we color was red, we color rr black and we are done black and we are done– Else (Else (vv and and rr were both black) we color were both black) we color rr double blackdouble black, which , which
is a violation of the internal property requiring a reorganization is a violation of the internal property requiring a reorganization of the treeof the tree
Example where the deletion of 8 causes a double black:Example where the deletion of 8 causes a double black:
6
3 8
4
v
r w
6
3
4
r
3030
Remedying a Double BlackRemedying a Double BlackThe algorithm for remedying a double black node The algorithm for remedying a double black node ww with with sibling sibling yy considers three cases considers three casesCase 1Case 1: : yy is black and has a red child is black and has a red child– We perform a We perform a restructuringrestructuring, equivalent to a , equivalent to a transfer transfer , and , and
we are donewe are done
Case 2Case 2: : yy is black and its children are both black is black and its children are both black– We perform a We perform a recoloringrecoloring, equivalent to a , equivalent to a fusionfusion, which may , which may
propagate up the double black violationpropagate up the double black violation
Case 3Case 3: : yy is red is red– We perform an We perform an adjustmentadjustment, equivalent to choosing a , equivalent to choosing a
different representation of a 3-node, after which either different representation of a 3-node, after which either Case 1 or Case 2 appliesCase 1 or Case 2 applies
Deletion in a red-black tree takes Deletion in a red-black tree takes OO(log (log nn)) time time
3131
Red-Black Tree ReorganizationRed-Black Tree Reorganizationremedy double redremedy double redInsertionInsertion
double red removed double red removed or propagated upor propagated upsplitsplitrecoloringrecoloring
double red removeddouble red removedchange of 4-node change of 4-node representationrepresentationrestructuringrestructuring
resultresult(2,4) tree action(2,4) tree actionRed-black tree actionRed-black tree action
restructuring or restructuring or recoloring followsrecoloring follows
change of 3-node change of 3-node representationrepresentationadjustmentadjustment
double black removed double black removed or propagated upor propagated upfusionfusionrecoloringrecoloring
double black removeddouble black removedtransfertransferrestructuringrestructuring
resultresult(2,4) tree action(2,4) tree actionRed-black tree actionRed-black tree action
3232
LocatorsLocatorsA locators identifies and tracks A locators identifies and tracks a (key, element) item within a a (key, element) item within a data structuredata structureA locator sticks with a specific A locator sticks with a specific item, even if that element item, even if that element changes its position in the data changes its position in the data structurestructureIntuitive notion:Intuitive notion:– claim checkclaim check– reservation numberreservation number
Methods of the locator ADT:Methods of the locator ADT:– keykey(): returns the key of the (): returns the key of the
item associated with the item associated with the locatorlocator
– elementelement(): returns the element (): returns the element of the item associated with the of the item associated with the locatorlocator
Application example:Application example:– Orders to purchase Orders to purchase
and sell a given stock and sell a given stock are stored in two are stored in two priority queues (sell priority queues (sell orders and buy orders and buy orders)orders)
the key of an order is the key of an order is the pricethe pricethe element is the the element is the number of sharesnumber of shares
– When an order is When an order is placed, a locator to it placed, a locator to it is returnedis returned
– Given a locator, an Given a locator, an order can be canceled order can be canceled or modifiedor modified
3333
Locator-based MethodsLocator-based MethodsLocator-based priority queue Locator-based priority queue methods:methods:– insertinsert(k, o): inserts the item (k, o): inserts the item
(k, o) and returns a locator for (k, o) and returns a locator for itit
– minmin(): returns the locator of (): returns the locator of an item with smallest keyan item with smallest key
– removeremove(l): remove the item (l): remove the item with locator lwith locator l
– replaceKeyreplaceKey(l, k): replaces the (l, k): replaces the key of the item with locator lkey of the item with locator l
– replaceElementreplaceElement(l, o): replaces (l, o): replaces with o the element of the item with o the element of the item with locator lwith locator l
– locatorslocators(): returns an iterator (): returns an iterator over the locators of the items over the locators of the items in the priority queuein the priority queue
Locator-based dictionary Locator-based dictionary methods:methods:– insertinsert(k, o): inserts the (k, o): inserts the
item (k, o) and returns its item (k, o) and returns its locatorlocator
– findfind(k): if the dictionary (k): if the dictionary contains an item with key contains an item with key k, returns its locator, else k, returns its locator, else return the special locator return the special locator NO_SUCH_KEYNO_SUCH_KEY
– removeremove(l): removes the (l): removes the item with locator l and item with locator l and returns its elementreturns its element
ImplementationImplementationThe locator is an The locator is an object storingobject storing– keykey– elementelement– position (or rank) of position (or rank) of
the item in the the item in the underlying structureunderlying structure
In turn, the position In turn, the position (or array cell) stores (or array cell) stores the locatorthe locatorExample:Example:– binary search tree binary search tree
with locatorswith locators
3 a
6 d
9 b
1 g 4 e 8 c
3535
Positions vs. LocatorsPositions vs. Locators PositionPosition– represents a “place” in a represents a “place” in a
data structuredata structure– related to other positions related to other positions
in the data structure in the data structure (e.g., previous/next or (e.g., previous/next or parent/child)parent/child)
– implemented as a node implemented as a node or an array cellor an array cell
Position-based ADTs (e.g., Position-based ADTs (e.g., sequence and tree) are sequence and tree) are fundamental data storage fundamental data storage schemesschemes
LocatorLocator– identifies and tracks a identifies and tracks a
(key, element) item(key, element) item– unrelated to other unrelated to other
locators in the data locators in the data structurestructure
– implemented as an object implemented as an object storing the item and its storing the item and its position in the underlying position in the underlying structurestructure
Key-based ADTs (e.g., priority Key-based ADTs (e.g., priority queue and dictionary) can be queue and dictionary) can be augmented with locator-augmented with locator-based methods based methods
