Advanced Data Structures and Advanced Data Structures and ImplementationImplementation• Top-Down Splay TreesTop-Down Splay Trees• Red-Black TreesRed-Black Trees• Top-Down Red Black TreesTop-Down Red Black Trees• Top-Down DeletionTop-Down Deletion• Deterministic Skip ListsDeterministic Skip Lists• AA-TreesAA-Trees• TreapsTreaps• k-d Treesk-d Trees• Pairing HeapsPairing Heaps
Top-Down Splay TreeTop-Down Splay Tree
• Direct strategy requires traversal from the Direct strategy requires traversal from the root down the tree, and then bottom-up root down the tree, and then bottom-up traversal to implement the splaying tree.traversal to implement the splaying tree.
• Can implement by storing parent links, or Can implement by storing parent links, or by storing the access path on a stack.by storing the access path on a stack.
• Both methods require large amount of Both methods require large amount of overhead and must handle many special overhead and must handle many special cases.cases.
• Initial rotations on the initial access path Initial rotations on the initial access path uses only uses only O(1)O(1) extra space, but retains the extra space, but retains the O(log N)O(log N) amortized time bound. amortized time bound.
X
Y
YL Yr
XR
If Y should become root, then X and its right sub tree are made left children of the smallest value in R, and Y is made root of “center” tree. Y does not have to be a leaf for the Zig case to apply.
L R L R
X
XR
Y
YL Yr
Case 1: Zig
Case 2: Zig-ZigCase 2: Zig-Zig
X
Y
ZL Zr
XR
L R
ZYR
The value to be splayed is in the tree rooted at Z. Rotate Y about X and attach as left child of smallest value in R
L R
X
XRYR
Y
ZL Zr
Z
Case 3: Zig-Zag Case 3: Zig-Zag (Simplified)(Simplified)
X
Y
ZL Zr
XR
L R
ZYL
The value to be splayed is in the tree rooted at Z. To make code simpler, the Zig-Zag rotation is reduced to a single Zig. This results in more iterations in the splay process.
L R
X
XR
Y
ZL Zr
ZYL
Splay Trees ImplementationSplay Trees Implementation
• See page 457 for generic SplayTree See page 457 for generic SplayTree class implementation.class implementation.
• Page 458 shows Top-Down splaying Page 458 shows Top-Down splaying method.method.
• Refer to page 459 for insertion Refer to page 459 for insertion method into the Top-Down Splay Tree.method into the Top-Down Splay Tree.
• Page 460 shows deletion method from Page 460 shows deletion method from the tree.the tree.
Reassembling the Splay TreeReassembling the Splay Tree
X
XL XR
L R X
L R
XL XR
When the value to be splayed to the root is at the root of the “center” tree, we have reached the point where we are ready to reassemble the tree. This is accomplished by a) making XL the right child of the maximum element in L, b) making XR the left child of the minimum element in R, and then making L and R the left and right children of X
Operation 1: Zig-Zig
L RA
B
C
D
E
F
G
H
X
Ar
Br
Cr
Dl
Er
Fl
Gl
Hl
Xl Xr
Rotate B around A and make L child of minimum element in R (which is now empty)
L RC
D
E
F
G
H
X
Cr
Dl
Er
Fl
Gl
Hl
Xl Xr
A
B
ArBr
L is still empty, and R is now the tree rooted at B. Note that R contains nodes > X but not in the right subtree of X.
Operation 2: Zig-Zag
L RC
D
E
F
G
H
X
Cr
Dl
Er
Fl
Gl
Hl
Xl Xr
Just perform Zig (simplified Zig-Zag)
D
Dl
L R
C
Cr
A
B
ArBr
L was previously empty and it now consists of node D and D’s left subtree
A
B
ArBr
E
F
G
H
X
Er
Fl
Gl
Hl
Xl Xr
L R
After X reaches root:
X
Xl Xr
C
Cr
A
B
ArBrE
Er
G
Gl
F
Fl
H
Hl
D
Dl
This configuration was achieved by doing Zig Zig (of F, G) followed by a Zig (node H)
Reassemble – XL becomes right sub tree of H, XR becomes left sub tree of E, and then L, R reattached to X
X
Xl
G
Gl
F
Fl
H
Hl
D
Dl
C
Cr
A
B
ArBrE
Er
Note that this is not the same tree as was obtained by doing BU splaying.
Red-Black TreeRed-Black Tree
• Popular alternative to the AVL tree.Popular alternative to the AVL tree.• Operations take Operations take O(log N)O(log N) time in worst case. time in worst case.• Height is at most Height is at most 2log(N+1)2log(N+1)..• A A red-black treered-black tree is a binary search tree with one is a binary search tree with one
extra attribute for each node: the extra attribute for each node: the colorcolor, which is , which is either red or black.either red or black.
• The root is black.The root is black.• If node is red, its children must be blackIf node is red, its children must be black..• Every path from a node to a Every path from a node to a nullnull reference reference
must contain the same number of black must contain the same number of black nodesnodes..
• Basic operations to conform with rules are color Basic operations to conform with rules are color changes and tree rotations.changes and tree rotations.
Theorem 1 – In a red-black tree, at least half the nodes on any path from the root to a leaf must be black.
Proof – If there is a red node on the path, there must be a corresponding black node.
Theorem 2 – In a red-black tree, no path from any node, N, to a leaf is more than twice as long as any other path from N to any other leaf.
Proof: By definition, every path from a node to any leaf contains the same number of black nodes. By Theorem1, a least ½ the nodes on any such path are black. Therefore, there can no more than twice as many nodes on any path from N to a leaf as on any other path. Therefore the length of every path is no more than twice as long as any other path.
Theorem 3 –A red-black tree with n internal nodes
has height h <= 2 lg(n + 1).Proof: Let h be the height of the red-black tree with root x. By Theorem 1,
bh(x) >= h/2From Theorem 1, n >= 2bh(x) - 1Therefore n >= 2 h/2 – 1
n + 1 >= 2h/2lg(n + 1) >= h/22lg(n + 1) >= h
Bottom-Up InsertionBottom-Up Insertion
• Cases:Cases:• 0: X is the root – color it black0: X is the root – color it black• 1: Both parent and uncle are red – color parent 1: Both parent and uncle are red – color parent
and uncle black, color grandparent red, point X to and uncle black, color grandparent red, point X to grandparent, check new situationgrandparent, check new situation
• 2 (zig-zag): Parent is red, but uncle is black. X 2 (zig-zag): Parent is red, but uncle is black. X and its parent are opposite type children – color and its parent are opposite type children – color grandparent red, color X black, rotate left on grandparent red, color X black, rotate left on parent, rotate right on grandparentparent, rotate right on grandparent
• 3 (zig-zig): Parent is red, but uncle is black. X 3 (zig-zig): Parent is red, but uncle is black. X and its parent are both left or both right children and its parent are both left or both right children – color parent black, color grandparent red, rotate – color parent black, color grandparent red, rotate right on grandparentright on grandparent
Top-Down Red-Black TreesTop-Down Red-Black Trees
• In T-Down insertion, the corrections are In T-Down insertion, the corrections are done while traversing down the tree to the done while traversing down the tree to the insertion point.insertion point.
• When the actual insertion is done, no When the actual insertion is done, no further corrections are needed, so no need further corrections are needed, so no need to traverse back up the tree.to traverse back up the tree.
• So, T-Down insertion can be done So, T-Down insertion can be done iteratively which is generally faster.iteratively which is generally faster.
• Insertion is always done as a leaf (as in Insertion is always done as a leaf (as in ordinary BST insertion).ordinary BST insertion).
ProcessProcess
• On the way down, when we see a node On the way down, when we see a node XX that has two red children, we make that has two red children, we make XX red red and its two children black.and its two children black.
• If If XX’s parent is red, we can apply either the ’s parent is red, we can apply either the single or double rotation to keep us from single or double rotation to keep us from having two consecutive red nodes.having two consecutive red nodes.
• XX’s parent and the parent’s sibling cannot ’s parent and the parent’s sibling cannot both be red, since their colors would both be red, since their colors would already have been flipped in that case.already have been flipped in that case.
Example: Insert 45Example: Insert 45
30
70
85
5
60
80
10
90
15
20
50
40 55
65Two red childrenTwo red children
Example (Cont.)Example (Cont.)
30
70
85
5
60
80
10
90
15
20
50
40 55
65flip colors - two red nodesflip colors - two red nodes
Example (Cont.): Do a single Example (Cont.): Do a single rotationrotation
30
70
85
5
60
80
10
90
15
20 50
40 55 65
Example (Cont.): Now Insert Example (Cont.): Now Insert 4545
30
70
85
5
60
80
10
90
15
20 50
40 55 65
45
NoteNote
• Since the parent of the newly inserted Since the parent of the newly inserted node was black, we are done.node was black, we are done.
• Had the parent of the inserted node been Had the parent of the inserted node been red, one more rotation would have had to red, one more rotation would have had to be performed.be performed.
• Although red-black trees have slightly Although red-black trees have slightly weaker balancing properties, their weaker balancing properties, their performance in experimentally almost performance in experimentally almost identical to that of AVL trees.identical to that of AVL trees.
Implementation of Top-Down Implementation of Top-Down Red-Black TreesRed-Black Trees
• See pages 464-467See pages 464-467
Top-Down DeletionsTop-Down Deletions
• Recall that in deleting from a binary search tree, Recall that in deleting from a binary search tree, the only nodes which are actually removed are the only nodes which are actually removed are leaves or nodes with exactly one child.leaves or nodes with exactly one child.
• Nodes with two children are never removed. Nodes with two children are never removed. Their contents are just replaced.Their contents are just replaced.
• If the node to be deleted is red, there is no If the node to be deleted is red, there is no problem - just delete the node.problem - just delete the node.
• If the node to be deleted is black, its removal will If the node to be deleted is black, its removal will violate property.violate property.
• The solution is to ensure that any node to be The solution is to ensure that any node to be deleted is red.deleted is red.
Deterministic Skip ListsDeterministic Skip Lists
• A probabilistically balanced A probabilistically balanced linked list.linked list.
• Invented in 1986 by Invented in 1986 by William Pugh.William Pugh.
• Definition:Definition: Two elements Two elements are linked if there exists at are linked if there exists at least one link going from least one link going from one to another.one to another.
• Definition: The gap size Definition: The gap size between two elements between two elements linked at height linked at height hh is equal is equal to the number of elements to the number of elements of height of height h-1h-1 between between them.them.
Skip ListSkip List
3 6NIL
79
12 17 1921
25 26
36
7 9 12 17 19 2125
26NIL
d) xtra pointers every eighth item - full structure
e) skip list - same link distribution, random choice
Search timeSearch time
• In the deterministic version (a-d):In the deterministic version (a-d):– in a, we need to check at most n nodesin a, we need to check at most n nodes– in b, at most in b, at most n/2n/2+1 nodes+1 nodes– in c, at most in c, at most n/4n/4+2 nodes+2 nodes– in general, at most in general, at most log Nlog N nodes nodes
• Efficient search, but impractical Efficient search, but impractical insertion and deletion.insertion and deletion.
LevelsLevels
• A node with k A node with k forward pointers is forward pointers is called a called a level klevel k node.node.
• If every (2If every (2ii)th node )th node has a pointer 2has a pointer 2ii nodes ahead, they nodes ahead, they have the following have the following distribution:distribution:
levellevel percentpercent
11 5050
22 2525
33 12.512.5
…… ……
Central idea in skip lists Central idea in skip lists
• Choose levels of nodes Choose levels of nodes randomlyrandomly, but , but in the same proportionsin the same proportions (as in e). (as in e).
• A node’s A node’s i i th forward pointer, points th forward pointer, points to the next node of level i or higher.to the next node of level i or higher.
• Insertions and deletions require only Insertions and deletions require only local modifications.local modifications.
• A node’s level never changes after A node’s level never changes after first being chosen.first being chosen.
InsertionInsertion
• To perform insertion, we must make To perform insertion, we must make sure that when a new node of height sure that when a new node of height hh is added, it doesn’t create a gap of is added, it doesn’t create a gap of four heights of four heights of hh node (in 1-2-3 node (in 1-2-3 deterministic skip list).deterministic skip list).
• See page 269 fig. 12.19See page 269 fig. 12.19• For implementation of Skip List see For implementation of Skip List see
pages 472-474.pages 472-474.
AA-TreesAA-Trees
• Also known as binary B-tree (BB-tree).Also known as binary B-tree (BB-tree).• BB-tree is a red-black tree with one extra condition: BB-tree is a red-black tree with one extra condition:
any node may have at most one red child.any node may have at most one red child.• Some conditions to make it simpler (p.475):Some conditions to make it simpler (p.475):
- only right child can be red- only right child can be red- code functions recursively- code functions recursively- instead of color store information in small - instead of color store information in small integer:integer:
- one if the node is a leaf- one if the node is a leaf- the level of its parent, if the node is red- the level of its parent, if the node is red- one less then the level of its parent, if - one less then the level of its parent, if
the the node is blacknode is black
AdvantagesAdvantages
• AA-trees simplify algorithms by:AA-trees simplify algorithms by:
- eliminating half of the restructuring cases- eliminating half of the restructuring cases
- simplifying deletion by removing an - simplifying deletion by removing an annoying caseannoying case
•if an internal node has only one child, if an internal node has only one child, that child must be a red right childthat child must be a red right child
•We can always replace a node with the We can always replace a node with the smallest child in the right sub tree (it smallest child in the right sub tree (it will either be a leaf or have a red child)will either be a leaf or have a red child)
Links in AA-treeLinks in AA-tree
• A A horizontalhorizontal link is a connection between a link is a connection between a node and a child with equal levels.node and a child with equal levels.
• HorizontalHorizontal links are right references. links are right references.• There cannot be two consecutive There cannot be two consecutive
horizontal links.horizontal links.• Nodes at level 2 or higher must have two Nodes at level 2 or higher must have two
children.children.• If a node has no right horizontal link, its If a node has no right horizontal link, its
two children are at the same level.two children are at the same level.
ExampleExample
30 70
85
5
60
8010
90
15
20
50
35 40 6555
Insertion in AA-treeInsertion in AA-tree
• A new item is always inserted at the A new item is always inserted at the bottom level.bottom level.
• In the previous example, inserting 2 will In the previous example, inserting 2 will create a horizontal left link.create a horizontal left link.
• In the previous example, inserting 45 In the previous example, inserting 45 generates consecutive right links.generates consecutive right links.
• After inserting at the bottom level, we may After inserting at the bottom level, we may need to perform rotations to restore the need to perform rotations to restore the horizontal link properties.horizontal link properties.
skewskew – remove left horizontal – remove left horizontal linkslinks
P X
A B C
P X
A B C
splitsplit – remove consecutive – remove consecutive horizontal linkshorizontal links
X P
A B
GX
P
A B
G
More on More on skew & splitskew & split
• skewskew removes a left horizontal link. removes a left horizontal link.• skewskew might also create consecutive might also create consecutive
right horizontal links.right horizontal links.• First we must apply First we must apply skewskew and then and then
use use split,split, if necessary. if necessary.• After a After a splitsplit, the middle node , the middle node
increases a level, which may create a increases a level, which may create a problem for the original parent.problem for the original parent.
Implementation of AA-treesImplementation of AA-trees
• Refer to pages 476 – 480 for detailed Refer to pages 476 – 480 for detailed implementation techniques.implementation techniques.
• See page 477 & 479 for more See page 477 & 479 for more examples of left and right rotations examples of left and right rotations of AA-trees.of AA-trees.
TreapsTreaps
• Binary search tree.Binary search tree.
• Like skip list, it uses random Like skip list, it uses random numbers and gives numbers and gives O (log N)O (log N) expected time for any input.expected time for any input.
• Slower than balanced search tree.Slower than balanced search tree.
• Although deletion is much slower, it Although deletion is much slower, it is still is still O (log N)O (log N) expected time. expected time.
Definition of a treapDefinition of a treap
• Each node stores an item, left and right Each node stores an item, left and right link, and a priority that is randomly link, and a priority that is randomly assigned when the node is created.assigned when the node is created.
• Treap is a binary search tree with the Treap is a binary search tree with the property that the node priorities satisfy property that the node priorities satisfy heap order: any node’s priority must be at heap order: any node’s priority must be at least as large as its parents.least as large as its parents.
• See pages 481-483 for implementation See pages 481-483 for implementation details.details.
k – d Treesk – d Trees
• Multidimensional b-tree.Multidimensional b-tree.• Branching of odd levels Branching of odd levels
is done with respect to is done with respect to the first key, and the first key, and branching on even levels branching on even levels is done with respect to is done with respect to the second key.the second key.
• Root is arbitrary chosen Root is arbitrary chosen to be an odd level.to be an odd level.
• Can be visually Can be visually represented:represented:
Some facts about k-d treesSome facts about k-d trees
• Can have any number of dimensions.Can have any number of dimensions.• In practice searches tend to be very In practice searches tend to be very
efficient.efficient.• For a randomly constructed tree, the For a randomly constructed tree, the
average running time of a partial average running time of a partial match query is match query is O (M+kN^(1-1/k))O (M+kN^(1-1/k))..
• See pages 484-485 for See pages 484-485 for implementation details.implementation details.
Pairing HeapsPairing Heaps
• A min (max) pairing heap is a min A min (max) pairing heap is a min (max) tree in which operations are (max) tree in which operations are done in a specified manner.done in a specified manner.
3
5 3
8
1
2 1
4
1
4 5
6
2
3
InsertInsert
• Create Create 11-element max tree with new item -element max tree with new item and meld with existing max pairing heap.and meld with existing max pairing heap.
3
6 7
9
6
7+ insert(2) =
3
6 7
9
6
72
Insert (Cont.)Insert (Cont.)
• Create Create 11-element max tree with new item -element max tree with new item and meld with existing max pairing heap.and meld with existing max pairing heap.
3
6 7
9
6
7
+insert(14) =
3
6 7
9
6
7
14
IncreaseKey (Node, IncreaseKey (Node, theAmount)theAmount)
• Since nodes do not have parent Since nodes do not have parent fields, we cannot easily check fields, we cannot easily check whether the key in the Node whether the key in the Node becomes larger than that in its becomes larger than that in its parent.parent.
• So, detach the Node from sibling So, detach the Node from sibling doubly-linked list and meld.doubly-linked list and meld.
IncreaseKey (Node, IncreaseKey (Node, theAmount)theAmount)
• If theNode is not the root, remove sub tree rooted at theNode from its sibling list.
1
2 6
9
4
5 1
6
1
3 2
4 2
4
3
3
theNode
IncreaseKey (Node, IncreaseKey (Node, theAmount)theAmount)
2
18
3
3
1
2 6
9
4
5 1
6
1
3 2
4
IncreaseKey (Node, IncreaseKey (Node, theAmount)theAmount)
1
2 6
9
4
5 1
6
1
3 2
4
2
18
3
3
Pairing heapsPairing heaps
• See pages 488 – 491 for See pages 488 – 491 for implementation details.implementation details.
