0 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Institute of Theoretical Informatics - Algorithmics
Advanced Data StructuresSimon Gog – [email protected]
KIT – University of the State of Baden-Wuerttemberg andNational Research Center of the Helmholtz Association www.kit.edu
Predecessor data structures
1 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
We want to support the following operations on a set of integers from thedomain U = [u].
insert(x) Add x to S. I.e. S′ = S ∪ x.delete(x) Delete x from S. I.e. S′ = S \ x.member(x) = |x ∈ S|predecessor(x) = maxy |y ≤ x ∧ y ∈ Ssuccessor(x) = miny |y ≥ x ∧ y ∈ S
where x is an integer in U and S the set of integers of size n stored in thedata structure.
minS = successor(0)maxS = predecessor(u − 1)
Solution know from „Algo I”: Balanced search trees. E.g. red-black trees.In all comparison based approaches at least one operation takesΩ(log n) time. Why?
Predecessor data structures
2 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Ω(log n) bound can be beaten in the word RAM model:Memory is organized in words of b ∈ O(log u) bitsA word can be accessed in constant timeWe can address all data using one wordStandard arithmetic operations take constant time on words (i.e.addition, subtraction, division, shifts . . .)
We first concentrate on the static case: The set S is fixed. I.e. no insertand delete operations.
x-fast trie (Willard, 1982)
3 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Conceptional: Complete binary tree of hight w = dlog ue
0
0 0
0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Operations member/successor/prodecessor can be answered inO(w) time by traversing the tree
x-fast trie (Willard, 1982)
3 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Conceptional: Complete binary tree of hight w = dlog ue
0
0 0
0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1
1
1
1
1
Operations member/successor/prodecessor can be answered inO(w) time by traversing the tree
x-fast trie (Willard, 1982)
3 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Conceptional: Complete binary tree of hight w = dlog ue
0
0 0
0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1
1
1
1
1
1
1
1
1
1
Operations member/successor/prodecessor can be answered inO(w) time by traversing the tree
x-fast trie (Willard, 1982)
3 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Conceptional: Complete binary tree of hight w = dlog ue
0
0 0
0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
Operations member/successor/prodecessor can be answered inO(w) time by traversing the tree
x-fast trieFrom O(log u) to O(log log u)...
4 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
0
0 1
0 1 2 3
0 1 2 3 4 5 6 7
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 150 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0
0
0
1
3
0
1
2
5
10
0
1
3
6
12
x-fast trieFrom O(log u) to O(log log u)...
5 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
0
0
0
1
3
0
1
2
5
10
0
1
3
6
12
Generate a perfect hash table hi for each level i (keys are the presentnodes)Note: Node numbers (represented in binary) are prefixes of keys in S
x-fast trieFrom O(log u) to O(log log u)...
5 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
h0
h1
h2
h3
h4
0
0
0
1
3
0
1
2
5
10
0
1
3
6
12
Generate a perfect hash table hi for each level i (keys are the presentnodes)Note: Node numbers (represented in binary) are prefixes of keys in S
x-fast trieQuery time from O(log u) to O(log log u)...
6 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Member queries can be answered by a binary search using prefixes ofthe searched key.There are w prefixes, i.e. binary search takes O(log w) or O(log log u)timeSpace: w ·O(n) words, i.e. O(n log u) bits
Predecessor/Successor queries
For each node store a pointer to the maximal/minimal leaf in itssubtreeUse a double linked list to represent leaf nodes.Solving predecessor: Search for a node v which represents thelongest prefix of x with any key in S. Two cases:
Minimum in subtree of v is larger x : Return element to the left of the leaf.Maximum in the subtree of v is smaller than x . Return maximum.
y-fast trie (Willard, 1982)Space from O(n log u) to O(n) words...
7 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Split S into nw blocks of O(log u) elements B0,B1, . . . ,Bd n
w e−1.
maxBi < minBi+1 for 0 ≤ i < d nw e − 1
Let ri = maxBi be a representative of block Bi .Build x-fast trie over representatives.
B0 B1 . . .
Total space: nw ·O(w) + O(n) = O(n) words
y-fast trieSpace from O(n log u) to O(n) words...
8 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Use sorted array to represent BiA member query is answered as follows
Search for successor of x in x-trie of ri ’sLet Bk be the block of the successor of xSearch in O(log w) = O(log log u) time for x in Bk
How does predecessor/successor work?
y-fast trie
9 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Changes to make structure dynamicuse cuckoo hashing for x-fast trieuse balanced search trees of size between 1
2w and 2w for Bisrepresentative is not the maximum, but any element separating twoconsecutive groups
Summary:
Operation static y-fast trie dynamic y-fast triepred(x)/succ(x) O(log log u) w.c. O(log log u)insert(x)/delete(x) O(log log u) exp. & am.construction O(n) exp.
Van Emde Boas Trees
10 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Conceptual bitvector B of length u with B[i ] = 1 for all i ∈ SSplit B into u/
√u blocks (blue blocks) B0,B1, ...
Set bit in R[i ] if there is at least one bit set in Bi
Also store the minimum/maximum of S
0 0 0 0Summary R
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0B= 1 1 1 1 1 1 1
1 1 1
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
min = 1max = 14
Here: u = 16
Van Emde Boas Trees
11 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Define van Emde Boas tree (vEB tree) recursivelyI.e. use vEB to represent Bi ’s and RvEB(u) denotes the vEB tree on a universe of size uBase case: u = 2. Only one node and variables min, max.
Technicalities↑√u = 2d(log u)/2e
↓√u = 2b(log u)/2c
high(x) =⌊
x↓√u
⌋(block that contains x)
low(x) = x mod ↓√u (relative position of x in Bhigh(x))
Van Emde Boas Trees
12 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Predecessor(x , B) (first attempt)
Let y = hight(x) and z = Predecessor (low(x),By )
If z 6= ⊥ return z + y · ↓√
uLet b = Predecessor (high(x),R)
If b 6= ⊥ return max(Bb) + b · ↓√
uReturn ⊥
Problem
Recurrence for time complexity: T (u) = 2T (√
u) + O(1)Solution (Master Theorem or drawing recursion tree): T (u) = Θ(log u)Now: avoid one of the recursive calls
Van Emde Boas Trees
13 Simon Gog:Advanced Data Structures
Institute of Theoretical InformaticsAlgorithmics
Predecessor(x , B) (second attempt)
If x > max return maxLet y = high(x), if min(By ) < x return Predecessor (low(x),By )
Let b = Predecessor (high(x),R)
If b 6= ⊥ return max(Bb) + b · ↓√
uReturn ⊥
Recurrence for time complexity: T (u) = T (√
u) + O(1)Solution (Master Theorem or drawing recursion tree):T (u) = Θ(log log u)