Answering Top-k Queries with Multi-Dimensional Selections: The Ranking Cube Approach
Dong Xin, Jiawei Han, Hong Cheng, Xiaolei Li
Department of Computer ScienceUniversity of Illinois at Urbana-Champaign
VLDB 2006
2
Outline
• Introduction• Ranking cube• Answering top-k queries by Ranking Cube• Ranking Fragments• Performance study• Discussion and Conclusions
3
Multi-Dimensional Ranking Analysis• Consider an online used car database R
Type (e.g., sedan, convertible)Maker (e.g., Ford, Hyundai)Color (e.g., red, silver)PriceMileage
select top 10 * from Rwhere type = “convertible” order by price + mileage asc
select top 10 * from Rwhere type = “convertible” and color = “red”order by price + mileage asc
select top 10 * from Rwhere type = “convertible” and color = “red” and maker = “Ford”order by price + mileage asc
Roll Up
Drill Down
4
OLAP with Ranking?• Data cube revisited
– Pre-compute multi-dimensional group-bys– Traditional measures: SUM, COUNT, AVG
• Materializing all top-k results is not feasible– Different k values– Various ranking functions
• e.g., order by (price-20k)^2 + (mileage-10k)^2 asc
• Our Proposal: Ranking Cube– Semi-online computation with semi-offline materialization– Support a broad class of ranking functions
5
More on Rank-Aware Data Cube
• Given a relation R– A1, A2, …, As are selection dimensions– N1,N2,…,Nr are ranking dimensions– {Ai} and {Nj} are not exclusive
• Our goal: efficiently answering top-k queries in a multi-dimensional space
Ranking function f(x) satisfies:
1. Given the sub-domain of x, the extreme point x* can be computed
2. Given a sub-domain of x, the upper and lower bounds of f(x) can be computed
6
Rank-Aware Query Processing• Rank-aware materialization for linear functions
– Onion [Chang et al, SIGMOD’00], PREFER [Hristidis et al. SIGMOD’01], Robust Indexing [Xin et al. VLDB’06]
• Rank-aware query transformation– Map rank query to range query [Chaudhuri et al. VLDB’99, Bruno et al. TO
DS’02]
• Rank-aware query optimization– TA [Fagin et al. PODS’ 01], RankSQL [Li et al. SIGMOD’05], Boolean+Ran
king [Zhang et al. SIGMOD’06]
• Rank aggregate– RankAgg [Li et al. SIGMOD’06], ObjectFinder [Kaushik et al. SIGMOD’06]
• Rank query with Joins– Ranked Join indices [Tsaparas et al. ICDE’03], Rank-Join [Ilyas et al, VLD
B’03, SIGMOD ’04]
• And more…
7
What’s New with Ranking Cube• An effort made to enrich the data cube
– Inherit the power of multi-dimensional analysis
– A new rank-aware materialization without assuming particular (e.g., linear) function
• Top-k query processing based on Rank Cube– Transfer a top-k query to a sequence of selection queries
– Block-level access instead of tuple-level access
– No modification needed in DBMS
8
Outline
• Introduction• Ranking cube• Answering top-k queries by Ranking Cube• Ranking Fragments• Performance study• Discussion and Conclusions
9
Ranking Cube• Intuition
– Given a ranking function, the ranking cube should be able to:• Quickly locate the most promising data region• How many tuples are there, and which tuples?• Efficient data retrieval
• Approach– Step 1: Create logical block space for rank analysis
• Group geometrically closed tuples into blocks by data partitioning
– Equi-depth– R-tree– Clustering
• Compute (logical) block ID for each block• The logical block space constitutes the basis for data cubing
10
Ranking Cube (cont.)• Approach (cont.)
– Step 1: Create logical block space for rank analysis• Each tuple is associated with a (logical) block ID
– Step 2: Compute measures in ranking cube• Group-by with selection dimensions• Straight-forward measure: logical block IDs, as well as the lis
t of tuple ID (TID) inside• Alternative measure: Compressed version (will discuss later)
– Step 3: Create physical block space for efficient I/O• The size of the logical block differs in each cuboids due to th
e multi-dimensional selections• Group nearby logical blocks into a physical block for efficient
data retrieval
11
Constructing Ranking Cube
A1 A2 {B: TID}
1 1 {1: 1,4} {5: 3}
1 2 {11: 2}
.. .. …
Expected Logical Block
Size P
Measure in Ranking Cube
A cell in ranking cube
Generating Logical Block Dimension
A1,A2: Selection Dimensions
N1,N2: Ranking Dimensions
Create Logical Block Space
N1
N2
Data Cubing
Table for data cubing Block table
12
Constructing Ranking Cube (cont.)A1 A2 {B:TID}
1 1 {1:1,4}
.. .. …
A1 {B:TID}
1 {1: 1, 4}, {5: 3}
.. …
The sizes of TID list in different cuboids are not balanced due to the different
cardinality of each dimension
Physical block: Merge nearby logical blocks
Logical block: Original block
partitions
A1 A2 B’ {B:TID}
1 1 1 {1:1,4} ,{5:3}
1 1 3 {9:17}
.. .. .. …
Physical Block
13
Outline
• Introduction• Ranking cube• Answering top-k queries by Ranking Cube• Ranking Fragments• Performance study• Discussion and Conclusions
14
Query Processing (1)
• Data access methodsGet physical block from Ranking Cube:
Clustered index on Cell identifiers (A1, A2, B’)
Get logical block from Block Table:
Clustered index on B
A1 A2 B’ {B:TID}
1 1 1 {1:1,4} ,{5:3}
1 1 3 {9:17}
.. .. .. …
15
Query Processing (2)Locate the first logical block (b1)The target physical block (t1, t3, t4) is retrieved(t1,t4) is returned, t3 is buffered
Query processing works on logical block space
Data accessing works on physical block space
Ranking Cube maintains the mapping between logical block and physical block
S list: maintains the current top answers
H list: maintains the best possible unseen answers
Locate the second logical block (b5)The target physical block (t1, t3, t4) is identifiedt3 was buffered, thus is directed returned
Select top 2 * from R where A1=1 and A2=1 order by N1+N2 asc
16
Query Processing (3)• Determine next logical blocks to be retrieved
– First logical block: analyzed by ranking function– Continuing logical blocks
• Found in neighboring blocks (for convex functions)• Decompose the space and analyze each of them (for other
functions)
Ranking Function:
N1+N2
First Block
Second Block
Ranking Function:
(N1-0.5)^2+(N2-0.5)^2
First Block
Second Block
17
Outline
• Introduction• Ranking cube• Answering top-k queries by Ranking Cube• Ranking Fragments• Performance study• Discussion and Conclusions
18
Ranking Fragments (1)
• Curse of dimensionality?
ABCD
ABC ABD ACD BCD
AC
BC
AD
BD
CD
A DB C
ABPartition dimensions into several groups
Materialize low dimensional cuboids offline
Assembly high dimensional cuboids online
Mining Cube Approach
[Li et al, VLDB’04]
19
Ranking Fragments (2)
Materialized:
Cuboids A1, Cuboids A2
To Assemble:
Cuboids A1A2
Do not assemble the whole cuboids
Assemble the required cells only
Requested Cell in Cuboids A1A2
Cuboids A1:
Request (A1=1, B=b1), return { t1, t4, t10,…}
Cuboids A2: Request (A2=1, B=b1), return {t1, t4, t3,…}
Merge two lists for cuboids A1A2:
Request (A1=1,A2=1, B=b1), return { t1, t4,}
20
• Decouple the cubing and partitioning modules• Unified logical block space for all cuboids
– Reduces computation and space comparing with m-dim indices– Makes the online fragment assembly easier– Advanced partitioning methods for high-dimensional and structural
data
• Compressing TID list – Lossless compression: e.g., dictionary encoding, null suppression– Lossy compression: e.g., bloom filter– High-level summary: e.g., count (mean) in each logical block– Compressing across cuboids: e.g., correlation between cells
• Block-level data access instead of tuple-level access
Ranking Cube: Beyond the index
21
Outline
• Introduction• Ranking cube• Answering top-k queries by Ranking Cube• Ranking Fragments• Performance study• Discussion and Conclusions
22
Experimental Results
• Performance study– Baseline: SQL Server
• Index on each dimension– Query transformation [Chaudhuri et al. VLDB’99]
• Transform a ranked query to a range selection query• Multi-dimensional index
– Ranking cube (fragment)• Index on cube cells (cuboids)• Index on block ids (block tables)
23
Experiment Setting• Implementation details
– C# + MS SQL Server 2005– Store all ranking cubes, block tables in SQL Server
• Synthetic data sets
• Real data set: Forest CoverType – 12 selection dimensions, 3 ranking dimensions, 3.5M tu
ples
24
Execution Time w.r.t. K
Synthetic data
Default parameter setting
25
# Dimensions in Ranking Function
Synthetic data with 4 ranking dimensions
Partitioning was built on all 4 ranking dimensions
The number of ranking attributes in queries are varied from 2 to 4
26
Number of Data Tuples
Vary the size of the database from 1M to 10M
Very promising performance on large datasets
27
Ranking Fragments
Forest CoverType data
Partition selection dimensions into groups with size 3
Build ranking fragments on each group
Vary the fragment size
28
Space Usage
1. Space usage grows linearly with number of selection dimensions
2. Most space is used to store the cell identifiers in the relational table
3. The space usage can be greatly reduced by storing the ranking cube out of the relational table
Build ranking fragments with group size 2
29
Conclusions and Future work• OLAP with Ranking
– Ranking Cube as semi-offline materialization– Ranked query processing by semi-online computation
• Current status– Extended to multi-relational ranked queries using
multi-rank-cube
• Future work– Apply compression techniques– Exploit and compare different partitioning strategies– Support more query types