Post on 24-Apr-2020
transcript
Chapter 6
External Memory Structures
1
Acknowledgements• Disk storage, Wikipedia. https://en.wikipedia.org/wiki/Disk_storage
• B-tree, Wikipedia. https://en.wikipedia.org/wiki/B-tree
• R-tree, Wikipedia. https://en.wikipedia.org/wiki/R-tree
• Multimedia Databases and Data Mining. Primary key indexing – B-trees. Christos Faloutsos – CMU
• Spatial Access Methods, Chapter 26 of book. Dr Eamonn Keogh, Computer Science & Engineering Department, University of California –Riverside, Riverside,CA 92521
2
Chapter Outline• External Disk Storage
• Working with External Data
• B-tree
• B+-tree
• R-tree
3
External Storage
4
Computer Architecture
5
Types of External Memory• Magnetic Tape
• Optical• CD (Compact Disc)
• CD-ROM
• CD-R
• CD-RW
• VCD (Video Compact Disc)
• DVD (Digital Video/Versatile Disc)
• Magnetic Disk• RAID
6
Magnetic Disk• Metal or plastic disk coated, on one or both sides,
with magnetizable material
• Data read and written through a magnetic head (coil) by means of induction
7
8
Disk Data Layout
Data Organization and Formatting
• Concentric rings or tracks• Gaps between tracks, reduce gap to increase capacity
• Same number of bits per track (variable density)
• Constant angular velocity
• Tracks divided into sectors
• Data read/written in blocks• Minimum block size is one sector
• May have more than one sector per block
9
10
Finding Sectors• Must be able to identify start of track and sector
• Format disk• Additional information not available to user
• Marks tracks and sectors
11
Multiple Platters• One head per side
• Heads are joined and aligned
• Aligned tracks on each platter form cylinders
• Data is striped by cylinder• reduces head movement
• increases speed (transfer rate)
Speed• Seek time
• Moving head to the right track
• (Rotational) latency• Waiting for data to rotate under head
• Access time = Seek + Latency
• Transfer rate: speed of copying bytes from disk
12
Total time
Look at External Storage
• A file partitioned into blocks of records
RAID• Redundant Arrays of Independent Disks
14
B-tree
15
16
B-trees
Eg., B-tree of order 3:
1 3
6
7
9
13
<6
>6 <9>9
17
B - tree properties:• each node, in a B-tree of order n:
• Key order• at most n pointers
• at least n/2 pointers (except root)
• all leaves at the same level
• if number of pointers is k, then node has exactly k-1 keys
• (leaves are empty)
v1 v2 … vn-1
p1 pn
18
Queries• Algo for exact match query? (eg., ssn=8?)
1 3
6
7
9
13
<6
>6 <9>9
19
Queries• Algo for exact match query? (eg., ssn=8?)
1 3
6
7
9
13
<6
>6 <9>9
20
Queries• Algo for exact match query? (eg., ssn=8?)
1 3
6
7
9
13
<6
>6 <9>9
21
Queries• Algo for exact match query? (eg., ssn=8?)
1 3
6
7
9
13
<6
>6 <9>9
22
Queries• Algo for exact match query? (eg., ssn=8?)
1 3
6
7
9
13
<6
>6 <9>9
H steps (= disk
accesses)
23
Queries
• what about range queries? (eg., 5<salary<8)
• Proximity/ nearest neighbor searches? (eg., salary ~ 8 )
1 3
6
7
9
13
<6
>6 <9>9
24
Queries
• what about range queries? (eg., 5<salary<8)
• Proximity/ nearest neighbor searches? (eg., salary ~ 8 )
1 3
6
7
9
13
<6
>6 <9>9
25
B-trees: Insertion• Insert in leaf; on overflow, push middle up
(recursively)• split: preserves B - tree properties
26
B-trees: Insertion
Easy case: Tree T0; insert ‘8’
1 3
6
7
9
13
<6
>6 <9>9
27
B-trees: Insertion
Tree T0; insert ‘8’
1 3
6
7
9
13
<6
>6 <9>9
8
28
B-trees: Insertion
Hardest case: Tree T0; insert ‘2’
1 3
6
7
9
13
<6
>6 <9>9
2
29
B-trees: Insertion
Hardest case: Tree T0; insert ‘2’
1 2
6
7
9
133
push middle up
30
B-trees: Insertion
Hardest case: Tree T0; insert ‘2’
6
7
9
131 3
22Ovf; push middle
31
B-trees: Insertion
Hardest case: Tree T0; insert ‘2’
7
9
131 3
2
6
Final state
32
B-trees: Deletion
• Case1: delete a key at a leaf – no underflow
• Case2: delete non-leaf key – no underflow
• Case3: delete leaf-key; underflow, and ‘rich sibling’
• Case4: delete leaf-key; underflow, and ‘poor sibling’
33
B-trees: Deletion
Easiest case: Tree T0; delete ‘3’
1 3
6
7
9
13
<6
>6 <9>9
34
B-trees: Deletion
Easiest case: Tree T0; delete ‘3’
1
6
7
9
13
<6
>6 <9>9
35
B-trees: Deletion
Easiest case: Tree T0; delete ‘3’
1
6
7
9
13
<6
>6 <9>9
15-826 Copyright: C. Faloutsos (2012) 36
B-trees: Deletion
• Case1: delete a key at a leaf – no underflow
• Case2: delete non-leaf key – no underflow
• Case3: delete leaf-key; underflow, and ‘rich sibling’
• Case4: delete leaf-key; underflow, and ‘poor sibling’
15-826 Copyright: C. Faloutsos (2012) 37
B-trees: Deletion
• Case2: delete a key at a non-leaf – no underflow (eg., delete 6 from T0)
1 3
6
7
9
13
<6
>6 <9>9
Delete &
promote, ie:
38
B-trees: Deletion
• Case2: delete a key at a non-leaf – no underflow (eg., delete 6 from T0)
1 3 7
9
13
<6
>6 <9>9
Delete &
promote, ie:
39
B-trees: Deletion
• Case2: delete a key at a non-leaf – no underflow (eg., delete 6 from T0)
1 7
9
13
<6
>6 <9>9
Delete &
promote, ie:3
40
B-trees: Deletion
• Case2: delete a key at a non-leaf – no underflow (eg., delete 6 from T0)
1 7
9
13
<3
>3 <9>9
3FINAL TREE
41
B-trees: Deletion
• Case2: delete a key at a non-leaf – no underflow (eg., delete 6 from T0)
• Q: How to promote?
• A: pick the largest key from the left sub-tree (or the smallest from the right sub-tree)
• Observation: every deletion eventually becomes a deletion of a leaf key
42
B-trees: Deletion
• Case1: delete a key at a leaf – no underflow
• Case2: delete non-leaf key – no underflow
• Case3: delete leaf-key; underflow, and ‘rich sibling’
• Case4: delete leaf-key; underflow, and ‘poor sibling’
43
B-trees: Deletion
• Case3: underflow & ‘rich sibling’ (eg., delete 7 from T0)
1 3
6
7
9
13
<6
>6 <9>9
Delete &
borrow, ie:
44
B-trees: Deletion
• Case3: underflow & ‘rich sibling’ (eg., delete 7 from T0)
1 3
6 9
13
<6
>6 <9>9
Delete &
borrow, ie:
Rich sibling
45
B-trees: Deletion
• Case3: underflow & ‘rich sibling’
• ‘rich’ = can give a key, without underflowing
• ‘borrowing’ a key: THROUGH the PARENT!
46
B-trees: Deletion
• Case3: underflow & ‘rich sibling’ (eg., delete 7 from T0)
1 3
6 9
13
<6
>6 <9>9
Delete &
borrow, ie:
Rich sibling
NO!!
47
B-trees: Deletion
• Case3: underflow & ‘rich sibling’ (eg., delete 7 from T0)
1 3
6 9
13
<6
>6 <9>9
Delete &
borrow, ie:
48
B-trees: Deletion
• Case3: underflow & ‘rich sibling’ (eg., delete 7 from T0)
1 3
9
13
<6
>6 <9>9
Delete &
borrow, ie:
6
49
B-trees: Deletion
• Case3: underflow & ‘rich sibling’ (eg., delete 7 from T0)
1
3 9
13
<6
>6 <9>9
Delete &
borrow, ie:
6
50
B-trees: Deletion
• Case3: underflow & ‘rich sibling’ (eg., delete 7 from T0)
1
3 9
13
<3
>3 <9>9
Delete &
borrow,
through the
parent
6
FINAL TREE
51
B-trees: Deletion
• Case1: delete a key at a leaf – no underflow
• Case2: delete non-leaf key – no underflow
• Case3: delete leaf-key; underflow, and ‘rich sibling’
• Case4: delete leaf-key; underflow, and ‘poor sibling’
52
B-trees: Deletion
• Case4: underflow & ‘poor sibling’ (eg., delete 13from T0)
1 3
6
7
9
13
<6
>6 <9>9
53
B-trees: Deletion
• Case4: underflow & ‘poor sibling’ (eg., delete 13from T0)
1 3
6
7
9<6
>6 <9>9
54
B-trees: Deletion
• Case4: underflow & ‘poor sibling’ (eg., delete 13from T0)
1 3
6
7
9<6
>6 <9>9
A: merge w/
‘poor’ sibling
55
B-trees: Deletion
• Case4: underflow & ‘poor sibling’ (eg., delete 13from T0)
• Merge, by pulling a key from the parent
• exact reversal from insertion: ‘split and push up’, vs. ‘merge and pull down’
56
B-trees: Deletion
• Case4: underflow & ‘poor sibling’ (eg., delete 13from T0)
1 3
6
7
<6
>6
A: merge w/
‘poor’ sibling
9
57
B-trees: Deletion
• Case4: underflow & ‘poor sibling’ (eg., delete 13from T0)
1 3
6
7
<6
>6
9
FINAL TREE
B+-Tree
58
59
B+-trees: Motivation
if we want to store the whole record with the key –> problems (what?)
1 3
6
7
9
13
<6
>6 <9>9
60
Solution: B+-trees • They string all leaf nodes together
• AND
• replicate keys from non-leaf nodes, to make sure every key appears at the leaf level
61
B+ trees
1 3
6
6
9
9
<6
>=6 <9>=9
7 13
R-Tree
62
Spatial Data
• Given such a database we can easily answer queries by using SQL, such as
• List all Mexican restaurants.• List all Grade A restaurants.
• However, classic databases do not allow queries such as• List all Mexican restaurants within five miles of UCR• List the pizza restaurant nearest to 91 and 60.
• These kinds of queries are called spatial queries• Nearest neighbor queries • Range queries • Spatial joins
63
B876,65878-1342ITA4Sues Pasta
A123,32878-1333MEX3Tinas Mexican
A34,764848-1298US2Joes Bugers
D244,365888-1212ITA1Marios Pizza
GradeLocationPhoneTypeIDName
B876,65878-1342ITA4Sues Pasta
A123,32878-1333MEX3Tinas Mexican
A34,764848-1298US2Joes Bugers
D244,365888-1212ITA1Marios Pizza
GradeLocationPhoneTypeIDName
Indexing Spatial Data• So, we call always index 1-dimensional data (if you can sort
it, you can index it), such that we can answer 1-nearest neighbor queries by accessing just O(log(n) ) of the database. (n is the number of items in the database). (i.e. the B-tree)
• But we cannot sort 2 dimensional data…
• Solution: R-Tree• introduced by Guttman in the 1984 SIGMOD conference.
64
R-Trees• R-trees are a N-dimensional extension of B+-trees,
useful for indexing sets of rectangles and other polygons.
• Supported in many modern database systems, along with variants like R+ -trees and R*-trees.
• Basic idea: generalize the notion of a one-dimensional interval associated with each B+ -tree node to an N-dimensional interval, that is, an N-dimensional rectangle.
• Will consider only the two-dimensional case (N = 2) • generalization for N > 2 is straightforward, although R-
trees work well only for relatively small N
65
R-Trees• A rectangular bounding box is associated with each
tree node.• Bounding box of a leaf node is a minimum sized rectangle
that contains all the rectangles/polygons associated with the leaf node.
• The bounding box associated with a non-leaf node contains the bounding box associated with all its children.
• Bounding box of a node serves as its key in its parent node (if any)
• Bounding boxes of children of a node are allowed to overlap
• A polygon is stored only in one node, and the bounding box of the node must contain the polygon• The storage efficiency or R-trees is better than that of k-d
trees or quadtrees since a polygon is stored only once
66
MBR• Suppose we have a cluster of points in 2-D space...
• We can build a “box” around points. The smallest box (which is axis parallel) that contains all the points is called a Minimum Bounding Rectangle (MBR)
67
MBR = {(L.x,L.y)(U.x,U.y)}
MINDIST• The formula for the distance between a point and
the closest possible point within an MBR
68
MBR = {(L.x,L.y)(U.x,U.y)}Q = (x,y)
MINDIST(Q,MBR)
if L.x < x < U.x and L.y < y < U.y then 0elseif L.x < x < U.x then min( (L.y -y)2 , (U.y -y)2 )elseif ….
MINDIST Example
69
MINDIST(point, MBR) = 5 MINDIST(point, MBR) = 0
MINDIST Example• Suppose we have a query point Q and one known
point R. Could any of the points in the MBR be closer to Q than R is?
70
MBR = {(6,1),(8,4)}
Q = (3,5)
R = (1,7)
Constructing MBR• Each MBR can be represented with just two points.
The lower left corner, and the upper right corner.
• We can further recursively group MBRs into larger MBRs….
71
R1
R2R5
R3
R7R9
R8
R6
R4
R10 R11
R12
Constructing R-Tree• …these nested MBRs are organized as a tree (called
a spatial access tree or a multidimensional tree).
72
R10 R11 R12
R1 R2 R3 R4 R5 R6 R7 R8 R9
Data nodes containing points
R10 R11
R12
Constructing R-Tree• At the leave nodes we
have the location, and a pointer to the record in question
• At the internal nodes, we just have MBR information
73
R10
R1 R2 R3
(3,4) 77(1,3) 88(2,3) 22(5,4) 13
(2,2) 47(3,0) 86(7,9) 52
(5,1) 32(1,4) 45(5,6) 27(7,8) 73
{(1,3),(5,4)} {(2,0),(7,9)} {(1,1),(7,8)}
{(1,0),(7,9)}
Search in R-Tree• To find data items (rectangles/polygons)
intersecting (overlaps) a given query point/region, do the following, starting from the root node:• If the node is a leaf node, output the data items whose
keys intersect the given query point/region.• Else, for each child of the current node whose bounding
box overlaps the query point/region, recursively search the child
• Can be very inefficient in worst case since multiple paths may need to be searched• but works acceptably in practice.
• Simple extensions of search procedure to handle predicates contained-in and contains
74
Insertion in R-Tree• To insert a data item:
• Find a leaf to store it, and add it to the leaf• To find leaf, follow a child (if any) whose bounding box contains
bounding box of data item, else child whose overlap with data item bounding box is maximum
• Handle overflows by splits (as in B+ -trees) • Split procedure is different though (see below)
• Adjust bounding boxes starting from the leaf upwards
• Split procedure:• Goal: divide entries of an overfull node into two sets such
that the bounding boxes have minimum total area • This is a heuristic. Alternatives like minimum overlap are possible
• Finding the “best” split is expensive, use heuristics instead• See next slide
75
Splitting an R-Tree Node• Quadratic split divides the entries in a node into two
new nodes as follows1. Find pair of entries with “maximum separation”
• that is, the pair such that the bounding box of the two would has the maximum wasted space (area of bounding box – sum of areas of two entries)
2. Place these entries in two new nodes3. Repeatedly find the entry with “maximum preference” for
one of the two new nodes, and assign the entry to that node• Preference of an entry to a node is the increase in area of
bounding box if the entry is added to the other node
4. Stop when half the entries have been added to one node• Then assign remaining entries to the other node
• Cheaper linear split heuristic works in time linear in number of entries,• Cheaper but generates slightly worse splits.
76
Deleting in R-Trees• Deletion of an entry in an R-tree done much like a
B+-tree deletion.• In case of underfull node, borrow entries from a sibling
if possible, else merging sibling nodes
• Alternative approach removes all entries from the underfull node, deletes the node, then reinserts all entries
• As always, deletion tends to be rarer than insertion for many real world databases.
77
End of Chapter 6