Database System Architecture and Performance

CSCI 6442

©Copyright 2015, David C. Roberts, all rights reserved

2

Agenda

Database performance goals DBMS use of disk Searching B-trees DBMS Architecture

3

DBMS Architecture

Data is stored on disk Disk is necessary for database to be

reliably available Disk is millions of times slower than

anything that happens in RAM Number of disk accesses is a good

measure of DBMS cost for an operation

4

Disk

o Disk is composed of fixed-length records, rotating around

o To access information, we need to move the head and wait for the disk to rotate

o We wait the same time whether we use one byte or all the record

o We call this fixed length record a page

5

Efficient Use of Disk

For efficient use of disk, we want to use all the information contained in a single page

We will look at how we organize disk in order to reduce the number of disk accesses for a search

6

Disk vs. RAM

RAM is accessible in any order Any sort of structures can be used Data structure courses usually cover

data structures for RAM We’ll talk about how to make efficient

use of disk

7

Disk as Pages

Disk is composed of fixed-length records, rotating around

To access information, we need to move the head and wait for the disk to rotate

We wait the same time whether we use one byte or all the record

We call this fixed length record a page

8

Search Methods

Linear search Binary search Binary tree-structured search N-ary trees B-trees Hashing

9

Linear Search

Elements are stored in arrival order Search starts at the beginning,

continues until desired value is found Average number of accesses for n

elements is approximately n/2

10

Binary Search

Elements are stored in order by value to be searched

Search starts at midpoint With each probe, half of candidates

are removed Average number of probes is log2n

11

Disadvantages of Binary Search

Elements must be kept in order Inserting one element may require

reorganization of entire list If stored, search jumps from bucket to

bucket

12

Using Linked Structure for Binary Search

Using links we can separate physical organization from search sequence

Avoids possible need to reorganize the entire store because of a single update

Accelerates update, still allows fast search

13

Example Binary Search Tree

14

Problems with Binary Search Tree

Each node is likely to be on a different page, making inefficient use of disk accesses

What if, instead of just one key at each node, we could store a whole page full of keys?

Then we would use disk efficiently and have a very shallow tree

15

Balance

16

Balance

A tree is said to be balanced if the length of all the paths from the root to the leaves differ by no more than one.

17

B-tree

We allow nodes to be incompletely filled in order to maintain perfect balance

We grow the tree from the bottom; when a node is over-full we split it and put an added node one level up

Deletions are the reverse of additions

18

B-tree

19

B-tree

Data Store

We understand that with each entry there is an address in storage. Having understood that, we omit them from the rest of the diagrams

20

B-tree

21

B-tree

1

22

B-tree

1 4

23

B-tree

1 4 6

24

B-tree

1 4 6 8

25

B-tree

5

1 4 6 8

26

B-tree

1 4

5

6 8

When a node is full, to add a value we split the node and put the middle value in the level above.

27

How It Really Looks

28

B-tree questions

How large should node size be? How many values should it contain?

Are the values indexed by a b-tree properly called keys?

How full are b-tree nodes, on the average, after the system has been operating for a while?

29

B-plus tree

B+ trees have all indexed values represented in the leaves

Other nodes do not have pointers to rows, only pointers to other nodes

B+ trees provide very high density of indexes

30

B+ tree

Index Set

Sequence Set

31

B+ Tree Add Algorithm

The insert algorithm for B+ Trees

Leaf Page Full

Index Page FULL

Action

NO NO Place the record in sorted position in the appropriate leaf page

YES NO

1. Split the leaf page 2. Place Middle Key in the index page in sorted order. 3. Left leaf page contains records with keys below the middle key. 4. Right leaf page contains records with keys equal to or greater than

the middle key.

YES YES

1. Split the leaf page. 2. Records with keys < middle key go to the left leaf page. 3. Records with keys >= middle key go to the right leaf page. 4. Split the index page. 5. Keys < middle key go to the left index page. 6. Keys > middle key go to the right index page. 7. The middle key goes to the next (higher level) index.

IF the next level index page is full, continue splitting the index pages.

32

B+ Tree Delete Algorithm

The delete algorithm for B+ Trees

Leaf Page Below

Fill Factor

Index Page Below

Fill Factor

Action

NO NODelete the record from the leaf page. Arrange keys in ascending order to fill

void. If the key of the deleted record appears in the index page, use the next key to replace it.

YES NOCombine the leaf page and its sibling. Change the index page to reflect the

change.

YES YES

1. Combine the leaf page and its sibling. 2. Adjust the index page to reflect the change. 3. Combine the index page with its sibling.

Continue combining index pages until you reach a page with the correct fill factor or you reach the root page.

33

Hashing

Develop a function that maps data values into a range of storage addresses

For each search value, use a function to compute a hash value and store the associated data at that address

To search, just compute the hash value and look at that address

34

Hashing

Instead of storing the data at the hash address, store a pointer to the data

The table of pointers is called a hash table

Using hashing for a search locates a stored value in just one access

Number of accesses to locate a value is independent of n

35

Hashing Question

Why are b-trees the most used index method for database systems and not hashing, given that hashing is faster?

Hint—think about the disadvantages of hashing

Database System Architecture

37

DBMS and Applications

ApplicationProgram

Buffer

ApplicationProgram

Buffer

ApplicationProgram

Buffer

ApplicationProgram

Buffer

DatabaseManagement

System

38

DBMS Software Architecture

ApplicationProgram

Buffer

ApplicationProgram

Buffer

ApplicationProgram

Buffer

ApplicationProgram

Buffer

SystemGlobalArea

DatabaseSystem

39

Database System Architecture

Lexical Analyzer

Syntax Analyzer

SQL Tokens

Executor

Quads

Results

40

Executor Software Architecture

SQL Executor

Table Management

Row Management

Page Management

Node Management

Index Management

Data Store

41

DBMS and Applications

ApplicationProgram

Buffer

ApplicationProgram

Buffer

ApplicationProgram

Buffer

ApplicationProgram

Buffer

DatabaseManagement

System

42

DBMS Software Architecture

ApplicationProgram

Buffer

ApplicationProgram

Buffer

ApplicationProgram

Buffer

ApplicationProgram

Buffer

SystemGlobalArea

DatabaseSystemCode

43

Inside the Database System

Lexical Analyzer

Syntax Analyzer,

CodeGenerator

SQL Tokens

Executor

Quads

Results

44

Executor Software Architecture

SQL Executor

Table Management

Row Management

Page Management

Node Management

Index Management

Data Store

Pages

Disk is divided into physical records called “pages”

A page can be an index page (ie b-tree) or a data page

Index page contains one node of a b-tree

Data page contains rows of tables

45

Page Allocation

Pages are initially considered all unallocated

In response to requests, they are allocated and marked allocated

When freed, they are chained onto a list of free pages

46

Database Extents

Database needs to be able to extend over disk boundaries Size may require it Growth may require it

Typically it’s managed as “extents”, each of which is a file to the OS file system

Multiple files are mapped into a single sequence of page IDs

47

48

Extents

SQL Executor

Table Management

Row Management

Page Management

Node Management

Index Management

Data Store

Extent Management

The Database

Extent 3

Extent 2

Extent 1

Row

<<tid>,<rid>,<cid><cli><cv>, … ,<cid>,<cli>,<cv>, … >>49

Startup

At startup, DBMS creates an empty system catalog

Catalog has images of some tables; once images are established, then SQL can be used to create other tables

50

System Catalog

System Catalog tells you how the database system works

When the system starts with a new database, it lays down part of the system catalog from an image

The rest of the system catalog is created by SQL statements

Many SQL statements reference or change the system catalog

51

Database Performance

52

Join Processing

For a non-join query be sure there are indexes on columns used in predicates

Joins are the issue in database performance

We need to understand how they are performed so that we can make them efficient

53

“Optimization”

More properly called access path selection “Optimizer” selects a strategy for processing Approaches:

Cost-based: estimate total cost to process by different approaches, choose lowest estimate

Heuristic: use rules to decide how to process Cost-based is typically used by all database

systems today

54

The Optimizer

Selects indexes to use Chooses the order of using indexes Chooses algorithms to use Decides when to apply predicates

55

Classes of Predicates

Predicate: condition in the WHERE clause

Predicates are combined using AND, OR to make WHERE clauses

Classes of predicates: Sargable: search arguments that can be

processed close to the data Residual: not sargable, such as complex

use of nesting

56

Access Paths

Five possible access paths:Table scanNon-selective index scanSelective index scanIndex only accessFully qualified unique index

Each of these types of scans has different cost estimates for its use

57

Predicate Selectivity

Selectivity function f(p): % of rows retrieved on average by predicate p

Number of rows retrieved is strongly related to cost n = number of rows in table

58

Form of P f

column = value 1/n

column != value 1-1/n (nearly 1)

column > value (high value - search value)/high value - low value)

p1 or p2 f(p1) + f(p2)

p1 and p2 f(p1) * f(p2)

Join Processing

Cartesian Product: for each row of inner table, inspect join value for every row of outer table. n2 operations

Nested loop: for each row of inner table, use index to retrieve matching rows of outer table. > 2n operations

Merge join: single pass through indexes on join columns for both tables. 2n operations

59

Join Order

For JOIN queries, the “outer” table is access first, “inner” second

Order for joining tables must be selected

Most selective first Least costly joins first

60

Query Transformations

Queries and subqueries may be transformed

We’ll ignore this for now, look at the bigger picture

61

Database Statistics

System catalog includes various database statisticsMax, min valuesCardinality of each tableData distribution

Statistics must be updated

62