Preference-Aware Query and Update Scheduling in Web-databases Huiming Qu Department of Computer...

transcript

Preference-Aware Query and Update Scheduling

in Web-databases

Huiming Qu

Department of Computer ScienceUniversity of Pittsburgh

Joint work with Prof. Alex Labrinidis

@HKUST, 12/18/20062Huiming Qu, University of Pittsburgh

Applications Applications

Online stock info. Weather, news… Inventory monitoring Network monitoring

Web-DB workloads Real-only queries Write-only updates Intensive and/or bursty

We focus on scheduling queries and

updates to enhance Web-Database performance

WWW Browser

WWW Server

Web-database

Data Warehouse

Queries

Updates

Why scheduling? Users care about

Timeliness and Staleness

Impact of scheduling A simple test

FIFO FIFO-UH (Update High) FIFO-QH (Query High)

2PL-HP [Abbott92] No best for both dimensions

Combining performance metrics Set constraint on one metric and optimize another [Kang04] Construct a single metric based on weighted aggregation [Abadi05]

FIFO-QH[23, 0.26]

FIFO [322, 0.07]

FIFO-UH [11591, 0]

1.E+00

1.E+01

1.E+02

1.E+03

1.E+04

1.E+05

0 0.1 0.2 0.3

Staleness (#uu)

Outline Introduction Quality Contracts

Framework for unifying timeliness and staleness metrics through user preferences

QUTS Query Update Time Share Scheduling according to

Quality Contract framework

Experiments Related work Conclusions

Quality Contracts (QC) Captures user preference

Between quality metrics Among different queries

worthto user

quality metric #i

max #i

min #i

period of max worth to user

period of some worth to user

point of zero worth to user

period of some refund for user period of max

refund for user

Quality Contract Convert performance on

individual metric to the “worth” to users

Quality Contract (QC) – cont. Each Query is

attached with a QC monotonically non-

increasing positive QC functions

QC<type, qosmax, rtmax, qodmax, uumax>

Optimization goal system profit, P

More details in [Labrinidis06] [Qu07]

Response Time (ms)

Staleness (# UU)

+qosmax

uumaxrtmax

qodmax

Response Time (ms)

Staleness (# UU)

+qosmax

uumaxrtmax

QoS function QoD function

Step QC

QoS function QoD function

Linear QC

Outline Introduction Quality Contracts QUTS: Query Update Time Share Scheduling Experiments Related work Conclusions

Profit guided scheduling Considers both value and constraint

Scheduling according to utility function in real time systems VRD [haritsa93]

Value/constraint inflated priorities for queries and updates

value constraint

Queries qosmax rtmax

Updates qodmax uumax

OK Incomparable

qodmax

Response Time (ms)

Staleness (# UU)

+qosmax

uumaxrtmax

Baseline algorithms Single priority queue

FIFO w/ incomparable priorities, queries and updates cannot be put

together

Dual priority queue [adelberg95] Separate update queue & query queue Prioritize queries and updates correspondingly UH (Update High) QH (Query High)

Eliminate hard-wired preference for either QoS or QoD! Profit-guided resource allocation (which queue to execute)

QUTS (Query Update Time Share) Dual priority queue with dynamic concatenation

Two-level scheduling scheme (meta-scheduling) Q1: how much CPU should be allocated to queries? Q2: how to deploy CPU allocation?

Query CPU percentage ρ (0 ≤ ρ ≤ 1) Approximate system profit P using ρ

Q1 - how much CPU to assign?

Optimal solution to maximize P

QoSmax profit

QoS QoD

Query share QoDmax profitUpdate share Query share

Quadratic function with linear constraints

Q2 - how to deploy CPU allocation? Periodically compute ρ and alleviate random variation

With in a single adaptation period , the percentage of CPU assigned to queries should be ρ.

Q2 - how to deploy CPU allocation? Adaptation period ω is divided to N atom time For each , the system refreshes the system state

according to ρ Query High: Update High:

Length of atom time Too small → too many interruptions Too large → miss opportunities

ρ = 0.5

Experiment Setup Real stock web site traces

on April 24, 2000 # queries: 82129 # updates: 496892 # stocks: 4602

Query execution time 5~9ms

Update execution time 1~5ms

Default atom time 10ms

Default adaptation time 1000ms

Algorithms FIFO UH QH QUTS

Meta Scheduling FIFO Update high Query high Profit guided

Query priorities FIFO VRD (Value over Relative Deadline)

Update priorities FIFO VRD (Value over Relative Deadline)

Scheduling algorithms compared through simulation

Experiment Design 1. Performance

• Compare QUTS with FIFO, UH, and QH under the entire spectrum of QC;

2. Adaptability• Evaluate the adaptability of QUTS to changing

workloads (QCs);

3. Sensitivity• Evaluate the sensitivity of QUTS to its two

parameters (ω and ).

1. Performance – QC setup QC <qosmax, rtmax, qodmax, uumax> Performance metric

Profit Percentage

QODmax% 0.1 0.2 … 0.9

QOSmax% 0.9 0.8 … 0.1

qodmax $10 ~ $19 $20 ~ $29 … $90 ~ $99

qosmax $90 ~ $99 $80 ~ $89 … $10 ~ $19

rtmax 50ms ~ 100ms

uumax 1

Response Time (ms)Q

Staleness (# UU)

+qosmax

uumaxrtmax

qodmax

9 sets of QC groups

1. Performance – All QC groups

FIFO UH QH QUTS

QOS% QOD%

Linear QCs

FIFO UH QH QUTS

QOS% QOD%

Step QCs

QUTS gives near maximal profit from both QoS and QoD.

Step QCs and linear QCs follow the same trend.

1. Performance – Various QC groups

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9QODmax%

eQOS% QOD% QOSmax%

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9QODmax%

QOS% QOD% QOSmax%

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

QODmax%

QOS% QOD% QOSmax%

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

QODmax%

QOS% QOD% QOSmax%

QUTS consistently achieves near maximal profit under the whole spectrum of QC

2. QUTS Adaptability Time is divided into four segments. In odd segments, each QC has

qodmax = 5 x qosmax

In even segments, each QC has qosmax = 5 x qodmax

QUTS adapts to QC changes fast!

3. QUTS Sensitivity

0.1 1 10 100

Adaptation Period (s)

Adaptation Period ω

Performance varies little for a wide range of ω

Atom Time

Better when ≥ transaction execution time

1 5 10 50 100 500 1000

Atom Time (ms)T

( = 10ms) (ω = 1000ms)

Related work Web-databases

[Adelberg et al. 1995] [Challenger et al. 2000] [Luo et al. 2002] [Datta et al. 2002] [Labrinidis et al. 2004] [Qu et al. 2006] [Labrinidis et al. 2006] [Xu et al. 2006] …

Real time databases [Abbott et al., 1988] [Sha et al., 1991] [Haritsa et al., 1993] [Ramamritham et al., 1994] [Burns et al., 2000] …

Stream Processing [Carney et al., 2002] [Das et al., 2003] [Babcock et al., 2004] [Sharaf et al., 2005] [Abadi et al., 2005] …

Economic Models [Ferguson et al., 1996] [Stonebraker et al., 1996] …

Conclusions We proposed

a Quality Contract framework to capture user preferences in web-database systems, and

a meta schedule scheme, QUTS, to dynamically concatenate query queue and update queue according to user queries.

We showed with extensive simulation study on real data traces that QUTS outperforms the baseline algorithms in the whole

spectrum of Quality Contract setup, QUTS adapts fast to the changing workloads, and QUTS has little sensitivity to its two parameters.

Future work Quality Contract generation strategies

Synthetic workload Provide suggestions to users to select QCs I/O scheduling for disk-based Web databases

Thanks!

Comments & Questions?

Huiming Quhuiming@cs.pitt.edu

Workload characteristics

Questions Skewed queries Atom time -> number of tran. Why restarts Network delay comparable to database delay

when facing high loads of updates Scalability point of view

Plot the correlation on time (U vs. Q)

Preference-Aware Query and Update Scheduling in Web-databases Huiming Qu Department of Computer...

Documents