Control-based Quality Adaptation in Data Stream
Management Systems (DSMS)
Data Stream Management System (DSMS)
• Continuous data, discarded after being processed
• Continuously answers queries
• Applications– Financial analysis– Mobile services– Sensor networks– Network monitoring– More …
User
DSMS
User
User
DataQuery
Results
DSMS architecture
• Network of query operators (1 – 12)
• Each operator has its own queue
• Scheduler decides which operator to execute
• Query results pushed to clients
• For our purposes, DSMS can be viewed as a blackbox
4 11
73 10
QueryEngine
S1
S2 III
I
2
5
6
1
9
12
DataStreams
OutputStreamsII
8
DSMS
Data Input Rate
Average DelayLoad
Adaptor
Load Shedding
• Eliminating excessive load by dropping data items less QoS violations
• Basic algorithm (Tatbul et al., 2003):
• Key questions– When?– How much?– Where?
What’s missing?
• Current solutions focus on steady-state performance– Open-loop control ?
• Assuming there inputs reach steady states
• However, arrivals are bursty in practice – always in transient state
• The solution: closed-loop control
Load
Time
CPUcapacity
Why Closed-Loop Control
• Reduce the effects of modeling error, input and output disturbances
• Improve dynamic response• Stabilize unstable systems
a+dm1/ayr
di do
a+dmK_
yr
di do
1( )m m i oy r r d a d d d
a
( ) ( ) 1
1 ( ) 1 ( ) 1 ( )
1 1
m mi o
m m m
i o
K a d a dy r d d
K a d K a d K a d
r d dK K
Identification of Database System
Control of Database System
• Output is delay time• Incoming flow rate fluctuates and unknown• Uncertainties in cost factor
Control
CostFactor
r
fouty
_
1/Sqfin
_
Experiments
• Implemented a controller in a real DSMS – Borealis• With bursty synthetic and real data