Post on 28-Mar-2015
transcript
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
Improving Performance of Internet Services Through
Reward-Driven Request Prioritization
Alexander Totok and Vijay Karamcheti
Computer Science Department
New York University
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
Web Server Overload Conditions
Consequences increased request response times some requests are dropped successful session throughput suffers dramatically client dissatisfaction → reduced revenues
Current solutions work with static client identity session-based admission control (SBAC) service level agreements (SLA) service membership per-client history-based approach
– looks at a client’s previous visits to the web site
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
Maximizing Profit Brought By Internet Services
Service profit (reward) maximization shopping web site: number of items sold
Idea: assign higher execution priority to the requests, whose sessions are likely to bring more reward how to predict a session’s reward?
Our solution: Reward-Driven Request Prioritization (RDRP) predicts a session's activities by comparing requests seen in it
with aggregated client behavior uses Bayesian inference analysis to dynamically compute
request priority in real time contrasts with the per-client history-based approach
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
Service Usage Profiles (Patterns)
Session structure: first-order Markov chain corresponds to a typical service usage profile (pattern)
Shopping scenario for TPC-W application (web store selling books) “Mostly Buyers” profile – more buying activity “Mostly Browsers” profile – more browsing activity
Home Search[empty cart]
Item[empty cart]
Add To CartSearchItem
Cart Register
Buy RequestBuy Confirm
Exit[browser]
Exit[buyer]
0.75
0.10.050.2
1
0.05
0.60.25
0.6
0.2
0.4
0.6
0.40.2
0.30.3
0.4
0.4
1 1
1
0.2
Mostly Buyers
Home Search[empty cart]
Item[empty cart]
Add To CartSearchItem
Cart Register
Buy RequestBuy Confirm
Exit[browser]
Exit[buyer]
0.6
0.330.050.35
1
0.05
0.020.6
0.4
0.30.4
0.5
0.50.2
0.050.35
0.4
0.6
1 1
1
0.3
Mostly Browsers
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
What Information Does RDRP Use?
User load structure: {Profilei}; {pi} – percentage of sessions belonging to Profilek
on-line request profiling and clustering analysis [Menasce’99]
Request reward rewardi: per request type specified by the service provider web shopping scenario: reward(addToCart)=1
Relative request execution cost for prediction of future server resource consumption costi: per request type – average request processing time fine-grained profiling of request execution
Only request reward is specified by the service provider
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
cost_expected
Step 2
{costi}
{rewardi}
Profile K:pi,j
User load structure
p1 Profile 3Profile 2
Profile 1
p2p3
...
Step 1
How Does The Algorithm Work?
priority =reward_attained + reward_expected
cost_incurred +
request +its
session’srequest history
Bayesiananalysis
Step4:
Pr that the session
belongs to Profile KStep 3
reward_expectedcost_expected
Uses Markov properties of service
usage profiles
reward_expected and
cost_expected
if the session belongs to Profile K
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
Prototype Implementation in J2EE
Request priority used to allocate threads and DB connections
JBossJetty Web Server JBoss JCA
service
requestrequest
profiling info RDRP
Socket Listener Database Connection Manager
priority queues
Thread pool
Sched. . .
priority queues
DB conn. pool
Sched. . .
getThread(priority, timeout)
getDBConn(priority, timeout)
request profiling info
request
execution
Request Profiling Service
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
Evaluation
Shopping scenario for TPC-W user load with two usage patterns: “Mostly Buyers”/“Mostly Browsers” new sessions: bursty arrival process (B-model [Wang’02])
Techniques compared default – FIFO prioritization session-based admission control (SBAC) per-client history-based approach
– success depends on how well prediction of a session’s behavior works
– model different correlation between sessions’ rewards and assigned priorities:
c = 0 (coin flip)
c = 0.25
c = 0.5 (good oracle)
c = 0.75 (very good oracle)
c = 1 (perfect oracle)
Reward-Driven Request Prioritization (RDRP)
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
0%
25%
50%
75%
100%
125%
150%
175%
200%
225%
135% 170% 200% 250%
Load (% of server capacity)
Rel
ativ
e re
war
d
FIFO SBAC coin flip (c=0)oracle (c=0.25) good oracle (c=0.50) very good oracle (c=0.75)perfect oracle (c=1) RDRP
Overload (135%): Reward
similar
The bigger the better
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
0 1 2 3 4Number of items bought in the session (reward)
Re
qu
es
t re
sp
on
se
tim
e (
ms
)
SBAC coin flip (c=0) oracle (c=0.25)good oracle (c=0.50) very good oracle (c=0.75) perfect oracle (c=1)RDRP
Overload (135%): Response Times
The smaller the better
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
Underload (80%): Response Times
0
500
1000
1500
2000
2500
3000
3500
4000
0 1 2 3 4
Number of items bought in the session (reward)
Req
ues
t re
spo
nse
tim
e (m
s)
FIFO coin flip (c=0) oracle (c=0.25)good oracle (c=0.50) very good oracle (c=0.75) perfect oracle (c=1)RDRP
The smaller the better
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
Discussion
Main distinguishing features of RDRP tries to predict future session’s reward is oriented towards session completion works in an abstract application-generic manner
May also take into account differences in user think times helps to distinguish between different service usage profiles in
the Bayesian inference analysis
What if clients do not show stable behavioral patterns?
IWQoS’2006, New Haven, CT, June 19 – 21, 2006
file
Thank You!