Date post: | 05-Jan-2016 |
Category: |
Documents |
Upload: | abigail-bradford |
View: | 217 times |
Download: | 0 times |
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
1
OPEN VERSUS CLOSED:OPEN VERSUS CLOSED:A CAUTIONARY TALEA CAUTIONARY TALE
Bianca Schroeder Bianca Schroeder Adam WiermanAdam Wierman Mor Harchol-Balter Mor Harchol-Balter
Computer Science DepartmentComputer Science DepartmentCarnegie Mellon UniversityCarnegie Mellon University
To appear at NSDI 2006presenter:吳泰廷
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
2
standardsystem
newsystem
old
new
new system hassmaller response time!
This comparisonrequires testing the two systems
on realistic workloads
THE RESEARCH PROCESSTHE RESEARCH PROCESS
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
3
INTRODUTIONINTRODUTION
Need system models that “accurately represent" the real system.
Representing a system accurately involves many things: bottleneck resource behavior, the scheduling of requests at that bottleneck, workload parameters such as the distribution of service request demands…….
One factor that researchers typically pay little attention to is whether the job arrivals obey a closed or an open system model.
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
4
We show that closed and open system models yield significantlydifferent results, even when both models are run with the same load and service demands.
Conclude with guidelines for choosing a system model.
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
5
think
receivesend
server
CLOSED SYSTEM MODEL CLOSED SYSTEM MODEL
User requests web page, receives page,reads page, clicks on new link
Closed SystemClosed System
N=MPL(multiprogramming level)
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
6
1:01.12 ip1 GET a.gif HTTP/1.01:01.20 ip2 GET b.htm HTTP/1.01:01.25 ip1 GET c.jpg HTTP/1.01:01.27 ip1 GET d.txt HTTP/1.01:01.28 ip3 GET a.htm HTTP/1.01:01.35 ip4 GET d.gif HTTP/1.01:01.45 ip2 GET e.htm HTTP/1.0::
Trace driven
OPEN SYSTEM MODELOPEN SYSTEM MODEL
service demands
x x x
server
new arrivals
arrival times
file sizes from trace
next arrivaltime from trace
Open SystemOpen System
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
7
Distribution driven
Use distributions ofinterarrival times and service demands (typically using trace info)
x x x
server
new arrivals
OPEN SYSTEM MODELOPEN SYSTEM MODEL
interarrival time dist.
service demand dist.
sample dist.
sample dist.
Open SystemOpen System
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
8
OPEN MODELOPEN MODEL
CLOSED MODELCLOSED MODEL
Arrivals are independentof completions
Arrivals are completelydependent on completions
There is no max number ofsimultaneous users
There is a fixed populationof users, called the
Multi-Programming-Level (MPL)
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
9
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
10
WEB WORKLOADWEB WORKLOADGENERATORSGENERATORS
CLOSED MODELCLOSED MODEL
OPEN MODEL OPEN MODEL
SurgeSPECWeb
TPC-WSclientRUBiS
WebBenchWebjamma
1. Workload generators for thesame purpose use differentsystem models!
2. It’s often not clear which model workload generatorsuse!
Do you use an open or closed Do you use an open or closed model?model?
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
11
NEITHER THE NEITHER THE OPEN OR OPEN OR CLOSEDCLOSED
MODEL IS MODEL IS COMPLETELY COMPLETELY
REALISTICREALISTIC
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
12
x x xnew arrivals
server
thinksend receive
leave system
with probability qreturn to the system
PARTLY-OPEN MODELPARTLY-OPEN MODEL
PARTLY-OPEN SYSTEMPARTLY-OPEN SYSTEM
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
13
What is the impact of the choice of an open or
closed model?
OUR GOALOUR GOAL
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
14
HOW DO WE HOW DO WE COMPARE COMPARE OPEN AND OPEN AND
CLOSED CLOSED SYSTEMS?SYSTEMS?
CLOSEDCLOSED
OPENOPEN
1. Fix the service distribution acrossthe systems
2. Fix the load across the systems
load depends onlyon mean arrival rate and mean service demands
load depends onMPL, think times, mean ofservice demands, variabilityof service demands …
adjust load using thethink time
adjust load using thearrival rate
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
15
How do open and closedHow do open and closed response times compare? response times compare?
FCFS schedulingopen Poisson arrival processclosed Exponential think times
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
16
load0 0.25 0.5 0.75 1
mea
n r
esp
on
se
tim
e1000
100
10
FCFS schedulingopen Poisson arrival processclosed Exponential think times
Open
Closed (MPL=10)
CLOSED CLOSED <<<< OPENOPEN
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
17
load0 0.25 0.5 0.75 1
mea
n r
esp
on
se
tim
e1000
100
10
Open
Closed (MPL=10)
Closed (MPL=100)
Closed (MPL=1000)CLOSED CLOSED OPENOPEN
FCFS schedulingopen Poisson arrival processclosed Exponential think times
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
18
OPEN MODELOPEN MODEL CLOSED MODELCLOSED MODELVSVS
CLOSED CLOSED OPEN AS MPL OPEN AS MPL GROWSGROWS
Schatte [36, 37] proves formally that as Ngrows to infinity, a closed FCFS queue converges to anopen queue.
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
19
low variability high variability
mea
n r
esp
on
se
tim
e
1500
1000
500
Open
Closed (MPL=10)Closed (MPL=100)
Closed (MPL=1000)Web
Workloads
How quickly does Closed How quickly does Closed Open?Open?
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
20
There principlesThere principles
1.For a given load, mean response times are significantly lower in closed systems than in open systems.
2. As the MPL grows, closed systems become open, but convergence is slow for practical purposes.
3.While variability has a large effect in open systems, the effect is much smaller in closed systems.
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
21
What is the impact of the choice of an open or
closed model?
OUR GOAL OUR GOAL
It matters a lot!
1. What is the impacton the effectivenessof scheduling?
2. What is the impactin practice?
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
22
FCFS (First-Come-First-Served)
PS (Processor-Sharing)
PESJF (Preemptive-Expected-Shortest-Job-First)
SRPT (Shortest-Remaining-Processing-Time-First)
PELJF (Preemptive-Expected-Longest-Job-First)
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
23
SCHEDULING IS A KEY SCHEDULING IS A KEY COMPONENT OF SYSTEM COMPONENT OF SYSTEM
DESIGNDESIGN
Improved design Shortest Remaining
Processing Time (SRPT)
Standard design Processor Sharing (PS)
WEB SERVERSWEB SERVERS
Does the effectiveness of schedulingDoes the effectiveness of schedulingdepend on the system model (open vs. closed)?depend on the system model (open vs. closed)?
Compare using a workload generator
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
24
SCHEDULING IN OPEN SCHEDULING IN OPEN SYSTEMSSYSTEMS
OPENm
ean
res
po
ns
e t
ime
1000
600
300
0
load0 .25 .5 .75 1
PLJFFCFSPSSRPT How do the closed How do the closed
results compare?results compare?
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
25
CONTRASTING THE IMPACT CONTRASTING THE IMPACT OF SCHEDULINGOF SCHEDULING
OPEN CLOSEDm
ean
res
po
ns
e t
ime
1000
600
300
0
load0 .25 .5 .75 10 .25 .5 .75 1
load
PLJFFCFSPSSRPT
PLJFFCFSPSSRPT
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
26
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
27
Three priciplesThree priciples1. While open systems benefit significantly from scheduling with respect to response time, closed systems improve much less.
2. Scheduling only significantly improves response time in closed systems under very specific parameter settings: moderate load (think times).
3. Scheduling can limit the effect of variability in both open and closed systems.
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
28
What is the impact of the choice of an open or
closed model?
OUR GOALOUR GOAL
It matters a lot!Especially when evaluating
scheduling policies
What is the impactin practice?
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
29
OPEN VS OPEN VS CLOSEDCLOSED
IN PRACTICEIN PRACTICE
3 CASE STUDIES3 CASE STUDIES
1. Serving static web content
2. Database backend ofan e-commerce site
3. Auctioning web site
testbedimplementation
trace-basedsimulation
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
30
Case studyCase study Open generatorOpen generator Closed Closed generatorgenerator
Scheduling Scheduling policiespolicies
Static web (LAN)Sclient on
World Cup trace
Modified Sclient on World Cup
tracePS, SRPT
E-commerce Modified TPC-W TPC-W PS, PESJF
Auctioning
Trace-based simulation
(top 10 auction site trace)
Trace-based simulation
(top 10 auction site trace)
PS, SRPT
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
31
OPEN VS OPEN VS CLOSEDCLOSED
IN PRACTICEIN PRACTICE
OPEN CLOSED
mea
n r
esp
on
se
tim
e
PSSRPT
PS
SRPT
300
200
100
load0 .25 .5 .75 10 .25 .5 .75 1
load
STATIC WEB SERVERSTATIC WEB SERVER
Different models give different conclusion about benefits of SRPT
MPL=50
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
32
OPEN CLOSED
PS
PS
mea
n r
esp
on
se
tim
e10
8
4
0
load load
PS
SRPT
PS
SRPT
20
14
7
0
load0 .25 .5 .75 10 .25 .5 .75 1
load
PESJFPESJF
E-COMMERCE SITEE-COMMERCE SITE
AUCTION SITEAUCTION SITE
MPL=50
MPL=50
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
33
What is the impact of the choice of an open or
closed model?
OUR GOAL OUR GOAL TODAYTODAY
It matters a lot in practice!Especially when evaluating scheduling policies
How can we identify whether to use an open
or closed model?
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
34
A MORE REALISTIC A MORE REALISTIC ALTERNATIVEALTERNATIVE
x x xnew arrivals
server
thinksend receive
leave system
with probability qreturn to the system
PARTLY-OPEN MODELPARTLY-OPEN MODEL
What parameters affect the load?What parameters affect the load?Does think time affect the load?Does think time affect the load?
How do think times affect response times?How do think times affect response times?
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
35
FITTING A PARTLY-OPEN FITTING A PARTLY-OPEN MODELMODEL
12 ip1 GET a.gif HTTP/1.020 ip2 GET b.htm HTTP/1.025 ip1 GET c.jpg HTTP/1.027 ip1 GET d.txt HTTP/1.028 ip3 GET a.htm HTTP/1.035 ip4 GET d.gif HTTP/1.045 ip2 GET e.htm HTTP/1.0::
Trace
service demands
file sizes from trace
PARTLY-OPEN PARTLY-OPEN
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
36
FITTING A PARTLY-OPEN FITTING A PARTLY-OPEN MODELMODEL
12 ip1 GET a.gif HTTP/1.020 ip2 GET b.htm HTTP/1.025 ip1 GET c.jpg HTTP/1.027 ip1 GET d.txt HTTP/1.028 ip3 GET a.htm HTTP/1.035 ip4 GET d.gif HTTP/1.045 ip2 GET e.htm HTTP/1.0::
Trace
PARTLY-OPEN PARTLY-OPEN
Fitting the interarrival times
• Distinguish userse.g. use ip address in a web trace
• Identify user session boundaries Use periods of inactivity of length > timeout
<Ti,fi1,fi2….>
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
37
CHOOSING A TIMEOUT VALUECHOOSING A TIMEOUT VALUE
Nu
mb
er
of
ses
sio
ns
2e5
1e5
00 30min
Timeout length
financial
world cup
dept store
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
38
THE EFFECT THE EFFECT OFOF
THINK TIMETHINK TIME
STATIC WEB SERVERSTATIC WEB SERVER
1 10 100 1000
mean think time
300
200
100
0
mea
n r
esp
on
se
tim
e
SRPT
PS
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
39
CLOSEDCLOSEDOPENOPENq1q0
x x xnew arrivals
server
thinksend receive
leave system
with probability qreturn to the system
PARTLY-OPEN MODELPARTLY-OPEN MODEL
number of requests per visit ↑number of requests per visit ↓ ? ?
A MORE REALISTIC A MORE REALISTIC ALTERNATIVEALTERNATIVE
Workload generators are only Open/Closed!Workload generators are only Open/Closed!
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
40
THE THE TRANSITION TRANSITION FROM OPEN FROM OPEN
CLOSED CLOSED
STATIC WEB SERVERSTATIC WEB SERVER
300
200
100
00 5 10 15 20
PS open
PS closed
PS
SRPTmea
n r
esp
on
se
tim
e
mean number of requests per visit
OPEN CLOSED
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
41
THE PARTLY-OPEN THE PARTLY-OPEN SYSTEM IN SYSTEM IN PRACTICEPRACTICE
mean number of requests per visit
mea
n r
esp
on
se
tim
e STATIC WEBSTATIC WEB
PSSRPT
0 5 10 15 20
200
100
0
E-COMMERCE SITEE-COMMERCE SITE
0 5 10 15 20
9
6
3
0
PSPESJF
AUCTIONINGAUCTIONING
0 5 10 15 20
15
10
5
0
PSSRPT
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
42
PS
SRPT PS SRPT
OPENOPEN CLOSEDCLOSED
PSSRPT
PARTLY-OPENPARTLY-OPEN
VSVS
THESE DIFFERENCES ARE THESE DIFFERENCES ARE IMPORTANT IN PRACTICEIMPORTANT IN PRACTICE
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
43
Two PrinciplesTwo Principles
1. A partly-open system behaves similarly to an open
system when the expected number of requests per
session is small (≤ 5) and similarly to a closed system
when the expected number of requests per session is
large (≥ 10 as a rule-of-thumb).
2. In a partly-open system, think time has little effect on
mean response time.
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
44
CHOOSING A SYSTEM MODELCHOOSING A SYSTEM MODELWeb workloads
Open or closed?
Use a partly-open model...
1. Large corporate web 2. CMU web server3. Online department store 4. Science institute (USGS)5. Online gaming site6. Financial service provider 7. Supercomputing web site 8. Kasparov-DeepBlue match 9. Site seeing “slashdot effect” 10. Soccer world cup
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
45
CHOOSING A SYSTEM MODELCHOOSING A SYSTEM MODELWeb workloads
Open or closed?
Use a partly-open model...
...to decide which is more accurate
1. Large corporate web 2. CMU web server3. Online department store 4. Science institute (USGS)5. Online gaming site6. Financial service provider 7. Supercomputing web site 8. Kasparov-DeepBlue match 9. Site seeing “slashdot effect” 10. Soccer world cup
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
46
HOW TO HOW TO CHOOSECHOOSE
A A SYSTEM SYSTEM MODELMODEL
Gathera
trace
How many simult. users are
there?
Fit a partlyopen modelto the trace
OPEN ≈ CLOSED
>>1000
else
What is theexpected num.
of visits?
OPEN CLOSED???
<5 5-10 >10
Me
an
nu
m. o
f vi
sit
s
15
10
5
00 30min
Timeout length
world cup
dept store
financial
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
47
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
48
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
49
CHOOSING A SYSTEM MODELCHOOSING A SYSTEM MODEL<5 expected visits
>10 expected visits
CLOSED
5-10 expected visits
Web WorkloadsOPEN
PARTLYOPEN
1. Large corporate web 2. CMU web server3. Online department store 4. Science institute (USGS)5. Online gaming site6. Financial service provider 7. Supercomputing web site 8. Kasparov-DeepBlue match 9. Site seeing “slashdot effect” 10. Soccer world cup
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
50
CHOOSING A SYSTEM MODELCHOOSING A SYSTEM MODEL<5 expected visits
1. Large corporate web 2. CMU web server
4. Science institute (USGS)6. Financial service provider
8. Kasparov-DeepBlue match 9. Site seeing “slashdot effect”
>10 expected visits5. Online gaming site
10. Soccer world cup CLOSED
5-10 expected visits3. Online department store
7. Supercomputing web site
Web WorkloadsOPEN
PARTLYOPEN
Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department
51
CONCLUSIONCONCLUSION The differences in behavior of closed, open,and partly-open
systems.
These principles underscore the importance of choosing the appropriate system model.
Our findings provide guidelines for choosing whether an open or closed model is the better approximation based on characteristics of the workload.
Understanding the appropriate system model is essential to understanding the impact of scheduling.