1
PerfCenter and AutoPerf: Tools and Techniques for Modeling and Measurement of the Performance of Distributed Applications
Varsha Apte
Faculty Member, IIT Bombay
2
Example: WebMail Application (ready to be deployed)
IMAP server
Ad Server
Authentication Server,
SMTP Server
Web Server
WAN
User request
Several interacting components
3
Several Usage ScenariosExample: Login
Browser Web Authentication IMAP SMTPUser/Password
Verify_passwd
Send_to_auth
GenerateHtml
list_messageCall list_message of IMAP server
GenerateHtml
0.2
0.8
Res
pons
e T
ime
Performance Goals during deployment:•User perceived measures: response time, request drops (minimize)•System measures: throughput, resource utilizations (maximize)
4
Deploying the application in a Data Center
What should be the configuration of the Web server? (Number of threads, buffer size,…)
On which machines should IMAP server be deployed? The Web Server?
How will the network affect the performance? (LAN vs WAN)
How many machines? Machine configuration? (how many CPUs, what speed, how many disks?)
Determining host and network architecture
5
Input specifications
Machines and Devices
Software Components
Network Params
Deployments
Scenarios
Parser
Queuing Model
Simulation Tool Analytical Tool
Ref MASCOTS 07
Output Analysis
PerfCenter: Modeling Tool
Inbuilt functions and constructs aid datacenter architect to analyze and modify the model
PerfCenter generates underlying queuing network modelPerfCenter solves
the model
Architect specifies the model
PerfCenter code can be downloaded from http://www.cse.iitb.ac.in/perfnet/perfcenter
*
6
Capacity analysis for WebMail
Maximum throughput achieved is 30 requests/sec
Graph for Response time performance with increase in number of users
7
Autoperf: a capacity measurement and profiling tool
Focusing on needs of a performance modeling tool
8
Input requirement for modeling tools Usage Scenarios Deployment details Resource Consumption Details – e.g.
“login transaction takes 20 ms CPU on Web server”Usually requires measured data
9
Performance measurement of multi-tier systems
Two goals: Capacity Analysis:
Maximum number of users supported, transaction rate supported, etc
Fine grained profiling for use in performance models
10
Measurement for Capacity analysisMeasurement for Capacity analysis
Clients running load generators
System Test EnvironmentSystem Test Environment
Servers running system under test
GenerateRequests
E.g.:HttperfFloodSilk PerformerLoadRunner
11
…….Measurement for capacity .Measurement for capacity analysis – Answers providedanalysis – Answers provided
12
Given such data, models can “extrapolate” and predict performance at volume usage (e.g. PerfCenter).
Measurement for modelingMeasurement for modeling
Web ServerApp Server 1
App Server 2
ResourceConsumption profile
10ms
20ms
40ms
45ms
LAN
Clients running load generators
GenerateRequests
13
Generate Load
ProfileServers
1 2
3Collect clientstatistics Collect server
statistics
Client
Correlate & display
Introducing: Introducing: AutoPerfAutoPerf Servers
AutoPerfAutoPerf
14
AutoPerfAutoPerfDeployment Information of servers
Web transaction workload description
Fine grained server sideResource profiles
15
Future enhancements PerfCenter/AutoPerf:
Various features which make the tools more user-friendly
Capability to model/measure performance of virtualized data centers
Many other minor features Skills that need to be learned/liked:
Java programming (both tools are in Java) Discipline required to maintain/improve large software
Working with quantitative data
16
What is fun about this project?
Working on something that will (should) get used.
New focus on energy and virtualization –both exciting fields
Many, many, algorithmic challenges Running simulation/measurement in efficient
ways
17
Work to be done by RA
Code maintenance Feature enhancement Write paper(s) for publication, go to conferences,
present them Creating web-pages and user groups, answering
questions Help in popularizing the tool, demos, etc Pick a challenging problem within this domain as
M.Tech. project, write paper (s), go to conferences!
18
Thank you/Questions
This research was sponsored by MHRD, Intel Corp, Tata Consultancy Services and IBM faculty award 2007-2009
PerfCenter code can be downloaded from http://www.cse.iitb.ac.in/perfnet/perfcenter
19
Request Arrival
Get SoftServer Get DeviceGet Network
Link
InstanceFree
BufferFull
Drop
ServiceRequest
EnQueue
Y
YN
N
SoftServerDevice
Service ReqNetworkLinkService Req
Queue Class
Simulator: Queue ClassAll resources like devices, soft servers and network link are abstracted queues
Discrete event simulator implemented in Java
Supports both open and
closed arrivals
20
Simulator: Synchronous callsServer1 Server2 Server3 Server4
Thread Busy
Thread waiting
Server1-t
Server2-t
Server3-t
PerfCenter Stack
User
21
Simulator Parameters
PerfCenter Simulates both open and closed systems
loadparmsarate 10end
loadparms noofusers 10 thinktime exp(3)end
Model parameters
modelparmsmethod simulationtype closednoofrequest 10000confint falsereplicationno 1end
Independent replication method for output analysis
22
Deployments dply3 and dply4
H1
Cpu%
H2
Cpu%
H3
Cpu%
H4
CPU%
IMAP
Host Disk%
H2
Disk%
dply3 77.6 17.7 23.9 NA 44.2 18.7
dply4 53.8 18.4 27.7 NA 47.1 19.5
23
Deployment summary
H1
Cpu
%
H2
Cpu
%
H3
Cpu
%
H4
CPU
%
IMAP
Host Disk
%
H2
Disk
%
dply1 98.1 8.2 NA NA 41.2 8.8
dply2 67.5 15.9 48.6 75.0 80.6 17.0
dply3 77.6 17.7 23.9 NA 44.2 18.7
dply4 53.8 18.4 27.7 NA 47.1 19.5
24
Simulator: Dynamic loading of Scheduling policy
/Queue/SchedulingStartergy/
/FCFS.class
/LCFS.class
/RR.class
host host[2]cpu count 1cpu schedp fcfscpu buffer 9999end
host host[2]cpu count 1cpu schedp rrcpu buffer 9999end
25
Using PerfCenter for “what-if” analysis
Scaling up Email Application To support requests arriving at rate
2000 req/sec
26
Step 1 Step 2
diskspeedupfactor2 =20diskspeedupfactor3=80deploy web H4set H1:cpu:count 32set H4:cpu:count 32set H3:cpu:count 12set H2:cpu:count 12cpuspeedupfactor1=2cpuspeedupfactor3=4cpuspeedupfactor4=2
host H5
cpu count 32
cpu buffer 99999
cpu schedP fcfs
cpu speedup 2
end
deploy web H5
set H2:cpu:count 32
set H3:cpu:count 18
27
Summary
H1 H2 H3 H4 H5
Step1 CPU Count 32 12 12 32
Cpu Utilization 88 100 75.8 87.7
CPUSpeedUp 2 1 4 3
Disk Speedup 20 80
Disk utilization 51.6 46.5
Step2 CPU Count 32 32 18 32 32
Cpu Utilization 64.5 55.9 57.0 63.1 58.7
CPUSpeedUp 2 1 4 2 2
Disk Speedup 20 80
Disk utilization 58 52.6
28
Identifying Network Link Capacity
256Kbps 1Mbps
Lan1 to lan2 20.1% 5.1%
Lan2 to lan1 18.7% 4.8%
29
Limitations of standard tools Do not perform automated capacity analysis
Need range of load levels to be specified Need duration of load generation to be specified Need the steps in which to vary load to be specified Report only the throughput at a given load level, but not
the maximum achievable throughput and saturation load level.
Tools should take as input a better workload description (CBMG) rather than just the percentage of virtual users requesting each type of transaction.
Do not perform automated fine grained server side resource profiling.