Software Performance Analysis

Software Performance Analysis

Course Review

SOFT 437

2

Scope

• Introduction to performance

– Performance Concepts

– Performance Misconceptions

– SPE

– Modeling strategies

• SPE models

– Software execution model

– System execution model

SOFT 437 – Course Review

3

Scope

• Software contention

• Capacity planning

• Performance-oriented design

– Principles

– Design patterns

– Performance patterns

– Performance antipatterns


Chapter 1

5

What is Performance?

• Performance refers to the response time or

throughput as seen by the users

• Response time is the time required to respond to a

request

– For example, time required for an ATM withdraw

transaction

• Throughput is the number of requests that can be

processed in some specified time interval

– An ATM network may be required to process 100,000

transactions per hour


What are the two dimensions of

software performance timeliness?

• Responsiveness

• Scalability

SOFT 437 – Course Review 6


Responsiveness

• Responsiveness is the ability of a system to meet its

objectives for response time or throughput

• Responsiveness is a measure of

– How fast the system responds to an event (example:

time for an ATM withdraw transaction)

– The number of events that a system can process in a

given period (example: the number of transactions

during an hour)

• Responsiveness has both an objective and a

subjective (user perceived) component


Scalability

• Scalability is the ability of a system to continue to

meet its response time or throughput objectives as

the demand for its functionality increases.R

esp

on

se T

ime

ThroughputKnee

Intermittent connectivity

Mobility

Limited resources

Resources VS Demands

9SOFT 437 – Course Review

10

Is SPE Necessary?

Fast state-of-the-art hardware and software

technology increase the risk of performance

problems

Resource Requirements


11

Performance Misconceptions

• Performance problems are rare

• Hardware is fast and inexpensive

• It’s too expensive to build responsive software

• You can tune the software later, if necessary

• Efficiency implies “tricky code”


12

Systems with High Performance

Requirements?

• Customer service functions

– Reservation systems, merchandise-checkout systems

• Real-time systems

– Such as flight-control systems

• Employee support systems

– Inventory control systems, computer-aided design

systems


13

Software Performance Engineering

• Software Performance Engineering (SPE) is a

systematic, quantitative approach to constructing

software systems to meet performance objectives

– Begins early in the software lifecycle

– Uses quantitative methods

– Identifies problems before developers invest

significant time in implementation

– Used through detailed design, coding, and testing


14

Waterfall Model


15

Identify Critical Use Cases

• The critical use cases are important to the operation

of the system, or are important to responsiveness as

seen by the user

• The selection of use cases is risk driven

– a risk (e.g. if performance goals are not met, the system will fail

or be less than successful)

• Example ATM use cases include:

– reloading a currency cassette

– customer transaction (e.g., withdraw, deposit)

– going off-line


16

Select Key Performance Scenarios

• It is unlikely that all of the scenarios for each critical use case will be important from a performance perspective

• The key performance scenarios are

– Executed frequently

– Critical to the perceived performance of a system

• Each performance scenario corresponds to a workload

• Workload intensity specifies the level of usage for the scenario (arrival rate)

• Example:

– Specify the workload intensity of a customer transaction, that is the number of customer transactions or their arrival rate during the peak period


Chapter 3-5

Software Execution Graphs

19

Use Case Diagrams

• A use case diagram shows

– a set of use cases,

– the actors that interact with use cases

– the relationships

• An actor can be represented by stick figures or stereotyped icons

• A use case is represented by an ellipse that contains the name of the use case

System

Boundary

Use Case

Association

Actor


20

Sequence Diagrams

• Sequence diagrams demonstrate the behaviour of

objects in a use case by describing the objects and

the messages they pass.

Lifeline

notation

Activation


23

Basic

Sequence

Diagrams

&

Message

Control

Flow

• Figure 3-6

Object lifeline: represents the

existence of an object over time

Messages


24

Synchronization

• UML provides different types of arrowheads to

represent communications among objects

Synchronous

message

Return

Asynchronous

messageProcess is Idle


25

Parallel

composition

clause

?


26

User IM: Server: Buddy IM:

startChat()

chatStatus()

Alt

Chat Buddy

Sequence Diagram

Chat Buddy

Sequence Diagram

startChat()

chatStatus()

•SERVER UNAVAILABLE

•INACTIVE

•AWAY

•REFUSED

•ACCEPTED

Send IM

Receive IM

Change Status

Buddy Status Change

loop[done == false]


28

Software Execution Model

• Constructed early in the development process to ensure that the chosen software architecture can achieve the required performance objectives

• Captures essential performance characteristics of the software

• Provides a static analysis of the mean, best and worst-case response time

• Characterizes the resource requirements of the proposed software alone, in the absence of other workloads, multiple users or delays due to contention for resources


29

Software Execution Model (con’t)

• Software execution models are generally sufficient

for identifying serious performance problems at the

architectural and early design phases

• We can refine software execution model in the

critical areas

The absence of problems in the software model does

not mean that there are none


30

The Execution Graphs• Execution graphs are one type of software execution

model

– Visual representation that helps to communicate execution behavior

• An execution graph is constructed for each performance scenario

• The graphs consist of nodes and arcs

– Nodes represent processing steps – a collection of operation invocations and program statements that perform a function in the software system

– Arcs represent the order of execution


31

Basic Execution Graph Notation


32

Expanded Nodes

• Expanded nodes represent processing steps

elaborated in another subgraph

• Expanded nodes show additional processing details

that are identified as the design evolves

Expanded Nodes


33

Repetition Nodes

• Repetition nodes represent one or more nodes that

are repeated

• Repetition factor associated with the node that

specifies the number of times the processing steps

repeat

• An arc connects the last node repeated with the

repetition node

Repetition

Node

Repetition

Factor


34

Case Nodes

• Case nodes represent conditional execution of

processing steps

• Attached nodes represent the steps that may be executed

• A case node has more than one attached

• Each attached node has an execution probability

Case

Node

Attached

Nodes



ATM Use Case

OperatorTransactions

CustomerTransactions

CommandFunctions

Customer

HostBank

Operator

ATM


Example of Key Performance Scenarios

RequestTransactionprocessDeposit

processWithdrawal

processBalanceInquiry


Withdrawal Scenario: Sequence Diagram

:User :ATM :HostBank

cardInserted

requestPin

aPin

requestTransaction

response

requestAccount

account

requestAmount

amount

transactionRequest

transactionAuthentication


Software Execution Model Analysis

• Primary purposes of software execution model analysis are:

Quick check on the system architecture

Assess Performance Alternatives

Identify critical performance issues

Derive parameters for the system model


Basic Solution Algorithms

• Basic structures are:

Sequences

Loops

Cases


Graph Reduction for Sequential

Structures

t1

t2

tn

t

t = t1 + t2 + … + tn


Authentication Transaction Scenario

validateUser

ValidateTransaction

sendResult

Node Time

validateUser 20

validateTransaction 50

sendResult 30

20

50

30

AuthenticateTransaction

100

Table 1: Processing Overhead


Authenticate Transaction Scenario (cont.)

validateUser

ValidateTransaction

sendResult

Proc. U 1

DB 2

Msgs 0

Proc. U 2

DB 3

Msgs 0

Proc. U 2

DB 1

Msgs 1

Device CPU Disk

Quantity 1 1

Service Unit Kinstr. Ph.I/O

Network

1

Msgs.

Proc. U 20 0

DB 500 2

Msgs 10 2

0

1

1

Service time 0.00001 0.02 0.01

Table 2: Processing Overhead

Software Resource

Requirement MatrixProcessing StepsProcessing Overhead Matrix

Hardware Resources

43

Mapping between software resource

requirements and computer device usage


STEP 1: use the processing

overhead matrix to calculate the

total computer resources required

per software resource for each node.


validateUser

ValidateTransaction

sendResult

Proc. U 1

DB 2

Msgs 0

Proc. U 2

DB 3

Msgs 0

Proc. U 2

DB 1

Msgs 1

Device CPU Disk

Quantity 1 1

Service Unit Kinstr. Ph.I/O

Network

1

Msgs.

Proc. U 20 0

DB 500 2

Msgs 10 2

0

1

1

Service time 0.00001 0.02 0.01

Table 2: Processing Overhead Proc. U 1

DB 2

Msgs 0

20 0

500 2

10 2

0

1

1

Total 1020 4 0

Kinstr. Ph.I/O Msgs.

45

XXX


STEP 2: compute the total computer

resource requirements for the graph


The Total Resource Requirements

Processing Step CPU

KInstr

Physical

I/O

Network

Messages

validateUser 1,020 4 0

validateTransaction 1,540 6 0

sendResult 550 4 1

Total: authorizeTransaction 3,110 14 1

Table 3: Total Computer Resource Requirements for authorizeTransaction


STEP 3: compute the total response

time for the task.


The Total Execution Time

Processing Step CPU

Kinstr

Physical

I/O

Network

Messages

validateUser 1,020 4 0

validateTransaction 1,540 6 0

sendResult 550 4 1

Total: authorizeTransaction 3,110 14 1

Service Unit 0.00001 0.02 0.01

Total: Response Time 0.0311 0.28 0.01

Table 3: Total Response Time for authorizeTransaction

49

0.3211msec

X


Graph Reduction for Loop

Structures



System Execution Model


53

Observations

• Component based development technologies are largely adopted in the development of Web applications

• The development environments require little training of distributed object technologies or even in computer science

• Many Web applications are developed without the benefit of serious analysis, design or SPE

• The development process takes “fix-it-later” approach

• Both responsiveness and scalability are important for Web ApplicationsSOFT 437 – Course Review

54

Performance Concerns

• Forecasting the volume of activity

• Choice of mechanism for executables

• Allocation of executables to processing nodes

• Mechanisms for accessing the database

• Mechanisms for handling persistent data

• Mechanisms for providing access and/or data security

• Instrumentation for characterizing Web application usage

• Location of the database

• Interfaces to legacy systems

• Impact of downloading applets or other code


The Essence of Distributed Systems

Object Placement

Communication

Synchronization


56

Types of System Interactions

• Four types of system interactions are typically supported in

middleware

– Synchronous

– Asynchronous

– Deferred synchronous, and

– Asynchronous callback communication

• Severs provide services to clients

• Clients request services on servers

• The roles of servers and clients refer to a particular

interaction, and may be reversed in subsequent interactions



Chapter 6

System Execution Graph


60

Sources of Contention for

Resources

• Multiple users of an application or transaction

executing at one time, e.g. several ATM customers

do a withdrawal simultaneously

• Multiple applications or systems executing on the

same hardware resources at one time

• The application under consideration can have

separate concurrent processes

• The application may be multi-threaded to handle

concurrent requests for different external processes


61

System Model Basics

• The system execution model represents the key

computer system resources as queues and servers

– A server represents a component of the environment

that provides some service to the software (e.g., a

processor, a disk, or a network element)

– A queue represents jobs waiting for service (e.g.,

when a request arrives, if the server is busy, the job

making the request must wait until the server is free)



63

Modelling Formulas

Arrival rate, λ

Mean service time, S

We calculate the following average values:

Throughput, X = λ

Utilization, U = XS

Residence time,

Queue length, N = X * RT

U

SRT

1


64

Little’s Law

Queue length = Throughput * Residence

time

(N = X * RT)

• The specifications and results from the simple

model are average values

• The metrics for a specific job may differ from the

average

• The difference depends on the distribution of

arrivals and service requirements


Both waiting and executing jobs

65

Utilization on Multiple Resources

• Utilization of single resources

U=X * S

Utilization of multiple resources

Ui = U/m


66

Service Demand Law

• The service demand on a device i

iii SVD

0X

UD i

i



Web Application Architecture

• Web applications adopt

client/server architecture

or multi-tier architecture

• The Web is a collection

of Internet servers

– Web server stores electronic

files

– Application server houses

business logic

Web Server

N Tier

Internet

Application

Server

Web Server

Internet

Application

Server

Application

Server

Web Server

71

Total Response Time for a Web Request

• Software contention: time spent by a request waiting

to obtain a software resource

• Hardware contention: time spend waiting to use a

hardware resource such as the CPU and I/O device

• Usage of hardware resources: time spent using

hardware resources such as the CPU or I/O devices,

i.e., the sum of the service demands at all hardware

resources



Contention for Software in Multi-tier Web Sites


Contention for Software in Multi-tier Web Sites

ResponseTime = SoftwareContention + Execution

Time

SoftwareContention = WaitWS + WaitAS + WaitDS

ExecutionTime = HardwareContention + TotalDemands

HardwareContention = HdwWaitWS + HdwWaitAS + HdwWaitDS

TotalDemands = DemandsWS + DemandsAS + DemandsDS

Time spent by the request waiting to obtain access to a software resource (e.g., thread)

Time spent by the request waiting to obtain access to a hardware resource (e.g., CPU)

74

• Consider now the

case in which the

queue for threads is

limited.

• A maximum of J

requests can be in

the queue for

threads at any time.

• A request that finds

J requests in the

queue is rejected.

Queue with Limited Size



Capacity Planning

• Capacity planning is the process of

– predicting when future load levels will saturate the system

– determining the most cost-effective way for delaying system

saturation as much as possible

• Future load levels are a function of

– Natural evolution of existing workloads

– Deployment of new applications and services

– Changes in customer behaviors

• Traffic surges due to new situations (i.e., breaking news, TV ad

campaigns…)

• Customer navigational patterns (new functionality)


Adequate Capacity

• Adequate capacity for an e-business site is a function of the three

following elements

– Service level agreement (SLA), Technology, and Cost Constraints


Technology– Specified technologies and standards

• The choice of technologies and standards place constraints on how adequate

capacity is provided


Cost

– Cost constraints

• Indicated how much capacity can be made available


Workload Characterization

• Is the process of precisely describing, in a qualitative and quantitative manner, the global workload of an e-business site.

• Maps the e-business functions into transactions and batch programs

• Build Client/Server Interaction Diagrams (CSIDs) for each e-business function

– Shows the transactions (nodes) with probability and size of the messages exchanged between the various servers and the network involved

Function: Option Selection


Workload Characterization (cont.)

Measurements

• Provide input data for SPE models, verify and

validate models

• Determine whether performance goals have actually

been met

• Monitor the performance of system over their life

time

It is much easier to make measurements than to know

exactly what you need to measure


Static vs. Dynamic Measurements

• Static measurements are made without executing the

software

– For example, measure the size of database table or a

network message

• Dynamic measurements require that the software be

executing during the measurement

• The measurements needed for SPE are primarily

dynamic rather than static


Terminology

• A state reflects the information about the systems hardware or software within a period of time when the computer system is running– For example, “idle” or “executing job X” are states of the CPU

• An event corresponds to a change in the system’s state– For example, the CPU changes from “idle” to “executing job X”

• There are two fundamental measurement techniques corresponding to states and events– Monitoring states

– Recording events


Monitoring States

• A performance monitor observes the activities of a

computer system and records performance

information about states of interest

• Monitors sample the system’s state by periodically

activating and recording the current state along

with the appropriate performance data about that

state

• The data collected by the monitor allows you to

identify “hot spots” – those portions of the code that

consume most of the CPU time


Types of Monitors

• Software monitors: programs that executes

independently from the software being measured

• Hardware monitors: external devices that are

attached to the computer system hardware via

external wires or probes

• System monitors observes the state of the overall

system

• Program monitors observes the states of the

particular program being measured


Types of Event Recording

Even recording can occur at different levels:

• Program event recording: the events are pertinent to

the program being measured

– the CPU time for database access, the elapsed time to

send a message

• System event recording: certain events are recorded

for all executing programs of interest.

– the CPU time for database accesses for teller and bank

analyst transactions,


Internal vs. External

• Both monitors and event recorders may be either

integrated into a program (internal) or external to it

• Internal measurement requires that code be inserted

into programs to detect events and record the

pertinent performance data

– Such as, compiler option or a preprocessor

• External monitors and event recorders execute

independently of the software to be measured


Three categories of performance

principles

• Performance control principles

– Performance Objectives

– Instrumenting

• Independent principles

– Centering

– Fixing-point

– Locality

– Processing vs. Frequency

• Synergistic

– Shared Resources

– Parallel Processing

– Spread-the-load



Performance Patterns

• The performance patterns describe best practices for

producing responsive, scalable software

• Performance patterns complement and extend the

performance principles

• Seven performance patterns address performance

and scalability

• Fast Path

• First Things First

• Coupling

• Batching

• Alternate Routes

• Flex time

• Slender Cyclic

Functions


Performance Patterns vs. Design

Patterns

• Each performance pattern is a realization of one or

more of the performance principles

• The performance patterns are at a higher level of

abstraction than design patterns

– A design pattern may provide an implementation of a

performance pattern

Performance Principles Centering Principles

Performance Patterns Fast Path

Design Patterns Proxy, predefined options

level of

abstraction

Antipatterns

The God class

Excessive Dynamic

Allocation

Circuitous Treasure

Hunt

One-Lane Bridge

Traffic Jam


Date post:	16-Jan-2022
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Software Performance Analysis

Documents