Performance Engineering Fundamentals

Practical Performance Analyst – 23rd June 2012

http://www.practicalperformanceanalyst.com

Agenda

What is Performance Engineering

Why is Performance Engineering Important

Performance Engineering Life Cycle

Holistic View of Performance

Activities involved in Performance Engineering

What does Performance mean to different stake holders

Roles performed by the Practical Performance Analyst

What is Proactive Performance Management

Cost of addressing Performance across the SDLC

Challenges in addressing Performance Engineering

What Is Performance Engineering

Definition - Performance Engineering (SPE) is a systematic and quantitative approach for the cost-effective development of software systems to meet stringent Non Functional Requirements (Performance - Capacity - Scalability - Availability - Reliability - etc).

Dr. Connie Smith in her book on Software Performance Engineering (URL - http://www.perfeng.com/) advocates that, Performance Engineering is a software-oriented approach, focused on optimal selection of application architecture, design, and implementation choices with the objective of meeting Non Functional Requirements.

Software Performance Engineering can also be defined functionally as the set of tasks or activities that need to be performed across the Software Development Life Cycle (SDLC) to meet the documented Non Functional Requirements.

Software Performance Engineering is often viewed as the art of building systems that meeting Non Functional requirements within the allocated time frame and budget constraints.

Why Is Performance Engineering Imp

Performance Engineering is important for the following reasons –

Determine End User Experience desired by customer for the applications being developed

Understand Non Functional Requirements for the applications

Determine Performance Targets for the Developers across the various different application tiers

Work proactively with Developers to ensure Performance Targets are addressed across the Build phase

Ensure that the application architecture chosen will allow you to meet the Non Functional Requirements

Determine the infrastructure capacity required to meet your Non Functional Requirements & IT SLA’s

Identify any major architectural or platform concerns early in the SDLC

Identify application bottlenecks through early Performance Testing (Before completion of SIT and start of UAT)

Model the application performance at design stage to validate your Non Functional Requirements

Proactively model application performance through Performance Test and once the application goes live

Proactively manage application infrastructure requirements once the application goes live

Proactively monitor application performance in production (OS, Application, Transactional) to track SLA’s and capture required metrics for purposes of Performance Modelling & Capacity Management

Work closely with business to forecast application performance and associated infrastructure requirements to meet growth in business workload

Prevent a meltdown on go live or in production

Prevent fire fighting in production and rather focus your time and energy on building required application functionality

Performance Engineering Life Cycle

Software Development Life Cycle

Functional Requirements Gathering

Architecture & Design

Build Application

System Test,

System Integrated Test & UAT

Deploy Into Production

Performance Engineering Life Cycle

Non Functional Requirements Gathering

Design for Performance &

Performance Modelling

Unit Performance Test &

Code Optimization

Performance Test

Monitoring & Capacity Management

Holistic View Of Performance

Txn Performance

- Resp Times, etc.

Application Performance – Operations/Sec, Messages/Sec,

Transactions/Sec, etc.

Infrastructure Performance – CPU Utilization, Memory Utilization, Disk IOPS, etc.

Network Performance – Packet Loss, Jitter, Packet Re-ordering, Delay, etc.

Activities Involved in PE

Software Development Life Cycle

Functional Requirements Gathering

Architecture & Design

Build Application

System Test,

System Integrated Test & UAT

Deploy Into Production

Performance Engineering Life Cycle

Non Functional Requirements Gathering

Design for Performance &

Performance Modelling

Unit Performance Test &

Code Optimization

Performance Test

Monitoring & Capacity Management

What does PE mean to different stake holders

User Expectation

Business Gains the ability to grow services and provision additional customers onto the existing or new platform as business grows

CTO Intends to see an application platform delivered that meets the organizational strategic requirements and scales as business workload grows

CFO Intends to get the most bang for his buck

Program Manager Is able to deliver an application on time, on budget while meeting the Non Functional Requirements as agreed with the customer

Developer Intends to write code that meets the Performance Targets documented by the overall Non Functional Requirements

Systems Administrator Expects to inherits an application that optimally uses infrastructure resources, doesn’t fall over in production, is easy to manage, doesn’t have performance bottlenecks, scales as expected to meet user workload and meets business & IT SLA’s

Capacity Planner Recommends infrastructure requirements based on business workload and application architecture. Works proactively with application teams to recommends application optimization and additional infrastructure required

Performance Tester Validates application performance based on defined Non Functional Requirements and expects to see the application meet defined Non Functional Requirements

End Customer Expects to use an application that provides a great end user experience

Roles Performed by the Practical Performance Analyst

Phase Role Performed

Requirements Gathering Determine Business Volumes & Growth plans Determine Non Functional Requirements

Design Validate Application Architecture Validate Infrastructure Architecture Validate Infrastructure Design Determine Infrastructure Capacity Requirements

Build & Optimization Set performance targets for developers Validate outcome of Unit Performance Tests Recommend optimization to Code Recommend optimization to Application Design Recommend optimization to Infrastructure Design

SIT & UAT Begin validating Application Performance for functionality that’s available Begin Tier Performance Tests for functionality that’s available Test, tune & optimize Performance for functionality that has been released

SVT Execute End to End Performance Tests Validate Performance End to End Identify applicable bottlenecks Use Diagnostics tools to identify bottlenecks, tune & Optimize application Ensure application meets its Non Functional Requirements Evolve Performance Monitoring Requirements

Pre-Go live Setup Performance Monitors for Network, OS, Application, Business Txn Set alerting for various Performance metrics Setup capture Performance metrics for purposes of Capacity Management

Go-Live & Post Go Live Monitor application for potential breaches in SLA’s Identify hotspots using low overhead transactional tracing & diagnostics tools Model application performance and predict capacity impacts for growing business workload

What is Proactive Performance Management

Performance Requirements

Analysis

Performance Modelling &

Capacity Planning

Build & Optimization

Performance Testing

Performance Monitoring

Capacity Management

Costs of Addressing Performance Across the SDLC

Requirements Gathering Phase – Lowest cost since you don’ t have designs yet and you are starting from a fresh slate. You’ve got the opportunity to document the appropriate Non Functional Requirements based on business requirements.

Design Phase – The application designs and infrastructure designs are coming together. You still have an opportunity to influence Non Functional Requirements. You haven’t missed the bus and have the opportunity to influence the design process.

Build Phase – Slight more expensive since the designs have been locked down, your developers have begun writing code according to specifications. Not having Non Functional Requirements means that you can’t really set developer performance targets. You will start feeling the pain once you get into SIT & UAT since you developers are only focussed on application functionality and don’t have visibility of any of the Non Functional Requirements

Testing Phase – Defects found at this stage are expensive to fix since they require you to raise defects and go through the whole change management and QA process. Changes to application or infrastructure design are very expensive to fix. Re-design of any application components due to performance bottlenecks could set the program back by weeks or months.

Go Live – The most expensive to fix. Your application is live, performance issues at this stage impact the business’s credibility, your credibility and the credibility of the whole QA process across the program. You should consider changing jobs and taking up something you are better at before you get fired.

Challenges in Addressing Performance

Lack of understanding of Performance Engineering concepts

Lack of willingness to invest in Performance Engineering tasks

Lack of skills to support Performance Engineering roles

Focus on application functional requirements and application design at the early stages of the program

Expensive tools to perform tasks related to Performance Engineering across the Software Delivery Life Cycle

Gaps between SVT & Production infrastructure

Delays due to delayed code drops, delayed SIT/UAT phases that reduce effective Performance Testing

Lack of diagnostics and monitoring frameworks to proactively identify performance bottlenecks

Lack of Capacity Management tools to proactively manage performance and provision infrastructure capacity

Tooling for Performance

Performance Requirements Gathering

Capacity Planning

Performance Modelling

Performance Testing

Performance Monitoring

Capacity Management

Thank You

trevor@practicalperformanceanalyst.com