10th August 2016John Clegg at ScaleConf

Performance as a Feature

The story of the Impossible Mission Force (IMF) Team at Xero

Performance as a Feature

Performance thinking as a part of dev culture

The Impossible Mission Force

Our team’s ultimate goal is to “self destruct”

Why “Performance as a

Feature” is important?

Customer satisfaction

Speed affect conversion rates

Cost to serveCost to Serve

Mobile experience

Why don’t we all do “Performance as a

Feature” ?

Premature optimisation is EVIL

Premature optimization is the root of all evil

- Donald Knuth (1974)

The right optimisation is NOT EVIL

“We should forget about small efficiencies, say about 97% of the time: Premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.”

- Donald Knuth (1974)

We’ll look at performance at the end...

Functional vs non-functional requirements

Features Features Features

We don’t know it’s a problem?

Product growth

Customers uses of the system gets bigger

People growth

How to make “Performance as a

Feature” happen !

1. Get business buy in2. Get devs to take ownership3. Education + culture

Make “Performance as a Feature” happen!

Business buy in

Prove there is a problem

Quantify investment

Reduce Risk: Product Spikes

Product Spike● A discovery story used to analyse or

answer a question○ Yes - Further define story and

continue ○ No - Save analysis

● Time-boxed● Quantified against our goals

Performance is a feature

Adapt as your product / business grows

Get devs to take ownership

Metrics

How fast are your pages?

Measure all things

Understand what metrics are important

Outliers

Decode metrics for the business

The “Pain” ratio

Find out what percentage of users will experience a given “percentile” time in a user session

p = percentile

n = Average number of pages per session

1 - (1-p)n

The “Pain” ratio example

Example Scenario:

Average number of pages per session = 20 pages

95th percentile total page time for your site = 6s

1 - (1-p)n1 - (1 - 0.05)20 = 1 - 0.358 => 64.2

%64% chance that a user will hit a 6s

page

Get developers theright tools

What’s going on in my stack?

Make performance tests easy

Performance A/B tests

Continuous testing

Feature flagging

Scientist

Education + Culture

Training + Workshops

Performance “Coach”

Work with the teams

Carrot + Stick

What does success look like?

Teams take ownership

Added to “Definition of Done”

Code Quality

Takeaways

Enable teams to take ownership

Enable teams to take ownership

1. Get the right tooling in place2. Own the metrics from the start3. Automate performance testing

Show business the ROI

Show business the Return on Investment

1. Prove you have a problem2. Quantify Investment3. Reduce Risk - Spikes

Education and Training

Education and Training

1. Make it easy to measure & test2. “Hands on” training3. “Coach” the teams

IMF team pic

Resources

MetricsYouTube “How to NOT measure latency” - Gil Tene

YouTube “Fast and Resilient Apps” - Illya Grigorik (Google I/O 16)

Tools“Scientist Measure twice, cut once” - http://githubengineering.com/scientist/

“Feature Toggles” - Martin Fowler - http://martinfowler.com/articles/feature-toggles.html

Pictures

https://www.flickr.com/photos/christian-dl/3754582699

https://www.youtube.com/watch?v=WhgcDeARanQ

https://www.flickr.com/photos/whatknot/15781706315

https://www.flickr.com/photos/developersteve/28219486931/

https://www.flickr.com/photos/deanhochman/13623115913

https://www.flickr.com/photos/srvega/15140121578/

https://www.flickr.com/photos/jarodcarruthers/8489647601

https://www.flickr.com/photos/87007001@N04/15126712086/

https://www.flickr.com/photos/conalg/17250403565

https://www.flickr.com/photos/99783447@N07/9431062947

https://www.flickr.com/photos/62141688@N08/16645686899/

https://www.flickr.com/photos/thomashawk/45974145

https://www.flickr.com/photos/abeles/1390183044

Questions