10th August 2016John Clegg at ScaleConf
Performance as a Feature
The story of the Impossible Mission Force (IMF) Team at Xero
Performance as a Feature
Performance thinking as a part of dev culture
The Impossible Mission Force
Our team’s ultimate goal is to “self destruct”
Why “Performance as a
Feature” is important?
Customer satisfaction
Speed affect conversion rates
Cost to serveCost to Serve
Mobile experience
Why don’t we all do “Performance as a
Feature” ?
Premature optimisation is EVIL
Premature optimization is the root of all evil
- Donald Knuth (1974)
The right optimisation is NOT EVIL
“We should forget about small efficiencies, say about 97% of the time: Premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.”
- Donald Knuth (1974)
We’ll look at performance at the end...
Functional vs non-functional requirements
Features Features Features
We don’t know it’s a problem?
Product growth
Customers uses of the system gets bigger
People growth
How to make “Performance as a
Feature” happen !
1. Get business buy in2. Get devs to take ownership3. Education + culture
Make “Performance as a Feature” happen!
Business buy in
Prove there is a problem
Quantify investment
Reduce Risk: Product Spikes
Product Spike● A discovery story used to analyse or
answer a question○ Yes - Further define story and
continue ○ No - Save analysis
● Time-boxed● Quantified against our goals
Performance is a feature
Adapt as your product / business grows
Get devs to take ownership
Metrics
How fast are your pages?
Measure all things
Understand what metrics are important
Outliers
Decode metrics for the business
The “Pain” ratio
Find out what percentage of users will experience a given “percentile” time in a user session
p = percentile
n = Average number of pages per session
1 - (1-p)n
The “Pain” ratio example
Example Scenario:
Average number of pages per session = 20 pages
95th percentile total page time for your site = 6s
1 - (1-p)n1 - (1 - 0.05)20 = 1 - 0.358 => 64.2
%64% chance that a user will hit a 6s
page
Get developers theright tools
What’s going on in my stack?
Make performance tests easy
Performance A/B tests
Continuous testing
Feature flagging
Scientist
Education + Culture
Training + Workshops
Performance “Coach”
Work with the teams
Carrot + Stick
What does success look like?
Teams take ownership
Added to “Definition of Done”
Code Quality
Takeaways
Enable teams to take ownership
Enable teams to take ownership
1. Get the right tooling in place2. Own the metrics from the start3. Automate performance testing
Show business the ROI
Show business the Return on Investment
1. Prove you have a problem2. Quantify Investment3. Reduce Risk - Spikes
Education and Training
Education and Training
1. Make it easy to measure & test2. “Hands on” training3. “Coach” the teams
IMF team pic
Resources
MetricsYouTube “How to NOT measure latency” - Gil Tene
YouTube “Fast and Resilient Apps” - Illya Grigorik (Google I/O 16)
Tools“Scientist Measure twice, cut once” - http://githubengineering.com/scientist/
“Feature Toggles” - Martin Fowler - http://martinfowler.com/articles/feature-toggles.html
Pictures
https://www.flickr.com/photos/christian-dl/3754582699
https://www.youtube.com/watch?v=WhgcDeARanQ
https://www.flickr.com/photos/whatknot/15781706315
https://www.flickr.com/photos/developersteve/28219486931/
https://www.flickr.com/photos/deanhochman/13623115913
https://www.flickr.com/photos/srvega/15140121578/
https://www.flickr.com/photos/jarodcarruthers/8489647601
https://www.flickr.com/photos/87007001@N04/15126712086/
https://www.flickr.com/photos/conalg/17250403565
https://www.flickr.com/photos/99783447@N07/9431062947
https://www.flickr.com/photos/62141688@N08/16645686899/
https://www.flickr.com/photos/thomashawk/45974145
https://www.flickr.com/photos/abeles/1390183044
Questions