Date post: | 22-Nov-2014 |
Category: |
Internet |
Upload: | cliff-crocker |
View: | 591 times |
Download: | 1 times |
Making More Happen – How to Measure Revenue in Milliseconds
Cliff Crocker@cliffcrockerSOASTAVP Product, mPulse
Performance is a Business Problem
Funnel Optimization is Everything
• Merchandizing• SEO• Multivariate testing• Pricing• Ad Campaigns & targeting• User Experience
Performance IS User Experience!!
Yahoo! - 2008Slower pages cause increase in abandonment
Increase of 400ms causes 5-9% increase in user abandonment
http://www.slideshare.net/stubbornella/designing-fast-websites-presentation
Shopzilla - 2009Faster pages increase site conversion rate
A reduction in Page Load time of 5s increased site conversion 7-12%!
http://assets.en.oreilly.com/1/event/29/Shopzilla%27s%20Site%20Redo%20-%20You%20Get%20What%20You%20Measure%20Presentation.ppt
Walmart - 2012Impact of slow pages on conversion rate
http://minus.com/msM8y8nyh#1e
SF WebPerf – 2012Up to 2% conversion increase for every second
SIMULATION
Unfortunately, it’s not always that straightforward
Not All Pages are Created EqualTop of funnel impact (Browse Pages)
For a typical eCommerce site, conversion rate drops by up to
50% when “browse” pages increase from 1
to 6 seconds
Not All Pages are Created EqualBottom of funnel impact (Checkout pages)
However, there is much less
impact to conversion
when “checkout”
pages degrade
Getting StartedThe challenge of change
• Organizational challenges• Process challenges• Visibility• Expertise• Focus & prioritization
Steve SkroceStaplesSr. Manager, Site [email protected]
How Did I Get Here
Developer SEO Performance
SiteSpeed & StaplesProcess change
• A top goal for Staples eCommerce• It’s about customer experience
• Speed is essential to UX• Because Staples is highly matrixed
• Visibility into the organization is key• Approach as a discipline and institutionalize the process
First Things FirstEstablishing a framework for SiteSpeed
• You can’t manage what you can’t measure• Earliest work involved selecting the proper tools
• Business and Engineering led team• Already had synthetic transaction monitoring tool• Backend tools were also available• RUM tool was missing from the set
• Needed RUM info if we were going to affect our customers’ experiences
First Things FirstStakeholders
• Assembling a cross-functional team• Business
• Decides which features and functionality lives on the site• Engineering
• How something gets built can make or break page load time• Marketing
• Asset optimization is important• Merchandising
• On-site content• Analytics
• Analytics team ties everything together• And others…
Why Is Speed Important to Staples?Customer experience
Bounce Rate increases as load time increases
Why Is Speed Important to Staples?Sales
Conversion rate decreases as load time increases
The First PassIdentify business functionality
• Governance meetings to evaluate functionality• Cataloged all site features• Weekly meeting to discuss one or two features in detail
• Feature owner comes prepared to review all aspects
• Full Page Load vs. Time to Interactive• How Staples defines them• FPL ensured wide exposure / awareness through the organization
• Everyone was involved from the beginning• Set the expectation of accountability
The First PassOptimizing for speed
• Onsite promotional banners were too “heavy”• Inconsistent image optimization
• Depended on the individual, not the process• 25k and 250k
• Put a new process in place
The First PassA/B Testing
• Testing practices were inefficient• Two vehicles for A/B testing• Eliminate one
• Process improvement opportunity for ending campaigns• Eliminate lingering artifacts
The First PassFocus on third-parties
• Needed “sanity” around third party calls• Categorization of calls
• Ads• Marketing• Analytics• Recommendations• Social Buttons• Etc.
• Removed some calls, optimized others• Further work needs to be done
The First PassLooking at the back-end
• Database calls were inefficient• Some pages went from 200 calls to 10
EvolutionWhat we’ve learned
• Go bolder, sooner• Sprint model allowed us to deliver something every
release• Something, not something substantial• Not enough time for big, bold changes
• We needed to step-back and reevaluate
How fast are we, really?Getting situational awareness
• Examples of pagespeed for Staples• Median vs. percentiles
• Make it simple for a wide audience to understand the metrics• Key page performance – target specific pages to focus on
• Home, Product, Search, Class, Login, Cart
Make Faster Page Load Times HappenHow do we get there?
• mPulse• WebPagetest.org• Dev Tools• NetPanel
• “Motivational” emails from the top
Make ‘Faster Page Load Times’ HappenHow do we get there?
• Confirmation of what we already knew:• Front-end vs. back-end time
X
JS refactoring Reduced k size by 48% Reduced lines of code by 52%
CSS refactoring Reduced k size by 83% Reduced lines of code by 88%
Early ResultsHard work pays off
Early ResultsHard work pays off
• Shaved 1 full second off of Home page median• Shaved 6 seconds off of the 98th%• Conversion improved by roughly 10%