Testing & Optimisation
What is it and how can it help your business?
Irene Kalkanis, February 2014
We’ll find out:• What ‘testing and optimisation’ means
• How to figure out whether you need it
• How to choose the right tool
• How to make sure you get value
What is Optimisation Testing? Increase conversions from landing pages through to sale or goal
completion
Decrease bounce rate from your landing page
Make your content / services / products easier to find and easier to buy
NOT Search Engine Optimisation (SEO)
NOT a substitute for web analytics
What is an A/B test?
Buy Buy
50% conversion 20% conversion
What is an A/B/n test?
50% conversion 20% conversion 80% conversion 60% conversion
Copy ---------------------------------------- Buy
Copy --------------------------------------------------
Add to
Basket
… you can add as many variations as you like, each one needs to be designed separately
Read more
Buy
What is an MVT test?
Buy Buy
50% conversion 20% conversion 20% conversion 70% conversion
Add to Basket Save Buy now
Buynow
… add more versions of each element and the tool will generate many different combinations of these elements
A/B/n Testing MVT TestingNo real constrictions on which creative or page design can be tested
Test many small page elements and measure their impact on conversion rate
Test dramatic lay out and design differences Measure effect of interaction between elements
Less traffic required to reach clear statistical significance
Make small, incremental changes to your site to reduce the ‘yuck’ effect
Ability to analyse effects of different elements and combinations in detail
A/B/n Testing MVT TestingTests need to be planned carefully to ensure that we understand why one version wins over another
Requires much more traffic to reach statistical significance
All test pages must be designed manually Usually give approximate results
Can lead to sudden, major changes which can result in the ‘yuck’ effect
The methodology of focusing on small elements can constrain creativity
Can’t test major layout changes easily
Pros
Cons
Do I need to test my site?
The first step is admitting you have a problem
Greenpeace website 2004 Can you spot the call to action?
Greenpeace website 2014 DONATE!! LIKE!! PROTECT TIGERS!!!!
Every website can be optimisedYou’re already optimising by trial and error, a testing tool will let you:
• make decisions informed by what your audience actually does rather than what they tell you, or your own ‘gut feeling’
• learn what works much faster
• test changes on small portions of your audience before going live
• test your changes and hypotheses in a scientific way
These tools offer both MVT and A/B/n testing
Some tools are more focused on UX
Some tools are more focused on marketing
Many will design variants, set up the tests and analyse the results for you
Different tools & set ups work better for different sites, vendors will be able to tell you what they can do for your site
Remember that you don’t have to use the same vendor for different tools
Make sure you get value from your tool• Invest in a good support structure
- It takes a village: optimisation testing can be work intensive and needs buy in from stakeholders up front
- You may need creative and analysts to build and understand your tests- Heat mapping tools help you dig deeper into A/B/n tests
• Don’t test everything- Agree on specific hypotheses you want to test- Agree exactly what a ‘conversion’ means to your business - Focus your tests on landing pages and key conversion journeys
• Don’t test forever- If you haven’t reached a statistically significant result within a pre-agreed period of time there’s no reason to
make a change
Crazy egg heatmap – confetti view
Expedia drops optional form, earns $12m profit
Two optimisation case studiesfrom e-consultancy.com
ASOS halves cart abandonment by losing the word ‘register’
These tools also offer targeting and ‘personalisation’
Summary1. A/B and MVT testing: a way to optimise you site by test different versions at the same
time2. Picking the right tool: get information and quotes from vendors, seek non-partial advice
from an expert3. Get value: use tools wisely on well known problems, build a good team of creative and
analysts around the tool