Date post: | 13-Feb-2017 |
Category: |
Marketing |
Upload: | ebiquity-na |
View: | 35 times |
Download: | 0 times |
The Best Test: Optimizing for Success
#EMPOWERBL16
Presenter
Optimization Practice Lead, Stratigent
TIM WALKER
#EMPOWERBL16
This session will cover…
Why You Need to Test
To Test or Not to Test3
1
Testing Basics2
Bridging Testing to Personalization4
#EMPOWERBL16
Why You Need To Test
#EMPOWERBL16
Goal of testing
Finding the optimal customer experience…
Across medium (Web, app)
Across the spectrum of personas
#EMPOWERBL16
Testing can help…
Perfect website design and flow
Maximize conversions
Provide insight into user behavior
Feedback on new features
#EMPOWERBL16
Testing stats
Source: Forrester’s Q3 2015 Global Online Testing Platform Customer Online Survey
#EMPOWERBL16
Pre-testing considerations
People
How much of an effect will this test have on the
users?
Process
Do you have the right technology and skills in place to develop and
execute the test?
Technology
Do you have the right technology and skills in place to develop and
execute the test?
#EMPOWERBL16
Testing Basics
#EMPOWERBL16
Testing Process
1.Ideation & prioritizatio
n
3.Execution
2.Implementation
4.Analysis
5.Iterate
#EMPOWERBL16
Types of Tests
A/B test
Multivariate test
Multi-page tests
split up site traffic in a balanced way, and then
show one t of users version A and the second version B
setup to test many different individual changes on a
page all at once
These tests involve changing the experience
across multiple pages
CHANNELSlow high
COMP L E X I T Y
high
#EMPOWERBL16
Client Example |A/B Testing
Client was concerned that not having a clearly defined 'guest' sign in option for the checkout flow was driving away customers from converting
#EMPOWERBL16
The new design led to a higher conversion level
Client Example |A/B Testing
Tested different versions comparing the default with a new design that had guest sign-in front and center
#EMPOWERBL16
Client Example |Multivariate Testing
#EMPOWERBL16
Client Example |Multivariate Testing
#EMPOWERBL16
To Test or Not to Test
#EMPOWERBL16Define what metrics to determine test success
Translate a test idea into a formal hypothesis
Define what metrics to determine test success
Grade each hypothesis
#EMPOWERBL16
Three dimensional grading
Reach
Impact
Technical difficulty
How many users will see this test?
How much of an effect will this test have on the users?
Do you have the right technology and skills in place to develop and execute the test?
#EMPOWERBL16
Metric selectionMetric selection is key to gauging test results, ensure metrics that align with test goals & can detect anomalies are selected.
Conversions | the primary metric of a test - measures what you are trying to accomplish (e.g. orders, leads, sign-ups)
Conversion
Secondary Interactions
Tactical Engagement
Secondary Interactions | measure intermediate steps towards conversion or post conversion (e.g. cart additions, lead quality)
Tactical Engagement | these metrics monitor conversion related values (e.g. units per order, average order value, revenue, profit)
#EMPOWERBL16
When not to test.
Your site design isn’t stable
Don’t know what they are trying to achieve
Don’t have the resources aligned in order to run a testing program
Aren’t mature enough to execute a complex test
#EMPOWERBL16
Bridging Testing to Personalization
#EMPOWERBL16
Testing as a precursor to personalization
We don’t have to treat all users the same.
If the retailor could predict the price the customers were willing to pay, then lower friction and high conversion value. An optimization program could have produced $75 in revenue versus $50
User Example: Coupons for everyone
Same shirt, but two customers are willing to pay different prices…
A retailer generally releases coupons for 50% off retail price
Customer B: $25
Customer A: $50 (retail price)
Customer B pays $25
Customer A pays $25
#EMPOWERBL16
Personalization modeling
Rules based personalization
Build out individual personas
Personalization based on visit history
Personalization based on integration of online/offline data
Customized Personalization
Predictive Personalization
Using 1st hit attributes
Triggers can change experience on the flyCustomers fit into audiences
Customers fit into behavioral segments
Advanced targeting
Look-a-like Models
Segments fit onto customers
Machine learning
#EMPOWERBL16
Client Example |A/B Testing
Client needed to test the size of tiles when page viewed on mobile phones
Saw a 36% increase in click-thru rate with wider tile format
A test was run on both layouts
#EMPOWERBL16
Q & A
THANK YOULinkedIn.com/Ebiquity
@ebiquityglobal
www.ebiquity.com