What’s Worth A/B Testing?Presented live September 30, 2010Featuring Anne HollandPublisher, WhichTestWon.com
Plus Guest Star: Mike VolpeVP of Marketing, HubSpot
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored By:
Who is HubSpot?160+ Employees in Cambridge, MA (MIT)
$33m in VC (3 rounds)Marketing Software at $3K-15K/year
Complicated & Confusing
Easy & Integrated
Blogging
Social Media
Lead Nurturing
Landing Pages
SEO
Email Marketing
AnalyticsLead Intelligence
Sales
Tools
3,200 Customers
Q1 2007 Q2 2010
What’s Worth A/B Testing?Presented live September 30, 2010Featuring Anne HollandPublisher, WhichTestWon.com
Plus Guest Star: Mike VolpeVP of Marketing, HubSpot
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Admin Note:Attendees will receive a Special Report PDF and an on-demand mp3 via email.
Sponsored By:
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Part I. Why Bother Testing?
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Why Bother Testing?Most sites are designed by:
Owner’s “gut”Pre-formatted templatesCopying competitors“Best practices”
Not by testing.
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Better Results From the Same Traffic
Typical results for previously untested pages:B2B lead generation forms +30-40%eCommerce page tests +20-25%
Old Site Tested Site
WhichTestWon.comTesting = Competitive Advantage
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
Marketers who run A/B tests on email campaigns
Marketers who run A/B or Multivariate tests on web pages
How Many Marketers Use Tests to Optimize Response Rates?
n=505; survey conducted August 2010 ©2010 Online Marketing Connect Aprimo & WhichTestWon com
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
How A/B Testing Works
Version A Version B
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Example: A/B Homepage Test
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Results
• 125% lift in clicks to the download offer page
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Part II. Which Web Pages Are Most Worth Testing?
First, Test Where the Money Is…
#1. Lead gen form pages#2. Other offer pages#3. Landing pages for
expensive traffic
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
$0 x 30% = $0$100 x 30% = $130
Form PagesWhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Results WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
• 41.8% lift in lead gen form submissions
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Offer Pages
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Results
• 62.6% lift in paid subscriptions
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Part III. Which Page Elements Are Most Worth Testing?
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Glance From 4+ Feet Away…
Your Page
15” monitor1024 x 768 resolutionHint: 980 pixels across, 520 down• Buttons & response links• Headline wording• Layout Clarity • Forms • Large images
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Test Buttons & Response Links
Results
• 32.5% registration gain
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Test Offer HeadlinesWhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Results WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
• 113% lift in free trial sign-ups
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Copy (Taxonomy) Test
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Results
• 42% lift in lead gen form completions on the next page (for the word “Enterprise”)
Test Offers on Page TemplatesWhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Results WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
• 72.72% lift in lead gen form submissions
Test Key ImagesWhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Results WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
• 37.2% lift in lead gen form completions
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Part IV. Quick Guidelines: Testing in Practice
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
How Much Does Testing Cost?
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
How Long Should a Test Run For?
Before-and-After
WhichTestWon.comWhich Traffic Should You Use?
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
How Conclusive Should Results Be?
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Part V. Convincing the CEO to Let You Test
When the HIPPO Rules…
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
If You Measure $$ Per Conversion
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
If You Measure Cost Per Conversion
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Lastly, Top 2 Safeguards:
1. Politically Correct Tests2. Yank a Dud Test Quickly
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
How HubSpot Supports Testing
1. Easy to Build Landing Pages & Forms2. Easy to Use Reports3. Closed Loop Conversion Rate Data
Mike VolpeVP of Marketing, HubSpot
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Easy to Build Landing Pages
Easy to editNo HTML required
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Easy to Build Forms
Easy to editDrag and dropNo HTML requiredAdd custom fields
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Easy to Use Reports
One Click Away from:• Visits by Source• Leads by Source• Customers by Source
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Easy to Use Reports
Easily measure landing page conversion rates
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Closed Loop Conversion Rate Data
Visit to Lead %andLead to Customer %
WhichTestWon.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Closed Loop Conversion Rate Data
All Conversion Rates
Broken Down Automaticallyby Source
Complicated & Confusing
Easy & Integrated
Blogging
Social Media
Lead Nurturing
Landing Pages
SEO
Email Marketing
AnalyticsLead Intelligence
Sales
Tools
30 Day Free Trial
Thank you!
Anne Holland, PublisherWhichTestWon.comEmail: [email protected]: @WhichTestWon
Mike Volpe, VP MarketingHubSpotEmail: [email protected]: @mvolpehttp://www.HubSpot.com
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com