Post on 23-Jan-2018
transcript
Marketo’s Secrets to Campaign Optimization
Mike Madden
Sr. Marketing Manager
Marketo
Proprietary & Confidential
Housekeeping
Page 2© Marketo, Inc. 10/10/2017 Proprietary & Confidential
• This webinar is being recorded! Slides and recording will be sent to you after the webinar concludes.
• Have a question? Use the chat box and I’ll get to your questions after the webinar.
• Posting to social? Use our hashtag - #mktgnation
• There is a brief survey after the webinar
Proprietary & Confidential
What is A/B Testing?
1
Testing Methodology
2
The 5 Tests
3
Conclusion
4
Agenda
Page 3© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Page 4© Marketo, Inc. 10/10/2017 Proprietary & Confidential
What is A/B Testing?
A/B testing, also known as split testing, compares two versions of the same campaign on a certain channel. These channels include (but are not limited to):
Website
Social Media
Emails
Mobile Apps
Landing Pages
Digital Ads
Page 5© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Why is A/B Testing Important?
A/B testing increases engagement with your buyers
A/B testing enhances campaign effectiveness and optimizes programs for a company’s
target audience
A/B testing enhances a marketer’s
awareness and expertise about
audience preferences
Page 6© Marketo, Inc. 10/10/2017 Proprietary & Confidential
The A/B Testing WorkflowChoose one element to
test
Ask a question /
write a hypothesis
Decide on the sample
group
Define what
success looks like
Set up the test
Look at the test results
Determine the winner
Implement the new control
Page 7© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Be Sure to Use Statistical Significance Calculators!
Page 8© Marketo, Inc. 10/10/2017 Proprietary & Confidential
1 EMAIL SEND TIME
Page 9© Marketo, Inc. 10/10/2017 Proprietary & Confidential
What is the best time of day to send an email?
Page 10© Marketo, Inc. 10/10/2017 Proprietary & Confidential
The Time Test Set Up
Total Audience
All marketable names, last
engagement date in past 1 year,
marketing job titles
5am (Control)
10am
1pm
Details:
• Split total audience 3 ways
• Backed into numbers to see at least
1,000 opens per test (for sig.)
• Sent each email from a different
campaign to ensure exact timing on
the send
• Tested the time testing over 7
different email campaigns
• Over 200,000 emails sent
• Tested in early, mid, and late buying
stages
Page 11© Marketo, Inc. 10/10/2017 Proprietary & Confidential
The Results
Results by significance:
• 1pm has a 14% higher open rate than 5am (100% statistical significance)
• 1pm has a 5% higher click to open rate than 5am (statistically insignificant)
• 1pm has a 20% higher click through rate than 5am (100% statistical significance)
• 1pm has a 7% higher unsubscribe rate than 5am (statistically insignificant)
1pm gives us 20% higher email engagement
Sent Delivered % Delivered Opened % Opened Clicks CTR CTO Unsub. Unsub %
5am (Control) 70128 68791 98.1% 5983 8.7% 306 0.4% 5.1% 98 0.14%
10am 68034 66763 98.1% 6227 9.3% 342 0.5% 5.5% 112 0.17%
1pm 68067 66754 98.1% 6642 9.9% 356 0.5% 5.4% 102 0.15%
Page 12© Marketo, Inc. 10/10/2017 Proprietary & Confidential
‒ It may take a while to find significance, but keep testing
‒ Make the times different enough to see some results
Proprietary & Confidential
Page 13© Marketo, Inc. 10/10/2017 Proprietary & Confidential
2 GENERIC VS PERSONALIZED FROM NAME
Page 14© Marketo, Inc. 10/10/2017 Proprietary & Confidential
What the Industry Says About Personalization
Personalized emails deliver 6x higher transaction rates. - Experian
Personalized email messages improve click-through rates by an average of 14% and conversions by 10%. - Aberdeen
When asked to prioritize one capability that will be most important to marketing in the future, 33% of marketers answered: “personalization.” - AdAge
Page 15© Marketo, Inc. 10/10/2017 Proprietary & Confidential
What Was Our Normal Personalization Process
These are tokens in Marketo, which personalize values based on fields in the database.
They give us two options:
1)1) Personalized
• From Name: Lead Owner’s Name
• From Address: Lead Owner’s Email Address
• Reply-to: Lead Owner’s Email Address
2) Default
• From Name: Team Marketo
• From Address: marketoteam@marketo.com
• Reply-to: marketoteam@marketo.com
Page 16© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Is a Personalized Email Really Better?
Page 17© Marketo, Inc. 10/10/2017 Proprietary & Confidential
The Simple Personalization Test
Best regards,
Mike Madden
Sr. Marketing Manager
1.650.376.2300
Best regards,
Team Marketo
Marketo, Inc.
1.650.376.2300
Control
Test
v
s
Page 18© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Testing Details
16 total campaigns
107,840 emails sent
Tests in early, mid, and late stage buying
stages
Page 19© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Where to Introduce Personalization?
Winner: Generic
Results:
• 10% higher open rate
• 26% higher click to open rate
• 38% higher click through rate
100% statistical significance
Early Stage
Page 20© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Where to Introduce Personalization?
Mid StageEarly Stage
Winner: Generic
Results:
• 10% higher open rate
• 26% higher click to open rate
• 38% higher click through rate
100% statistical significance
Winner: Generic
Results:
• 3% lower open rate
• 30% higher click to open rate
• 26% higher click through rate
100% statistical significance
Page 21© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Where to Introduce Personalization?
Late StageMid StageEarly Stage
Winner: Generic From Name
Results:
• 10% higher open rate
• 26% higher click to open rate
• 38% higher click through rate
100% statistical significance
Winner: Generic From Name
Results:
• 3% lower open rate
• 30% higher click to open rate
• 26% higher click through rate
100% statistical significance
Winner: Generic From Name
Results:
• 4% higher open rate
• 5% higher click to open rate
• 9% higher click through rate
66% statistical significance
Since these numbers aren’t significant and these are late stage leads, we stick with the Personalized From Name.
Page 22© Marketo, Inc. 10/10/2017 Proprietary & Confidential
‒ Introducing personalization in the beginning of a buyer’s journey might be worse for the buyer.
‒ Make the right business decision even if a test says otherwise.
‒ Use this test to find the right handoff point for personalization
Proprietary & Confidential
Page 23© Marketo, Inc. 10/10/2017 Proprietary & Confidential
3 HTML VS. TEXT-BASED EMAILS
Page 24© Marketo, Inc. 10/10/2017 Proprietary & Confidential
HTML or Text-Based Emails: Which is Better?
Page 25© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Emails Are Like Landing Pages
www.marketo.com Social Links
3 Main Offer Links
Page 26© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Testing Details
5 total campaigns
99,702 emails sent
Tested with different
offers, but held
templates constant
Page 27© Marketo, Inc. 10/10/2017 Proprietary & Confidential
The HTML vs. Text Email A/B Test
vs
Page 28© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Data at First Glance
Over the course of 5 different email drops, we have found text emails to do the following:
• Roughly the same open rate as HTML emails
• 11% higher click to open rates (95% statistical significance)
• 8% higher click through rates (86% statistical significance)
Page 29© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Deeper Dive into Unique Link Performance
Email Name Link Clicks % of Clicks Buyers % of Buyers
HTML Email Twitter Link 16 12.8% 8 9.1%
HTML Email Facebook Link 13 10.4% 5 5.7%
HTML Email Marketo Logo Link 1 0.8% 1 1.1%
HTML Email Offer Link 95 76.0% 74 84.1%
Total 125 100% 88 100%
Page 30© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Final Data
Looking solely at unique offer clicks, the data strongly favors text-based emails…
• Roughly the same open rate as HTML emails
• 21% higher click to open rates (100% statistical significance)
• 17% higher click through rates (99% statistical significance)
Page 31© Marketo, Inc. 10/10/2017 Proprietary & Confidential
‒ Sometimes the pretty emails aren’t the winners
‒ On the bright-side, text-based emails require less creative resources
‒ Test these out across asset types. You may find that HTML emails work better for some programs
Proprietary & Confidential
Page 32© Marketo, Inc. 10/10/2017 Proprietary & Confidential
4 WEBINAR OPTIMIZATION
Page 33© Marketo, Inc. 10/10/2017 Proprietary & Confidential
We Have 3 Types of Buyers
Behavior Score 0 - 5
Early Stage
Behavior Score 6 - 19
Mid Stage
Behavior Score > 19
Late stage
Page 34© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Those Leads Plotted by Behavior Score
0 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000
Early
Mid
Late
Leads Broken Down by Stage
Page 35© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Who registers for our webinars? Who attends them?
Page 36© Marketo, Inc. 10/10/2017 Proprietary & Confidential
An Early Stage Webinar
Abstract:
As Seth Godin put it, why waste a sentence saying nothing? If you want content opened, read, and clicked, it all starts with the subject line. Listen to Mike Madden, Sr. Demand Generation Programs Manager at Marketo, for this on-demand webinar, 3 Hacks to Boost Email Open Rates, to learn three simple and surprising tips to launch your open rates sky high!
Page 37© Marketo, Inc. 10/10/2017 Proprietary & Confidential
After the 1st Email Invite, Here’s What Happened…
5
185
1037
0 200 400 600 800 1000 1200
Early
Mid
Late
Registered Leads Broken Down by Stage
Page 38© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Another Early Stage Webinar
Abstract:
Whether you’re new to social media marketing or a seasoned professional, there are a few big mistakes that you’ve probably made in the past, might be making now, and can avoid making in the future. Learn from Lisa Marcyes, Sr. Social Media Marketing Manager at Marketo, as she discusses the top eight social media marketing mistakes—from strategy, to content, to metrics—and how you can avoid them.
Page 39© Marketo, Inc. 10/10/2017 Proprietary & Confidential
And Yet We See It Again!
155
687
2716
0 500 1000 1500 2000 2500 3000
Early
Mid
Late
Registered Leads Broken Down by Stage
Page 40© Marketo, Inc. 10/10/2017 Proprietary & Confidential
8 Biggest Mistakes Social Attendee List
23
184
889
0 100 200 300 400 500 600 700 800 900 1000
Early
Mid
Late
Attended Leads Broken Down by Stage
Page 41© Marketo, Inc. 10/10/2017 Proprietary & Confidential
8 Biggest Mistakes Social – Attendance Rates
15%
27%
33%
0% 5% 10% 15% 20% 25% 30% 35%
Early
Mid
Late
Attendance Rates Broken Down by Stage
Page 42© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Let’s go back to the 3 Hacks to Boost Open Rates Webinar…
Page 43© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Email Performance Broken Down by Stage
Stage Sent Delivered % Delivered Opens Open Rate Clicks CTR CTO Unsub. % Unsub. Registered LP Conversion
Early 96955 94954 97.9% 11160 11.8% 147 0.2% 1.3% 158 0.2% 1 0.68%
Mid/Late 51969 50546 97.3% 13720 27.1% 1959 3.6% 14.3% 49 0.1% 1208 61.66%
Index Index Index Index Index
100 100 100 100 100
231 2420 1084 53 9065
Page 44© Marketo, Inc. 10/10/2017 Proprietary & Confidential
3 Hacks Webinar – Final Attendee Breakdown
207
726
1745
53
253
544
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Early
Mid
Late
Final Attendee/Registration List by Stage
Attended
Registered
31% attendance
35% attendance
26% attendance
Page 45© Marketo, Inc. 10/10/2017 Proprietary & Confidential
‒ It’s all about investment
‒ Give people what they want, when they want it
‒ Put more focus on early stage
Proprietary & Confidential
Page 46© Marketo, Inc. 10/10/2017 Proprietary & Confidential
5 UPDATING CTA TREATMENT
Page 47© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Ahh, Email Design Challenges…
Page 48© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Does CTA button treatment affect click to open rates?
Page 49© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Which CTA Button Would You Click?
Control:
• Squared corners
• All uppercase text
Test:
• Rounded corners
• Proper case italicized text
vs
Page 50© Marketo, Inc. 10/10/2017 Proprietary & Confidential
The Test Button Has Spacers for Alignment
Button Attributes:
• Table expands to 100%, not a fixed width
• 5 pixels of padding on top of the button
• 10 pixels of padding within the button to keep text centered
Page 51© Marketo, Inc. 10/10/2017 Proprietary & Confidential
The Test Button Renders More Consistently
And it just looks pretty!
Page 52© Marketo, Inc. 10/10/2017 Proprietary & Confidential
Testing Details
7 total campaigns
154,073 emails sent
Tested with different
offers, but held CTA Buttons constant
Page 53© Marketo, Inc. 10/10/2017 Proprietary & Confidential
The Results
Results by significance:
• New button has a 4% higher click to open (statistically insignificant)
• New button has a 25% lower unsubscribe rate than the control (100% statistical significance)
The rounded button with italicized text and proper spacing reduces our unsubscribe rate by 25%!
Sent Delivered % Delivered Opens % Opened Clicks CTR CTO Unsub. Unsub. %
Control 77227 75171 97.3% 11786 15.7% 1036 1.4% 8.8% 183 0.24%
New Button 76846 74797 97.3% 11744 15.7% 1073 1.4% 9.1% 136 0.18%
Page 54© Marketo, Inc. 10/10/2017 Proprietary & Confidential
‒ Sometimes you test for one thing and solve for another
‒ Use well-built HTML buttons (you can find them online)
Proprietary & Confidential
Questions?
https://www.linkedin.com/in/michaelmadden824