7/12/2016 1 Trust Your Data, Not Your Instincts Presented: Thursday, July 14, 2016 John Perell, Friends of the Smithsonian Sarah Stallings, National Geographic Society Laura Connors, National Parks Conservation Association Kerri Kerr, Avalon Consulting #Bridge16 John Perell Director of Direct Response and Shared Services Smithsonian Institution Sarah Stallings Director, Annual Giving National Geographic Society Laura Connors Vice President of Membership National Parks Conservation Association Kerri Kerr Senior Vice President Avalon Consulting Group Here with you today… #Bridge16 A B Which one won?
Transcript
7/12/2016
1
Trust Your Data, Not Your
Instincts Presented: Thursday, July 14, 2016
John Perell, Friends of the Smithsonian
Sarah Stallings, National Geographic Society
Laura Connors, National Parks Conservation Association
Kerri Kerr, Avalon Consulting
#Bridge16
John Perell
Director of Direct Response and Shared Services
Smithsonian Institution
Sarah Stallings
Director, Annual Giving
National Geographic Society
Laura Connors
Vice President of Membership
National Parks Conservation Association
Kerri Kerr
Senior Vice President
Avalon Consulting Group
Here with you today…
#Bridge16
A
B
Which one won?
7/12/2016
2
#Bridge16
A
B
Which one won?
#Bridge16
A B
Which one won?
#Bridge16
If we just trusted our instincts,
marketers probably wouldn’t
use the data‐driven best
practices we’ve come to rely on.
Introduction
7/12/2016
3
#Bridge16
In today’s session
Confirming all assumptions with data, from top‐level trends to individual
campaign performance.
Dispelling myths through testing or other sources.
Accessing the data needed to
strategically guide our programs.
Introduction
#Bridge16
Three‐pronged approach: Using data to guide your program
• Using readily available benchmarking reports.
Industry‐level
• Creating donor‐level metrics to provide information about the profile of your program.
Program‐level
• Developing a solid testing plan to guide campaign‐level strategy.
Campaign‐level
#Bridge16
INDUSTRY‐LEVEL INFORMATION
7/12/2016
4
#Bridge16
Industry‐level trends: Using data to confirm assumptions
Many industry‐level data points exist to confirm or dispel common myths:
Overall Trends
• Should we be raising more money online?
• Should we be acquiring more millennials?
Benchmarking
• Is my program “normal” and how do we stack up against other organizations?
#Bridge16
Top‐level industry trends provide guidance on how organizations are raising
money, specifically when it comes to questions about online fundraising.
Industry‐level trends: Direct marketing fundraising
Online up 9.2% in 2015.
Source: Blackbaud 2015 Charitable Giving Report
#Bridge16
Industry‐level trends: Direct marketing fundraising
Less than 10% of giving is online. Direct mail is 80%.
Source: Blackbaud’s 2014 State of the Not‐for‐Profit Industry Report
7/12/2016
5
#Bridge16
Industry‐level trends: Who is giving?
“Should we be raising money from younger donors?”
Readily available industry reports can illustrate the age trends at a top level.
Younger donors (under 40) don’t respond well to direct marketing. Typically they need to “grow” into
being donors.
The long‐term value of millennials does not
compare to that of older donors – the upfront cost is too much to offset with
future revenue.
#Bridge16
Using age overlay data and analyzing productivity in appeals and
acquisition can illustrate the impact of older donors.
Industry‐level trends: Who is giving?
#Bridge16
Industry Benchmarking
Using readily available benchmarking reports allows for comparison to the
industry and individual sectors.
Provides quarterly industry
updates and offers individual
organization benchmarking.
Provides annual online
benchmarking tools.
Provides online and other
benchmarking information.
Provides industry‐level trends
and insights.
7/12/2016
6
#Bridge16
PROGRAM‐LEVEL METRICS
#Bridge16
Program‐level trends: Creating donor‐level metrics to illustrate areas of opportunity
Analyze your file to confirm assumptions, dispel myths, & answer key questions.
Reviewing key metrics like retention provides insight into the health and
stability of the file:
• If problems exist, this view can also help identify downward trends.
For the Friends
program, retention
is very strong
overall!
7/12/2016
7
#Bridge16
Friends of the Smithsonian: Major Donor Pipeline
Analysis of $1,000+ members by join level illustrates that 84% started below
that level.
• For many organizations, the direct marketing program provides a strong
pipeline to the major giving program.
Average time to upgrade
is 8‐9 years for most
programs.
#Bridge16
Friends of the Smithsonian: Age profile
Performing an age append answers the often‐asked question:
“How old are our members?”
• For Friends of the Smithsonian, members skew older than the industry profile.
• Generally speaking,
direct‐marketing‐
responsive members
are older, and younger
groups will “grow into”
giving.
#Bridge16
Overlaying the number of other organizations your members give to provides
insightful information:
• For Friends of the Smithsonian, members are very philanthropic and give to
many other organizations.
This view highlights
competition in the mailbox
but can also serve as a
reminder: members do not
belong to any one
organization.
Friends of the Smithsonian: Giving profile
7/12/2016
8
#Bridge16
National Parks Conservation Association: Estimating the break‐even point for acquisition
Calculating the break‐even point for your acquisition program can provide a
comparison to the industry as a whole.
3‐5 years to break even
is considered strong by
industry standards!
#Bridge16
Quantitative and Qualitative analysis
Focus groups and member surveys can answer questions like:
What messages and topics do our members respond
best to?
Why do my members give to my organization?
What are key sources of
information for my members?
#Bridge16
Quantitative and Qualitative analysis
Rather than using guesswork or anecdotal feedback, quantitative and
qualitative information can give real information and remove assumptions.
Note: Don’t take this analysis at
face value – use it to inform
your testing strategy!
7/12/2016
9
#Bridge16
CAMPAIGN‐LEVEL DATA
#Bridge16
Campaign‐level trends: Using testing to guide your program
When in doubt about the best techniques for your direct marketing program,
test to find the data‐driven answers you need.
Testing is essential to a
healthy, productive
direct marketing
program.
#Bridge16
Campaign‐level trends: Using testing to guide your program
Each organization is different and not all best practices are universal.
Use testing to determine what works for your organization.
Test making incremental or large changes, seeing their impact, while minimizing overall risk.
Testing helps find ways to improve performance, can save money, and improve the
bottom line.
7/12/2016
10
#Bridge16
How to set up your test
Ensure the statistical
significance of your test – use a sample‐size calculator to
accurately project your panels.
Ensure the goal of the test is clear –what will be the measure of success?
• 20% lift in response rate? 10% lift in average gift? 15% reduction in cost?
Test one element at a time to
isolate the impact of the variable
change.
• Carrier? Copy? Ask string?
Choose tests where there is a reason to believe the change will improve the campaign.
• What information informs the testing decision?
To ensure valid testing results, it is important to follow the guidelines below:
#Bridge16
How to measure your test
It is important to accurately read the results of your test, beyond just
“eye‐balling”: Look for differing test results for both the overall campaign and by audience segment.
A difference in results may be insignificant and within the margin of error.
Use a calculator designed to identify significant differences before determining a winning test.
Use roll‐out costs to extrapolate the true impact of the test at full quantities.
#Bridge16
TESTING OVER TIME
(aka Be Patient!)
7/12/2016
11
#Bridge16
Background: NPCA uses a premium in its acquisition program to drive new joins at the $15 level. The long‐term value of premium donors was in question. Additionally, donor‐level analysis showed a clear break at new joins above $25.
Five‐year donor value jumps by 93% for new joins at the $25 first gift.
NPCA: Ask string testing
#Bridge16
Average gift still slightly favored the premium package.
Net revenue was virtually tied with the cost of the premium included.
• NPCA performed a 6‐month test: $15 premium control vs. no premium.
• As expected, the upfront performance indicated a decline in new joins.
• Long‐term value analysis would be the true measure of results.
NPCA: Premium vs. No Premium Testing
#Bridge16
With a premium at $15.
While Year 1 net revenue was virtually tied, Year 2 net revenue strongly favored premium joins due to higher initial volume of premium acquired donors.
Evaluating the new joins from each group at Year 1 and Year 2 unveiled more interesting findings about the test.
NPCA: Premium vs. No Premium Testing
With a premium at $15.
7/12/2016
12
#Bridge16
$2,266$2,159
$0
$500
$1,000
$1,500
$2,000
$2,500
Control ‐ $15 Test ‐ $25
Year 1 Net Revenue
• $25 had lower response, as expected.
• Year 1 net revenue was essentially the same.
NPCA: Ask string testingNPCA tested a $25 ask against the $15 control to measure upfront results, long‐term net revenue, and ROI.
#Bridge16
$28,515
$25,253
$0
$5,000
$10,000
$15,000
$20,000
$25,000
$30,000
Control ‐ $15 Test ‐ $25
Year 3 Net RevenueHowever…
• Over three years, net
revenue favored the
control ($15 ask).
• While a higher‐value
donor is being acquired
with the $25 ask, that
value is not quite high
enough in this test to
offset the far lower initial
join quantity.
NPCA: Ask string testing
#Bridge16
Utilize individual list return on investment metrics. • Don’t be fooled by upfront performance!
36
NPCA: List Analysis
7/12/2016
13
#Bridge16
Test and analyze co‐op modeled lists as part of your acquisition audience.• For many organizations, co‐op lists generate high retaining and high LTV members when
compared to outside lists.• There are vast differences among co‐ops – assess both short‐term and long‐term
performance to determine the most productive universe.
37
NPCA: List Analysis
#Bridge16
TESTING IN CAMPAIGNS
#Bridge16
National Geographic Society: Pilot Campaign Creative Testing
Package #1: From the Field/Mike Fay
7/12/2016
14
#Bridge16
National Geographic Society: Pilot Campaign Creative Testing
Package #2: RSVP/Invitation
#Bridge16
National Geographic Society: Testing Results
RSVP package beat From the Field package at a 99% statistical confidence level:
• 290% stronger response rate
• 21% cheaper package
• $53.72 cost to acquire (return the investment within 2‐4 years)
• Strong initial average gift translates into higher long‐term value
Every list performed better in the RSVP package
Winner
!
WHICH ONE WON?Package Net Response Average Gross Total Net Net/
Mailed Rate Gifts Gift Income Cost Income Donor
From the Field 130,707 0.10% 141 $56.14 $7,916 $63,804 ($55,888) ($396.37)
