Precision Performances. Uncompromising Results.
Precision Performances. Uncompromising Results.
B2B Internet Research:the Key to SuccessRoland Klassen
Tim Sinke
Precision Performances. Uncompromising Results.
Who are we andwhat’s our bias?
Precision Performances. Uncompromising Results.
Acrobat Research• CATI and Web – 3 offices – Toronto, Sudbury,
Canso, NS
• Web since 1997
• Majority of Business is CATI
• Background is B2B, traditional focus in IT
• Approach has been to attempt to take an impartial view
Questions to Ask
Precision Performances. Uncompromising Results.
Questions to Ask
Precision Performances. Uncompromising Results.
Population to be Sampled• Identifiable?
• Online?
• Reachable on phone?
• Over-surveyed?
• Already exists in a panel?
Questions to Ask
Precision Performances. Uncompromising Results.
Type of Info to be Collected?• Complexity?
• Length?
• Stimuli needed?
• Mixed-mode?
Questions to Ask
Precision Performances. Uncompromising Results.
Geographical coverage?
Questions to Ask
Precision Performances. Uncompromising Results.
Does it need to be Projectable?• Limitation based on company
size/industry?
• Does it have to be matched to D&B SICs/NAICs/etc?
Questions to Ask
Precision Performances. Uncompromising Results.
Other Questions• Response Rate
• Time in field
• Incentives
• Budget Expectations
– Faster and cheaper, right?• Maybe
Questions to Ask
Precision Performances. Uncompromising Results.
Other Questions...
• CAN the survey be self-administered?
• SHOULD the survey be self-administered?
Questions to Ask
Precision Performances. Uncompromising Results.
Sample Sources
Precision Performances. Uncompromising Results.
Internal Lists
• Customers
• Employees
• Prospects
Sample Sources
Precision Performances. Uncompromising Results.
Drop-in/Intercepts• Web-site specific
• For general web sample – dropping – click-through rates are dropping off to nothing
Sample Sources
Precision Performances. Uncompromising Results.
Online Panels• Main general panels in NA are SSI,
Greenfield, e-Rewards, Harris Interactive
– Get specifics on their panel build and profile processes
• Specialty panels
Sample Sources
Precision Performances. Uncompromising Results.
A word on major online panels
• Educate yourself
• Review your screening criteria
• Are the numbers available?
• Geographies
Sample Sources
Precision Performances. Uncompromising Results.
Who are you talking to?• Went to www.ask.com – typed “online
market research pros and cons” got these…
Sample Sources
Precision Performances. Uncompromising Results.
Sample Sources
Precision Performances. Uncompromising Results.
Sample Sources
Precision Performances. Uncompromising Results.
A Quick Review of one Player • Used SSI as an example
– Not meant to support or criticize
• Following are excerpts from their website:
Sample Sources
Precision Performances. Uncompromising Results.
What it says:• SurveySpot panelists are recruited from many sources. These
include banner ads on the Web and other Internet methods, and by RDD telephone. All SurveySpot members are recruited exclusively by permission-based techniques. SSI does not use unsolicited e-mail (or spam) in building the panel.
• SurveySpot panel demographics are not based on predictive techniques. They are created from self-reported, respondent-specific information. Researchers are assured that they are reaching the exact target market they seek meaning more cost-effective online research [EMPHASIS ADDED]
Sample Sources
Precision Performances. Uncompromising Results.
Industry Select Offered:• Industry
– Agriculture SIC 01-09– Mining SIC 10-14– Contracting SIC 15-17– Manufacturing SIC 20-39– Utilities SIC 40-49– Wholesale SIC 50-51
Sample Sources
– Retail SIC 52-59– Financial SIC 60-67– Services SIC 70-79– Professionals SIC 80-89– Government SIC 91-97
Precision Performances. Uncompromising Results.
What are the issues with this?• Respondents don’t know their SIC Code
• How are SIC Codes mapped and assigned? Primary/secondary? To what level of detail?
• Does it matter to you?
• Do they match back to D&B if you’re extrapolating to the industry?
Sample Sources
Precision Performances. Uncompromising Results.
Recruit-to-Web
Precision Performances. Uncompromising Results.
Background• What is R2W?
– CATI recruit – web completion– User ID/Password protected– Appropriate follow ups – based on
response/completion stage
• Why use R2W?
– Can’t reach everyone through a panel– You want to be sure of the respondent
Recruit-to-Web
Precision Performances. Uncompromising Results.
Examples• Most senior C-Level IT person in Global 500
corporations – US and Europe
• Office equipment DMs mailed physical samples to take a follow-up survey
• Telecom decision makers recruited in cross-section of businesses in specific geographies
• Results – in Survey Stats segment
Recruit-to-Web
Precision Performances. Uncompromising Results.
General Process• (CATI) Recruit
– Custom email – immediate or batch– Follow up – 48-72 hours– Follow up – 1 week– Optional after that
Recruit-to-Web
Precision Performances. Uncompromising Results.
Applications• Can be effective; however
• Use to
– Support definite online requirement – can’t be done on the phone
– Refusal conversion and/or multi-modal survey• If “standard” tool – could become more expensive
than CATI
• More rigorous in defining and controlling the respondent
Recruit-to-Web
Precision Performances. Uncompromising Results.
Survey Stats
Precision Performances. Uncompromising Results.
Customer/Internal Lists• Response rates – 5 to 75%
• Lower end tends to be lower than low end of CATI CSAT
• What is less intrusive – email (suspected SPAM) or CATI (suspected telemarketing)?
– Subject line/intros/”warm-ups” are key
Survey Stats
Precision Performances. Uncompromising Results.
Web Panel• Cost on CPI – response rate “isn’t your
problem” – however, feasibility is
• Be careful on over-selling
Survey Stats
Precision Performances. Uncompromising Results.
Factors Influencing Response Rates
• Survey Length
• Respondent Level
• Incentive
• Reminders
• Company Size
• External stimuli/downloads required – obstacles
• Geography
Survey Stats
Precision Performances. Uncompromising Results.
Examples of Survey Response• Historical
– Used to target/achieve 50-60% conversion on R2W studies
– Declining – now aggressive target is 35-50%
Survey Stats
Precision Performances. Uncompromising Results.
C-Level in Global 500 Corporations
• R2W – High Incidence; most senior contact; 40 minutes, $300 US incentive. Conversion – US – 42.8%; Euro 16.9%
Survey Stats
367
15791 14
125
2459
9
325
0 100 200 300 400 500 600
US
UK
IT
GE
FR
Precision Performances. Uncompromising Results.
International Office Equipment Study
• R2W – High Incidence; 30 minutes, incentive $50 US; 40 Euros (had to scale up to 100 Euros). Conversion – US – 58.2%; Euro 20.9%
Survey Stats
569 327
402 91
244 65
258
33
0 200 400 600 800 1000
US
UK
GE
FR
Precision Performances. Uncompromising Results.
Production Environment Study
• Web Panel - Europe
Survey Stats
Precision Performances. Uncompromising Results.
Telecom DM (internet)• R2W – Small bus Telecom DMs; 51 minutes $35
incentive. Conversion 35.5%
Survey Stats
685
243
0 100 200 300 400 500 600 700
Recruits
CPs
Precision Performances. Uncompromising Results.
A Different View• From e-Rewards.
• Demographic Profile Variable analysis of response rates, based on:
– Tot emails sent: 5.7 m– Tot surv resp: 1.3 m
(NOTE: Not sent on B2B studies specifically)
Survey StatsOccupation Resp Rate
Exec/Upper Mgt 17.9%
Sales 18.8%
Military 20.2%
Bus Owner/Self Empl 21.6%
Crafts/Constr 21.8%
Farming/Primary 23.0%
MEAN 23.0%
Mid Mgt 23.2%
Prof/Tech 24.3%
Retired 24.5%
Student 24.8%
Other 25.0%
Artistic/Literary 25.7%
Clerical 26.4%
Unemployed 27.0%
Doctor 27.9%
Teacher 30.2%
Homemaker 30.3%
Precision Performances. Uncompromising Results.
Quality of Data• Lower response rates – higher bias potential?
• Who are the non-responders?
• Equally important – who are the responders?
– Who’s answering – quality/validation checks?
• Web response rates declining, but not yet worse than telephone or mail
Survey Stats
Precision Performances. Uncompromising Results.
Incentives
Precision Performances. Uncompromising Results.
What’s required – how much and how?
• Panels – costs included in CPI – may change for B2B
• Pressure is increasing
Incentives
Precision Performances. Uncompromising Results.
Amounts to Pay• Low-level - $1 per
minute
• Mid-level - $1.50-$2.00 per minute
• High-level - $35+ (short); $40+ (mid); $50+ (long) – add $2.00 per minute above 20 minutes in length
Incentives
0
5
10
15
20
25
30
35
40
45
50
Low Level Mid Level High Level
Short (<10 min) Med (15 min) Long (20+min)
Precision Performances. Uncompromising Results.
How to Pay• Cash (cheque)
• E-Certificate (simpler, but expectation of instant delivery)
• Close survey and review data prior to issuing incentives
• Notify respondents of timeframes
• Create a centralized database and 800 number/central email for queries
Incentives
Precision Performances. Uncompromising Results.
KSF – Summary Checklist
Precision Performances. Uncompromising Results.
Check points for Success (1)• Question research objectives
• Multi-modal?
• Pick sample partner carefully
• Put sample partner “through the wringer”
• Analyze data for comparison and validation
KSF – Summary Checklist
Precision Performances. Uncompromising Results.
Check points for Success (2)
• Are respondents engaged? Devise tests
– Look for repetition– Average length– Volume of DK/RF
• Work in partnership with data collection partner
• Be ready to adjust on the fly
• Always, always, always pilot qualitatively
• Remember, you’re only as good as your sample
KSF – Summary Checklist
Precision Performances. Uncompromising Results.
Thank you. Questions?