Date post: | 21-Jan-2015 |
Category: |
Business |
Upload: | clearsaleing |
View: | 1,573 times |
Download: | 0 times |
©2009 Third Door Media, Inc.
Attribution Management Forum 3.0: How To Build Accurate Models
To Solve Attribution
Tuesday May 5, 2009 1 PM EDT
Speakers: Adam Goldberg, Dr. Purush Papatla
©2009 Third Door Media, Inc.
Viewing Tips
Turn Off Pop-Up BlockersTechnical difficulties?
Click on “Help?” link Use Q+A box
Submitting questions to speaker Q+A session at end of webcast Use “Ask a Question” box to submit
questions Send questions at any time
©2009 Third Door Media, Inc.
Adam Goldberg, Chief Innovation Officer
Co-Founded ClearSaleing Inc. in 2006Columbus, OH
Started Google’s inside sales organization in 2003-2006New York City, NY
Started Actuate’s inside sales organization in 2000-2003San Francisco, CA
Worked for Oracle Corp. in 1998-2000 in Major Account SalesRedwood Shores, CA
Speaker and Trainer at events such as: (SMX), (SES), (DMA)
©2009 Third Door Media, Inc.
Dr. Purush Papatla
Ph.D. from Kellogg School of Management at Northwestern University
Associate Professor, MarketingSheldon B. Lubar School of Business
President and Founder; Vetra AnalyticsPublished in top-tier marketing journals
» Marketing Science» Journal of Marketing Research» Journal of Business Research» Journal of Retailing» Journal of Interactive Marketing
©2009 Third Door Media, Inc.
EVOLUTION OF ONLINE ADVERTISING
Number of Clicks
Online Conversions
OfflineConversions
AttributionManagement
PortfolioManagement
©2009 Third Door Media, Inc.
Last Click
Pur
chas
e P
ath™
Even Attribution
Exclusions
Rules Based Attribution
Mathematical Attribution Models
ATTRIBUTION MANAGEMENT HIERARCHY
©2009 Third Door Media, Inc.
THE CONSUMER BUYING CYCLE
©2009 Third Door Media, Inc.
THE CONSUMER BUYING CYCLE
©2009 Third Door Media, Inc.
THE CONSUMER BUYING CYCLE
©2009 Third Door Media, Inc.
Poll Question 1
• Currently, how are you attributing conversion credit to your various ad sources?
1. Last click2. Other attribution method
©2009 Third Door Media, Inc.
Recap of Modeling Framework From “Measuring the Immeasurable”
www.AttributionManagement.com
©2009 Third Door Media, Inc.
CONSUMER DECISIONS
©2009 Third Door Media, Inc.
DECISION INFLUENCER
What we know
Our Communications┼Paid Search┼Banner Ads┼e-mail┼Onsite Promotions┼Comparison Shopping┼Affiliate ad
Consumer Search┼Organic search┼Site visits to us
What we don’t know yet
┼Competitor Communications┼Consumer search
┼ Site visits to competitors┼ Product trials┼ …….
┼Other sources┼ Social Media┼ Word of mouth┼ Opinion sites┼ Expert opinions┼ Traditional Mass Media
Uncertainty
©2009 Third Door Media, Inc.
MODELING CONSUMER DECISIONS
┼ Build a mathematical model to predict consumer decisions┼ Using data on influencers that we are able to track and measure
┼ Representing data on influencers that we can’t yet track and measure - our
uncertainty - through a statistical distribution
┼ Calibrate the model on observed consumer decisions┼ Purchase - yes/no
┼ Purchase size - dollar volume, # of units
┼ Repeat purchases
┼ Word of mouth
┼ Etc.
┼ Test the model’s quality by comparing predicted and actual behavior
©2009 Third Door Media, Inc.
Consumer’s Decision = f (Our Communications, Consumer Search, Competitor Communications, Other Sources)
= f ([Paid Search, Banner Ads, e-mail, Onsite Promotions, Comparison Shopping, Affiliate ads], [Organic search, Site visits to us], [uncertainty])
CONSUMER DECISION MODEL
©2009 Third Door Media, Inc.
MEASURING THE EFFECTS OF KNOWN FACTORS?
We assume that each of the known influencers
has an influence potential
©2009 Third Door Media, Inc.
MATHEMATICAL MODEL FOR CONSUMER’S DECISION
* The β’s are the attribution weights
©2009 Third Door Media, Inc.
We calibrate the model on data from the ClearSaleing platform
The data includes but is not limited to:
Purchase Path™ data
Record of consumer’s decisions• Purchase/non-purchase
• Product(s) purchased• Amount spent• Repeat visits and purchases
GETTING THE ATTRIBUTIONS
©2009 Third Door Media, Inc.
Calibrate the model on the ClearSaleing data
• Find the values of β’s which will help us predict consumer decisions as accurately as possible
Model is calibrated using:
• Maximum Likelihood • Bayesian methods
The β’s are the attribution weights!
GETTING THE ATTRIBUTIONS
©2009 Third Door Media, Inc.
MODELING THE INFLUENCE POTENTIAL
Influence potential of an influencer = f (# of exposures,
when each of the exposures occurred,
decay rate of the effect of exposures)
©2009 Third Door Media, Inc.
Poll Question 2
• What challenges have you run into when trying to build an attribution model?
1. Our technology cannot track beyond the last ad clicked2. We cannot build a sound mathematical model3. We cannot incorporate offline, social media, and word of mouth advertising4. All of the above5. We haven’t tried to build an attribution model
©2009 Third Door Media, Inc.
PROGRESS SINCE LAST WEBINAR
1. Selection of businesses for the first
round of model testing
2. Identification of unique influencers
3. Set up the data for calibrating and
testing the model
4. Calibrate and test multiple versions of
the model
©2009 Third Door Media, Inc.
SELECTING BUSINESSES
Selecting businesses that:
• Have a high level of ad spend
• Wide array of advertising sources (paid search, email, banner, etc)
• We have a least 6 month of dataWe have 2+ years of data in some cases
• Seasonal variations
©2009 Third Door Media, Inc.
SELECTED BUSINESS VERTICALS
We will be developing and testing the model on nine businesses in the following verticals
• Retail – web only
• Retail – multi-channel
• Insurance
• Financial Services
©2009 Third Door Media, Inc.
Identification of unique influencers
PROGRESS SINCE LAST WEBINAR
©2009 Third Door Media, Inc.
INFLUENCER CATEGORIES
We organized the influencers into the following categories:
• Direct
• Organic Referrers (e.g., Google)
• Paid Search
• Comparison Shopping
• Display advertising
• Affiliate
• Social Media
• Video
©2009 Third Door Media, Inc.
UNIQUE INFLUENCERS
Each category was further sub-categorized into a number of unique influencers:
• Direct - 1
• Organic Referrers – 11 (Google, Yahoo, MSN, etc)
• Paid Search Engine – 11 (Ex: Brand vs. Non-Brand)
• Comparison Shopping – 3 (Ex: Model Number vs. Product Name)
• e-mail - 3 (Ex: Direct Response vs. Brand)
• Display advertising - 4
• Affiliate - 2
• Social Media - 1
• Video - 1
©2009 Third Door Media, Inc.
OVERALL
We have:
• 9 categories of influencers• 37 types of unique influencers across the nine
categories
Our model develops attributions for these 37 unique influencers across the nine businesses.
©2009 Third Door Media, Inc.
PROGRESS SINCE LAST WEBINAR
Set up the data for calibrating and testing the model
©2009 Third Door Media, Inc.
PROGRESS SINCE LAST WEBINAR
135 predictors
64,653 Purchase Paths™• 11,353 paths resulting in a purchase• 53,300 abandoned paths that did not end in a purchase
o A path was defined as abandoned based on some proprietary criteria
Model can explain abandonment too
Another frontier: Attributions for abandonment
©2009 Third Door Media, Inc.
To date, we have calibrated over 70 versions of the model
We plan to calibrate and test the model at least 500 more times in various forms before firming up our
conclusions• 45,000 models run across the nine data sets
Testing
• Do the estimated attributions make intuitive sense?• Is the model able to predict consumer behavior?
o Can it predict purchases?o Can it predict non-purchases?
RESULTS TO DATE
©2009 Third Door Media, Inc.
Intuitive assessment of attributions
• Findings: not yet firmed up since we have 37 unique influencers to assess across hundreds of model runs
Predictive Testing
• 85% or more of the purchases being predicted correctly• 95% or more of the non-purchases predicted correctly• Lift charts for calibration and prediction samples
o Top decile indices average between 450 and 500
RESULTS TO DATE
©2009 Third Door Media, Inc.
Consumer ratings and reviews
Social networks
Blogging
Social commerce
Instant messaging
You Tube
RSS and multiple feeds
TYPES OF NON-CLICK/ PASSIVE INFLUENCERS
©2009 Third Door Media, Inc.
Why do we need to?
• Benefits of including non-click influencers in attribution models
• Risks of not-including non-click influencers in attribution models
Challenges
• How do we include them, if they don’t click and we don’t track??
ATTRIBUTIONS FOR NON-CLICK INFLUENCERS
©2009 Third Door Media, Inc.
Use survey data on the use of non-click influencers
• Statistically infer the likelihood of the use of each type of passive influencers by different demographic, lifestyle and psychographic segments
• Vetra is currently working on this approacho Vetra Passive Survey™
• Vetra Passive Survey™ can be used to include passive influencers in attribution models
VETRA PASSIVE SURVEY™
©2009 Third Door Media, Inc.
Monitor sources of passive influence • YouTube• Facebook• Myspace• epinions.com
Statistically infer the proportions of buyers who engage in discussions and exchanges regarding products
• Vetra has completed preliminary work on a model for this inference
Vetra Passive Proportions™ can also be used to include passive influencers in attribution models
VETRA PASSIVE PROPORTIONS™
©2009 Third Door Media, Inc.
Vetra and ClearSaleing will:
• Continue to test a number of attribution models and influencers
• Analyze the performance of models across different verticals
• Identify the best attribution models for different verticals
• Expand attribution models to include passive influencers usingo Vetra Passive Survey™o Vetra Passive Proportions™
ONGOING RESEARCH ON ATTRIBUTION AND NEXT STEPS
©2009 Third Door Media, Inc.
Poll Question 3
• What is your timeframe in switching from a last click model to an advanced attribution model?
1. Less than 6 months2. Within a year3. Within 2 years4. More than 2 years5. No timeframe
©2009 Third Door Media, Inc.
©2009 Third Door Media, Inc.
Adam Goldberg- www.attributionmanagement.com [email protected]
www.ClearSaleing.com
QUESTIONS?
Upcoming SMN webcast:May 12: Ask iProspect: Strategies &Tactics for World-Class SEO
May 14: Search Marketing for Small Business: The Basics for Online Success