Digital travel summit channel attribution 2014 04-01

transcript

C O N F I D E N T I A L

Marketing Channel Attribution Modeling To Improve Conversion Rates

Jonathan Isernhagen

Digital Travel Summit Day 1

Digital Travel Summit

4/1/2014

Agenda

1) Marketers’ mission

2) Calculating channel ROIa) spendb) VCM

3) Attributiona) algorithmic

i. data assemblyii. types of models

b) arbitrary

4) Options for action

jonathan.isernhagen@travelocity.com @jon_isernhagen www.linkedin.com/in/isernhagen

CEO’s/Our duty to the (publically-traded) company

• Maximize brand awareness/sentiment?

• Maximize client loyalty?

• Maximize co-worker satisfaction?

• Drive traffic to the site?

• Maximize shopper movement down the funnel?

• Maximize transaction volumes?

• Maximize shareholder value?

• Maximize individual channel ROI?

• “Give back" to the community through charitable actions?

Example: Apple, Inc.

“Apple's Board of Directors oversees the Chief Executive Officer and other senior management in the competent and ethical operation of Apple on a day-to-day basis and assures that the long-term interests of shareholders are being served.

Source: http://investor.apple.com/governance.cfm

Increase spending until $1 out brings $1 back…

….spend-spend-spend-spend-spend-stop

…which is the point at which incremental ROI = 0%

ROI = (VCM – Spend) Spend

Abbreviation Term Definition

ROI Return on Investment Indicator of investment profitability. Positive = good.

VCM Variable Contribution Margin The amount of profit driven by a given transaction.

Spend Channel Spend The amount spent driving traffic to the site during the period in question

Calculated over a specified time period of investment and return.

Agenda

b) arbitrary

Which spend do you include?

Marketing ≈ Farming

1)Seed = impressions

2)Transactions = fruit

…butmarketers plant

& harvest constantly.

spendspendspendspend

How changes in spend can mess up ROI

- 10 -

profit

impact impact

impact

Jan Feb Mar

Variable Contribution Margin (“VCM”)

- 11 -

1) What is the value of each transaction?

2) Which channels deserve credit for which transactions?

= Transaction revenue - variable non-marketing expenses:

• Revenue: – Supplier Commissions; – GDS incentives; – Overrides (lumpy: average/booking must be assumed)– Media (not transaction-driven, but has to be modeled in

somewhere)– Attached bookings / Lifetime value: try to gauge value without

double-counting

• Expenses: – Website hosting/capacity costs– Data processing expenses– Other expenses which vary by transaction or site activity volume

- 12 -jonathan.isernhagen@travelocity.com @jon_isernhagen www.linkedin.com/in/isernhagen

Which marketing treatment(s) triggered the purchase?

"Half the money I spend on advertising is wasted; the trouble is, I don't know which half.“

-John WanamakerFather of Modern

Advertising

“Hard to Know” ≠ “Random”

We may not find the correct answer, but the answer does exist.

Google Analytics attribution options

Google offers several attribution mechanisms out of the box

Choosing the wrong model will have consequences.

How badly do you want to know?

Raylan: You'll pay to find that out.Boyd: What are you packing?

Choosing your attribution strategy

Do you want/need true attribution*?

Do you havedata guys?

Do they havebandwidth for this?

Use your sitemetric solution’s

attribution

YesHire a vendor

Can you accessyour data?

*Do you have:1) Large enough budget?2) Multiple channels?3) Belief in ROI “knowability?”

Agenda

b) arbitrary

• Sends• Opens

Collecting the necessary data

Analysis space

• Clicks• Visitors

• Transactions• VCM• Media impressions

Site metrics tool

Back office system

• Channel-specific phone #s

Email service provider

Call Center IVR

• Impressions• Clicks• Spend

Display ad server

e.g. SAS, Revolution Analytics,SPSS, Teradata Warehouse Miner

SEM bid management tool

• GRPs• Spend

Television plan

Meta search feeds

Manual spend entry table

Connecting the necessary data

Transactions Profiles Customers

Sessions

∞ 1∞ 1Clicks:TransactionsClicks ∞ 11 ∞

∞ 1∞ 1

Opens∞1

TV planImpressions

Ad server

Impressions

Back officeSite monitoring tool CRM system

Profile deduplication is crucial

Profiles Customers∞ 1

CRM system

1) App registration information tied to desktop profile

2) Logged-in customer information on multiple devices

3) E-mails sent to same address opened on multiple devices

4) Third-party services with network visibility

Impressions are also important

This example overcredits impressions (excludes other channels) but it gives some idea of the sub-surface portion of the

impressions iceberg.

Forrester 2012Q2 survey of attribution vendors

Algorithmic attribution per Visual IQ:Top Down (MMO) & Bottom Up (Fractional Attribution)

SUMMARY LEVEL DATA USER LEVEL DATA

CHANNELS Offline + digital cross channel Digital media channels

ROLE Strategic: Optimize spend across channelsTactical: Generates granular

media recommendations

OUTPUT Cross channel insight Full fractional attribution

CAPTURES Seasonality and external factorsInterplay between digital touch

point and channels

PREDICTABLE GRANULARITY

Conversions at aggregate level Propensity to convert at user level

“Given all the history we know, how likely is this shopper to convert soon?”

Datasong answers with a 2-stage model:

-Are Brand shoppers (BSEM, BSEO, DTI) likely to book? Yes

-Did your last TV spot cause their brand loyalty? Maybe

Algorithmic attribution per DataSong: Survival Modeling

Model 2Accuracy 81%

Model 1Accuracy 68%

0 0.10.20.30.40.50.60.70.80.9 10

NonConverters Converters

0 0.10.20.30.40.50.60.70.80.9 10

Algorithmic attribution per DataSong: Survival Modeling

1) Axes show the time since last channel exposure2) Dots represent converters vs. nonConverters3) Orange line represents the relationship between 2

variables, e.g.:a) time since last email andb) time since last affiliate visit

4) Responders: everyone above the line. We count the folks above the line and see what our accuracy is

5) Attribution: once satisfied with a model(s), we’d:a) take a given order, b) see the time since last email and affiliate and based on

the timing, and;c) where we are on the orange line, so;d) we have a means to allocate which was more causal.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

Survival Modeling: DIY

Recommended by one of our PhD statisticians.

“SAS usage is not necessary.”

C O N F I D E N T I A L- 28 -

Algorithmic attribution per Google: Interaction Method (“Shapley value”)

Brand SEM Email

Conversion

Brand SEM Email

Conversion

Every combination (“coalition”) of clicks is tested.

“How important is each player to the overall cooperation?”

http://en.wikipedia.org/wiki/Shapley_value

Display

Running the process

1) For each site visitora) Assemble visit historyb) Create variables to represent:

i. Channel impressionsii. Channel clicksiii.Past purchases

2) Regress or use machine learning algorithma) Ascertain which channel touches predict bookingb) Give VCM credit to causal channels

3) Calculate ROIa) Use each channel’s VCM and spendb) Where ROI is positive, spend upc) Where ROI is negative, cut spend or change tactics

Output

Channel* Desktop Tablet Mobile

Brand 20%

Brand SEM 62% 51% 38%

Display -5% -12% -7%

Display - Retargeting 26% 25% 29%

Email 250%

Meta search 18% 22% 10%

Non-brand SEM -30% -18% -40%

SEO 500% 520% 390%

Social media -5% -15% 15%

- 30 -

*of impression/click, not necessarily of consumer conversion

Agenda

b) arbitrary

“Round up all the clicks!”

• Offline bookings• Multiple profile activity• Impressions (Display or Television/Radio/etc)• Cleared-cookie clicks

- 32 -

Leaves out:

55% of shoppers clear cookies every 3 months

Cookie clearing is a serious problem causing the true impact of online channels to be massively understated.

“The Consumer Side of Analytics” - Forrester/Jupiter Research 2009

Bracketing

1) Assume for the sake of argument that one of the site tool’s attribution schemes reflects reality, but you don’t know which one.

2) Find (or create) the scheme that is most and least generous to digital channels. a) Last click: usually the least generous to digital channelsb) 3-tier: a scheme we created to favor NBSEM and Meta/TravelSearch

3) Assume that the true attribution (& ROI) lies between these extremes

4) Adjust spend and see which scheme better predicts reality.

Tactical decision-making

Channel Easy decisions Hard decisions Tactics

Advertising heavily in target-rich geographies before/during periods of high demand.

• national vs. regional• constant vs. bursts• more air time vs fresh

creative

• A/B test across “identical” regions, observe rise in brand transactions.

• Get Comscore to measure brand awareness/intent after broadcast.

SEMSpending to capacity on Brand SEM

Whether to spend up on marginal NBSEM keywords

A/B test whether ↓ BSEM spend is offset by ↑ BSEO volume

Emailing up to 1x/week, hitting each site visitor with the message he or she is most likely to convert from.sending messages with exceptional deals.

• Whether to upgrade to a rules-base ESP system.

• Whether to saturation-bomb your e-mail list.

• Whether to push a loss-leading promo code.

Split the list, quantify lifetime value, A/B test:• Unsubscribe impact

of saturation bombing• Uplift impact of offer

personalization.• Long-term behavior

of code recipients

Take-away lessons

- 36 -

1) Be intentional about your attribution decisions, and creative about how to test your assumptions.

2) For top competitors in each vertical, the question is no longer “should we do algorithmic attribution?” but rather “How can we operationalize our attribution model?”

3) If your data aren’t yet in shape to support attribution & other ad hoc analyses, get that project underway.

Attribution options - vendors

Provider MethodCost

Timing Comments1x monthly

Adobe Attribution model

Varies - 4-10 wksConsulting arm willing to address attribution questions

as one-off modeling projects.

Adometry Econometric model

Low High 12 wksForrester-acclaimed industry co-leader.

Highly display-focused.

Datasong

Attribution model w/Customer

scoring

Comprehensive model which pulls in both online and offline spend impacts.

MedPer-customer demand curve to predict the expected

return on every campaign

Google Analytics

Interactions model

- - nowHas delivered digital-only channel interactions model

to beta clients.

Gauge LLC Simple regression

High - 10 wksGeneral consulting analysts, not on the efficient

frontier for this deliverable.

Kenshoo Unknown algorithm

- Med 4 wksPrimarily targeted towards SEM spend, but can be

expanded to comprise other digital channels

Visual IQEconometric

and Attribution models

High - 4-8 wksEconometric model. Offers cross-channel allocation

guidance, including offline spend.

Med-High

- 3-6 wksAttribution model. Offers within-channel campaign-

level spend guidance. No offline.

Funnelism doesn’t drive investment decisions

Facebook visitor tracking

Visitor action Volume Trackable Useful

Ad link click Small Yes No

Digital travel summit channel attribution 2014 04-01

Internet