#SMX #11B @bgTheory

Brad Geddes, Founder, AdAlysis

The Mad Scientists: Quality Score Under the Microscope

#SMX #11B @bgTheory

What Is Quality Score

#SMX #11B @bgTheory

Ad Rank

BidEstimate:50%

QualityScoreEstimate:>40%

AdExtensionsEstimate:<10%

#SMX #11B @bgTheory

The Breakdowns for Visible QS

LandingPages39%

ExpectedCTR39%

AdRelevance22%

#SMX #11B @bgTheory

Visible Quality Score Formula

LandingPageExperience

AdRelevance CTR

Aboveaverage 3.5 2 3.5Average 1.75 1 1.75Belowaverage 0 0 0

Quality Score = 1 + Landing Page Experience weight + Ad Relevance weight + CTR weight

#SMX #11B @bgTheory

Expected CTR

#SMX #11B @bgTheory

BrandTerm:•  6%belowaverage•  15%average•  46%Aboveaverage

Why You Can’t Project What is a ‘Good CTR’

#SMX #11B @bgTheory

Non-brandterm•  3.5%belowaverage•  8.5%average•  14.5%aboveaverage

Why You Can’t Project What is a ‘Good CTR’

#SMX #11B @bgTheory

Ad Relevance: Inverse Document Frequency

#SMX #11B @bgTheory

Lodging Heatmap Ad Description

% O

ff

Airf

are

Airl

ine

s

Airp

ort

Ca

ll to

A

ctio

n

Ch

ea

p /

A

ffo

rda

bl

e

De

als

/ D

isco

un

ts

De

stin

atio

n

Lod

gin

g

Off

icia

l Si

te

On

line

Pla

n

Pric

e /

Pr

icin

g

Serv

ice

s

Sup

erla

tive

[A

dje

ctiv

e /

A

dve

rbs]

Ad

Titl

e

% Off

Agency

Airfare

Airlines

Airport

Brands

Call to Action

Cheap / Affordable

Deals / Discounts

Destination

DKI [Dynamic Keyword Insertion

Lodging

Official Site

Online

Param Insertion

Plan

Price / Pricing

Sale

Services

Superlative [Adjective /

Adverbs]

Source: Microsoft Internal Data. Total Impressions Generated = 298 Million, Total Ads Analyzed = 149k,

Analysis Period = 12/1/2014 – 11/30/2015. Note: The categories represent groups of similar

keywords.

Great Good

Insufficient data

Poor

#SMX #11B @bgTheory

Reading a Heatmap Ad Description

% O

ff

Airf

are

Airl

ine

s

Airp

ort

Ca

ll to

A

ctio

n

Ch

ea

p /

A

ffo

rda

ble

D

ea

ls /

Disc

ou

nt

s De

stin

ati

on

Ad

Titl

e

% Off

Airfare

Airlines

Airport

Call to Action

Cheap / Affordable

Deals / Discounts

Lodging

Official Site

Online

Search for Any Flight Online Cheap Flights to SJC

Search for Your Favorite Hotel 10% off Popular Destinations

Some offers are so common, no one cares

#SMX #11B @bgTheory

Below Average Ad Relevance

Usuallyanindicationthatyouneed:

• Betterorganization• DifferentAds

#SMX #11B @bgTheory

Sometimes You Don’t Want to Be Completely Relevant

0.00%

0.50%

1.00%

1.50%

2.00%

2.50%

3.00%

3.50%

4.00%

3 4 5 6 7 8 9

QualityScore

ConversionRatesbyQualityScore

#SMX #11B @bgTheory

Mixing B2B & B2C Ads

#SMX #11B @bgTheory

Non-Linear CTRs are Almost Always Landing Page Issues

#SMX #11B @bgTheory

Landing Pages Appear to Have Little CPC Weight

-10

-5

0

5

10

15

AverageCPCDifferentialBetweenBelowAverage&AboveAverageLandingPages

CPCDifferential

Linear(CPCDifferential)

#SMX #11B @bgTheory

Todeterminethisinfo,youwouldneedAnauctionwith:•  Thesamequery•  Thesameuser•  Thesameuser’spagehistory•  Thesamecompetitors•  Thesamebids•  Thesameuserlocation•  Thesamedevicetype•  Thesamelandingpage•  Thesamead•  Theexactsameadextensions(foreveryone)•  Thesameadposition•  Yetmorenon-variances•  RelyonGoogletreatingtheauctionexactlythesameeverytime•  Withtheonlyvariationbeingthequalityscoreofthewordinyouraccount

whileeveryoneelse’sremainedexactlythesame

QS Has Little CPC Weight For Ads Outside of Position 1

#SMX #11B @bgTheory

The Reality: As QS Goes Up; so does position. This means you have to beat a new competitor CPC fluctuates; but rarely just goes down

CompeBtor1

YourInfo

CompeBtor2

CompeBtor3

CompeBtor4

$1

$1

$1.25

$1.25

$0.50

7

X

6

X

4

X

10X

7X

X

X

X

X

X

0

6

4

0

5

WeighBng Bid0.5 QS0.4 AdExt0.1

= 5.6

= 6

= 5

= 10

= 3.3

AdRank

#SMX #11B @bgTheory

LandingPagesseemtoaffectyourabilitytoentertheauctionmorethanyourCPCs.Forthisdata:OnlyusedConversionOptimizerCampaignssomanualbidswouldn'tbeavariable.

Visible vs Auction QS

VisibleQS:

WhatyouseeinAdWordsasaguide

AuctionQS:

Whatissubmittedrealtimeintotheauction

#SMX #11B @bgTheory

Does Quality Score Affect Impression Share?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10

Impression

Sha

re

QualityScore

ImpressionSharebyQS

#SMX #11B @bgTheory

Do Below Average Landing Pages Affect IS?

0%

10%

20%

30%

40%

50%

60%

70%

80%

1 2 3 4 5 6

ImpressionSharebyLandingPageQuality

BelowAverageLP Average/AboveLP

#SMX #11B @bgTheory

Analysis for you: CPC by QS (What’s Your Discount?) Some companies (especially at position 1) do have discounts

#SMX #11B @bgTheory

Analysis for you: CPA vs QS (What’s the Financial Impact of Increasing?)

#SMX #11B @bgTheory

Finding Ad Groups That Need Assistance

https://youtu.be/Df7Lo7CMQiI

#SMX #11B @bgTheory

How To Improve

#SMX #11B @bgTheory

•  Adrankis:MaxBid,QualityScore,&AdExtensions•  QualityScoreismadeupof3factors

•  ExpectedCTRhasastrongcorrelationbetweenaverageCPCsandfinancialimpactinthesameposition.

•  AdRelevancehasapartialcorrelationbetweenitsfactors&CPCs•  It’sagoodindicationyouneedbetterorganizationordifferentads

•  LandingPageshaveaweakcorrelationbetweentheirfactors&CPCs•  Theyhaveacorrelationbetweentheirfactorsandimpressionshare

•  Asqualityscoregoesup,sodoesposition,causingyoutocompetewithdifferentadvertisersandthusrarelyseeadirectinfluenceonCPCs.

•  AsQualityScoreincreases,sodoesimpressionsharehencedirectcorrelationsbetweenallfactorsandCPCsareimpossiblesincethecompetitiondiffers.

•  IncreasingQSisaboutorganizationandthenadtesting

Wrap-Up

#SMX #11B @bgTheory

LEARN MORE: UPCOMING @SMX EVENTS

THANK YOU! SEE YOU AT THE NEXT #SMX

Let’s Connect @bgTheory in/ewhisper Powerful Ad Testing Made Simple

Ad Testing Software