Date post: | 16-Apr-2017 |
Category: |
Marketing |
Upload: | frederik-hyldig |
View: | 251 times |
Download: | 2 times |
Statistical Concepts Every PPC Specialist Should Know
Frederik HyldigHead of PPC | s360 | @frederikhyldig
Who am I?Frederik Hyldig
Frederik Hyldig
Head of PPC at s360 - one of the leading search agencies in Denmark
Works with some of the largest advertisers in Denmark and Scandinavia
Writes for PPC Hero, Wordstream, Moz, SEMrush, 3Q Digital, and the Certified Knowledge blog
SEO & PPC inall the Nordic
countries
www.s360.dk
Some of the companies we work withFrederik Hyldig
In this presentation:- Statistics: a sexy skill- The Pareto principle- The law of large numbers- Relative and absolute numbers- Simpson’s paradox- How (not) to lie with statistics
Frederik Hyldig
A sexy skill
Frederik Hyldig
Frederik Hyldig
At least according to this guy...
Frederik Hyldig
I keep saying the sexy job in the next ten years will be statisticians. People think I’m joking, but who would’ve guessed that computer engineers would’ve been the sexy job of the 1990s? The ability to take data—to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it—that’s going to be a hugely important skill in the next decades [...]
- Hal Varian, Chief Economist at Google
The ability to take data- to be able to understand it- to process it- to extract value from it- to visualize it- to communicate it
Frederik Hyldig
Reflected in the quality of your work
Reflected in the relationships with clients (or bosses)
Frederik Hyldig
Data isn’t going anywhere. Our top skill category last year, statistical analysis and data mining, is still sitting comfortably at #2.
It is the only skill category that is consistently ranked in the top 4 across all of the countries we analyzed.
We still live in an increasingly data-driven world, and businesses are still aggressively hiring experts in data storage, retrieval and analysis.
-The 25 Skills That Can Get You Hired in 2016
https://blog.linkedin.com/2016/01/12/the-25-skills-that-can-get-you-hired-in-2016
Frederik Hyldig
Meaning businesses will be like...
Statistical Concepts Every PPC Specialist Should Know
Frederik Hyldig
The Pareto principle
Frederik Hyldig
Concept 1
The Pareto principle Frederik Hyldig
For many events, roughly 80% of the results come from 20% of the effort.
The Pareto principle Frederik Hyldig
Vilfredo ParetoItalian economist
Also known as the 80/20 rule
The Pareto principle Frederik Hyldig
stickfigureeconomics.blogspot.com/2012/06/understanding-pareto-principle-aka-8020.html
The Pareto principle Frederik Hyldig
stickfigureeconomics.blogspot.com/2012/06/understanding-pareto-principle-aka-8020.html
The Pareto principle Frederik Hyldig
How is this relevant in PPC?
Frederik Hyldig
The Pareto principle Frederik Hyldig
Focus on the few most important campaigns, ad groups, or keywords.
● Improving ads● Restructuring accounts● Negative keyword● Etc.
The Pareto principle Frederik Hyldig
Example:477 campaigns367 conversions
According to the Pareto principle:
293 conversions (80% of the 367 conversions)
should come from
95 campaigns (20% of 477 total campaigns)
Frederik Hyldig
92% of the conversions (338) came from 20% of the campaigns
The Pareto principle
The Pareto principleFrederik Hyldig
Concept:
Take away:A very large part of the results will often come from a small part of the activities.
Prioritize your work on the few keywords/campaigns with the largest impact.
For many events, roughly 80% of the results come from 20% of the effort.
The law of large numbers
Frederik Hyldig
Concept 2
The law of large numbers Frederik Hyldig
If you repeat a random experiment often enough, the average of the outcomes will converge towards the expected value.
The law of large numbers Frederik Hyldig
Coin tossing example: 10 tosses
The law of large numbers Frederik Hyldig
Coin tossing example: 1,000 tosses
The law of large numbers Frederik Hyldig
The law of large numbers Frederik Hyldig
>
Frederik Hyldig
Frederik Hyldig
If the statisticians who reported to the Gates Foundation had asked about the characteristics of the worst schools, they would have found that bad schools also tend to be smaller than average. The truth is that small schools are not better on average; they are simply more variable.
- Daniel Kahneman
How is this relevant in PPC?
Frederik Hyldig
The law of large numbers Frederik Hyldig
Sorted by highest conversion rate
These are all very small cities
The law of large numbers Frederik Hyldig
Sorted by lowest conversion rate
These are also small cities
The law of large numbers Frederik Hyldig
Seems like we have our most valuable hours at night!
Frederik Hyldig
Frederik Hyldig
The law of large numbers Frederik Hyldig
No.... It’s just more variable due to low volume
Frederik Hyldig
ROAS
#1 #2 #3 #4 #5 #6 #7 #8
Look at the return in country #1! Let’s allocate more of our spend to this account!
Countries
Frederik Hyldig
Ad spend
#1 #2 #3 #4 #5 #6 #7 #8
Not so fast! The volume is much lower than the other countries and therefore we should expect more variance.
Countries
The law of large numbers
Frederik Hyldig
Concept:
Take away:When randomness is involved (i.e: always) you should expect more
variance from low volume.
Put more trust in numbers when volume is higher.
If you repeat a random experiment often enough, the average of the outcomes will converge towards the expected value.
Relative and absolute numbers
Frederik Hyldig
Concept 3
Relative and absolute numbers
A new drug reduces the risk of getting a dangerous disease by 25%
Frederik Hyldig
Relative and absolute numbers Frederik Hyldig
No drug: 20 in 1,000 get disease (2.0% risk)
With drug: 15 in 1,000 get disease (1.5% risk)
25 percent fewer people get the disease
Actual risk of getting disease is only 0.5 percentage points lower
What you will read in the newspapers
What you really want to know when deciding
How is this relevant in PPC?
Frederik Hyldig
Relative and absolute numbers Frederik Hyldig
Relative and absolute numbers Frederik Hyldig
Relative and absolute numbers
Frederik Hyldig
Concept:
Take away:Percentage must not be confused with percentage points.
Percentages can be misleading when the absolute numbers are omitted.
Simpson's paradox
Frederik Hyldig
Concept 4
Simpson’s paradox Frederik Hyldig
A trend appears in different groups of data but disappears or reverses when these groups are combined.
Simpson's paradox Frederik Hyldig
1973 admission figures for the University of California, Berkeley
Simpson's paradox Frederik Hyldig
1973 admission figures for the University of California, Berkeley - by departments
How is this relevant in PPC?
Frederik Hyldig
Simpson’s paradox Frederik Hyldig
Variant 1: Low prices on XX Variant 2: Save 20% on XX
Ad group 1
Ad group 2
Ad group 3
Ad group 4
Simpson’s paradox Frederik Hyldig
Ad 2 is winning in every ad group
But when aggregating the numbers, ad 2 is actually worse!
Simpson’s paradox Frederik Hyldig
Ad 1 volume is concentrated in the two ad groups with generally higher CTRs.
Simpson’s paradox Frederik Hyldig
Simpson’s paradox Frederik Hyldig
Even though ads are shown evenly, a group with high volume can still cause Simpson’s paradox
Simpson’s paradox
Frederik Hyldig
Concept:
Take away:Beware when aggregating data.
Make sure to run properly randomized trials (rotating ads evenly).
How to lie with statistics
Frederik Hyldig
Concept 5
not
How (not) to lie with statistics Frederik Hyldig
Graph is from “How to Lie With Statistics” by Darrell Huff
Manipulating graphs
How (not) to lie with statistics Frederik Hyldig
Graph is from “How to Lie With Statistics” by Darrell Huff
Manipulating graphs
How (not) to lie with statistics Frederik Hyldig
Graph is from “How to Lie With Statistics” by Darrell Huff
Manipulating graphs
How (not) to lie with statistics Frederik Hyldig
Graph is from “How to Lie With Statistics” by Darrell Huff
Manipulating graphs
Same numbers. Different impressions.
How (not) to lie with statistics Frederik Hyldig
Manipulating graphs
Pay attention to the vertical axis in AdWords - especially when looking at shorter intervals.
How (not) to lie with statistics Frederik Hyldig
Manipulating graphs
A graph should start a zero.
How (not) to lie with statistics
An increase by 20% followed by a decrease by 20% does not bring you back to the starting value.
● 100 + 20% is 120
● 20% of 120 is 24
● 120 - 20% is 96
Frederik Hyldig
Shifting baseline
How (not) to lie with statistics Frederik Hyldig
Shifting baseline
How not to lie with statistics
Frederik Hyldig
Concept:
Take away:Pay attention to the vertical axis when looking at (or creating) graphs.
Be aware of shifting bases when increasing and decreasing.
Summary- The Pareto principle- The law of large numbers- Relative and absolute numbers- Simpson’s paradox- How (not) to lie with statistics
Frederik Hyldig
Thanks for listening!
Frederik HyldigHead of PPCE-mail: [email protected]: @frederikhyldig
s360SEO & PPC acrossall the nordic countriesWeb: www.s360.dk