So today I want to discuss 3 things
How good are we at ranking factors?
How to run tests & pitfalls to avoid
Some of the tests we’ve been running
Search: future of tv advertising
http://contentmarketinginstitute.com/2016/04/television-advertising-native/
Search: future of tv advertising
DA PA LRDs Rank
http://dis.tl/rankq-1a
91 47 2 ?
http://dis.tl/rankq-1b
86 57 9 ?
Search: future of tv advertising
DA PA LRDs Rank
http://dis.tl/rankq-1a
91 47 2 3
http://dis.tl/rankq-1b
86 57 9 8
Search: kettlebell technique
DA PA LRDs Rank
http://dis.tl/rankq-2a
41 42 12 ?
http://dis.tl/rankq-2b
76 44 20 ?
Search: kettlebell technique
DA PA LRDs Rank
http://dis.tl/rankq-2a
41 42 12 4
http://dis.tl/rankq-2b
76 44 20 6
http://www.macworld.co.uk/how-to/iphone/how-to-back-up-iphone-ipad-backup-3595444/
Search: do I need to backup my iphone
http://www.idownloadblog.com/2013/08/28/not-enough-icloud-storage-iphone-cannot-be-backed-up/
Search: do I need to backup my iphone
Search: do I need to backup my iphone
DA PA LRDs Rank
http://dis.tl/rankq-3a
77 59 16 ?
http://dis.tl/rankq-3b
67 34 3 ?
Search: do I need to backup my iphone
DA PA LRDs Rank
http://dis.tl/rankq-3a
77 59 16 7
http://dis.tl/rankq-3b
67 34 3 5
% of UK search result pairs predicted correctly
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO > 3 yrs SEO
% of UK search result pairs predicted correctly
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip
% of UK search result pairs predicted correctly
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip
% of UK search result pairs predicted correctly
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip DeepRank
IMEC Labs
IMEC Labs
Run experiments against the SERPs to understand aspects of Google’s algorithm.
3 Steps to DIY SEO Split-Tests
1. Create two buckets of pages.
2. Make a change to all pages in one bucket.
3. Analyse which bucket performs better.
1. Create two buckets of pages.
A good DIY approach is using GA segments. Easy to make (see link).
Suggestion: Create Segments by category (e.g. blog tag, product category).
2. Make a change to all pages in one bucket.
Control Variant
This will be your variant bucket.
Utilise your CMS where possible. Ask Dev for scalable approach.
3. Analyse which bucket performs better.
A naive approach can just compare absolute traffic, if you had a close match before.
But needs a big uplift and hard to create such Segments.
Better approach is using Google's Causal Impact library — read this great Lunametrics post.
Variant
Control
ConcertHotels.com: Test Setup
~20,000 location category pages
Pages
Title: <<Location>> Hotels, NY | ConcertHotels.com H1: <<Location>> Hotels
Before (Control)
Title: Hotels near <<Location>>, NY | ConcertHotels.com H1: Hotels near <<Location>>
After (Variant)
ConcertHotels.com: Results
2.5 weeks to get to significance. Gradual improvement in organic performance leading to steady amount of higher traffic.
Results
SmokyMountains.com: Test Setup
~100 lodging pages
Pages
schema.org markup: @type “WebSite”. Generic on all pages.
Before (Control)
schema.org markup: @type “LodgingBusiness”. Customised to each page.
After (Variant)
SmokyMountains.com: Results
Fewer test pages than previous test, so it is less smooth but is detected much quicker. Traffic uplift here is estimated to be ~5%.
Results
iCanvas.com: Test Setup
~3200 artist category pages
Pages
meta description: Shop our selection of canvas prints by Banksy, each hand-stretched over museum-quality bars and printed with brilliant, fade-resistant inks. Free shipping and returns. internal links: ~50 self referential links
Before (Control)
meta description: Banksy Prints on Canvas, including There Is Always Hope Balloon Girl, Life Is Beautiful and others. Free shipping and returns. internal links: <removed>
After (Variant)
iCanvas.com: Results
2 weeks to get significance. Number of pages is between the previous tests: a few days to be noticed but quite smooth.
Results
You can’t assume traffic equality between “buckets” of pages
This is why we build a counterfactual comparison using control pages. Use Google’s Causal Impact library to do it yourself.
Pay attention to: Amount of traffic
& number of pages
These two factors will determine how quickly you can test and what size uplifts you can detect.
New pages that appear during tests
The simplest approach is to just ignore all new pages that didn’t exist before the test started.
Different pages can have different seasonality
For example, “roses” pages on valentine’s day. You need to cut outliers.
You could damage conversions, so pay attention to those metrics too.
Tag people in your analytics depending if they arrive on Control or Variant
www.distilledodn.com
DistilledODN allows you to test exactly which changes to your website will result in an uplift in traffic from search engines.
Our new SEO A/B Testing platform is available.
@TomAnthonySEO
Image Credits
• Buckets - mamarazzi
• Rose - alicelingching
• Girders - JFB119
• Rubiks - Wallpaperzone