The Only Two Possible Outcomes of Analysis

Opportunity or OptimizeThe Only Two Possible Outcomes of Analysis

By Dean LevittFounder, Teacup Analytics

Who Am I?A curriculum vitae of sorts

• Born and raised on a farm in South Africa, near Johannesburg

• Built Mad Mimi, acq. by GoDaddy in 2014, now GoDaddy Email Marketing

• Previously Chief of Culture for Mad Mimi & Director of Online Support for GoDaddy

• Founder of Teacup Analytics

• I currently live in Tel Aviv, Israel

• I run, surf and travel

What Is Teacup Analytics?Teacup Analytics Simplifies Google Analytics for Small Businesses

• Launched in March 2016

• Teacup Analytics makes Google Analytics data simple

• Library of over 30 analyst-crafted reports

• Innovative grading system that qualitatively analyzes web traffic

• Monitors the impact that actions have on performance

The big question every analyst hears:

“Can you analyze my data and tell me where to improve?”

Why Analysis Actually MattersIt’s all about uncertainty reduction, isn’t it?

• Analysis is the breaking down of a complex topic to gain a better understanding



• The goal of analysis is to reduce uncertainty




• Increase confidence in business decisions

• Inspire new decisions that are backed by data




• The end result of analysis must be action!





• For data to be useful, it must drive decisions that lead to positive outcomes





• For data to be useful, it must drive decisions that lead to positive outcomes

• Without the commitment to acting, analysis is superfluous

“A good metric changes the way you behave”

Ben Yoskovitz, Lean Analytics

Analysis Needs ContextContext is like a lighthouse to your data

• Data should always be understood in context



• Is my analysis encouraging actions that meet current goals?




• There are many types of context:

TeamTask orOutput Department Company




• There are many types of context:

TeamTask orOutput Department Company

Which segments are successfully completing the task?

Is my team’s output better or worse than other teams?

Are other departments facing similar challenges?

Are we performing better or worse than last quarter?

Segment By Intent

Context By SegmentationSegmentation is foundational.

• The foundation for our analysis lies in context



• Context is provided by segmentation




• Segmentation implies intent





• Understanding intent provides actionable direction






• Intent trumps demographics






• Intent trumps demographics

• Intent is the step before conversion!

Ok, let’s get to the good stuff!

Ok, let’s get to the good stuff!(It’s about time)

The Only Possible OutcomesFirst, let’s give credit where it’s due

• Tim Wilson of Analytics Demystified first discussed this concept in a May 2015 blog

post titled Two Reasons For Analysis

• The premise is that analysis is effective:

• when there is a problem

• When there is a potential opportunity

http://analyticsdemystified.com/analysis/two-reasons-for-analysis/

“Indicators tend to direct your attention toward what they are

monitoring. It is like riding a bicycle: you will probably steer it

where you are looking.”

Andy Grove

The Only Possible OutcomesThe two, wait, three possible outcomes of analysis

Results

ActivityLow High

Below

Above


Results

ActivityLow High

Below

Above

CONTEXT


Results

ActivityLow High

Below

Above


Results

ActivityLow High

Below

Above

GrowthOpportunity

OptimizationNeeded

The Do-Nothing Zo

ne


Results

ActivityLow High

Below

Above

Can we increaseactivity?

What is causingpoor results?

The Do-Nothing Zo

ne

Easy Analysis and ActionThe quickest way to find actionable insight

• All analysis boils down to three possible actions



• Action 1: Do nothing!




• Activity is insignificant and results are poor

• Not worth time and effort

• Too hard to make an impact with current resources

• Already at full capacity and performing well




• Action 2: Optimize!





• Activity or volume is significant but conversions suck i.e. low quality

• This is a leaky bucket, with holes you should plug before adding water

• You’re improving for results, and not worrying about volume





• Action 3: Grow!





• Action 3: Grow!

• Activity is low but has great conversion rates, i.e. high quality

• Ripe for growth because the bucket is water-tight


Results

ActivityLow High

Below

Above

GrowthOpportunity

OptimizationNeeded

The Do-Nothing Zo

ne

Case Study 1: Web Traffic

Case Study 1: Web TrafficFinding opportunity online

Results

ActivityLow High

Below

Above

Direct: B

Organic search: A+

Referral: A

Paid Search: B

Email: A

Social: D


Results

ActivityLow High

Below

Above

Direct: B

Organic search: A+

Referral: A

Paid Search: B

Email: A

Social: D

CONTEXT


Results

ActivityLow High

Below

Above

Conversion rate: C

Bounce rate: B

Time on site: A+Site depth: A+


Results

ActivityLow High

Below

Above

Conversion rate: C

Bounce rate: B

Time on site: A+Site depth: A+

Conversion rate: A

Detour Into The 3rd DimensionBut what about cost?

• So, what about adding another dimension, like cost?



• Adding a 3rd dimension doesn’t actively change the analysis




• Instead, think about it like this:

• X-axis = input

• Y-axis = output




• Instead, think about it like this:

• X-axis = input

• Y-axis = output

• So, rather than a 3rd dimension, change axes to account for concerns like cost or time

Detour Into The 3rd DimensionIt’s not perfect but it can help with budget allocation

Return onInvestment

CostLow High

Below

Above

4

3

21

Case Study 2: Customer Success

Case Study 2: Customer SuccessAn approximated case study from my previous company

• At Mad Mimi, we decided to tackle churn



• There were a handful of features that customers needed to use to maximize value for

both customer and company



• There were a handful of features that customers needed to use to maximize value for

both customer and company

• Customers that utilized some features, had a higher LTV and lower churn rate


• Feature 1: Composing a newsletter

• About 90% of signups composed a newsletter – the activity

• However, only 10% who composed the newsletter upgraded from free to paid – the

result

• Let’s map this on the graph…


Upgrades

ActivityLow High

Below

Above

Compose newsletter


• Feature 2: Importing their list of customer email addresses

• About 60% of signups imported their email list – the activity

• And again, 60% of those folks upgraded - result

• Let’s map this on the graph…


ActivityLow High

Below

Above

Import list

Compose newsletter

Upgrades


• Feature 3: Customizing the design with available tools and images

• Well, only about 20% of customers really took the time to design their newsletters -

activity

• But 80% of customers with lovely, well-designed newsletters upgraded and were

retained over the long term – result

• To the graph…


ActivityLow High

Below

Above

Import list

Customization

Compose newsletter

Upgrades


• The analysis showed better designed newsletters resulted in higher LTV and retention



• The actions:

• Optimize the compose process to encourage customization

• Increase the number of well designed newsletter by offering free design



• The actions:



• In this instance the optimization and growth opportunities were correlated



• The actions:



• In this instance the optimization and growth opportunities were correlated

• The real life result: a huge increase in customer retention


ActivityLow High

Below

Above

Import list

Design tools

Compose newsletter

Upgrades

Case Study 3: Online Advertising

Case Study 3: Online AdsBudget allocation and tweaking the assets

• Online ads are very well suited to this technique



• This helps unearth the best performing ads that are worthy of budget increases




• It also uncovers ads that need to be optimized

• Improve copy, landing pages, keywords etc.




• It also uncovers ads that need to be optimized

• This works at a granular level per campaign or ad too

• Consider whether optimization or targeting is needed by device type, location etc.


ConversionRate

CTRLow High

Below

Above

Campaign 4: 15% Conv.

Campaign 3: 51% Conv.

Campaign 2: 72% Conv. Campaign 1: 75% Conv.

Quick Fire Examples

Employee RetentionKeeping support reps engaged and successful

CustomerSatisfaction

InteractionsLow High

Below

Above

Chris

Allyson

MatthewVanessa

Content MarketingFinding the highest performing content

ConversionsShares

Referrals etc.

Articles with target keywordLow High

Below

Above

Keyword 4: Email list

Keyword 3: Free Email newsletters

Keyword 2: Email marketing tipsKeyword 1: Free email marketing

Customer Contract RenewalBetter than A/B testing offers

Offer Accepted

Offers ClickedLow High

Below

Above

Offer 4: Free upgrade on renewal

Offer 3: 25% off 1st yr/3 yr contract

Offer 2: 20% disc. on 2 yr renewalOffer 1: 10% disc. on 1 yr renewal

Ecommerce Sales ChannelsWhat are the most valuable products?

Revenue Per Session

Items soldLow High

Below

Above

Paid search

Referral

EmailOrganic search

Product developmentWhere is time and energy best focused?

Business goals

Est. man-hoursLow High

Below

Above

Feature 4

Feature 3

Feature 2Feature 1

After The Insight…

“Data is a strategic asset but it’s only valuable if it’s used

constructively and appropriately to deliver results.”

Bernard Marr

Taking ActionOtherwise, why bother, really?

• What we’ve done is highlight which areas need optimization and which are ready for

significant growth



significant growth

• Both require investment of time and money, so please consider costs and returns

before acting



significant growth

• Both require investment of time and money, so please consider costs and returns

before acting

• When ready to act, set targets and measure

Measurement Matters MostAverage gains over time

• Setting a target should be based on certainty



• “Can I be certain that this action had a real, long term impact?”



• “Can I be certain that this action had a real, long term impact?”

• Consider the following aspects:

• Time

• Variability

• Avg. Behavior


• Step 1: Choose a time period (a month, a quarter)



• Step 2: Avg. out the results over this time

• For example: Avg. conversion rate

• Keep variability in mind by calculating Std. Deviation




• Step 3: Act!




• Step 3: Act!

• Step 4: Measure the changes but keep averaging out the results until you reach an

equal time period post action




• Step 3: Act!

• Step 4: Measure the changes but keep averaging out the results until you reach an

equal time period post action

• Step 5: Did your results beat the previous avg. by 1 std. deviation? Success!

The Rinse And RepeatIt’s the circle of life, sort of

• So, what do you do when you’ve optimized? Grow!



• And then?



• And then?

• Segment and optimize further



• And then?


• Or find another task to optimize



• And then?



• And then grow what you’ve optimized



• And then?



• And then grow what you’ve optimized

• Always aiming for the top right corner


Results

ActivityLow High

Below

Above

GrowthOpportunity

OptimizationNeeded

The Do-Nothing Zo

ne

Date post:	12-Apr-2017
Category:	Business
Upload:	teacup-analytics
View:	65 times
Download:	0 times