+ All Categories
Home > Technology > Metrics and Analytics, Guest Lecture, UCLA

Metrics and Analytics, Guest Lecture, UCLA

Date post: 06-May-2015
Category:
Upload: darren-levy
View: 978 times
Download: 0 times
Share this document with a friend
Description:
Provided hour lecture on metrics and analytics for Jaime Levy's UX Design course at UCLA Extension.
Popular Tags:
39
Analytics & Metrics UCLA Extension (Guest Lecture) Darren Levy - Senior Director of Revenue, Retention at Mylife.com - Founder of Gatherspace.com [email protected]
Transcript
Page 1: Metrics and Analytics, Guest Lecture, UCLA

Analytics & MetricsUCLA Extension (Guest Lecture)

Darren Levy- Senior Director of Revenue, Retention at Mylife.com- Founder of Gatherspace.com

[email protected]

Page 2: Metrics and Analytics, Guest Lecture, UCLA

True or False:

Technology is the hardest part of building

an online business?

Page 3: Metrics and Analytics, Guest Lecture, UCLA

True or False:

Knowing the metrics is someone else’s job.

Page 4: Metrics and Analytics, Guest Lecture, UCLA

Breakdown of today’s discussion

- Part 1 – Definitions, why they are so critical- Part 2 – What metrics to focus on?- Part 3 – Optimizing metrics- Part 4 – Case Studies

Page 5: Metrics and Analytics, Guest Lecture, UCLA

I. Why do we need metrics?

Page 6: Metrics and Analytics, Guest Lecture, UCLA

What are metrics, why do we need them?

- Small units of measurement, typically ratio’s.- How are we doing? - Is business healthy?- Is the customer happy, is the product working?

Page 7: Metrics and Analytics, Guest Lecture, UCLA

99.1% Pure

Page 8: Metrics and Analytics, Guest Lecture, UCLA

…and what about analytics?

- Combining metrics, finding patterns of behavior.- What should we be working on? - Should we pivot?

Page 9: Metrics and Analytics, Guest Lecture, UCLA

Google Analytics – be cautious

- Storing all of your proprietary data with Google- Limitations on customizations- When shit hits the fan, getting customer support- You NEED to slice and dice and query your data

Page 10: Metrics and Analytics, Guest Lecture, UCLA

Why are metrics so importantfor YOU?!

- Be the Expert, be the Hero, the “go to” person- Don’t rely so much on others- Because we’re all Liars

Page 11: Metrics and Analytics, Guest Lecture, UCLA

Ice Cream Glove

Page 12: Metrics and Analytics, Guest Lecture, UCLA

Tightly ingrained withinLean Startup Process

Hypothesis

Experiment

Metrics

Validation

Page 13: Metrics and Analytics, Guest Lecture, UCLA

II. What metrics to focus on?

Page 14: Metrics and Analytics, Guest Lecture, UCLA

Vanity Metrics

- Not actionable- Not understandable- Examples:

• traffic • time on site• cumulative registrations• # of visits• Overall conversion rate

Page 15: Metrics and Analytics, Guest Lecture, UCLA

Single Key MetricOMTM

- Analysis Paralysis (don’t overdo it!)- For startups, business rally around a metric- Changes over time as business progresses

Page 16: Metrics and Analytics, Guest Lecture, UCLA

Customer Lifecycle – 5 Steps to SuccessAARRR!

Page 17: Metrics and Analytics, Guest Lecture, UCLA

Dave McClure’s Pirate MetricsAARRR!

Page 18: Metrics and Analytics, Guest Lecture, UCLA

AARRR Lifecycle

Acquisition Visit Site(or landing page, or widget)

* CPC* Keyword Rank* Ad CTR

CTR ~100%Value=.01

Category Lifecycle Experience Metrics Est Value

Activation Jump through initial hoop, signup (“a ha” moment)

* CTR CTR ~5%Value=$3

Retention Repeat Visitor, Open up emails

* Avg # of Visits* Engagement Score

CTR ~2%Value=$5

Revenue Customer generates revenue* RPV* AOV* ARPU

CTR ~1%Value=$25

Page 19: Metrics and Analytics, Guest Lecture, UCLA

Lean AnalyticsGrowth Stages (“Gates”)

Page 20: Metrics and Analytics, Guest Lecture, UCLA

Single Key MetricsFor Startups

How much do we spend to get a customer?Cost per Acquisition ($) CPA

How much do our customers pay us?Average Revenue per User ($) ARPU

At what rate are customers cancelling?Churn (%) Churn

How much is a customer worth over time?Lifetime Value per Customer ($)

LTV

Page 21: Metrics and Analytics, Guest Lecture, UCLA

Subscription v. Transactional Models

Subscription Business Transactional Business

VPR (value per reg)Churn / LTV / ARPUActivation & Engagement

Shopping Cart AbandonmentAverage Order Value (AOV)Upsell & Product Page CTR

Page 22: Metrics and Analytics, Guest Lecture, UCLA

III. How do we do this?

Page 23: Metrics and Analytics, Guest Lecture, UCLA

How do we get Metrics?

- Google Analytics- DIY – recording events, activities in a SQL db- AB testing- Other tools – Heatmaps

Page 24: Metrics and Analytics, Guest Lecture, UCLA

Types of AB Testing

- Simple AB testing- Multivariate testing- Price testing

Page 25: Metrics and Analytics, Guest Lecture, UCLA

Simple AB Testing

- Changing 1 element at a time?- Effecting CTR at any point in a funnel

Page 26: Metrics and Analytics, Guest Lecture, UCLA

Multivariate Testing

- More simultaneous variables- Takes longer to test- Better results than “simple”

Page 27: Metrics and Analytics, Guest Lecture, UCLA

Price Testing

- Testing different prices or terms- Not measuring CTR, measuring RPV- Pay attention to “Accumulation of Revenue”- Pricing Strategies (freemium, subsc., bundling)

Page 28: Metrics and Analytics, Guest Lecture, UCLA

Ugly Friend Concept

Page 29: Metrics and Analytics, Guest Lecture, UCLA

Famous Price Test (Ugly Friend)

- Decoy (ugly friend) pushes price selection- In this example, decoy pushed sales by 30%

Page 30: Metrics and Analytics, Guest Lecture, UCLA

Apple iPhone Pricing

~67% ~20% ~13%

Page 31: Metrics and Analytics, Guest Lecture, UCLA

10 Things you MUST Test

1. Calls to Action (CTA)2. Propositions3. Copy & Content4. Visual Media5. Funnel testing6. Forms 7. Emails 8. Pricing9. Shipping 10. Personalization

Page 32: Metrics and Analytics, Guest Lecture, UCLA

AB Test Example

Page 33: Metrics and Analytics, Guest Lecture, UCLA

Cohort AnalysisMeasuring engagement and value over time

Page 34: Metrics and Analytics, Guest Lecture, UCLA

Optimization via Heatmaps

Page 35: Metrics and Analytics, Guest Lecture, UCLA

Quantitative v. Qualitative Metrics

- Example of qualitative (fire alarm in middle of night, 99.97% up time, yet pissed off customers)- Why you need both. (user testing, surveys, call your customers)

Page 36: Metrics and Analytics, Guest Lecture, UCLA

IV. Case Studies

Page 37: Metrics and Analytics, Guest Lecture, UCLA

Price Test @ Dollarshaveclub.com

Control Test

4 7

7% higher CLTV4.4% higher 24 hr Net

Page 38: Metrics and Analytics, Guest Lecture, UCLA

Case Study: Netflix & House of Cards

- Goal to be “HBO” of online TV- Cost of $100m on production

- Analytics told them * Most customers watched David Fincher’s social network all the way* British version of “House of Cards” was highly watched* Significant overlap of British House of Cards watch Kevin Spacy films and Fincher films.

- Data so far * Q1 2013, brought in 3m new custmers* in Q1 alone, paid for cost of production* Increasing CLTV

Page 39: Metrics and Analytics, Guest Lecture, UCLA

Q&A


Recommended