Date post: | 12-Jul-2015 |
Category: |
Presentations & Public Speaking |
Upload: | prodactive |
View: | 259 times |
Download: | 0 times |
Product Analytics 101A introductory view on analytics for
product design, development and management
George @ Voulgaris .net @gevou
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
How many products will you be selling in 12 months from today?
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Short tale instead of intro “Sales Forecast: The Quest for Holy Grail” - or - “the 50 shades of finance”
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
1. Tools
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Advertising
Which ad is driving the most sign ups?Advertising tools allow you to add event tracking and ad retargeting to better understand your site visitors, where they came from, and if your campaigns are working.
Analytics
What are customers doing before they churn?Analytics tools allow you to visualize your data, see how customers use your product, and identify trends, so you can make better business decisions.
Marketing Automation
Did that newsletter send traffic to your site?Email marketing tools make it easy to send newsletters and other marketing materials to your customers. Integrating with these tools makes it easy to setup drip campaigns for your product.
Sales & Business
What sales leads and customers need the most attention today?Sales & business tools help you track leads throughout the sales funnel, manage customer relationships, and run outbound campaigns.
Customer Support
How can I answer any questions my customer have?Customer support tools help you communicate with current customers to answer their questions, provide support, and identify recurring issues.
Developer Tools
Why couldn’t new users sign up for accounts yesterday and how much money did we lose?Performance and error reporting tools help you improve the health and stability of your application by automatically logging and tracking errors and alerting you when there is any downtime.
User Testing
What is the best homepage copy for driving sales?Testing tools give you insight into how people are using your product, so you can optimize your design. Experiment with variations using A/B tests and understand behavior with heat maps to see what version works best.
source: segment.com
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Advertising
Which ad is driving the most sign ups?Advertising tools allow you to add event tracking and ad retargeting to better understand your site visitors, where they came from, and if your campaigns are working.
Analytics
What are customers doing before they churn? Analytics tools allow you to visualize your data, see how customers use your product, and identify trends, so you can make better business decisions.
Marketing Automation
Did that newsletter send traffic to your site? Email marketing tools make it easy to send newsletters and other marketing materials to your customers. Integrating with these tools makes it easy to setup drip campaigns for your product.
Sales & Business
What sales leads and customers need the most attention today? Sales & business tools help you track leads throughout the sales funnel, manage customer relationships, and run outbound campaigns.
Customer Support
How can I answer any questions my customer have? Customer support tools help you communicate with current customers to answer their questions, provide support, and identify recurring issues.
Developer Tools
Why couldn’t new users sign up for accounts yesterday and how much money did we lose? Performance and error reporting tools help you improve the health and stability of your application by automatically logging and tracking errors and alerting you when there is any downtime.
User Testing
What is the best homepage copy for driving sales? Testing tools give you insight into how people are using your product, so you can optimize your design. Experiment with variations using A/B tests and understand behavior with heat maps to see what version works best.
source: segment.com
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
few examples
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
inspectlet.com
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
inspectlet.com
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
inspectlet.com
Replay user activity* for the ultimate insight on how users actually use your app
*be careful with your T&Cs and the possibility of sensitive data
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
mixpanel.com
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
mixpanel.com
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
mixpanel.com
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Add Push notifications, Surveys, Revenue analysis…
mixpanel.com
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
… + pretty much any data you want to send from your App
mixpanel.com
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Advanced Segmentation, In-depth profiling, Direct Engagement / Communication
intercom.io
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
… combined with automated messaging.
intercom.io
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
“Q: Is this the:
a) best b) recommended c) endorsedd) cutest
set of tools?”
A: These are just examples. I do not endorse any - I just happen to use them at the moment. Do your own research for best fitness to your case.
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
With all those beautiful apps, tools, analyses, graphs, functions, automations you should be able to build the perfect products and businesses! Right?
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
What tools do you use? (or fooling around with) How and Why? What are your challenges at the moment?
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
2. What matters? (because observation is not a business by itself… mostly)
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
why
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
There is a great difference in focus for startups and established businesses. What is it?
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
There is a massive difference in focus between Customer Discovery and Customer Development stages. What is it?
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
where
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Inbound Marketing
website & purchase funnel in product
UX
Customer Support
Customer engagement inc. churn, retain, upsell
Company
The Long Funnel gets even longer
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Inbound Marketing
website & purchase funnel in product
UX
Customer Support
Customer engagement inc. churn, retain, upsell
Company
The Long Funnel gets even longer
Layered Inbound Marketing diagram source: Inbound Marketing Handbook - MarketingSherpa
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Inbound Marketing
website & purchase funnel in product
UX
Customer Support
Customer engagement inc. churn, retain, upsell
Company
Advertising
Analytics
Marketing Automation
Sales & Business
Customer SupporDeveloper Tools
User Testing
BI
The Long Funnel gets even longer
Layered Inbound Marketing diagram source: Inbound Marketing Handbook - MarketingSherpa
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
when
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Which stages of the entire journey / funnel are you currently monitoring in your business? How?
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
What matters most depends on the product / business maturity stage and the user engagement stage.
There are few frameworks that can help.
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ...
Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ...
Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates...
Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics...
Do users promote your product?Email, widgets, campaigns, likes, RTs, affiliates...
Acquisition Activation Retention
Revenue Referral
Dave McClure’s Pirate Metrics
Source: http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Lean Analytics (A.Croll & B. Yoskovitz recommended reading)
Stage
EMPATHY I’ve found a real, poorly-met need that a reachable market faces.
STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for.
VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives.
REVENUE The users and features fuel growth organically and artificially.
SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem.
Gate
The
five
stag
es
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Eric Rees’ (The Lean Startup) Sustainable growth is a requirement for any business.
“New customers come from the actions of past customers”
The three growth engines: Virality Make people invite others as a necessary side effectRate of growing StickinessKeep people coming back / addictive / network effects / lock-in Rate of churn (compound)Price Reinvest revenue of past to get new customers. Rate of expenditure
Both AARRR & Lean Analytics frameworks resonate with…
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
how
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Lean Analytics (Adapted version from the book by A.Croll & B. Yoskovitz recommended reading)
Pick a KPI
Draw a line in the sand
Identify ways to improve this KPI
Make a guess Find relevant patterns in your
user data
Design experiments
(e.g. A/B)
Implement what seems to work
Find correlations between those
patterns & users
data available
no data
Analyze the behavior of the KPI
Line-in-the-sand was crossed
Line-in-the-sand was NOT crossed
Give up or pivot Go back to Discovery.
Talk to customers
Try again
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Lean Analytics (Adapted version from the book by A.Croll & B. Yoskovitz recommended reading)
Pick a KPI
Draw a line in the sand
Identify ways to improve this KPI
Make a guess Find relevant patterns in your
user data
Design experiments
(e.g. A/B)
Implement what seems to work
Find correlations between those
patterns & users
data available
no data
Analyze the behavior of the KPI
Line-in-the-sand was crossed
Line-in-the-sand was NOT crossed
Give up or pivot Go back to Discovery.
Talk to customers
Try again
Identify business objectives & decision processes
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
what
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Good metrics are:
• Actionable Accessible Auditable
• Metrics need to change a behavior - or need to hurt
• Use rates
• One Metric That Matters: based on model and stage
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Some terms to familiarise yourself with:
• Qualitative versus quantitative metrics Qualitative metrics are unstructured, anecdotal, revealing, and hard to aggregate; quantitative metrics involve numbers and statistics, and provide hard numbers but less insight.
• Vanity versus actionable metrics Vanity metrics might make you feel good, but they don’t change how you act. Actionable metrics change your behavior by helping you pick a course of action.
• Exploratory versus reporting metrics Exploratory metrics are speculative and try to find unknown insights to give you the upper hand, while reporting metrics keep you abreast of normal, managerial, day-to-day operations.
• Leading versus lagging metrics Leading metrics give you a predictive understanding of the future; lag- ging metrics explain the past. Leading metrics are better because you still have time to act on them.
• Correlated versus causal metrics If two metrics change together, they’re correlated, but if one metric causes another metric to change, they’re causal. If you find a causal relationship between something you want (like revenue) and something you can control (like which ad you show), then you can change the future.
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Typically your focus will go through the fives stages of product/business maturity
Stage
EMPATHY I’ve found a real, poorly-met need that a reachable market faces.
STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for.
VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives.
REVENUE The users and features fuel growth organically and artificially.
SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem.
Gate
The
five
stag
es
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
More terms to familiarise with:
Average Revenue Per User (ARPU): Revenues / Users
Churn: users abandoned (month)
Lifetime: 1 / churn
Lifetime Value (LTV): ARPU * Lifetime
Customer Acquisition Cost (CAC): Total monthly acq costs / new monthly customers (e.g. 3xCAC < LTV)
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Even more terms to familiarise yourself with:
Invitation Rate = Invited / inviters
Acceptance Rate = Signed-up / Invited total
Viral Coefficient = Invitation Rate * Acceptance Rate (e.g. Users * VC - churn > 1)
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Empathy
Stickiness
Virality
Revenue
Scale
E-commerce SaaS MediaMobile
appUser-gencontent
2-sidedmarket
Interviews; qualitative results; quantitative scoring; surveys
Loyalty, conversion
CAC, shares, reactivation
Transaction, CLV
Affiliates, white-label
Engagement, churn
Inherent virality, CAC
Upselling, CAC, CLV
API, magic #, mktplace
Content, spam
Invites, sharing
Ads, donations
Analytics, user data
Inventory, listings
SEM, sharing
Transactions, commission
Other verticals
(Money from transactions)
Downloads, churn, virality
WoM, app ratings, CAC
CLV, ARPDAU
Spinoffs, publishers
(Money from active users)
Traffic, visits, returns
Content virality, SEM
CPE, affiliate %, eyeballs
Syndication, licenses
(Money from ad clicks)
Lean Analytics Important: drawing base-lines (in the sand)
A considerably more information-rich version is available at: bit.ly/BigLeanTable
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
What KPIs do you use at the moment? What is the example of the impact of these in your business?
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
3. What matters? Money-talks
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
TL;DR
1.Make more money than what you spend *
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
TL;DR
1.Make more money than what you spend *
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
TL;DR
1. Make more money than what you spend *
2. The (dream of) Hockey-stick growth includes a long painful period with resource stagnation (you are working on high growth ventures right?)
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
8
THE STARTUP FUNDING SPECTRUM
+
NET
CASH
PO
SITI
ON
Closed
ALPHA
Public
BETA
SCALE - STAGE
1st Revenue
Redistributed for educational purposes with permission from Maria Dramalioti Taylor, Managing Partner, x.Million Capital Ventures
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
TL;DR
1. Make more money than what you spend *
2. The (dream of) Hockey-stick growth includes a long painful period with resource stagnation
3. Finance people want numbers, charts, proof; be it investors or CFOs
4. Planning and playing with other people’s money involves accountability. Be accountable for the right thing
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
The “Business Plan” Approach (top-bottom)
• Do background / buy research
• Check macro - trends
• Make projections and forecasts Initially to convince… and find yourself accountable 12 months later…
The Data Driven Approach (bottom-up)
Deductively and quantitatively prove that:
• (Startups *) …you are narrowing-in a valid Business Model
• (Existing businesses / products ) …your key growth assumptions still hold-true; optimize
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
What planning, forecasting and optimization processes do you use at the moment? What are your challenges with these at the moment?
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
Hopefully by now we all agree that:
1. Make more money than what you spend *
2. The (dream of) Hockey-stick growth includes a long painful period with resource stagnation
3. Playing with other people’s money involves accountability. Be accountable for the right thing
4.Data-driven planning & decisions (*) reduce risk
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
4. Perfect
George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014
A few common problems when implementing a data-driven business
1. The asterisk in decisions was on purpose. data-driven decisions need analytics AND decision processes
2. The 2nd asterisk in decisions. What’s hard about pivoting? alignment, transparency, knowledge
3. Vanity metrics … and politics. Someone will look ugly.
4. Resourcinginability to explain / understand the importance
5. Culture (I know all) / Ethics (Morale in changing situations) Change is tough
George @ Voulgaris .net @gevou
fallingelephant.com visionmobile.com techtalkscentral.com