Metrics LessONs LeArNeD
Or
KiLLiNG Pre-MArKetFit PrODUct iLLUsiONs
@andreasklinger
LOOKK | CONDÉ NAST | 10-14-2011 | 1
ANDreAs KLiNGer
cO-FOUNDer & cWtFOLOOKK.cOM
BAcKGrOUND iN sOciAL/ONLiNe
LOOKK | CONDÉ NAST | 10-14-2011 | 2
VP PrODUctsiNce
2,5 MONths
MFP - Maria Francesca Pepe | Veleno | www.lookk.com/mfp-maria-francesca-pepe/veleno
4 Biggest (development) Problemswe had/have:
OVer eNGiNeereD
AND OVer cONcePteD
FeAtUres
sOLUtiON?FeAtUre
rOLLOUts iN A/B
NO DAtA
DriVeN stArtUP cULtUre
sOLUtiON?GOiNG MeNtAL
ON Metrics.
BLOcKeD PrOcesses
sOLUtiON? KANBAN
BOArD
LOOKK | CONDÉ NAST | 10-14-2011 | 4
+1
UNcLeArUser VALUe
PrOP(Pre-PMF)
sOLUtiON?cUstDeV
+UserGrOUP
iN FB
?
4 Biggest (development) Problemswe had/have:
OVer eNGiNeereD
AND OVer cONcePteD
FeAtUres
sOLUtiON?FeAtUre
rOLLOUts iN A/B
NO DAtA
DriVeN stArtUP cULtUre
sOLUtiON?GOiNG MeNtAL
ON Metrics.
BLOcKeD PrOcesses
sOLUtiON? KANBAN
BOArD
LOOKK | CONDÉ NAST | 10-14-2011 | 5
+1
UNcLeArUser VALUe
PrOP(Pre-PMF)
sOLUtiON?cUstDeV
+UserGrOUP
iN FB
?
Metrics
Mark Goldenberg | Woven Bird | www.lookk.com/mark-goldenberg/woven-bird
LOOKK | CONDÉ NAST | 10-14-2011 | 7
stArtUP
LOOKK | CONDÉ NAST | 10-14-2011 | 8
stArtUP
BeFOre PrODUct MArKet Fit
LOOKK | CONDÉ NAST | 10-14-2011 | 9
stArtUP
BeFOre PrODUct MArKet Fit
“BLUe OceAN”
LOOKK | CONDÉ NAST | 10-14-2011 | 10
stArtUP
“BLUe OceAN”
BeFOre PrODUct MArKet Fit
MADNess
LOOKK | CONDÉ NAST | 10-14-2011 | 11
BUZZ is OK
NUMBers Are OK
FeeDBAcK is OK
MeDiA LiKes YOU
YOU?
LOOKK | CONDÉ NAST | 10-14-2011 | 12
BUZZ is OK
NUMBers Are OK
FeeDBAcK is OK
MeDiA LiKes YOU
sOMethiNGis WrONG
trAFFic sPiKes
NO sticKYNess
PUshes DON’t LiFt trAFFic
YOU?
LOOKK | CONDÉ NAST | 10-14-2011 | 13
BUZZ is OK
NUMBers Are OK
FeeDBAcK is OK
MeDiA LiKes YOU
sOMethiNGis WrONG
trAFFic sPiKes
NO sticKYNess
PUshes DON’t LiFt trAFFic
YOU?
VANitY WhAt ActiONs WOrK?
VANitY Metrics
esPeciALLY:- registered Users- Facebook Fans
But also:- Visitors / Month- Users / Month
Always goes up.
or
@j�dfi\�`eÖl\eZ\[�by external factors than by internal.
VANitY Metrics
esPeciALLY:- registered Users- Facebook Fans
But also:- Visitors / Month- Users / Month
Are usually controlled by external ]XZkfij��GI�\÷\Zkj�fi�\m\e�J<F %
tell you nothing about the health of your Ylj`e\jj�Xe[�k_\�\÷\Zkj�f]�pfli�nfib%
(exception: it might show that viral or gX`[�kiXùZ�nfibj�$�jk`cc�[f\jek�_\cg�pfli�product)
eNGiNes OF GrOWth
sticKY (retention)
VirAL (referral)
PAiD (works technically like viral)
eric ries
eNGiNes OF GrOWth
sticKY (retention)
VirAL (referral)
PAiD (works technically like viral)
eric ries
in Discover/Validation Focus on retention and Activation
(in Validation/Growth Focus on revenue and referral)
eNGiNes OF GrOWth
VANitY Metrics
Pr resULts- created visits in Google Analytics- created userentries in Database- Disappeared.
cOMPetitiONs- created visits in Google Analytics- created userentries in Database- Disappeared.
VANitY Metrics
Pr resULts- created visits in Google Analytics- created userentries in Database- Disappeared.
cOMPetitiONs- created visits in Google Analytics- created userentries in Database- Disappeared.
DriLL DOWNreMOVe the DAtA schMUtZ
AND Get tO ActiONABLe NUMBers
DriLLDOWN
Metrics
ActiONABLe Metrics
Where are your weaknesses?
if it’s not a weakness you are working on the wrong thing.
ActiONABLeMetrics
Where are your weaknesses?
if it’s not a weakness you are working on the wrong thing.
start with the AArrr Funnel
AArrr
eXAMPLe - LOOKK
AcQUisitiON - User registeredActiVAtiON - User VotedreteNtiON - User Visited AgainreFerrAL - …reVeNUe - …
AArrr
eXAMPLe - LOOKK
AcQUisitiON - User registeredActiVAtiON - User Voted (eg 90%)reteNtiON - User Visited AgainreFerrAL - …reVeNUe - …
ALL NUMBers LOOKeD OK.
GOOGLe ANALYtics WAsALsO Nice tO LOOK At.
MetricsNeeD
tOhUrt
Lilach Eliyahu | Peek a Boo | www.lookk.com/lilach-eliyahu/peek-a-boo
AArrr
eXAMPLe - LOOKK
AcQUisitiON - User registeredActiVAtiON - User Voted (eg 90%)reteNtiON - User Visited AgainreFerrAL - …reVeNUe - …
<- DriLLDOWN!
Analyse what it truely means to be retentive or activated on your website.
eg. We changed activiation to “Users that voted for more than two designers”
everybody else is just voting for a friend.
Now: AArrrr = the PAiN
the PAiN = ActiONABLe
AArrr NeeDs tO hUrt
AcQUisitiONYou need a way to reach the user.
(Dave is wrong )�\^%�i\^`jk\i\[�"�ZfeÕid\[�\dX`c
ActiVAtiONthe User has seen the real core of your
product and used it. (segment in user groups if multifaced)
reteNtiON the user did an action that leads to value. (if possible close to money or
coreaction)
reFerrALNot only shared but invited users
came.
ONLY LOOKAt%
Or.
VALUes
Lilach Eliyahu | Peek a Boo | www.lookk.com/lilach-eliyahu/peek-a-boo
LessONs LeArNeD
ignore everything you currently do not work on within your product. (too much data is bad for your eyes)
eg. We ignore revenue atm.
Look for the 4 values that can tell you about your state of your product.
Only look at relative values displayed as % or .
Do not lie to yourself. if you are Pre-PMF you suck.
it’s all about iterative sucking less.
tOOLs
tOOLs
tL;Dr: All suck.
tOOLs
tL;Dr: All suck.
…but are useful for certain parts:
tOOLs
tL;Dr: All suck.
…but are useful for certain parts:
>ff^c\�8eXcpk`Zj�4�KiXùZ�8eXcpj`jKissmetrics, Mixpanel = People AnalysisYour own Database = drilldown
Be consistent throughout all tools:�$�N_\e&N_\i\�kf�d\Xjli\�X�ÕeXc�^fXc - Naming of goals - Logging and Naming of user types (eg. User, Designer, Admin, Visitor)
tOOLs
Google Analytics
Dashboards rock.
Unfortunately single
player only (yet)
- referrals - Optimize Online Marketing, - Bird eye view Website - suck for funnel analysis (no ad-hoc eg) - suck for consistancy (goals are easy to break)
Don’t do funnel testing in GA - only track the last step
Don’t go too crazy with customization
tOOLs
KissMetrics
“View this people” feature is genius
- insights on conversions- Quick reports on features/pages- suck (yet) at dashboards
Use it for funnel optimization.Lj\�`k�]fi�hl`Zb�Ö\o`Yc\�`ej`^_kj%Lj\�`k�kf�Õe[�lj\ij�^iflgj
if you are a saas use it for everything ;)
Don’t expect too much and also test mixpanel.
tOOLs
LOOKKButler
- sQL = quick, tables = cheap- You can drilldown to your actionable KPis- Dashboards for team members- suck at bird-eye view, not ad-_fZ�Ö\o`Yc\
Questions:Who are the most popular designers if you only count people who vote for more than one designer?
N_f�Xi\�fli�dfjk�`eÖl\ek`Xc�users?
cOMPAre reteNtiON
BehAViOr
Weekly cohorts
if you group them good for comparing behavioural patterns over time.eg. Visited Again
cOMPAre reteNtiON
BehAViOr
AArrr+
WeeKLY cOhOrts
iF BAseD ON reGistrAtiON WeeK
hArD tO ActiON ON
- Better: ActiON WeeK
tAKe AWAYs:
Use YOUr DB tO DriLLDOWNBe cONsisteNt BeYOND
tOOLsFOcUs ON reteNtiON AND
ActiVAtiON
shAreYOUr
LessONs LeArNeD.
@andreasklinger
MFP - Maria Francesca Pepe | Veleno | www.lookk.com/mfp-maria-francesca-pepe/veleno