Metrics LessONs LeArNeD

Or

KiLLiNG Pre-MArKetFit PrODUct iLLUsiONs

@andreasklinger

LOOKK | CONDÉ NAST | 10-14-2011 | 1

ANDreAs KLiNGer

cO-FOUNDer & cWtFOLOOKK.cOM

BAcKGrOUND iN sOciAL/ONLiNe

LOOKK | CONDÉ NAST | 10-14-2011 | 2

VP PrODUctsiNce

2,5 MONths

MFP - Maria Francesca Pepe | Veleno | www.lookk.com/mfp-maria-francesca-pepe/veleno

4 Biggest (development) Problemswe had/have:

OVer eNGiNeereD

AND OVer cONcePteD

FeAtUres

sOLUtiON?FeAtUre

rOLLOUts iN A/B

NO DAtA

DriVeN stArtUP cULtUre

sOLUtiON?GOiNG MeNtAL

ON Metrics.

BLOcKeD PrOcesses

sOLUtiON? KANBAN

BOArD

LOOKK | CONDÉ NAST | 10-14-2011 | 4

+1

UNcLeArUser VALUe

PrOP(Pre-PMF)

sOLUtiON?cUstDeV

+UserGrOUP

iN FB

?

4 Biggest (development) Problemswe had/have:

OVer eNGiNeereD

AND OVer cONcePteD

FeAtUres

sOLUtiON?FeAtUre

rOLLOUts iN A/B

NO DAtA

DriVeN stArtUP cULtUre

sOLUtiON?GOiNG MeNtAL

ON Metrics.

BLOcKeD PrOcesses

sOLUtiON? KANBAN

BOArD

LOOKK | CONDÉ NAST | 10-14-2011 | 5

+1

UNcLeArUser VALUe

PrOP(Pre-PMF)

sOLUtiON?cUstDeV

+UserGrOUP

iN FB

?

Metrics

Mark Goldenberg | Woven Bird | www.lookk.com/mark-goldenberg/woven-bird

LOOKK | CONDÉ NAST | 10-14-2011 | 7

stArtUP

LOOKK | CONDÉ NAST | 10-14-2011 | 8

stArtUP

BeFOre PrODUct MArKet Fit

LOOKK | CONDÉ NAST | 10-14-2011 | 9

stArtUP

BeFOre PrODUct MArKet Fit

“BLUe OceAN”

LOOKK | CONDÉ NAST | 10-14-2011 | 10

stArtUP

“BLUe OceAN”

BeFOre PrODUct MArKet Fit

MADNess

LOOKK | CONDÉ NAST | 10-14-2011 | 11

BUZZ is OK

NUMBers Are OK

FeeDBAcK is OK

MeDiA LiKes YOU

YOU?

LOOKK | CONDÉ NAST | 10-14-2011 | 12

BUZZ is OK

NUMBers Are OK

FeeDBAcK is OK

MeDiA LiKes YOU

sOMethiNGis WrONG

trAFFic sPiKes

NO sticKYNess

PUshes DON’t LiFt trAFFic

YOU?

LOOKK | CONDÉ NAST | 10-14-2011 | 13

BUZZ is OK

NUMBers Are OK

FeeDBAcK is OK

MeDiA LiKes YOU

sOMethiNGis WrONG

trAFFic sPiKes

NO sticKYNess

PUshes DON’t LiFt trAFFic

YOU?

VANitY WhAt ActiONs WOrK?

VANitY Metrics

esPeciALLY:- registered Users- Facebook Fans

But also:- Visitors / Month- Users / Month

Always goes up.

or

@j�dfi\�`eÖl\eZ\[�by external factors than by internal.

VANitY Metrics

esPeciALLY:- registered Users- Facebook Fans

But also:- Visitors / Month- Users / Month

Are usually controlled by external ]XZkfij��GI�\÷\Zkj�fi�\m\e�J<F %

tell you nothing about the health of your Ylj`e\jj�Xe[�k_\�\÷\Zkj�f]�pfli�nfib%

(exception: it might show that viral or gX`[�kiXùZ�nfibj�$�jk`cc�[f\jek�_\cg�pfli�product)

eNGiNes OF GrOWth

sticKY (retention)

VirAL (referral)

PAiD (works technically like viral)

eric ries

eNGiNes OF GrOWth

sticKY (retention)

VirAL (referral)

PAiD (works technically like viral)

eric ries

in Discover/Validation Focus on retention and Activation

(in Validation/Growth Focus on revenue and referral)

eNGiNes OF GrOWth

VANitY Metrics

Pr resULts- created visits in Google Analytics- created userentries in Database- Disappeared.

cOMPetitiONs- created visits in Google Analytics- created userentries in Database- Disappeared.

VANitY Metrics

Pr resULts- created visits in Google Analytics- created userentries in Database- Disappeared.

cOMPetitiONs- created visits in Google Analytics- created userentries in Database- Disappeared.

DriLL DOWNreMOVe the DAtA schMUtZ

AND Get tO ActiONABLe NUMBers

DriLLDOWN

Metrics

ActiONABLe Metrics

Where are your weaknesses?

if it’s not a weakness you are working on the wrong thing.

ActiONABLeMetrics

Where are your weaknesses?

if it’s not a weakness you are working on the wrong thing.

start with the AArrr Funnel

AArrr

eXAMPLe - LOOKK

AcQUisitiON - User registeredActiVAtiON - User VotedreteNtiON - User Visited AgainreFerrAL - …reVeNUe - …

AArrr

eXAMPLe - LOOKK

AcQUisitiON - User registeredActiVAtiON - User Voted (eg 90%)reteNtiON - User Visited AgainreFerrAL - …reVeNUe - …

ALL NUMBers LOOKeD OK.

GOOGLe ANALYtics WAsALsO Nice tO LOOK At.

MetricsNeeD

tOhUrt

Lilach Eliyahu | Peek a Boo | www.lookk.com/lilach-eliyahu/peek-a-boo

AArrr

eXAMPLe - LOOKK

AcQUisitiON - User registeredActiVAtiON - User Voted (eg 90%)reteNtiON - User Visited AgainreFerrAL - …reVeNUe - …

<- DriLLDOWN!

Analyse what it truely means to be retentive or activated on your website.

eg. We changed activiation to “Users that voted for more than two designers”

everybody else is just voting for a friend.

Now: AArrrr = the PAiN

the PAiN = ActiONABLe

AArrr NeeDs tO hUrt

AcQUisitiONYou need a way to reach the user.

(Dave is wrong )�\^%�i\^`jk\i\[�"�ZfeÕid\[�\dX`c

ActiVAtiONthe User has seen the real core of your

product and used it. (segment in user groups if multifaced)

reteNtiON the user did an action that leads to value. (if possible close to money or

coreaction)

reFerrALNot only shared but invited users

came.

ONLY LOOKAt%

Or.

VALUes

Lilach Eliyahu | Peek a Boo | www.lookk.com/lilach-eliyahu/peek-a-boo

LessONs LeArNeD

ignore everything you currently do not work on within your product. (too much data is bad for your eyes)

eg. We ignore revenue atm.

Look for the 4 values that can tell you about your state of your product.

Only look at relative values displayed as % or .

Do not lie to yourself. if you are Pre-PMF you suck.

it’s all about iterative sucking less.

tOOLs

tOOLs

tL;Dr: All suck.

tOOLs

tL;Dr: All suck.

…but are useful for certain parts:

tOOLs

tL;Dr: All suck.

…but are useful for certain parts:

>ff^c\�8eXcpk`Zj�4�KiXùZ�8eXcpj`jKissmetrics, Mixpanel = People AnalysisYour own Database = drilldown

Be consistent throughout all tools:�$�N_\e&N_\i\�kf�d\Xjli\�X�ÕeXc�^fXc - Naming of goals - Logging and Naming of user types (eg. User, Designer, Admin, Visitor)

tOOLs

Google Analytics

Dashboards rock.

Unfortunately single

player only (yet)

- referrals - Optimize Online Marketing, - Bird eye view Website - suck for funnel analysis (no ad-hoc eg) - suck for consistancy (goals are easy to break)

Don’t do funnel testing in GA - only track the last step

Don’t go too crazy with customization

tOOLs

KissMetrics

“View this people” feature is genius

- insights on conversions- Quick reports on features/pages- suck (yet) at dashboards

Use it for funnel optimization.Lj\�`k�]fi�hl`Zb�Ö\o`Yc\�`ej`^_kj%Lj\�`k�kf�Õe[�lj\ij�^iflgj

if you are a saas use it for everything ;)

Don’t expect too much and also test mixpanel.

tOOLs

LOOKKButler

- sQL = quick, tables = cheap- You can drilldown to your actionable KPis- Dashboards for team members- suck at bird-eye view, not ad-_fZ�Ö\o`Yc\

Questions:Who are the most popular designers if you only count people who vote for more than one designer?

N_f�Xi\�fli�dfjk�`eÖl\ek`Xc�users?

cOMPAre reteNtiON

BehAViOr

Weekly cohorts

if you group them good for comparing behavioural patterns over time.eg. Visited Again

cOMPAre reteNtiON

BehAViOr

AArrr+

WeeKLY cOhOrts

iF BAseD ON reGistrAtiON WeeK

hArD tO ActiON ON

- Better: ActiON WeeK

tAKe AWAYs:

Use YOUr DB tO DriLLDOWNBe cONsisteNt BeYOND

tOOLsFOcUs ON reteNtiON AND

ActiVAtiON

shAreYOUr

LessONs LeArNeD.

@andreasklinger

MFP - Maria Francesca Pepe | Veleno | www.lookk.com/mfp-maria-francesca-pepe/veleno