Date post: | 17-Oct-2014 |
Category: |
Technology |
View: | 3,108 times |
Download: | 0 times |
Experimenting
Enterprise
© 2013 BigVisible Solutions
in the
© 2012 BigVisible Solutions 2
David J BlandStartups consulted: 18Twitter followers: 2179Iterations of Coaching: 215
Speaker Vanity Metrics
Brian BozzutoBlogs & Articles Written: 77Presentations Delivered: 46Teams coached: 54
© 2013 BigVisible Solutions 3
This session doesn’t guarantee a 100 million dollar success, but it should help prevent a 100 million dollar failure.
© 2012 BigVisible Solutions 4
A business model used to be static and survived many years.
Radio
TV
Internet
iPod
0 5 10 15 20 25 30 35 40
38
13
4
3
2
Years to Reach a Market of 50 Million
© 2012 BigVisible Solutions 5
© 2012 BigVisible Solutions 6
The Lean Startup provides a scientific approach to creating and managing startups and get a desired product to customers’ hands faster.
-Eric Ries
© 2013 BigVisible Solutions 7
Entrepreneur Intrapreneur
© 2013 BigVisible Solutions 8
Entrepreneur Intrapreneur
Skills
© 2013 BigVisible Solutions 9
This is great in theory, but it’s not for B2B.
This is great in theory, but I’m a visionary.
EntrepreneurIntrapreneur
© 2013 BigVisible Solutions 10
Define the value proposition for:
Source: Adapted from Steve Blank
Buyer
User
Decider
Buyer
User
Decider
B2C B2B
11© 2013 BigVisible Solutions
Are your users/customers:
Source: Adapted from Steve Blank
Aware of problem?Actively seeking a solution?Hacking together a solution?Have $$$ for your solution?
12© 2013 BigVisible Solutions
Can We Build It?
Should We Build It?
13© 2013 BigVisible Solutions
Requirements = Hypotheses
© 2013 BigVisible Solutions 14Source: Adapted from Cynefin
Simple“Everyone Knows That”
Complicated“Let’s consult the expert”
Complex“Whoa, it all makes sense now…”
Chaos“That still doesn’t make sense”
Work with experts to inform our hypothesis
Run experiments to gain new knowledge
15© 2013 BigVisible Solutions
Safe Fail != Fail Safe
© 2012 BigVisible Solutions 16
Experiment withproduct & process
17© 2013 BigVisible Solutions
If you experiment just to see what happens, then you’ll succeed at seeing what happens.
18© 2013 BigVisible Solutions
Be careful how you structure your experiments
© 2013 BigVisible Solutions 19Image Source: Eric Ries The Lean Startup
Then here
Start hereLast here
© 2013 BigVisible Solutions 20Image Source: Eric Ries The Lean Startup
NowExecuteQuickly
© 2012 BigVisible Solutions 21
Experiment Template
For <customer segment> I predict <outcome>
when I run <experiment>
© 2012 BigVisible Solutions 22
Experiment Example
For new customers, I predict 10% increase in
conversion when I offer a “no questions asked” cancellation policy
© 2012 BigVisible Solutions 23
Experiment Guidelines
• Customer Segment (who)• Up to / Limit Of (how many)• Length of Time (how long)• Financial Exposure (how much)• Amplification & Dampening Measures
24© 2013 BigVisible Solutions
Experiment DashboardID Hypothesis Description Status Next Steps
01a
We predict that adding
a refund feature will
increase conversions
Story 13c adds refund label and call to
action button that puts
customer in contact with call center
agent.
Complete
0.2% increase in conversion on cohort. Meeting
with VP of Ops to discuss
strategy.
25© 2013 BigVisible Solutions
“This is all great, but how do we know what
experiments to run?”
© 2012 BigVisible Solutions 26
Iterative Product+ Static Business Model
Failed Organization
© 2012 BigVisible Solutions 27Source: Alex Osterwalder
© 2012 BigVisible Solutions 28
? ? ???
?
??
?
?
??
“Let’s map out our existing business model”
Source: Adapted from Alex Osterwalder
tip: tell a story
© 2012 BigVisible Solutions 29
? ? ???
?
??
?
?
??
“Let’s validate our riskiest assumptions”
Source: Adapted from Alex Osterwalder
© 2012 BigVisible Solutions 30
This is most likely your Riskiest Assumption
Source: Adapted from Alex Osterwalder
© 2012 BigVisible Solutions 31
This is most likely your other Riskiest Assumption
Source: Adapted from Alex Osterwalder
© 2012 BigVisible Solutions 32
What happens when we subtract costs from revenue?
Source: Adapted from Alex Osterwalder
33© 2013 BigVisible Solutions
“Okay, now we have a big list of assumptions, how do we decide where to start?”
© 2013 BigVisible Solutions 34
experiment
experiment
experiment
experiment
experiment
experiment
experiment
“Let’s dot vote!”(you could, but that’s pretty subjective)
© 2013 BigVisible Solutions 35
time
lear
ning
experimentexperiment
experiment
experiment
experiment
experiment
experiment
experiment
experiment
experiment
experiment
experiment
experiment
(ok better, but what are they tied to?)
© 2013 BigVisible Solutions 36
Now we have a focusing mechanism
focustime time
lear
ning
lear
ning
experiments experiments
experiments experiments
© 2013 BigVisible Solutions 37
And assumptions that need validation
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
© 2013 BigVisible Solutions 38
This is Experiment Mapping
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
© 2013 BigVisible Solutions 39
Customer Segment is our focus
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
© 2013 BigVisible Solutions 40
Will they pay for our product?
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
They willpay for our
product
© 2013 BigVisible Solutions 41
Can we scale to meet demand?
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
They willpay for our
product
We can scaleto meetdemand
© 2013 BigVisible Solutions 42
Can we use our distribution channels?
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
They willpay for our
product
We can useexisting
distributionchannels
We can scaleto meetdemand
© 2013 BigVisible Solutions 43
Are we solving a real need?
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
They willpay for our
product
Our productsolves a need
they have
We can useexisting
distributionchannels
We can scaleto meetdemand
© 2013 BigVisible Solutions 44
Which of these is the riskiest?
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
They willpay for our
product
Our productsolves a need
they have
We can useexisting
distributionchannels
We can scaleto meetdemand
© 2013 BigVisible Solutions 45
Can we do a concierge test?
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
They willpay for our
product
Our productsolves a need
they have
We can useexisting
distributionchannels
We can scaleto meetdemand
Conciergetest with 10customers
© 2013 BigVisible Solutions 46
We could build a prototype
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
They willpay for our
product
Our productsolves a need
they have
We can useexisting
distributionchannels
We can scaleto meetdemand
Conciergetest with 10customers
Build aprototype
© 2013 BigVisible Solutions 47
Maybe run a focus group?
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
They willpay for our
product
Our productsolves a need
they have
We can useexisting
distributionchannels
We can scaleto meetdemand
Conciergetest with 10customers
Build aprototype
Run a focusgroup with80 people
© 2013 BigVisible Solutions 48
Prototype will take time
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
They willpay for our
product
Our productsolves a need
they have
We can useexisting
distributionchannels
We can scaleto meetdemand
Conciergetest with 10customers
Build aprototype
Run a focusgroup with80 people
© 2013 BigVisible Solutions 49
Focus groups take a long time too
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
They willpay for our
product
Our productsolves a need
they have
We can useexisting
distributionchannels
We can scaleto meetdemand
Conciergetest with 10customers
Build aprototype
Run a focusgroup with80 people
© 2013 BigVisible Solutions 50
Concierge test is quick & we’ll learn
focus
assumption assum
ption
assumption assumption
experiments experiments
experiments experiments
time time
lear
ning
lear
ning
CustomerSegment
They willpay for our
product
Our productsolves a need
they have
We can useexisting
distributionchannels
We can scaleto meetdemand
Conciergetest with 10customers
Build aprototype
Run a focusgroup with80 people
© 2013 BigVisible Solutions 51
“Let’s swarm on this experiment”
highlearning
shorttimeframe
52© 2013 BigVisible Solutions
Just because you can measure it, does not mean you should measure it.
53© 2013 BigVisible Solutions
Auditable
Actionable
Accessible
Source: Eric Ries The Lean Startup
54© 2013 BigVisible Solutions
Auditable
“Be able to defend your metrics with confidence”
55© 2013 BigVisible Solutions
Auditable?
zombieproject
metrics
56© 2013 BigVisible Solutions
Actionable
“Does this metric incentivize a change in behavior?”
57© 2013 BigVisible Solutions
Actionable?
Facebook likes won’t influence your dog
58© 2013 BigVisible Solutions
Actionable
Number of steps should influence
your lifestyle
59© 2013 BigVisible Solutions
Actionable?
What does this tell you about your
product?
60© 2013 BigVisible Solutions
Accessible
“Can you easily access and share this data?”
61© 2013 BigVisible Solutions
Accessible?
Not accessible
62© 2013 BigVisible Solutions
Accessible
Accessible
Image Source: Lean.st
© 2013 BigVisible Solutions 63Source: Dave McClure Pirate Metrics
AcquisitionActivationRetentionReferralRevenue
© 2013 BigVisible Solutions 64Source: Dave McClure Pirate Metrics
AcquisitionActivationRetentionReferralRevenue
© 2013 BigVisible Solutions 65
“Metrics are people, too”
Source: Eric Ries The Lean Startup
© 2013 BigVisible Solutions 66
Aware
Hopeful
Satisfied
PassionateSource: Adapted from Cooper & Vlaskovits
© 2013 BigVisible Solutions 67
Aware
Hopeful
SatisfiedPassionate
Acquisition
Activation
RetentionReferral
© 2012 BigVisible Solutions 68
Start small.
© 2012 BigVisible Solutions 69
When you have stories around an uncertain feature, add a hypothesis that states the expected outcome on a specific metric.
© 2012 BigVisible Solutions 70
If you have a Kanban Board, add a Validate Column to
the very right of it.
© 2012 BigVisible Solutions 71
Problem Solution Process
Known Known Waterfall orAgile
Known Unknown Agile
Unknown Unknown Lean Startup
© 2012 BigVisible Solutions 72
Where are the unknown / unknown’s in your enterprise?
73
David J BlandExperimentor
Brian BozzutoPrincipal
© 2013 BigVisible Solutions
Thank You!