Learn-Measure-Build Craft Useful Experiments By Running Backwards

Jason Fraser & Janice FraserFebruary 2014

LUXR.CO JUNE 2012

LUXR.CO JUNE 2012

TWEET!

#LeanStartup

@clevergirl@luxrco

© 2013 LUXr Incorporated, www.luxr.co

June & Pete’s Wedding Cake

© 2013 LUXr Incorporated, www.luxr.co

http://saptstrength.com/

Pete June

© 2013 LUXr Incorporated, www.luxr.co

April

http://saptstrength.com/

May

June

July

August

© 2013 LUXr Incorporated, www.luxr.co

sedonaweddingcakes.com

© 2013 LUXr Incorporated, www.luxr.co

freebestpictures.blogspot.com

© 2013 LUXr Incorporated, www.luxr.co

thekitchn.com

© 2013 LUXr Incorporated, www.luxr.co

Zoe Clark, The Cake Parlour

© 2013 LUXr Incorporated, www.luxr.co

sedonaweddingcakes.com

Zoe Clark, The Cake Parlour

© 2013 LUXr Incorporated, www.luxr.co

LEAN Star tup

© LUXR.CO APRIL 2013

LEAN Star tup

© LUXR.CO APRIL 2013

Build measure

learn!

© LUXR.CO APRIL 2013

© LUXR.CO 2013

Lean Startup is NOT

Cheap Startup

Fast Startup

Shortcut Startup

© LUXR.CO APRIL 2013

An approach for building companies that are creating new products and services in situations of extreme uncertainty.

The approach advocates creating small products that test the entrepreneurʼs assumptions, and using customer feedback to evolve the product, thereby reducing waste.

Lean Startup is...

© LUXR.CO APRIL 2013

1. List your assumptions.2. Understand your customers.3. Experiment efficiently.4. Adjust direction based on evidence.

© LUXR.CO APRIL 2013

1. List your assumptions.2. Understand your customers.3. Experiment efficiently.4. Adjust direction based on evidence.

© LUXR.CO APRIL 2013

Go backward to go forward.

Learn

Measure

Build

© LUXR.CO 2013

Plot the difference

THINK

release

MAKErelease

MAKErelease

MAKE

RISK

= U

NVAL

IDAT

ED E

FFOR

T

TIME

© LUXR.CO 2013

Lots of little wigglesRI

SK =

UNV

ALID

ATED

EFF

ORT

TIME

© LUXR.CO 2013

RISK

= U

NVAL

IDAT

ED E

FFOR

T

TIME

Each wiggle is a learning cycle.

MAKE

releaseBUILD

LEARN

BUILD

MEASURE

LUXR.CO JUNE 2012

There’s a pattern to startup work:

ideation decisionmaking

ideation etc.

LUXR.CO JUNE 2012

In foolproof experiments, the

results should be self-evident.

LUXR.CO JUNE 2012

1. Choose the problem to tackle. Clarify what’s uncertain, what you need to learn.

2. Understand the possible answers.3. Brainstorm “what you could do to prove it”.4. Choose the most broadly differentiated set of

indicators.5. Run the experiment. 6. Write down the results...on a sheet of paper. 7. Project the results into your future.

7 steps to foolproof experiments

LUXR.CO JUNE 2012

The Workshop

LUXR.CO JUNE 2012

Product

What is the product weʼre working on?

Today’s Challenge

LUXR.CO JUNE 2012

A Real Product

What is the product weʼre working on?Weʼre going to make a Business Canvas.

Today’s Challenge

LUXR.CO JUNE 2012

ENGINE OF GROWTH

VALUE STREAM & MARKET STAKEHOLDERS DISTRIBUTION

COSTREVENUE

PROBLEM

CUSTOMER SOLUTION

VALUE PROP

1-Viral

2-Sticky

3-Paid

1. Choose the problem to tackle. Clarify what’s uncertain, what you need to learn.

LUXR.CO JUNE 2012

{Activity}

Identify assumptions10 assumptions, 1 per sticky

LUXR.CO JUNE 2012

{Activity}

Divide into 2 pilesWill kill the company in the next 6 months

if weʼre wrong

Everything else

LUXR.CO JUNE 2012

{Activity}

Review & de-dupe

LUXR.CO JUNE 2012

{Activity}

Stack rank the urgents

LUXR.CO JUNE 2012

{Activity}

Pick the top one

LUXR.CO JUNE 2012

✓ Choose the problem to tackle. Clarify what’s uncertain, what you need to learn.

2. Understand the possible answers.

Your Top RiskPeople don’t

like my product.

Possible Answers

Don’t like it.

Do like it.

“Meh.”

LUXR.CO JUNE 2012

✓ Choose the problem to tackle. Clarify what’s uncertain, what you need to learn.

✓ Understand the possible answers.3. Brainstorm “what you could do to prove it”.

7 steps to foolproof experiments

LUXR.CO JUNE 2012

Experiment Framework

For Example:

We believe people like [customer type] have a need for(or problem doing) [need/action/behavior].

The smallest thing we can do to prove that need is [experiment].

We will know we have succeeded when [quantitative/measurable outcome] or [qualitative/observable outcome].

Every experiment has three parts:

1. Hypothesis that is provable/disprovable

2. The experiment itself; the thing you build

3. An indicator of result

LUXR.CO JUNE 2012

{Activity}

Design an experiment to learn if this is true.

we believe that ________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

__________________________

Briefly describe it.

State how you will know if the hypothesis is valid or invalid. This can be quantitative evidence or qualitative.

How much time/money/effort will it take?

LUXR.CO JUNE 2012

{Activity}

Discuss, then pick the experiment to run.

we believe that ________________

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____________ in 2 days.

Decide

we believe that ________________

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____________ in 2 weeks.

we believe that ________________

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____________ in 2 months.

✔

LUXR.CO JUNE 2012

✓ Choose the problem to tackle. Clarify what’s uncertain, what you need to learn.

✓ Understand the possible answers.

✓ Brainstorm “what you could do to prove it”.4. Choose the most broadly differentiated set of

indicators.

7 steps to foolproof experiments

TODAY: Choose an experiment you

can do from this room. Not a survey.

LUXR.CO JUNE 2012

Experiments that were designed

assumption experiment measurable/observable outcome

busy people need a service to help them getmenial stuff done

fb ad, neighborhood targeting presents an offer, landing page. Phone number or tweet. No out

of pocket cost.

10% ctr on ad, 10% ctr on landing page. 2 days

busy people need a service to help them getmenial stuff done

find busy people (in offices). Set up a “shop”. Sit in their reception and see if we can get jobs. Prevail upon friend

who runs 300-person company

5% of employees will make a hire. 1 repeat booking

2 wks

busy people need a service to help them getmenial stuff done

ad in 10 different offices, diff types of companies 2 weeks.

LUXR.CO JUNE 2012

Yusuke Tell your MVP Story!

LUXR.CO JUNE 2012

1. Choose the problem to tackle. Clarify what’s uncertain, what you need to learn.

2. Understand the possible answers.3. Brainstorm “what you could do to prove it”.4. Choose the most broadly differentiated set of

indicators.5. Run the experiment. 6. Write down the results...on a sheet of paper. 7. Project the results into your future.

7 steps to foolproof experiments

LUXR.CO JUNE 2012

What did you learn?

Top 5 things

{Debrief}

LUXR.CO JUNE 2012

Design the next experiment:

2 days, 2 weeks, 2 months

{Next Steps}

LUXR.CO JUNE 2012

What’s the right size for the experiment?

{Choose}

LUXR.CO JUNE 2012

Techniques Checklist• Brainstorm quietly• Use Sticky Notes for ideas• Self-Edit• Go to the wall for team sharing• Stack-ranking• Roman Voting• Right-Sizing the work

LUXR.CO JUNE 2012

FISHBOWL