Use data to inform product decisions

Why we do need data to inform product development

Imagine this ...

... or this

Outline

Why

Driven vs Informed

What

5 things to be mindful of

Why do we need data?

Why do we need data?

What data do we need?

It’s about asking the right questions, you fool!!

Data & the product lifecycle

What do we want?

Assumption: Sitting on a tractor all day isn’t the best use of my time

Assumption: My users want to spend less time on the tractor so that they can spend more time on other tasks

Hypothesis: We believe this is true if the users of our MVP spend 20% more time on the farm

Approaches: One Single Metric, Prototypes, MVP, Direct Observations, Competitor Analysis

Question: Is there a market need for driverless tractors?

Data & the product lifecycle

How should it work?Questions: How should people find and use their personalised TV Guide? What should they do next?

Assumption: People will expect to find their personal TV Guide under the “Guide” tab. They will then switch to the ‘full’ TV Guide to see what else is on

Hypothesis: We believe that our users will discover new content on the app if their personal guide is easier to find. We know this is true if there’s a 30% increase in click-through rate from the personal TV Guide by Dec ’14

Approaches: A/B and MVT, behavioural plan & KPIs, user stories & metrics, prototypes and user testing

Data & the product lifecycle

How is it working?

Question: Is our product / feature meeting the hypothesis?

Assumption: Our users will use this feature to create and manage their holiday plans because it is so easy to use

Hypothesis: We know that our assumption is correct if we see a 30% increase in the number of users creating new holiday plans through this new feature by December ‘14

Approaches: Usage tracking, user testing, product retrospectives and refine or reject hypothesis

Data driven

Data drivenA/B or multi-variate test continuously !

Focus on the “One Metric That Matters” !

Build hypothesis around key KPI !

Optimise your product based on data !

Are we making a noticeable difference?

BUT... What data cannot tellIs it a good product idea? !

Metrics do not always offer you the full picture !

Data is one of the factors that feed into a decision !

We typically do not own all product decisions

Data informed

Data informed

Data

Users

Intuition

Competition

Technology

Brand

Strategy

BusinessRegulation

Time

Data informedData is one of the factors to consider !

Focus on the questions that you want answered !

You cannot replace intuition or creative ideas with data !

Assess impact on relevant areas

5 things to be mindful ofFocus on asking the right questions !

Data can’t replace intuition !

Listen to the data and act accordingly! !

Build and launch with data in mind !

Be clear on hypothesis, sample size and timings

SO ...

Embrace the data, don’t fear it!

Thanks!marcabrahamlondon@gmail.com

!

marcabraham.wordpress.com !

@MAA1 !

!

!

Related linkshttp://svpg.com/assessing-product-opportunities/ !http://www.romanpichler.com/blog/goal-oriented-agile-product-roadmap/

http://vimeo.com/14999991

http://www.realityisagame.com/archives/390/wooga-follows-zynga-in-metrics-driven-game-design/

http://marcabraham.wordpress.com/2013/05/03/book-review-lean-analytics/

http://www.kaushik.net/avinash/web-analytics-101-definitions-goals-metrics-kpis-dimensions-targets/

http://marcabraham.wordpress.com/2013/09/09/some-considerations-regarding-data-driven-design/

http://insideintercom.io/the-problem-with-data-driven-decisions/

http://www.webdesignerdepot.com/2013/05/the-perils-of-ab-testing/

http://andrewchen.co/2008/09/08/how-to-measure-if-users-love-your-product-using-cohorts-and-revisit-rates/

http://codeascraft.com/2012/06/21/building-websites-with-science/