Minimum viable research

transcript

• Live music start up • 2nd biggest live music site after Ticketmaster• Fans track their favourite artists so they never

miss them live• 7 million fans discover concerts on Songkick

every month

The usual things

• Daily stand ups• Kanban board• Short iteration cycles• Prototypes as the guide for what we are building. • Sketching hacks with designers, PM’s and devs• Post it notes and writeboards everywhere.• Define and measure the success of everything

Product development inspiration

• Flow Interactive - UCD• Jeff Gothelf and Josh Seiden - Lean UX• Marty Cagan - Silicon Product Group• Tom Chi - Experience Lead at Google X• Jeff Paton - Dual track Scrum• Jake Knapp - Google Ventures• Eric Reis and Steve Blank - Lean start up

Product discovery Product delivery

How we make stuff at Songkick

Customer discoveryListen Experiment Execute

Steve Blank - Customer Discover• Is this worth working on?• Is there anyone out there who wants this?• If we could build a perfect solution to this

problem, would anyone buy* it?

Steve Blank circa 1978

Customer discovery

Identify customers

Meet with them

Synthesis all the data / stories /artefacts you

collected

Communicate what you learnt to the

Idea for a productarea of product feature

Is this worth working on? Is there anyone out there who wants this?

Customers behaviours, attitudes, goals, motivations, frustrations and pain points

Input Output

Some validation that this is worth working on and there are people out there who

want this.

Product discovery

Ideas / concepts

Hypothesis

Design experiments

Run experiments

Analyse results

Product / Biz GoalCustomer problem

Validated learnings go into the product delivery

backlog

Input Output

How can we make it work? What solution(s) can we provide? Will people be able to use them?

Product discovery: hypothesisIf we show bands a share on Facebook button after they have added a tour date

then bands will share their events to Facebook,

because they want to tell their fans about their shows.

Product discovery

Ideas / concepts

Hypothesis

Design experiments

Run experiments

Analyse results

backlog

Input Output

Product discovery: experiments

• Build the solution• In-product prototype• Live data prototype (outside product)• Clickable prototype• Flat mock up• Paper prototype• + lots of other creative experiments - no code

or interface needed

Product discovery

Ideas / concepts

Hypothesis

Design experiments

Run experiments

Analyse results

backlog

Input Output

Product delivery

Validated product discovery ideas

It’s still a iterative process

Competitor product features

1000’s of users requesting the same

feature

Measure

Define success criteria (qual/quant/KPI)Prototype

Prioritise improvements

Refine

Inputs

How can we build this efficiently?How can we ensure what we are

building is of good enough quality?

Facebook share experimentIf we show bands a share on Facebook button after they have added a tour date

then bands will share their events to Facebook,

because they want to tell their fans about their shows.

Product discovery Product delivery

Right now as a UXD at Songkick this is how I spend my time

Customer discoveryListen Experiment Execute

Is this worth working on? Is there anyone out there

who wants this?

How can we make it work? What solution(s) can we provide?

Will people be able to use our solutions to solve the problem?

How can we build this efficiently?How can we ensure what we are

building is of good enough quality?

50% 20%30%

Get out the building

***Insert picture of me in London with backpack or map or something***

Stay in the building and hack your space

Shit everywhere

Server room

Free Window frames we found in the basement - lying around from office refurb projects

£300 - knock a hole in the wall, fit the frame and ordinary glass

£30 - Car window tint film (used by boy racers but also makes a pretty good budget two way mirror)

Free - mic I stole from the unused conference calling set up.

Meeting room Free - mic boom and screwdriver for over the shoulder mobile testing

Free - Old mini-hifi - donated by a Songkick developer and used as a pre amp to increase the volume of the sound from meeting room.

4 sets of headphones - £15 each but gradually upgraded to better sets (donated by people who work at Songkick)

4 way audio splitter £50 Maplin

Audio and video cables - Maplin £60

Observation room

Grand old duke of York

Songkick CEO Ian Hogarth

Get feedback from users as often as you can

Insights

We run 3 - 6 users every Wednesday

alot00

Hello! 0 people gets you 0 insights

3 websites, 2 mobile apps and Spotify appFan apps Data and

industryCrowd sourced

touring

Songkick.comiPhone appAndroid appSpotify app tourbox.songkick.com

detour-ldn.songkick.comdetour.songkick.com

3 websites, 2 mobile apps and Spotify appFan apps Data and industry Crowd sourced touring

Design team

Support team

Train the product managers to run research sessions

• Product managers have natural empathy for customers, are great communicators and enjoy talking to people

• The more people that can run sessions the more feedback you’ll get

Always know exactly what you are trying to learn before you start

Agree what are the most important things to learn and uncover

• Write them down so you can refer to them later.

• Structure sessions so 1/3 to 1/2 are around customer discovery

• Rest of session is for feedback on prototypes and evaluating things we recently shipped.

Experiment with remote research tools

Self-Moderated remote tools

Survey tools Click and memory test tools

Analytics

Google Analytics and log file analysis

Few example of the ones we use regularly

Remote tools are just another data point• Sit along side our qualitative research

• Help us build a clearer picture of our customers

Hustling the recruit

Recruiting is hard

• We all know that the research we run is only as good as the people who we choose to participate

• Getting the recruit right is important!

• We always write a recruitment screener

Recruiting yourself - you know your customers better than anyone• Existing customers - mine support desk, Twitter

and Facebook

• Target customers - place ads where they already hang out• Post ads on Gumtree• Go to concert venues• Get people who work in music shops to help• Post on musician sites / Contact music colleges• Identify participants who can help with future

recruits

The 2 for 1 recruiting trick

Place ad on Gumtree

Survey on Polldaddy

Include recruitment

screener questions and larger survey

questions

Include specific date, outline how

the session works and offer a small incentive

5 participants for your next study

200 responses for your survey.

Feedback to the team as soon as possible

Get observers to help out with the analysis

How we get feedback to the team fast• Allow 30 mins between each session

• Get observers to help out with the analysis

• Tweak prototypes or the live site between sessions to learn as much as possible

• Write top 5's after each session and share them with the team right away

A little on how I do analysis• Record audio of sessions with Pearnote

• Record all the sessions with person in picture, audio and what happened on screen but rarely get a chance to watch them :(

• Want to figure out a way to make highlights video fast - WIP.

• Report back on research goals + hypothesis in 24 hrs.

• Sometimes afterwards I build customer storyboards to use in persona work later.

Customer storyboards

Touring lifecycle

Customer insights

Just do it *JFDI*

Minimum viable research• Even with a small team, a little cash and limited

time you can find a way to run research every week.

1. Get out the building2. Stay in the building and hack your space3. Get feedback from your target audience as often as you can4. Always know what you're trying to learn before you start5. Experiment with remote research tools to fill in the gaps6. Hustle the recruit7. Get feedback to the team as soon as possible8. Just do it

By making research easy to do regularly and turning around analysis fast, user research is now a key part of how we make product and strategic business decisions.

Your responsibility as people designing products and services is to step out of your own perspective and immerse yourself in the world of the people you are designing for.

The problem is out there, not at your desk.

Jo Packer@jujupacker

jo@songkick.comjujupacker@googlemail.com

Thanks

Links to people who have inspired my approach to making productsFlow Interactive - UCDLUXR - Lean UXJeff Gothelf and Josh Seiden - Lean UXMarty Cagan - Silicon Valley Product GroupTom Chi - Experience Lead at Google XJeff Paton - Dual track ScrumJake Knapp - Design StaffEric Reis - Lean start upSteve Blank - Lean start up