Date post: | 11-Jan-2015 |
Category: |
Documents |
Upload: | colin-millerchip |
View: | 544 times |
Download: | 2 times |
Market research without totally screwing it up
Colin Millerchip
18 April 2011
Agenda• Introduction• Before you start• Market segments• Looking for problems• General techniques• Discovery-validation cycle• Research tools (qualitative, quantitative)• Risk management• Financial modelling• Questions?
Before we start…• This presentation is about market research, primarily to
discover new product / service opportunities• Market research is not the only way to find such
opportunities– For eg, technology-driven innovation, customer funded
development• Apologies in advance if this is overly biased towards
“mass market B2B”• … and apologies that these are very verbose slides
– They’re intended to be a resource that can be used off-line
Introduction• Research isn’t a well-defined path you walk along
– Each piece of research will need its own path – If you follow a checklist, you still need to keep your brain
engaged– Discovery-validation cycle will help you find some of the
“unknown unknowns”• You’ll never get to 100% knowledge
– If you try to do this, you’re going to be way too late– There are no guarantees – deal with it!
Before you start
• What are you trying to find out?– What questions you want to answer– Who you want to ask (market segment)
• Make sure to think about the business justification– What will the people making the go/no-go decision need to
know?– For eg: What’s the channel to market? Do people tend to
spend money in this area? Can we capture the opportunity (core competence)?
• How long do you have?– Use common sense: plan the phases of your research,
track progress against plan, modify as you learn more
Market segments
(sorry, this is a bit “marketing-y”)
• What is a market segment? A group of people who…– can be reached via the same channel– respond to the same message– are in contact with each other (soft requirement?)
• What’s it for? Focus!
• How do you identify market segments?– Top-down: take the whole marketplace and slice– Bottom-up: pick a few representative customers and
build up a definition around them
Market segments (cont.)
• Good segmentation criteria– have clear boundaries– are easy to determine– are easy to estimate in the general population– don’t change much over time
• Example criteria– Geographic– Business demographic (industry, size, etc)– Adopter categories– Product usage– Purchasing approaches
Looking for problems
(sorry, this is a bit “product-y”)• Pragmatic Marketing call this “the 5th P”
– The traditional “four P’s of marketing”: product, price, place, promotion
• We’re trying to find a problem to solve for people…– … that’s sufficiently painful, pervasive, pressing (urgent), and
profitable (for us to solve)• Typically* people want to do one of 3 things: make more
money; reduce costs (time ~= money); or reduce risks. – Qu: what are you looking to do for people?
http://www.pragmaticmarketing.com/srpm
* … in B2B; this isn’t the case with B2C
General techniques
• Face-to-face meetings are invaluable– It’s costly, but you must do it – NIHITO
• Listening not talking– Don’t end up doing a pitch of your ideas
• You’ll learn more from -ve than +ve feedback– Thank them for the praise, but encourage criticism
• Don’t tell customers that they’re wrong– … even if they are– You’re researching, not selling– (if necessary, you can gentle correct their misunderstandings at the end)
General techniques (cont.)
• Contextualise– ie, understand the context of the comments– eg, their job title, level of experience, job title, size of
team, size of company, etc
• Will help you understand results– eg, testers are like X, developers are like Y
• Will help you find target market segments– eg, small companies don’t have this problem
General techniques (cont.)
• Open-ended questions• Leading questions
– Unprompted vs. prompted feedback (both are useful)
• How painful is the problem?– Frequency of use case– How ‘workaroundable’– Need vs. want
• Repeatability– Try to ask the same things, in the same way
• Document it (if you need to)– Write up customer interviews straight away– Try to record interview (dictaphone)
General techniques (cont.)
• Selection Bias– 3 segments within any market segment: customers; evaluators
(know of you, haven’t bought); prospects (don’t know about you).
• Make sure you’re not missing a major segment in your research• Much easier said than done for ‘prospects’…
– For eg, your poll shows that 100% of database users back up databases… but could it be that only database administrators answered the poll?
• People lie!– People tend to be nice to you– They love your product ideas… but won’t part with cash when
you ship the product– Qu: how can you test their propensity to spend money to solve
their problem?
Discovery-validation cycle
• Start with qualitative research– Customer interviews, open-ended, 2-way discussions– Learn more about your research topic
• Move to quantitative research– Polls/surveys, specific question / answers– Gather statistically-valid data– This phase isn’t pure validation, there’s still some discovery going on
“If you want to know what people think and do, choose quant;
If you want to know why they think and behave that way, choose qual”
Qualitative Quantitative
Root Quality (properties / characteristics) Quantity (number / magnitude)
Nature Learning Confirming (some learning)
Style Open questions Closed / specific questions
Interaction Two-way exchange One-way collection
Number of contacts Few (10’s) Many (100’s)
Effort per contact Large Small
Qualitative research tools
• Customer interviews• Usability tests• Forum research• Meet people at events (user groups, conferences, etc)• Feedback via sales, support
– careful with this…
• … etc etc…
Quantitative research tools: polls
• Normally (but not always), you want the answers to be mutually exclusive, and complete (ie, cover the total range of responses)
• Make then engaging – for eg, sacrifice perfect response data for amusement
• Yes and No isn’t enough: what about “Don’t know”, or “What’s this?” or “Other”
• Incentivise people: less critical than for surveys (minimal time commitment), sharing current results is normally enough
I’ve used:
http://www.polldaddy.com/
Quantitative research tools: surveys
• Keep the survey fairly short– Resist temptation to ask about secondary issues
• Find out about the respondent – Demographics, company, experience
• Get their permission to follow up (“if you want to enter the prize draw, or are happy to be contacted about your responses, please leave your email address here”)
• Interesting questions– “Spend 100 points”– Forced choices (“pick one”; ranking (beware this can annoy people))
• Incentivise people: freebees, prize draw, share results
I’ve used:
http://www.surveymonkey.com/
http://www.questionpro.com/
Quantitative research tools: gotcha’s
• Careful with your wording!• Test it out before going live
– Monitor initial early responses, be prepared to change survey if required (eg, loads of people select “Other” option to a question)
– Use a web re-director can be helpful here, since not all changes can be made to an active/open survey
• Think about how you’ll react to various findings– If survey response data to a question suggests X, will you believe it? Or
will you doubt that people understood the question / answered it properly?
Other research tools
• Google adwords• Alexa / compete.com traffic ranking?
– Question over whether these are biased• Forum traffic stats (ours, competitors)• Social networking sites• Customer feedback mechanisms• Usage data from your service / product (eg, download
stats; quote / invoice stats; “dial home” pings; etc)• … etc etc…
Analysts data
• Can cut down leg work• Beware of…
– “Hockey-stick” curves (especially ones that are continually adjusted)
– Correlation between vendor spend with analysts, and their ranking in analyst’s “magic quadrant”
Risk management
• You can’t wait until you know for sure• Identify the risks – technical, commercial,
personnel, etc• Try to quantify these
– Likelihood / Frequency– Consequences– Cost to mitigate
• Put in place a mitigation plan
Financial modelling• FITA* may not be enough• Financial modelling, & related jargon
– Monte Carlo simulation / PDFs (eg, RiskAMP, @RISK)– Fully loaded costs, NPV, IRR, interest rate, cash flow forecast,
revenue model, Tornado chart, spider chartAre you scared yet? :o)
• Can drive the model backwards– “What revenue do I need to achieve to get my money back
in n years?”
* Finger in the air