Survey Analytics for Behavioural Change

transcript

SURVEY ANALYTICS FOR BEHAVIOURAL CHANGE

Leong Chee TungPartner

cheetung@triskeliongroup.co

WHAT WE’LL COVER

In the age of Big Data, why use surveys?

How to construct a good survey question

How frequently should surveys be conducted

What kind of analytics are possible

with survey dataWhat behavioral levers can be addressed with

surveys

BIG DATA

More data has been transferred in the last 1 sec than all data stored in 1996

Velocity

5 exabytes (1Bn GB) are being created every day

VolumeSurveys only 1 means of data collection, wearables, internet searches, clickthroughs, etc.

Variety

Source: Big Data: The Management Revolution – Harvard Business Review Oct 2010

BIG DATA, BIG PROBLEMS

Hard to concretely test

hypothesesProcessing

power intensiveTime

consuming

Correlation & Causation

Source: http://gizmodo.com/5977989/internet-explorer-vs-murder-rate-will-be-your-favorite-chart-

OTHER BIG DATA LIMITATIONS

Multiple methods of data collection – unstable predictive power (e.g. Google

Flu Trends)

Best used: extremely common. When less

common, not as good‘Echo-chamber’ effect: eg Google Translate & Wikipedia

Misleading degree of scientific precision

WHAT’S GOOD ABOUT SURVEYS?

Relatively cheap to collect & process data

Allows representative-ness for large populations

EXTENSIVE

High degree of control in structuring data collection

FLEXIBLE

Methodological techniques are well-established, tradeoffs (e.g. margin of error) are mostly known

DEPENDABLE

GOOD SURVEY QUESTIONS

ACTIONABILITY: e.g. Net Promoter Score

• Single question with strong correlations to outcomes that matter

• What should you do to change it?

SORTING EFFECT: Differentiates positive and negative outcomes

• Bad question “I believe I am paid fairly at work”

• Good question “I have the opportunity to do what I do best everyday”

LINKED TO BUSINESS OUTCOMES

BEST PRACTICES IN SURVEY CONSTRUCTION

Avoid loaded/leading words

Pay attention to question ordering

Avoid non-specific questions (e.g.. The culture of my organisation is the right one for the business environment we are in)

No double-barreled questions

Use unbalanced scales carefully (e.g. Strongly

Disagree, Moderately Disagree, Disagree, Agree,

Strongly Agree)

Question wording simple: “Do you have a best friend at

work?”

Depends On Purpose &

Data Stability

SURVEY FREQUENCY

Effective habit building: 2 – 8 months** Source: Phillippa Lally et al, How are Habits Formed: Modelling Habit Formation in the Real World, European Journal of Social Psychology, 16 Jul 2009

POSSIBILITIES

Predictive attrition modelling Performance/Engagement Causality

POSSIBILITIES (CONT’D)

Linkage to Customer metrics Sentiment analysis

“FEELINGS ARE FACTS”

Simon Cooper, former President for the Ritz Carlton Hotel Company

BEHAVIOURAL LEVERS

Source: Gallup

NEGATIVE EXAMPLE

COGNITIVE BIASES THAT SCREW UP

YOUR DECISIONS

Anchoring bias

Availability heuristic

Bandwagon effect

Blind-spot bias

Choice-supportive bias

Clustering illusion

Confirmation bias

Conservatism bias

Information bias

Ostrich effect

Outcome bias

Overconfidence

Placebo effect

Pro-innovation bias

Recency

Salience

Selective Perception

Stereotyping

Survivorship bias

Zero-risk bias

SUMMARY

Use surveys in conjunction with other data sources for best effect

Consider purpose & underlying data stability to determine frequency

Beware of cognitive biases in change efforts

SURVEY ANALYTICS FOR BEHAVIOURAL CHANGE

Visit us at http://triskeliongroup.co

Survey Analytics for Behavioural Change

Data & Analytics