Adapting conjoint to the mobile phenomenon

Chicago New York London Dubai New Delhi Bangalore SingaporeSan Francisco

Making Conjoint MobileAdapting Conjoint To The Mobile Phenomenon

Chris Diener, Rajat Narang, Mohit Shant, Mukul Goyal, Hem Chander

Sawtooth Conference 2013

2

Evolution of Market Research

Paper and Pen

Personal Computer

Laptop Computer

Smartphones/Tablet

Change in platforms followed increased mobility

Over the years, Market Research has adapted to various survey platforms

3

Smartphones sales went up by 33% in 2012.

Sale of Internet enabled phones in emerging markets will reach 1.2 billion by 2015, which will add 500M new internet users.

Both India and Africa average three mobile phones for every four people.

Number of tablet users are growing every year, adding to more mobility to lifestyle.

Why Research is going Mobile

Smartphone and mobile web penetration is growing day by day

4

Dawn of the era of Mobile Research

*“Catch the Wave” paper by Steve von Bevern, Research Now

“65% of the researchers expect to allocate up to 10% of their budgets to mobile”

Respondents prefer Smartphone surveys, primarily due to its “on the go” nature

25%18%

31%

4%

18%

0%

10%

20%

30%

40%

UK US Australia Canada Total

“I have run multiple research projects via mobile”

Market Research companies are adapting to this phenomenon the world over

5

App Based SMS Based

Web Browser Based

Different ways of conducting surveys on mobile platforms

6

Customer Satisfaction Studies

Ad Value Identification

Media & Message Effectiveness

Habits and Usage Studies

Customer Service Representative (CSR)

Concept Testing

Customer Attitude and Expectation

“For some projects with extensive or complex stimuli, e.g., many discrete choice studies, survey completion on smartphone might be ill advised”- Mobile Research Risk, Burke Inc, 2013

BUT!There is discernible skepticism for conducting conjoint on mobile platform

Different types of surveys using mobile platforms

Researchers are utilizing mobile platforms to conduct different types of short surveys

7

Complexity Representation on small screen

Short attention span

Too many tasks Large number of attributes in a survey

Smart phone penetration difference

Challenges in conducting conjoint on mobile platform

8

Adapting Techniques

– Back to Basics: Tested Pairwise Ratings and Partial Profile, which are simpler to evaluate

– Less is more: Fewer attributes per screen

– Tailor Made Methodologies: New techniques tailor-made for mobile platform to increase interactivity and ease of understanding like Shortened ACBC

Reduction ofattributes

Visual Transformations

– Simplify

– Simplification of tasks, leading to optimal use of screen space

– Reduction of concepts on screen

– Minimum scrolling to view tasks

– Optimize

– Coding optimized to fit the tasks on mobile screens

Reduction of concepts on screen

Possible Resolutions

9

US India– Topic: Determine the preferred multimedia tablet configuration

– Number of attributes tested: 9

– 8 random and 2 fixed tasks per respondent

– Platform - Desktop / Laptop (referred here as PC)

- Mobile / Tablet (referred here as Mobile)

– Census representation of Sample MethodologyPlatform

PC Mobile

CBC (3 concepts per screen) 200 200

PCR (Similar to CVA) NA 200

CBC (2 concepts per screen) NA 200

Partial Profile NA 200

Shortened ACBC NA 200

*We thank IndiaSpeaks and uSamp for providing the sample in India and US respectively

Conducted a Multi Country Survey

10

Heading 24pt Bold Calibri

PCR – Pairwise comparison rating

CBC Mobile (2 concepts)

CBC Mobile(3 concepts)

Partial Profile Shortened ACBC

–Similar to CVA technique in data collection

–Estimation done in two ways– Discrete Estimation– Chip Allocation

–2 concepts per screen (excluding none)

–3 concepts per screen (excluding none)

–Similar to CBC on PC

–6 attributes evaluated perconcept

–2 concepts per screen

–Only Pre-screening and consideration set sections

–Can we go back to the basics?

– Is less better?

–What is the impact of Visual Transformation?

–Do lower number of attributes improve respondent’s ability to focus?

–How well can simpler tasks predict respondents choice?

Rat

ion

ale

to t

est

De

scri

pti

on

Evaluated the following techniques on Mobile and compared them with CBC on a PC

11

‒ 9 point rating‒ Rating

responses transformed to:‒ Discrete

Choice‒ Chip

allocationfor estimation

‒ No frills, clear display of concepts

‒ Acts as None

‒ 2 Concepts per screen

PCR – Pairwise comparison rating

12

‒ None option shown at the bottom

‒ More width per concept

‒ Higher Font Size


13

‒ Optimized Coding to fit on one screen

‒ Optimal use of real estate



14

‒ Only 6 attributes per screen

‒ Brand and price on every screen


‒ Clear display of concepts with high font size

Partial Profile

15

Pre

scre

en

ing

Ne

ar N

eig

hb

orh

oo

d C

on

cep

ts

Scre

en

ing

Task

s

Bu

ild Y

ou

r O

wn

‒ Prescreening of attributes

‒ Simplified BYO task

‒ Well defined concepts

‒ 7 attributes in a concept

Shortened ACBC

16

– Robustness of results on mobile as compared to those on PC

– Adaptation of various techniques to the mobile platform

Researcher Perspective

– Average Time Taken

– Readability

– Ease of understanding

– Enjoyability

– Encouragement to give honest opinions

– Realism of tablet configuration

Respondent Perspective

Evaluation of research from multiple viewpoints

17

Results – Researcher’s Perspective

18

Technique

CBC PC 1 1

PCR – Chip Allocation 0.44 0.57

PCR – Discrete Estimation 0.64 0.79

CBC Mobile (2 concepts) 0.77 0.75


Partial Profile 0.83 0.78

ACBC 0.79 0.71

Correlation Analysis with utilities of CBC PC

Correlation analysis shows most of the methods are similar to results of CBC PC

19

Technique

CBC PC 63.5% 58.1%

PCR – Chip Allocation 53.9% 56.5%

PCR – Discrete Estimation 64.5% 66.1%

CBC Mobile (2 concepts) 82.2% 58.0%

CBC Mobile (3 concepts) 64.8% 54.6%

Partial Profile 78.5% 67.4%

ACBC NA 79.2%

Hold Out Task Accuracy

Statistical Significant Difference from CBC PC at 95% confidence level for each country

– Hold out tasks placed in middle and end of exercise

– They were of similar format as conjoint exercise

Hold Out task prediction rates are all in “well accepted” range.

4 Methods do better in hold out task when compared with that of PC

20

Technique

CBC PC 0.09 0.10

PCR – Chip Allocation 0.27 0.29

PCR – Discrete Estimation 0.06 0.06



Partial Profile 0.05 0.09

ACBC 0.10 0.10

MAE Analysis

MAE tells a similar story with 3 methods on mobile doing better than CBC on a PC

21

Results – Respondent’s Perspective

22

3 3.33.7 3.7 3.8

4.5

3.13.6

4.9

3.74.2

5.8

0

1

2

3

4

5

6

7

CBC PC CBC Mobile (2 Concepts)

PCR CBC Mobile (3 Concepts)

Partial Profile ACBC

In M

inu

tes

US

India

Avg. Time taken to complete trade-off exercise

Respondents take more time on mobile devices, but simpler tasks are quicker, for example CBC on Mobile – 2 tasks

ACBC, takes slightly more time, even more so in India, indicating that Indian market is not that familiar with these techniques

23

82% 85% 81% 76%85%

73%64%

52% 53%

72%

56% 52%

0%

20%

40%

60%

80%

100%

CBC PC PCR CBC Mobile (2 concepts)



US

India

Readability


Readability is not an issue and respondents find the survey legible on mobile

24

Ease of Understanding

81% 79%73%

62%69% 69%73% 78% 74%

68%76% 72%

0%

20%

40%

60%

80%

100%




US

India


Ease of understanding is comparable to that of PC for simpler tasks across both countries. India scores well on ease of

understanding for all techniques

25

Enjoyability

73%

59% 55%48%

58% 53%

68%78% 75%

65%74%

80%

0%

20%

40%

60%

80%

100%




US

India


Indian respondents enjoyed the survey on mobile more than their US counterparts

26

76% 75%64% 61%

71% 73%73%81% 82%

75%87%

81%

0%

20%

40%

60%

80%

100%




US

India

Encouragement to give honest opinions


Majority felt that they were encouraged to give honest opinions for all the methods evaluated on mobile platforms

27

75% 75% 73%63%

71% 67%69%77% 75% 72% 72% 74%

0%

20%

40%

60%

80%

100%




US

India

Tablet configuration looked realistic


Tablet configuration, which was similar for all techniques, gets very similar response from all respondents

28

Implications

29

Yes it is POSSIBLE!

Conjoint surveys on mobile platform provided robust data quality

Respondents could read and answer comfortably on mobile platforms

Respondents took slightly longer to complete the surveys on mobile platforms

Respondents felt that the product combinations looked realistic to them on mobile platforms

30

By combining simple techniques with basic aesthetics!

All methods customized for mobile platform performed well. Simplicity and optimal use of screen gives great results

“3 Tasks concept” should be avoided on mobile platform as respondent perceptions are less favorable -- though accuracy parameters do not differ much

“2 Tasks concept” is the preferred layout for mobile platform. Both full profile and partial profile perform well. However, full profile is easier to understand

PCR, when done through discrete estimation, performs well. ACBC, even though it gives good accuracy, perhaps should be avoided as it takes longer and scores lower on “Ease of understanding"

31

3232

If you need help with Analytics or Research, please write to us:[email protected]@[email protected]

For Media related queries [email protected]

For all other queries [email protected]

32

HEAD OFFICE

314 Marble Arch Tower,55 Bryanston Street,

London W1H 7AA

Phone : + 44 207 868 2240

UK OFFICE

DLF Cyber City SEZ,Building#14, 4th Floor, Tower B,DLF Phase-III, Sector 24 & 25A,

Gurgaon-122002,

Phone: +91.124.4953.400

INDIA OFFICE

AbsolutData AnalyticsMiddle East JLT

Office 1604, Tower BB1Mazaya Business Avenue

Jumeirah Lake Towers

Phone: +97150-1577257

DUBAI OFFICE

1851 Harbor Bay Parkway,Suite 125, Alameda,

California, USA – 94502

Phone: +1 510 748 9922Fax: +1 510 217 2387

mailto:[email protected]





Date post:	22-Jan-2018
Category:	Technology
Upload:	absolutdata-analytics
View:	5,861 times
Download:	0 times

Adapting conjoint to the mobile phenomenon

Technology