Concept Testing

Concept Testing Approaches

Conjoint Analysis

Evaluating with Customers

Concept Testing is used to help screen and refine new

product ideas Conjoint Analysis

used to determine the combination of attributes that maximizes appeal

Purposes of Concept Testing

To identify very poor concepts so that they can be eliminated.

To estimate (at least crudely) the sales or trial rate the product would enjoy (buying intentions, early projection of market share).

To help develop the idea (e.g. make tradeoffs among attributes).

Procedure for a Concept Test Prepare concept statement Clarify specific purposes Decide format(s) Select commercialization Determine price(s) Select respondent type(s) Select response situation Define the interview Conduct trial interviews Interview, tabulate, analyze

Concept Testing

A concept is composed of attributes and benefits for a particular usage situation

Attributes incorporate a specific product form and technology

DetermineDetermineCustomerCustomer InterestInterest

see Page and Rosenbaum (1992), “Developing an Effective Concept Testing Program for Durables,” J Product Innovation Mgmt

Concept Testing Cautions and Concerns

If the prime benefit is a personal sense (aroma, taste).

If the concept involves new art and entertainment. If the concept embodies a new technology that

users cannot visualize. If concept testing is mishandled by management,

then blamed for product failure. If customers simply do not know what problems

they have.

The Concept Statement

The Customer Value Proposition: FOR {the ideal customer} WHO {have the following problem} MY PRODUCT IS A {product category} THAT {key differentiating benefit} UNLIKE {the major competitor}

UsageUsageSituationSituation CustomerCustomer

ProductProduct

The Concept Statement

Format

The Concept Statement

Format: Narrative

Mail Concept Test -- Narrative

Here is a tasty, sparkling beverage that quenches thirst, refreshes, and makes the mouth tingle with a delightful flavor blend of orange, mint, and lime.

It helps adults (and kids too) control weight by reducing the craving for sweets and between-meal snacks. And, best of all, it contains absolutely no calories.

Comes in 12-ounce cans or bottles and costs 60 cents each.

1. How different, if at all, do you think this diet soft drink would be from other available products now on the market that might be compared with it?

Very different ( ) ( ) ( ) ( ) Not at all different

2. Assuming you tried the product described above and liked it, about how often do you think you would buy it?

More than once a week ( ) ( ) ( ) ( ) ( ) ( ) Would never buy it

The Concept Statement

Format Narrative Drawing / Diagram

Mail Concept Test – Drawing / Diagram

The Concept Statement

Format Narrative Drawing / Diagram Model / Prototype Information Acceleration

Information Acceleration

http://www.wharton.upenn.edu/learning/futureview/

Developing Concepts to Test

Time to prepareTime to prepare test materialstest materials

Number of itemsNumber of items testedtested 11MoreMoreMostMost

PreferredPreferred

LeastLeastPreferredPreferred

Paper &Paper &PencilPencil

ComputerComputer

PrototypePrototype

Working ModelWorking Model

e.g., www.acupoll.com

What is generally tested?

BUYERBUYER

Does it solve Does it solve a “problem”?a “problem”?

yesIs itIs it““believable”?believable”?

yesIs itIs it““unique”?unique”?

yes

Would it be Would it be bought at onebought at oneof several testedof several testedprice points?price points?

yes

Can measure potential customer reactions using: (1) 5-pt “definitely not” - “definitely” scales(2) sorting tasks

Considerations in the Concept Test

Core Idea vs. Positioning/Commercial Statement New Brand vs. Old Brand vs. No Brand Price Picture Category Purchase Measure Decisions:

Buyer Intent Frequency Price

Product Diagnostics Attribute Diagnostics

Ask the right people...

time

SalesSalesThe ChasmThe Chasm

Early MarketEarly Market Mainstream MarketMainstream Market

TechnologyTechnologyEnthusiastsEnthusiasts

VisionariesVisionaries PragmatistsPragmatists ConservativesConservatives

See (1) Rogers (1995) Diffusion of Innovations (2) Moore (1991) Crossing the Chasm (3) www.chasmgroup.com

Lead U

sers

and In

novat

ors

vs. M

ainstr

eam

Mar

ket

Ask the right questions...

How important is the product “experience”? Does the customer have to “touch & feel”

the product to understand the benefits offered?

““Simulate” the ExperienceSimulate” the Experience

How can concepts be tested?

Focus Groups One-on-One Personal Interviews Mall Intercept Phone Interviews Postal Surveys Internet Surveys Hybrids (e.g., phone-mail-phone)

Compare in terms of:Compare in terms of:sample control, concept flexibility, costsample control, concept flexibility, cost

Compare in terms of:Compare in terms of:sample control, concept flexibility, costsample control, concept flexibility, cost

see: (1) Pope (1993), Practical Marketing Research (2) McQuarrie (1996) The Market Research Toolbox

Typical Analysis

Category or IndustryPurchase Intent Concept Norm

Definitely Would Buy 27% 20%Probably Would Buy 43 40Top Two BoxTop Two Box 70% 70% 60% 60%

Might or Might Not Buy 22% Probably Would Not Buy 5Definitely Would Not Buy 3

Sales Potential Estimation

Translating concept test results into sales estimates Assumes awareness and availability Translating “Intent” into sales potential:

Develop the “norms” carefully for a specific market and for specific launch practices

Examples: Services: 45% chance that the “definitely would buys”

actually will buy; 15% for the “probably will”s Consumer Packaged Goods: 70-80% chance that the

“definites” will buy; 33% chance for the “probably will”s

Sales Potential Estimation

Sales Potential Estimation

Translating Intent into Sales Potential Example: Aerosol Hand Cleaner

After examining norms for comparable existing products, you determine that: 90% of the “definites” 40% of the “probables” 10% of the “mights” 0% of the “probably nots” and “definitely nots”

will actually purchase the product Apply those %age to Concept Test results:

Sales Potential Estimation

Translating Intent into Sales Potential Apply those %age to Concept Test results:

90% of the “definites” (5% of sample) = .045 40% of the “probables” (36%) = .144 10% of the “mights” (33%) = .033 0% of the last 2 categories = .000

Sum them to determine the %age who would actually buy: .045+.144+.033= .22

Thus, 22% of sample population would buy(remember: this % is conditioned on awareness & availability)

From Potential to Forecast

With Sales Potential Estimates: To remove the conditions of awareness and

availability, multiply by the appropriate percentages:

If 60% of the sample will be aware (via advertising, etc.) and the product will be available in 80% of the outlets, then:

(.22) X (.60) X (.80) = .11 11% of the sample is likely to buy

Summary of Concept Testing

Advantages relatively easy to get customer input can be used as an early screen for new product ideas

Limitations not that helpful for the design and development of

specific product forms not as reliable for discontinuous

innovations

Conjoint Analysis

Primary benefit in addition to (or in lieu of) concept tests:

forces a trade-off

Conjoint Analysis

Can be used to quantify the relative importance of attributes

Can be used to help determine the combination of attributes that maximizes appeal

Relatively easy for incremental innovation Requires experts or information acceleration

for discontinuous innovations

see (1) Page and Rosenbaum (1987), “Redesigning Product Lines With Conjoint Analysis,” J Product Innovation Mgmt (2) www.sawtooth.com {Sawtooth Software}

Major Assumptions

An offering is a bundle of attributes and benefits. An offering can be decomposed into a bundle of “features” for which “utility values” can be calculated.

The utility value of an offering is some simple function of the utilities of the offering’s “feature” levels.

Customers prefer the offering with the highest utility value.

Conjoint: Steps 1 and 2 Identify Relevant Attributes

Survey/Focus Group/Intuition Salsa Example

(Thickness, Color, Spiciness)

Identify Relevant Levels of Each Attribute Thickness: Regular, Thick, Extra-Thick Color: Red, Green Spiciness: Mild, Medium-Hot, Extra Hot

Create Profiles for each Combination

3 thickness (reg., thick, extra-thick) 2 color (red, green) 3 spiciness (mild, med/hot, extra hot) Leads to 3X2X3 = 18 Profiles

Conjoint: Step 3 Choose a Sample

Considerations: Consumer Involvement Typicality Diversity (if multiple segments) Expertise (if complex or discontinuous)

Conjoint: Step 4

Obtain Customer Judgements Rank Order

Sort into categories Rank the profiles within each category

Pair-wise Comparisons Use a computer package to quickly hone in on

important attributes

Thickness Spiciness Color ActualRanking*

Ranking asEstimatedby Model

Regular Mild Red 4 4Regular Mild Green 3 3Regular Medium-Hot Red 10 10Regular Medium-Hot Green 6 8Regular Extra-Hot Red 15 16Regular Extra-Hot Green 16 15Thick Mild Red 2 2Thick Mild Green 1 1Thick Medium-Hot Red 8 6Thick Medium-Hot Green 5 5Thick Extra-Hot Red 13 13Thick Extra-Hot Green 11 11Extra-Thick Mild Red 7 7Extra-Thick Mild Green 9 9Extra-Thick Medium-Hot Red 14 14Extra-Thick Medium-Hot Green 12 12Extra-Thick Extra-Hot Red 17 18Extra-Thick Extra-Hot Green 18 17

* 1 = most preferred, 18 = least preferred.

Conjoint Analysis Input: Salsa ExampleConjoint Analysis Input: Salsa ExampleFigure 7.2

Conjoint: Step 5

Compute Individual Value Systems Use MONANOVA for rank order data

Output in the form of standardized utilities

Regular Thick Ex-Thick

UT

ILIT

Y

2

1

0

-1

-2

Mild Medium-Hot Ex-Hot Red Green

Thickness Spiciness Color

0.161 0.913 -1.074 1.667 0.105 -1.774 -0.161 0.161

Conjoint Analysis: Graphical OutputConjoint Analysis: Graphical Output

Conjoint Analysis:Relative Importance of Attributes

0 20 40 60 80 100 %

Spiciness

Thickness

Color

59.8%

34.6%

5.6%

Conjoint: Step 6

Find the average utilities (part-worths) for each attribute Intuition: Find the attribute with the biggest range in utilities

across the different levels Use graphs/calculations for importance measures Be careful with averages

Segments may exist Cluster Analysis can tell you

Let’s consider golf balls...

• • distance and durabilitydistance and durability• • durability and pricedurability and price• • distance and controldistance and control

Conjoint Analysis Average Average Price

Driving Ball Life Distance

250 yards 54 holes $3.00

220 yards 36 holes $4.00

200 yards 18 holes $5.00

Your “Optimal” Product Design

$5/sleeve

Driving Distance of 200 yards

Average Ball LifeAverage Ball Lifeof 54 holesof 54 holes

See also Titleist’s Ball-Fitting and Wilson’s Custom Fit

How can conjoint analysis be conducted?

One-on-One Personal Interviews written or verbal concept descriptions multimedia presentation of concepts

RTI’s TradeOff VR; Sawtooth’s Sensus TradeOff; MIT’s Information Acceleration networked computer facilities

Moskowitz Jacobs

Mail written concept descriptions disk by mail

Internet the future??

Summary of Conjoint Analysis

Advantages the relative importance of product features can be quantified

using customer input only need to test a relatively small number of actual product

designs

Limitations output is usually not directly linked to actual purchase