Pruebas cualitativas de consumidores para la innovacion · ORDER OF PRESENTATION IN THE SESSION 1st...

transcript

Pruebas cualitativas de consumidores para la innovacion

Jean-Xavier Guinard

University of California, Davis

jxguinard@ucdavis.edu

Outline

• Consumer research model

• Consumer testing methodology – an overview, including…• 9-point hedonic scale• Penalty analysis (DOL vs. JAR attributes)• Preference mapping• Focus groups• Projective techniques• Ethnography

• Consumer-driven optimization and innovation methodologies• Hedonext• Quali-Quanti Method• Repertory Grid Method• Check-All-That-Apply (CATA)• Means End Chain Analysis• Empathic Design• Design Thinking

Consumer Research Model

Product Variables•Sensory properties•Functionalities•Marketing mix•…

Consumer Variables•Demographics•Psychographics•Genetics & physiology•…

Context Variables•Physical context•Convenience/effort•Societal pressures•…

Consumption Behavior•Preferences (likes & dislikes)•Choice•Purchase/Repeat purchase•Consumption•Pleasure/SatietyCulture

A multivariate problem requires:•Multivariate designs and statistics•Qualitative and quantitative approaches

Guinard; 2010

New Product Development

• Innovation• “Me-too”

• “Me-better”

• Breakthrough Innovation

The Tools of Consumer Testing

• Quantitative tools

• Qualitative methods

Quantitative Testing

• Screener

• Scorecard or questionnaire

• Exit survey

The Tools of Consumer Testing

• Sensory variables• 9-point hedonic scale (for overall degree of liking, and

liking of specific attributes)

• Liking scale (0=not at all to 10=very much)

• Just right scale (too low - just right - too high)

• Purchase intent (5 points) (purchase intent)

The Tools of Consumer TestingHedonic & Purchase Intent Scales

Just About Right (JAR) Scales

Penalty Analysis of Caramel Chew

20 30 40 50 60 70

% of panelists

COLOR: Too Dark

OVERALL FLAVOR: Too Weak

OVERALL FLAVOR: Too Strong

CARAMEL FLAVOR: Too WeakCARAMEL FLAVOR: Too Strong

SWEETNESS: Not Sweet Enough

FIRMNESS: Too Firm

Other Tools for Consumer Testing

• Preference Tests• Paired-preference test (binomial statistics; two-tailed

probabilities)

• No-preference option?• If the no-preference option is used, analyze with chi-squared

• Preference ranking

Other Scales Used in Consumer Tests

• Frequency (frequency of use)

• Agree - disagree (5 points) or true / false (attitudes)

• Level of importance (5 points) (expectations)

• Pricing scales (price)

• Associations brand - attributes

First Order, Carry Over and Contrast Effects• First-order effect: the 1st sample gets a higher hedonic

rating• Use ‘dummy’ or ‘warm-up’ sample

• Use fully randomized design

• Be aware of carry-over and contrast effects• Randomize

ORDER OF PRESENTATION IN THE SESSION

1st 2nd 3rd 4th 5th 6th 7th 8th

Condition 1

5.93 5.30 5.47 5.77 5.65 5.49 5.53 5.40

Condition 2

5.79 5.60 5.45 5.58 5.45 5.51 5.62 5.63

Exit Survey

• Focuses on consumer variables

• Critical to any consumer segmentation piece, i.e., it is used in preference mapping to characterize the preference segments

• Typical elements:• Demographics (age, gender, income, occupation, etc.)

• Usage (“How often do you consume…?” “Which brand(s) do you buy?” Etc.)

• Attitudes and beliefs

• Psychographics (“Do you prefer aisle or window”? Etc.)

Preference Mapping

• The purpose of preference mapping techniques is to:

1. Understand market segmentation

2. Identify drivers of liking for the uncovered preference segments

3. Characterize the preference segments in terms of demographics, psychographics and usage

Preference Mapping

• For a set of products representative of the segment under investigation, collect hedonic ratings by consumers (and information about the consumers) and descriptive ratings by a trained panel and/or instrumental measurements.

• Conduct preference clustering, and internal and external preference mapping analyses of the data.

The Data Matrix

ConsumersP

s Hedonic Ratings

9-point Hedonic Scale or Other Liking Scale

Preference Clustering

• Cluster analysis of the matrix of consumer hedonic ratings across products.

• Identifies segments in the consumer population (based on similarity of their likes and dislikes).

Dendrogram for Overall Liking (Actual)

Average Distance Between Clusters

Example:How many clusters?

Internal Preference Mapping

• Principal component analysis of the matrix of consumer hedonic ratings across the products (based on the covariance matrix).

• The outcomes are biplots of the consumers and of the products.

Olive Oil Drivers of liking: hedonic ratings by consumers vs. sensory attributes from descriptive analysis

Dimension 1 (29.65%)

Preference Mapping - EVOOs

3 preference clusters

Olive Oil Preference Segments

Dimension 1 (29.65%)

Cluster 1 N=33Butter, ripe fruit, grassy, spicy. Balance group tend to like a broad number of products

Cluster 2 N=48Negative drivers of liking: bitter, pungent, astringency and spicy.Attributes liked: butter, winey, fusty, rancid.Preferred oils: I2, I3, U11

Cluster 3 N=29Negative drivers of liking: mint, herbs, tropical fruit.Attributes liked: nutty, tea, ripe fruitPreferred Oils: C2, U9, U1, U4, I4, I3

Melons – Preference Mapping

-1.0 -0.5 0.0 0.5 1.0

PC1 (17.31%)

G1 (N=26)

G2 (N=101)

G3 (N=44)acclaim

aphrodite

caldeo

caldeoDF

carigold

globstar

goldex

magellan

proteo

proteoDF

rml0499

saguaro

sol-real

-15 -10 -5 0 5 10 15

F1 (17.31 %)

Observations (axes F1 and F2: 29.67 %)

Melons – Preference Mapping

Acclaim

Aphrodite

Caldeo

CaldeoDF

CariGold

GlobstarGoldEx

Magellan

PrimoProteo

ProteoDF

rml0499

Saguaro

Solreal

-10 -5 0 5 10

(a) Overall liking Preference map (N=171)

Series1

Series5

Series9

Series13

Series17

Series21

Series25

Series29

Series33

Series37

Series41

Series45

Series49

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58

(a) Overall liking: Contour plot

90%-100%

80%-90%

70%-80%

60%-70%

50%-60%

40%-50%

30%-40%

20%-30%

10%-20%

0%-10%

Qualitative Methods

• Focus groups

• Projective techniques (word associations, psycho-drawing, role playing)

• Individual interviews (Means End Chain Analysis)

• Ethnography

Focus Groups

• 7-10 consumers per group

• 3 to 4 groups

• Professional moderator

• Group of observers

• 2 hours max.

• Script (7-10 key questions)

Focus Groups

• Requires a professional moderator who can:• Manage group dynamics

• Keep his/her personal opinions in check

• Data analysis is difficult

• 3 to 4 groups are required

• Recruitment is complex

• Significant cost

• Time-consuming

Focus Groups

• Stimulate and accesses interactions among participants

• The moderator can probe for more information on critical aspects

• Valid and credible (the information comes from the consumer)

• Gives access to many consumers in a limited amount of time

Projective Techniques

• Word associations• “If I say __________ , what is the first

word that comes to your mind?”

• Psycho-drawing• “Please draw how you feel after using

this product/service.”

• Role playing• “Pretend you are shopping with your

partner and approaching the organic produce section... Start talking.”

Projective Techniques

Ethnography - Direct Observation

• Observation of the consumer in his/her natural environment and in context

• Product-consumer interaction research

• Professional ethnographers or self-reporting

• Video, phone camera, Skype, Zoom (optional)

• Netnography (digital ethnography)

Product-Consumer Interaction research

• Unilever have placed video cameras in kitchens to observe the tea making process

… and video cameras in showers to observe the shower process to develop shampoo and shower gels

Inquiry versus Observation:What’s Different?

Inquiry

1. People can’t ask for what they don’t know is technically possible.

2. People are generally highly unreliable reporters of their own behavior.

Observation

1. Well-chosen observers have deep knowledge of corporate capabilities, including the extent of the company’s technical expertise.

2. Observers only rely on real actions rather than reported behavior.

Source: Leonard & Rayport, Harvard Bus. Rev., Nov/Dec 1997

Inquiry

3. People tend to give answers they think are expected or desired.

Observation

3. People are not asked to respond to verbal stimuli; they give non-verbal cues of their feelings and responses through body language, in addition to spontaneous, unsolicited comments.

Inquiry

4. People are less likely to recall their feelings about intangible characteristics of products and services when they aren’t in the process of using them.

Observation

4. Using the actual product or a prototype, or engaging in the actual activity for which an innovation is being designed, stimulates comments about intangibles as smells or emotions associated with the product’s use.

Inquiry

5. People’s imaginations -and hence their desires -are bounded by their experience; they accept inadequacies and deficiencies in their environment as normal.

6. Questions are often biased and reflect inquirers’ unrecognized assumptions.

Observation5. Trained, technically

sophisticated observers can see solutions to unarticulated needs.

6. Observation is open ended and varied; trained observers tend to cancel out one another’s observational biases.

Inquiry

7. Questioning interrupts the usual flow of people’s natural activity.

8. Questioning stifles opportunities for users to suggest innovations.

Observation

7. Observation, while almost never totally unobtrusive, interrupts normal activities less than questioning does.

8. Observers in the field often identify user innovations that can be duplicated and improved for the rest of the market.

What We Learn from Observation

• Triggers of use

• Interactions with the user’s environment

• User customization

• Intangible attributes of the product

• Unarticulated user needs

The Repertory Grid Method

• The purpose is to uncover the language consumers use to describe products, as a means of understanding how they perceive, and feel about them.

• Three steps:• Language generation through triadic elicitation

• Scaling of own concepts by each consumer

• Data analyzed by Procrustes Analysis

Triadic Elicitation

Check-All-That-Apply (CATA)

• Consumers, are asked to check all the attributes that apply in the description of the sensory attributes of each product from a pre-established list of attributes

• Frequencies of selection of each attribute are computed and analyzed with multivariate statistics

• Does not provide a measure of intensities but adequately characterizes the sensory features of a set of products

• Analysis: • Correspondence analysis (CA)

CATA Question Example

• CATA attributes• Randomize order (consider pros and cons)

• 10-17 CATA attributes is optimal

• Unimodal vs bimodal vs multimodal

CATA and Correspondence Analysis

Comparison of Descriptive Analysis (PCA) and CATA (Correspondence Analysis) Sensory Maps for Coffees

Hedonext®

• New method invented by Jean-Marc Sieffermann (AgroParisTech)

• Designed for the evaluation of the acceptance/liking of consumer products

• Sequential evaluation of a set of products with consumer electing to stop when he/she has found a product to their liking and taking product home as reward

• Product must have adequate reward value

• Research collaboration with J.-M. Sieffermann and Eleonore Loescher(LVMH) on olive oils

How do we check whether a new consumer testing method is valid, reliable and viable?• We compare it to the standard in the field – in this case, the 9-point

hedonic scale (‘Hedonic’)…

• …with a ‘premium’ food product: 8 extra virgin olive oils• From California, Italy and Spain

• Price range $8-$36; average price $15

1 2 3 4 5 6 7 8 9Dislike

ExtremelyDislike

Very MuchDislike

ModeratelyDislikeSlightly

Neither LikeNor Dislike

LikeSlightly

LikeModerately

LikeVery Much

LikeExtremely

Extra Virgin Olive Oils

• Bertolli (Imported Italian blend)

• California Olive Ranch (California)

• Castillo de Canena (Spain)

• Colavita (Imported Italian blend)

• McEvoy (California)

• Cobram Estate (California)

• Kirkland (Costco brand) (Imported Italian blend)

• Villa Campestri (Italy)

Hedonext® vs. Hedonic• Northern California consumers, users and likers of olive oil

• Hedonext®• 139 consumers• Mutually Orthogonal Latin Squares (MOLS) design (block of 112 consumers) [Wakeling &

MacFie, 1995]• Chosen oil given as reward• Measure of liking/preference: number of selections for each oil

• Hedonic• 141 consumers, same demographics (age, gender, ethnicity, education, income) and olive oil

usage as Hedonic population• 9-point hedonic scale, overall liking• MOLS design (block of 112 consumers) – same as for Hedonext• Cobram Estate EVOO (California) given as reward• Measures of liking/preference: Mean degree of liking (ANOVA & Fisher’s LSD), preference

mapping (PCA) and preference clustering (CA)

Sensory Theater, Robert Mondavi Institute, UC Davis

Welcome to the Olive Oil Tasting!

Please, read carefully the instructions of this test.

• You will evaluate and give us your opinion on a maximum of 8 extra virgin olive oils.

• You will have 1 olive oil sample at a time.

• Make sure the 3-digit code on your sample matches the code in your score sheet.

• Look, smell, taste and feel the sample. Then, indicate your opinion about the olive oil.

• After tasting an oil, if you want to stop raise the RED sign. If you want to try another oil, raise the GREEN sign.

• After the olive oil evaluation(s), please, complete the Exit Survey.

• When you are done, get the Extra Virgin Olive Oil Gift you have chosen with our experimenters.

[ ] I like this olive oil and do not wish to try any other. I understand that this is the oil I will have as a reward.[ ] I want to continue and try another olive oil.

Similar rankings between Hedonext® and Hedonic

• 3 top chosen oils (#8, 4 & 1), with oil #8 as the clear winner

• 3 oils tied in the middle (#3, 5 & 6)

• 2 oils at the bottom (#7 & 2)

Hedonext®Number of selections

HedonicMean liking on 9-point hedonic scale

Oil # DOL (9-point) LSD (p<0.05)

8 6.69 a

6 6.37 ab

1 6.14 b

4 6.01 b

3 5.61 c

2 5.23 cd

5 5.04 d

7 4.90 d

Selection Position

• Most frequent selection (28%) @ position #8 (last oil tasted) – unusual for Hedonext®

• Could be due to the nature (CLT) and brevity (under 20 minutes) of the test [vs. HUT]

• Very few selections (<12%) @ positions #1-3 – typical for Hedonext®• Also typical for Hedonext®, peak half-way (@ positions 4, 5 & 6) - 44% of

selections

Olive Oil Selection (First 7 and First 6)

• Same outcome to what we saw when all 8 positions were considered

What did you think about this tasting protocol?

Wordcloud based on the frequency. The bigger the word the more quoted it was.

Most quoted :• Like/liked• One (the one)• Taste/tasting• Think

Other remarkable words:• Best/prefer• Game• Choice/chosen/choose• decision• risk

Verbatims Hedonext® - Comments (1/2)

• It was interesting because I had to take a gamble between if I wanted to tryanother, or stick to the one I liked. (C6)

• It felt like a psychology test. I really wanted to keep tasting but I didn't want to bypass really delicious oils. I ended up going for one that really stuck out. (C17)

• It was interesting to have to make a decision on whether or not to continue, without tasting all the samples. I had to think critically about how much I liked the oil. (C44)

• I feel strong feelings of regret. It is a hard balance between wanting to choose oneand sticking with it rather than seeing the rest. (C72)

• I felt like stopping at a favorite made me more critical. (C83)• I liked it! It made me really think about each choice and evaluate how I liked the

flavors independent of the samples. Made me think about my preferences versus sample to sample. Interesting protocol! ( C116)

• Interesting experience--the element of risk adds to it, makes the decision more high stakes which forces you to think about it more. (C121)

Verbatims Hedonext® – ‘Game’ Aspect

• It is interesting. Reminds me of a TV game show. (C46)• The protocol is like a game show--risk/reward equation. Odd given the premise which I

(mis)understood to be giving opinions about olive oils. (C52)• Unorthodox? Very unsure if this is the best I would like or if I passed up on a bottle I

would like. It was like a game to me. (C66)• It was a bit of a gamble though, since I was unsure how I would feel about future

samples and knew I would get to take one home. ( C79)• It was fine. Had the extra catch that you can't go back to like a previous one after you

have moved on which made it a bit of a guessing game if I would like a later onebetter than the current one. I would have liked a side-by-side comparison to better evaluate them. (C95)

Can We Do Preference Segmentation with Hedonext®?

• Yes!• It requires a product sensory map from:

• Descriptive analysis with experts,

• CATA with consumers, or

• List-All-That-Apply (LATA) with experts – what we used in this study

Sensory Map

• Expert judges from the Los Angeles Olive Oil Competition rated the quality of the oils on a 100-point scale and provided sensory descriptors of the oils (List-All-That-Apply)

• Outcomes

• Mean quality ratings

• MFA of LATA ratings

Preference Segmentation with Hedonext®• Start with 1 preference segment for each chosen product in the set

(i.e., 8 preference segments in this study)

• Combine segments of consumers who chose similar products from the sensory map

• We compiled 4 preference segments - consumers who chose:• Oils #1 or 8 (n=52)

• Oils #3, 4 or 6 (n=52)

• Oil #5 (n=15)

• Oils #2 or 7 (n=20)

• Similar segmentation solution to preference clustering (CA) and preference mapping (PCA) solutions from the Hedonic method

Preference Segmentation with Hedonic

Hedonext® Summary• Hedonext® works!

• Similar outcomes to the 9-point hedonic scale• Hedonext® product selections were consistent with mean hedonic ratings on the 9-point hedonic

scale• Preference segmentation outcomes were consistent with preference mapping outcomes from

hedonic ratings on the 9-point hedonic scale

• Most consumers enjoyed the method (interesting, game-like, element of risk)

• Caveats• Reward value needed for product under study• Full randomization and complete balance not quite possible

• To-do list• Develop and implement a statistical analysis for Hedonext® data• Determine CLT (short) vs. HUT (long) applicability• Develop a power analysis for Hedonext® – how many consumers (n) are required for a set

of p products?

Guinard, 2010

The Quali-Quanti Method

Product: 6 formulations of a frozen-fresh entrée/side dish (salmon, tuna, chicken, carrots, broccoli, mushroom)

Focus groups (week 1)

Home-use test (weeks 2 & 3)

Call-back groups (week 4)

Guinard, 2010

Focus Groups - Week 1

Four focus groups of 10 consumers each

Women, 21-60 years of age, from the Dallas area

Duration: 2 hours

Consumer habits and beliefs

Concept evaluation (concept board)

Evaluation of the product (including tasting)

Product naming

Package evaluation

Pricing

Home-Use Test: Weeks 2 & 3

Part I. Concept evaluation

5-point agree/disagree scale

“This product should be served as an entrée”

“I would serve this product to my family”

“I would order this product in a restaurant”

“This product is for working women”

“This product is high in antioxidants”

Part II. Attitudes and food habits

5-point agree/disagree scale“French cuisine is healthy”

“There are some good frozen dinners in the frozen food section of my supermarket”

“It is important to prepare a meal at home everyday”

Frequency scale“How often do you…?”

5-point importance scale (not at all/very important)

“An entrée should be made from all natural ingredients”

Part III. Product evaluation

Where, how was the product prepared and eaten?

Sensory quality 9-point hedonic scale

3-point just-right scale

Product/concept fit 5-point agree/disagree scale (same items as for

concept)

Call-back Groups - Week 4

Four groups of about 8 consumers each

Duration: 1.5 hours

Exploration of concept-product fit

User customization of the product

Evaluation of reformulated products

Guinard, 2010

Reactions to the Concept

Healthy product, nutritionally-balanced, with antioxidants

Convenient and easy to prepare

A product for working women, older adults and yuppies, rather than for teenagers or children

Would recommend the product to their friends

As an entrée or side dish, but not really as a snack or treat

A product for the family, but not quite for guests

A product for eating at work, in a cafeteria, or on an airplane, but not really in a restaurant

Guinard, 2010

Attitudes and Food Habits

French cuisine viewed as moderately healthy but good-tasting

Current frozen dinner market lacks good products.

An entrée should not have previously been frozen; but an entrée should be something that can be prepared in minutes.

Lunch is brought to the office, whereas frozen food is eaten for lunch or dinner between once a month and once a week.

Need for involvement in the preparation of a meal

Meals eaten away from home, and meals eaten with the entire family 2/3 times a week.

Results

OVERALL DEGREE OF LIKING - HOME USE TEST

123456789

PRODUCTS

Reactions to the Products

Broccoli received the highest hedonic ratings (7.7 on the 9-point hedonic scale) and tuna received the lowest (4.4).

The serving size was just right for the tuna, salmon, chicken and broccoli, but a bit too large for the carrots and mushrooms.

The innovative nature of the product was clearly felt for the salmon, broccoli and particularly the carrots and the mushrooms.

The chicken and fish products were viewed as entrees whereas the vegie products were viewed as side dishes.

Results

DEGREE OF LIKING OF APPEARANCE

123456789

Salmon

Chicke

Brocco

Carrots

Mushro

PRODUCTS

Focus Group Home Use

Results

DEGREE OF LIKING OF TASTE

123456789

Salmon

Chicke

Brocco

Carrots

Mushro

PRODUCTS

t)Focus Group Home Use

Guinard, 2010

Results

The home-use test confirmed the responses obtained during the focus groups:

Products which received high hedonic ratings in the focus groups were rated even higher in the home-use test (e.g., broccoli, mushrooms), and vice versa (e.g., tuna, chicken).

Guinard, 2010

Significance

This “Quali-Quanti” method:

Speeds up product development

Reduces biases inherent to quantitative testing protocols

Generates information regarding the concept, the product, and the goodness-of-fit between them

Accesses the beliefs and attitudes of the consumers

Uncovers potential product functionalities (e.g., health benefits, convenience of preparation, sensory pleasure)

Means End Chain Analysis

• Laddering interview technique:• “Why do you…?”

• “Why is it important that you that …?”

• Repeat

• Attributes – Consequences - Values

Means End Chain AnalysisConsumers of both imported and local oils

Empathic Design

• Observation

• Multidisciplinary team

• Harvard Business School

Empathic Design: The Process

1. Observation

2. Capturing the data

3. Reflection and analysis

4. Brainstorming for solutions

5. Developing prototypes of possible solutions

Empathic Design

• Observation• Consumer in his/her natural environment

• “Lead users”

• Team of observers: human-factors expert, product engineer, marketing expert

• Skills: open-mindedness, observational skills, curiosity

Empathic design

• Capturing data• Observation and open-ended questions (e.g., “why are

you doing that?”)

• Observers’ guide (e.g., “what problems is the user encountering?”)

• Video-cameras, notebooks, laptop computer

Empathic Design

• Reflection and analysis• Team of observers meet (with colleagues)

• Identify possible problems and needs of the consumers

Empathic Design

• Brainstorming for solutions• Transform observations into graphic, visual

representations of possible solutions

• Creative, yet disciplined process

• Defer judgement, build on ideas of others, stay focused, encourage wild ideas

• From simple conference room to virtual reality facilities

Empathic Design

• Developing prototypes of possible solutions• Physical representation of the new product concept (to

clarify concept for the developers)

• Put the concept before others

• Stimulate reaction and foster discussion with potential customers of the innovation

Empathic Design

Why does the methods work?• Teamwork, with representatives of all components of

the company

• Recognition of the heterogeneous nature of the consumer population

• Adequate statistical power

• Emphasis on observation

• Gets at unarticulated consumer needs

=> Breakthrough innovation

Design Thinking

• From Stanford’s d.school and IDEO (David Kelley)

• Consumer-based problem-solving method that also relies on observation of the consumer, and draws from empathy and experimentation

• The Design Thinking Process first defines the problem and then implements the solutions, always with the needs of the user demographic at the core of concept development.

• This process focuses on need finding, understanding, creating, thinking, and doing.

• At the core of this process is a bias towards action and creation which allows for continuous learning and improvement upon initial ideas.

Design Thinking

• The Design Thinking Process comprises 5 steps:• EMPATHIZE: Work to fully understand the experience of the user for whom you are

designing. Do this through observation, interaction, and immersing yourself in their experiences.

• DEFINE: Process and synthesize the findings from your empathy work in order to form a user point of view that you will address with your design.

• IDEATE: Explore a wide variety of possible solutions through generating a large quantity of diverse possible solutions, allowing you to step beyond the obvious and explore a range of ideas.

• PROTOTYPE: Transform your ideas into a physical form so that you can experience and interact with them and, in the process, learn and develop more empathy.

• TEST: Try out high-resolution products and use observations and feedback to refine prototypes, learn more about the user, and refine your original point of view.

Design Thinking

Virtual Reality

• Mimic the user’s environment and experience

• Technology dependent but potential savings

Consumer Research Model

Product Variables•Sensory properties•Functionalities•Marketing mix•…

Consumer Variables•Demographics•Psychographics•Genetics & physiology•…

Context Variables•Physical context•Convenience/effort•Societal pressures•…

Consumption Behavior•Preferences (likes & dislikes)•Choice•Purchase/Repeat purchase•Consumption•Pleasure/SatietyCulture

A multivariate problem requires:•Multivariate designs and statistics•Qualitative and quantitative approaches

Guinard; 2010

Pruebas cualitativas de consumidores para la innovacion · ORDER OF PRESENTATION IN THE SESSION 1st...

Documents