Design of Experimentsfor Research and Developmentfor Research and Development

Prepared for

Midwest SCCMidwest SCCbyCarr Consulting September 8 2009Carr Consulting September 8, 2009

OutlineOutline

What is DOE?St t f D i d E i t Structure of Designed Experiments

Examples– Factorial Experimentsp– Screening Studies– Optimization Studies– Mixture ExperimentsMixture Experiments

Summary

Carr Consulting 2

What is Design of Experiments?What is Design of Experiments? Design of Experiments (DOE) is: Design of Experiments (DOE) is:

– an efficient, systematic approach» Minimum number of runs to get an answer

R it t k i d» Resource commitment known in advance» Controllable level of precision

– to study the impacts of multiple, controllable factors» Ingredient types and levels» Process conditions» Packaging characteristics

– on key measures of product quality» Objective: Physical/chemical/sensory» Subjective: Consumer acceptance

Carr Consulting 3

» One-at-a-time or simultaneously

When to Use DOE?When to Use DOE?

EARLY -- Screening Studies EARLY -- Screening Studies.– So many variables, so little time.

MIDDLE -- Factorial Experiments.– It’s never that simple.

LATE -- Optimization Studies.Just tell me what’s the best and why?– Just tell me what s the best and why?

Carr Consulting 4

Why Use DOE?Why Use DOE?

Efficiency– Minimum number of samples to get the answer.p g

Sensitivity– “Hidden Replications” add power at no extra cost.

Robust Findings– Effects of each variable are assessed at multiple

levels of all other variables.

Carr Consulting 5

DOE ExamplesDOE Examples

Factorial Experiments Screening Studies Optimization Studies Mixture Experiments

Carr Consulting 6

F t i l E i tFactorial Experiments

ObjectiveObjective

OBJECTIVE: Assess the impact of four production variables on consumers’ acceptance of a shampoovariables on consumers acceptance of a shampoo.– Silicone Type (Type A or Type B)– Silicone Level (0.1% or 2.0%)– Pearlizer (Without or With)– Polymer Level (0.1% vs. 1.0%)

Carr Consulting 8

Technical ApproachTechnical Approach

A factorial experiment comprised of the 16 p ppossible combinations of factor levels was developed to assess the impact of the four formula variablesformula variables.

400 consumers evaluated four of the sixteen samples in a Home Use Test using a BIBD.

Carr Consulting 9

Sixteen Experimental RunsSixteen Experimental Runs

Run Silicone Type Silicone Level Pearlizer Polymer Level OVR Liking1 B 2.0 Without 0.1 812 A 0.1 Without 0.1 583 B 0 1 Without 0 1 503 B 0.1 Without 0.1 504 B 0.1 With 1.0 595 A 2.0 With 1.0 856 A 0.1 With 0.1 587 A 0.1 With 1.0 688 A 2.0 Without 0.1 908 A 2.0 Without 0.1 909 A 0.1 Without 1.0 7110 A 2.0 Without 1.0 9111 B 0.1 With 0.1 5112 B 2.0 With 1.0 7613 B 2.0 Without 1.0 8113 B 2.0 Without 1.0 8114 B 2.0 With 0.1 7815 A 2.0 With 0.1 8516 B 0.1 Without 1.0 62

Carr Consulting 10

Selecting Effects for the ModelSelecting Effects for the Model

One option for fitting the factorial model is to use a One option for fitting the factorial model is to use a probability plot to identify potentially significant effects.

Design-Expert® SoftwareOVR Liking

Shapiro-Wilk testW-value = 0.949p-value = 0.654A: Silicone Type

Half-Normal Plot

ab 95

99

B

Points that fall off the line, high and A: Silicone Type

B: Silicone LevelC: PearlizerD: Polymer Level

Positive Effects Negative Effects

f-N

orm

al %

Pro

ba

70

80

90

95

A

CDBD

line, high and to the right are likely to be significant

Hal

f

0102030

50be significant.

Carr Consulting 11|Standardized Effect|

0.00 5.98 11.96 17.94 23.92

Analysis of Variance Confirms th Si ifi f th Eff tthe Significance of the Effects

Source dfSum of

SquaresMean

Square F-Value P-Valueq qTotal 15 2852.36 A-Silicone Type 1 281.25 281.25 147.5 < 0.0001 B-Silicone Level 1 2289.20 2289.20 1200.3 < 0.0001

C Pearlizer 1 35 70 35 70 18 7 0 0015 C-Pearlizer 1 35.70 35.70 18.7 0.0015 D-Polymer Level 1 106.06 106.06 55.6 < 0.0001 BD 1 121.08 121.08 63.5 < 0.0001Residual 10 19.07 1.91

Carr Consulting 12

Significant Main EffectsSignificant Main Effects

90 90

Ove

rall

Liki

ng

70

80

Ove

rall

Liki

ng

70

80

O

A B

50

60

Without With

50

60

Silicone Type A is more well liked

Without Pearlizer is more well liked (slightly)

Silicone Type Pearlizer

Carr Consulting 13

more well liked. well liked (slightly).

Significant Main EffectsSignificant Main Effects

80

90

80

90

60

70

80

Ove

rall

Liki

ng

60

70

80

Ove

rall

Liki

ng

0.10 0.57 1.05 1.52 2.00

50

60

0.10 0.33 0.55 0.78 1.00

50

60

2.0% Silicone is more well liked than 0 1%

1.0% Polymer is more ll lik d th 0 1%

Silicone Level Polymer Level

Carr Consulting 14

well liked than 0.1%. well liked than 0.1%.

Significant Silicone-by-Polymer InteractionInteraction

2.0% Silicone is more well liked than 0 1% Sili

91.00

0.1% Silicone. At 2.0% silicone,

polymer level does 1.0% Polymer

Liki

ng70 50

80.75

p ynot matter.

At 0.1% Silicone, 1 0% Polymer is

Ove

rall

60.25

70.50

0.1% Polymer1.0% Polymer is more well liked than 0.1% Polymer

0 10 0 57 1 05 1 52 2 00

50.00

Carr Consulting 15

0.10 0.57 1.05 1.52 2.00

Silicone Level

Results & RecommendationsResults & Recommendations

Most well liked shampoo is made at:– Silicone Type A– Silicone Type A– High Silicone Level (2.0 %),– Without Pearlizer, and

Either low or high Polymer (0 1% or 1 0%)– Either low or high Polymer (0.1% or 1.0%). Confirmatory study including both

polymer levels should be conducted.Wh i th dd d t f 1 0%– Why incur the added cost of 1.0% polymer if you do not need to?

Carr Consulting 16

S i E i tScreening Experiments

Screening ExperimentsScreening Experiments

Early stages of researchM f t Many factors

Unknown effects Looking for factors with big effects Looking for factors with big effects Save resources for fine-tuning

experiments run later

Carr Consulting 18

How Screening Studies Save RunsHow Screening Studies Save Runs

F Fi d Si Four-way, Five-way and Six-way interactions are almost always impossible to interpret.Wh i t i

Main Effects and Interactions in a 26 Factorial Experiment

Why invest resources in your experimental designs to be able to estimate effects that you will not be able to understand?

EFFECTNumber of

EffectsNumber of

Runs NeededIntercept 1Main Effects 6 22not be able to understand?

Run a specially selected subset of the full factorial to save resources but still be able to

2-way Interactions 153-way Interactions 204-way Interactions 15 425-way Interactions 66-way Interactions 1resources but still be able to

study the effects that you are interested in.

6 way Interactions 1

Carr Consulting 19

ObjectiveObjective

Assess the impact of six production variables on consumers’ impressions of a sweet snack.p– Diameter (Small vs. Large)– Mouthfeel Ingredient (0.0% vs. 0.4%)

Moisture (2 25% vs 4 50%)– Moisture (2.25% vs. 4.50%)– Particulates (0.0% vs. 0.5%)– Color (3.5% vs. 7.0%)– Mold Position (Open vs. Closed)

Carr Consulting 20

Technical ApproachTechnical Approach

A statistically design variable-screening study comprised of eight experimental sample was p g p pdeveloped to assess the impact of the six production variables

108 consumers evaluated all eight samples 108 consumers evaluated all eight samples. Statistical analyses were conducted to measure

the impact of each product variable on all p pconsumer responses.

Carr Consulting 21

The Experimental DesignThe Experimental Design For each factor, both levels are replicated four times.p Within each level of one factor, the levels of the other

factors are changing. The effect of a factor is not connected to an arbitrary set of initial conditions.connected to an arbitrary set of initial conditions.

Run DiameterMouthfeel Ingredient Moisture Particulates Color

Mold Position

1 S ll 0 4 2 25 0 0 7 0 O1 Small 0.4 2.25 0.0 7.0 Open2 Small 0.0 2.25 0.5 7.0 Closed3 Large 0.4 2.25 0.5 3.5 Open4 Large 0.4 4.50 0.5 7.0 Closed5 Small 0.4 4.50 0.0 3.5 Closed6 Small 0.0 4.50 0.5 3.5 Open7 Large 0.0 2.25 0.0 3.5 Closed8 Large 0.0 4.50 0.0 7.0 Open

Carr Consulting 22

Alternative One at a Time StudyAlternative One-at-a-Time Study Requires one less run but provides only q p y

one comparison for each factor.– Screening Study provides four.

Also, comparisons may be influenced by h i f h b li di ichoice of the baseline conditions.

M thf l M ldRun Diameter

Mouthfeel Ingredient Moisture Particulates Color

Mold Position

1 Small 0.4 2.25 0.0 7.0 Open2 Large 0.4 2.25 0.0 7.0 Open3 Small 0.0 2.25 0.0 7.0 Open4 Small 0.4 4.50 0.0 7.0 Open5 Small 0.4 2.25 0.5 7.0 Open6 Small 0.4 2.25 0.0 3.5 Open7 Small 0.4 2.25 0.0 7.0 Closed

Carr Consulting 23

Analysis is Simple and DirectAnalysis is Simple and DirectColor Liking Size Likingg

babi

lity

95

99

g

abili

ty

95

99

rmal

% P

rob

7080

90

mal

% P

rob

7080

90

95A

F

Hal

f-Nor

0102030

50

Hal

f-Nor

0102030

50

Some responses clearly Other responses clearly|Standardized Effect|

0.00 0.13 0.26 0.39 0.53

|Standardized Effect|0.00 0.48 0.95 1.43 1.90

Carr Consulting 24

Some responses clearly have no significant effects.

Other responses clearly have significant effects.

Results Impact of FactorsResults - Impact of FactorsResponse Diameter

Mouthfeel Ingredient Moisture

Parti-culates Color

Mold Positionp

Acceptance Color Size + - Thickness + - APP Coatingg OVR Appear + - OVR Liking - - - PI - - Flavor

Mouthfeel - - Mouthfeel - - Crispness - - Crunchiness - -Intensity Color + +

Si + Size + - Thick - - Amt Coating + Fruity Flavor Creaminess

Carr Consulting 25

Crispness + + Crunchiness + +

ResultsResults

M ld P i i h d i Mold Position had greatest impact.– Closed is superior to open.

» More well liked overall.» More well liked in key attributes» More well liked in key attributes.

Diameter and Finished Moisture also important.– Larger diameter more well liked for size, thickness and

overall appearanceoverall appearance.– Smaller diameter more well liked overall, for purchase

intent and mouthfeel.– Low finished moisture more well liked for crispness,

hi d llcrunchiness and overall. Mouthfeel Ingredient, Particulates and Color are

not important.

Carr Consulting 26

O ti i ti D iOptimization Designs

Optimization StudiesOptimization Studies

Response Surface Methodology (RSM).– Designed regression analysis.g g y– Built on simple factorial experiments.– Predict responses at points between those

run in the studyrun in the study.» The response surface.

Mixture Experiments– An RSM study in which the levels of all

variables have to sum to a constant.

Carr Consulting 28

RSM ExampleRSM Example

PROJECT OBJECTIVEDetermine the formula for an orange flavored– Determine the formula for an orange flavored beverage with the current flavor system that has the greatest acceptability among consumers.

RESEARCH OBJECTIVE RESEARCH OBJECTIVE– Model the impact of sweetener, acid and flavor

levels on the overall liking for the product in order to d t i th l l th t i ld th t ll lik ddetermine the levels that yield the most well liked product.

Carr Consulting 29

ApproachApproach

Statistically Designed Study– Systematic changes to the levels of sweetener,

acid and flavoracid and flavor.– 15 experimental samples.

Designed Consumer Acceptance Test– Each respondent evaluates 3 of the 15 samples

in a balanced incomplete block design (BIBD).

Analysis y– Link formula and acceptability.– Identify the most well liked formula.

Carr Consulting 30

Three Factor RSMThree-Factor RSM

Carr Consulting 31

Samples and Data

S t A id Fl Liki

Samples and Data

Sweetener Acid Flavor Liking66.2 66.2 66.2 7.266.2 66.2 151 7.066.2 151 66.2 8.566.2 151 66.2 8.566.2 151 151 8.1151 66.2 66.2 9.2151 66.2 151 9.41 1 1 1 66 2 9 8151 151 66.2 9.8151 151 151 10.950 100 100 7.4200 100 100 8.7200 100 100 8.7100 50 100 8.0100 200 100 8.7100 100 50 10.4

Carr Consulting 32

100 100 200 9.9100 100 100 9.3

RSM ModelRSM Model

The Link between Formula and Acceptance

2

2

2

3.02)Swt(LOG63.0

3.02)Swt(LOG78.089.9Liking

2

3.02)Acid(LOG53.0

3.02)Acid(LOG42.0

Response SurfaceResponse Surface

Liking

10 5

8.5

10.5

200%6.5

100%

200%

Acid100%200%

4.5

Carr Consulting 34

50%50%

100%00%

Sweetener

Conto PlotContour Plot

Liking200ci

d

100 10130

Ac 100

88.5

99.5

Sweetener50 100 200

508

150

Carr Consulting 35

Sweetener

ResultsResults

Overall Liking is maximized with a 50% increase Overall Liking is maximized with a 50% increase in sweetener and a 30% increase in acid.

Flavor level has no significant impact on liking.

Carr Consulting 36

SummarySummary

Statistical DOE Statistical DOE– Easy to use.– Applicable at all stages.– Applicable to all research.– Efficient, powerful & rich in information

Carr Consulting 37

Mi t E i tMixture Experiments

Mixture ExperimentMixture Experiment

OBJECTIVE:– Determine the relative proportions of threeDetermine the relative proportions of three

components in a blend that deliver a desired set of sensory properties.

RESEARCH OBJECTIVE:– Model the sensory properties of the product

as a function of its composition. Identify the region of blend ratios within which all action t d d ti fi d i lt lstandards are satisfied simultaneously.

Carr Consulting 39

ApproachApproach

Seven experimental samples:– Three “Pure Blends”. – Three 50:50 Blends.– One 33:33:33 Blend.

Evaluated by 175 assessors using a 7-Pick-4 Evaluated by 175 assessors using a 7 Pick 4 BIBD (yields 100 evaluations/sample).

Sensory responses modeled using proportions of the components as predictorsof the components as predictors.

Carr Consulting 40

Three Component Mixture DesignThree Component Mixture Design

X1

X2 X3

Carr Consulting 41

Samples and DataSamples and Data

X1 X2 X3 Liking100 0 0 5.9

0 100 0 4.70 0 100 4.2

50 50 0 5 850 50 0 5.850 0 50 4.90 50 50 5.6

33 33 33 5 533 33 33 5.5

Carr Consulting 42

Mixture ModelMixture Model

Predictive Model Relating Overall Liking to Blend

Liki 5 9*X1Liking = 5.9*X1+ 4.7*X2+ 4 2*X3+ 4.2 X3+ 1.9*X1*X2- 0.7*X1*X30 3+ 4.5*X2*X3

Carr Consulting 43R2=0.98

Contour Plot of Overall LikingContour Plot of Overall Liking

X1 100DESIGN EXPERT PlotActual Components:

X1=100

Actual Components:X1 = X1X2 = X2X3 = X3

5.75

X2=0X3=0

5.05.5

X2=0X3=0

4.55.0

5.5

X3=100X2=100 X1=0Carr Consulting 44

X3 100X1=0

Optimized Blend RegionOptimized Blend RegionX1=100

DESIGN EXPERT PlotActual Components:X1 = X1

X1 100

X1 = X1X2 = X2X3 = X3

Linger: 3.25 X2=0X3=0Liking: 5.5

g

Flavor Str.: 2.75

Bitterness: 2

X2=0X3 0

O l Pl t

Liking: 5.5

X3=100X1=0X2=100

Carr Consulting 45

Overlay Plot

SummarySummary

Statistical DOE Statistical DOE– Easy to use.– Applicable at all stages.– Applicable to all research.– Efficient, powerful & rich in information

Carr Consulting 46