www.ifpress.com

Advances in Tetrad Testing

Sensometrics 2012

Rennes, France

John M. Ennis

The Institute for Perception

john.m.ennis@ifpress.com

Rune H.B. Christensen

Technical University of Denmark

rhbc@imm.dtu.dk

2

Discrimination Testing

Discrimination testing as important as ever:

Compliance with health initiatives

Cost reductions

Changes to ingredients, processes, packaging, handling, etc.

Quality control

Three challenges:

1. Identify sensitive methods for unspecified testing

2. Measurement:

a) Quantify sensory differences

b) Understand precision in measurement

3. Determine size of meaningful difference

3

The Tetrad Test - Methodology

Four samples presented:

Six possible presentation orders:

Guessing probability = 1/3

“Group the stimuli into two groups of

two samples based on similarity”

AABB, ABAB, ABBA

BBAA, BABA, BAAB

4

The Tetrad Test - History

Mentioned by Lockhart (1951) and Gridgeman (1954)

Revisited by O’Mahony, Masuoka, & Ishii (1994)

First experiments:

Masuoka, Hatjopolous, & O'Mahony (1995)

Delwiche & O'Mahony (1996)

Psychometric function derived by Ennis et al. (1998)

Support for Tetrad testing in IFPrograms™ (2009)

Sample size tables published by Ennis & Jesionka (2011)

Operational power-based comparison with Triangle test

by Ennis (2012)

Large-scale comparison with Triangle test by Garcia,

Ennis, & Prinyawiwatkul (2012)

Support for Tetrad testing in sensR (2012)

5

Experimental Results (1/3)

Masuoka, Hatjopoulos & O’Mahony (1995)

Beer samples varying in bitterness

9 judges with 12 replications: N=108 per condition

d' values not significantly different

0.33

0.43

0.53

0.63

0.73

Triangle Tetrad3-AFC

Proportion Correct

0.0

0.5

1.0

1.5

TriangleTetrad

3-AFC

d'

6

Experimental Results (2/3)

Delwiche & O’Mahony (1996)

Chocolate pudding varying in sweetness

13 judges with 12 replications: N = 156 per condition

d' values not significantly different

0.33

0.43

0.53

0.63

0.73

0.83

0.93

Triangle Tetrad3-AFC

Proportion Correct

0.0

0.5

1.0

1.5

2.0

2.5

TriangleTetrad

3-AFC

d'

7

Experimental Results (3/3)

Garcia, Prinyawiwatkul, Ennis (2012)

Apple juices varying in sweetness

404 children: 1 Tetrad, 2 Triangle evaluations

d' values not significantly different

0.33

0.43

0.53

0.63

TriangleTetrad

Proportion Correct

0.0

0.5

1.0

1.5

TriangleTetrad

d'

8

Thurstonian Theory

Psychometric function (Ennis et al.,1998)

30%

40%

50%

60%

70%

80%

90%

100%

0 1 2 3 δ

Perc

en

tag

e c

orr

ect

3-AFC Tetrad Triangle

9

Triangle/Tetrad – Possible Cases (d = 1.5)

W (f)

W S S

(d) W W S S

(a) W W S

(a) W W

• Correct

• Wrong

• Correct

• Wrong

• Wrong

S

(b) W W S

(c) W W S

(d) W W S

W (e)

W S

• Correct

• Wrong

• Wrong

• Wrong

• Correct

S

(b) W W S S

(c) W W S S

W (e)

W S S

• Wrong

48.0%

17.9%

28.5%

3.2%

2.3%

63.0%

20.2%

6.7%

6.9%

1.0%

2.1%

Pc = 50.3% Pc = 64.0%

Δ □

10

Suppose α = 0.05 and want 80% power

If δ = 1.5

Tetrad N = 20

Triangle N = 57

If δ = 1.0

Tetrad N = 65

Triangle N = 220

Tetrad sample sizes are

roughly 1/3 Triangle

sample sizes

See Ennis & Jesionka (2011)

for more information

0

50

100

150

200

250

1.5 1

Tetrad Triangle

Triangle/Tetrad – Sample Sizes

δ

11

Precision of Measurement (1/4)

Variance in estimate of δ (Bi, Ennis, & O’Mahony, 1997)

Variance is B value divided by sample size

0

5

10

15

20

25

30

35

0.5 1 1.5 2 2.5

Tetrad Triangle

δ

B V

alu

e

12

Precision of Measurement (2/4)

Tetrad test can be analyzed using GLM framework

(Brockhoff and Christensen, 2010):

Convenient access to statistical analysis

PC δ 𝑓□

−1

13

Precision of Measurement (3/4)

Relative likelihood (Christensen & Brockhoff, 2009)

Function shape gives improved estimate of precision

Example: N = 60, δ ~ 1

0.00

0.25

0.50

0.75

1.00

0.0 0.5 1.0 1.5 2.0 2.5

Tetrad Triangle

Rela

tive lik

elih

ood

δ

14

Precision of Measurement (4/4)

Expected widths of likelihood confidence intervals

N = 60, 95% confidence

0.0

0.5

1.0

1.5

2.0

0.0 0.5 1.0 1.5 2.0 2.5

Tetrad Triangle

δ

Exp

ecte

d W

idth

15

Comparative Examples (1/2)

Six pasta sauces for food service applications

Research to compare Triangle and Tetrad tests

Test sample sizes vary between 96 and 132

33%

43%

53%

63%

73%

Mild Savory Pesto Alfredo Neopolitan Meat

Tetrad Triangle

0.00

0.50

1.00

1.50

2.00

Mild Savory Pesto Alfredo Neopolitan Meat

Tetrad Triangle

Proportion correct

d' values

16

Comparative Examples (2/2)

Likelihood confidence intervals:

Tetrad test gives more precise estimate of sensory

difference in each case

0.0 0.5 1.0 1.5 2.0 2.5 δ

} Neopolitan

} Pesto

} Mild

} Savory

} Alfredo

} Meat

Tetrad

Triangle

17

Final Points

Future topics:

Equivalence

Unequal variance

Multivariate Tetrad model

Comparison to 2-AFCR

Decision rule investigation

Thanks to:

Daniel Ennis & Benoit Rousseau, The Institute for Perception

Pieter Punter, OP&P Product Research

Per Brockhoff, Technical University of Denmark

www.ifpress.com

Advances in Tetrad Testing

Sensometrics 2012

Rennes, France

John M. Ennis

The Institute for Perception

john.m.ennis@ifpress.com

Rune H.B. Christensen

Technical University of Denmark

rhbc@imm.dtu.dk