www.ifpress.com
Advances in Tetrad Testing
Sensometrics 2012
Rennes, France
John M. Ennis
The Institute for Perception
Rune H.B. Christensen
Technical University of Denmark
2
Discrimination Testing
Discrimination testing as important as ever:
Compliance with health initiatives
Cost reductions
Changes to ingredients, processes, packaging, handling, etc.
Quality control
Three challenges:
1. Identify sensitive methods for unspecified testing
2. Measurement:
a) Quantify sensory differences
b) Understand precision in measurement
3. Determine size of meaningful difference
3
The Tetrad Test - Methodology
Four samples presented:
Six possible presentation orders:
Guessing probability = 1/3
“Group the stimuli into two groups of
two samples based on similarity”
AABB, ABAB, ABBA
BBAA, BABA, BAAB
4
The Tetrad Test - History
Mentioned by Lockhart (1951) and Gridgeman (1954)
Revisited by O’Mahony, Masuoka, & Ishii (1994)
First experiments:
Masuoka, Hatjopolous, & O'Mahony (1995)
Delwiche & O'Mahony (1996)
Psychometric function derived by Ennis et al. (1998)
Support for Tetrad testing in IFPrograms™ (2009)
Sample size tables published by Ennis & Jesionka (2011)
Operational power-based comparison with Triangle test
by Ennis (2012)
Large-scale comparison with Triangle test by Garcia,
Ennis, & Prinyawiwatkul (2012)
Support for Tetrad testing in sensR (2012)
5
Experimental Results (1/3)
Masuoka, Hatjopoulos & O’Mahony (1995)
Beer samples varying in bitterness
9 judges with 12 replications: N=108 per condition
d' values not significantly different
0.33
0.43
0.53
0.63
0.73
Triangle Tetrad3-AFC
Proportion Correct
0.0
0.5
1.0
1.5
TriangleTetrad
3-AFC
d'
6
Experimental Results (2/3)
Delwiche & O’Mahony (1996)
Chocolate pudding varying in sweetness
13 judges with 12 replications: N = 156 per condition
d' values not significantly different
0.33
0.43
0.53
0.63
0.73
0.83
0.93
Triangle Tetrad3-AFC
Proportion Correct
0.0
0.5
1.0
1.5
2.0
2.5
TriangleTetrad
3-AFC
d'
7
Experimental Results (3/3)
Garcia, Prinyawiwatkul, Ennis (2012)
Apple juices varying in sweetness
404 children: 1 Tetrad, 2 Triangle evaluations
d' values not significantly different
0.33
0.43
0.53
0.63
TriangleTetrad
Proportion Correct
0.0
0.5
1.0
1.5
TriangleTetrad
d'
8
Thurstonian Theory
Psychometric function (Ennis et al.,1998)
30%
40%
50%
60%
70%
80%
90%
100%
0 1 2 3 δ
Perc
en
tag
e c
orr
ect
3-AFC Tetrad Triangle
9
Triangle/Tetrad – Possible Cases (d = 1.5)
W (f)
W S S
(d) W W S S
(a) W W S
(a) W W
• Correct
• Wrong
• Correct
• Wrong
• Wrong
S
(b) W W S
(c) W W S
(d) W W S
W (e)
W S
• Correct
• Wrong
• Wrong
• Wrong
• Correct
S
(b) W W S S
(c) W W S S
W (e)
W S S
• Wrong
48.0%
17.9%
28.5%
3.2%
2.3%
63.0%
20.2%
6.7%
6.9%
1.0%
2.1%
Pc = 50.3% Pc = 64.0%
Δ □
10
Suppose α = 0.05 and want 80% power
If δ = 1.5
Tetrad N = 20
Triangle N = 57
If δ = 1.0
Tetrad N = 65
Triangle N = 220
Tetrad sample sizes are
roughly 1/3 Triangle
sample sizes
See Ennis & Jesionka (2011)
for more information
0
50
100
150
200
250
1.5 1
Tetrad Triangle
Triangle/Tetrad – Sample Sizes
δ
11
Precision of Measurement (1/4)
Variance in estimate of δ (Bi, Ennis, & O’Mahony, 1997)
Variance is B value divided by sample size
0
5
10
15
20
25
30
35
0.5 1 1.5 2 2.5
Tetrad Triangle
δ
B V
alu
e
12
Precision of Measurement (2/4)
Tetrad test can be analyzed using GLM framework
(Brockhoff and Christensen, 2010):
Convenient access to statistical analysis
PC δ 𝑓□
−1
13
Precision of Measurement (3/4)
Relative likelihood (Christensen & Brockhoff, 2009)
Function shape gives improved estimate of precision
Example: N = 60, δ ~ 1
0.00
0.25
0.50
0.75
1.00
0.0 0.5 1.0 1.5 2.0 2.5
Tetrad Triangle
Rela
tive lik
elih
ood
δ
14
Precision of Measurement (4/4)
Expected widths of likelihood confidence intervals
N = 60, 95% confidence
0.0
0.5
1.0
1.5
2.0
0.0 0.5 1.0 1.5 2.0 2.5
Tetrad Triangle
δ
Exp
ecte
d W
idth
15
Comparative Examples (1/2)
Six pasta sauces for food service applications
Research to compare Triangle and Tetrad tests
Test sample sizes vary between 96 and 132
33%
43%
53%
63%
73%
Mild Savory Pesto Alfredo Neopolitan Meat
Tetrad Triangle
0.00
0.50
1.00
1.50
2.00
Mild Savory Pesto Alfredo Neopolitan Meat
Tetrad Triangle
Proportion correct
d' values
16
Comparative Examples (2/2)
Likelihood confidence intervals:
Tetrad test gives more precise estimate of sensory
difference in each case
0.0 0.5 1.0 1.5 2.0 2.5 δ
} Neopolitan
} Pesto
} Mild
} Savory
} Alfredo
} Meat
Tetrad
Triangle
17
Final Points
Future topics:
Equivalence
Unequal variance
Multivariate Tetrad model
Comparison to 2-AFCR
Decision rule investigation
Thanks to:
Daniel Ennis & Benoit Rousseau, The Institute for Perception
Pieter Punter, OP&P Product Research
Per Brockhoff, Technical University of Denmark
www.ifpress.com
Advances in Tetrad Testing
Sensometrics 2012
Rennes, France
John M. Ennis
The Institute for Perception
Rune H.B. Christensen
Technical University of Denmark
