Post on 28-Jul-2015
transcript
PSBE2-08
Research Methods
Week 5
1Tassos Sarampalis
Announcements
2
Announcements
Heymans Colloquium“The Ease of Lying” by
Dr. B.J. Verschuere (University of Amsterdam)Wednesday 14 March, 12.00–13.00 hrs
Room 033, Grote Rozenstraat 15http://www.rug.nl/psy/Agenda/121403_col_psy_verschuere
3
How to catch a criminal:Memory detection, not lie detection
Tuesday 13 March 20128:00 - 9.30 p.m. | Academiegebouw
Exam
4
Second hour of the final lecture will be devoted to questionsEmail them to a.sarampalis@rug.nl
The week’s reading
Chapter 7(pp 189 – 232)
5
Randomised Controlled Experiments+
Causal Inferences
6
Randomised Experiments
• Between-subjects designs
• Within-subjects designs
• Mixed factorial designs
7
Wait-list Control Group
8time
Group A
Group B
Sample
Test Treat Test Test
Test Test Treat Test
Wait-list Control Group
9
Group A
Group B
Sample
Test Treat Test Test
Group C
Group D
time
Test Test
Test Test Treat Test Test Test
Test Test Test Treat Test Test
Test Test Test Test Treat Test
Within-Subjects Designs
• Cannot be confounded by individual differences
But
• Can be confounded by the order of presentation
10
Counterbalancing
• Complete Design
Practice effects are balanced by administering conditions several times, changing the order
• Incomplete Design
Each condition is only given once
11
Counterbalancing
• Complete Design
- Block Randomisation- ABBA Counterbalancing
12
Counterbalancing
• Complete Design
- Block Randomisation
13
Subject First Block Second Block
1 A B
2 B A
3 A B
4 A B
5 B A
6 B A
7 A B
Counterbalancing
• Complete Design
- ABBA Counterbalancing
14
Subject First Block Second Block Third Block Fourth Block
1 A B B A
2 A B B A
3 A B B A
4 A B B A
5 A B B A
6 A B B A
7 A B B A
Counterbalancing
• Complete Design
- ABBA Counterbalancing
15
Subject First Block Second Block
Third Block Fourth Block
Fifth Block Sixth Block
1 A B C C B A
2 A B C C B A
3 A B C C B A
4 A B C C B A
5 A B C C B A
6 A B C C B A
7 A B C C B A
Counterbalancing
• Complete Design
- ABBA Counterbalancing
NOT appropriate when practice effects are non-linear
16
A B C C B A
Linear 0 +1 +2 +3 +4 +5
Non-linear 0 +6 +6 +6 +6 +6
Counterbalancing
• Complete Design
- ABBA Counterbalancing
NOT appropriate when practice effects are non-linearor when there are anticipation effects
17
Counterbalancing
• Incomplete Design
18
• Practice effects are balanced across subjects rather than for each subject• Each condition must be presented in each ordinal position, equally often
– All Possible Orders(good for fewer than 3 or 4 conditions)
– Selected Orders(Latin Square or random starting order with rotation)
Counterbalancing
• All Possible Orders
19
Subject Order
1 ABC
2 ACB
3 BAC
4 BCA
5 CAB
6 CBA
7 ABC
… …
Counterbalancing
• Random Order With Rotation
20
Subject Order
1 DCFBAE
2 CFBAED
3 FBAEDC
4 BAEDCF
5 AEDCFB
6 EDCFBA
7 DCFBAE
… …
Counterbalancing
• Random Order With Rotation
21
Subject Order
1 DCFBAE
2 CFBAED
3 FBAEDC
4 BAEDCF
5 AEDCFB
6 EDCFBA
7 DCFBAE
… …
Counterbalancing
• Latin Square
22
23
Gonville and
Caius CollegeCambridge, UK
Latin Squares
24
Ordinal Position
1 2 3 4
Sequence 1 A B C D
Sequence 2 B C D A
Sequence 3 C D A B
Sequence 4 D A B C
Latin Squares
Each condition appears in each ordinal position once
25
Ordinal Position
1 2 3 4
Sequence 1 A B C D
Sequence 2 B C D A
Sequence 3 C D A B
Sequence 4 D A B C
Balanced Latin Squares
• As before, each condition appears in each ordinal position once
• ALSO: each condition precedes and follows each condition exactly once
26
D C B A
C A D B
B D A C
A B C D
Balanced Latin Squares
• As before, each condition appears in each ordinal position once
• ALSO: each condition precedes and follows each condition exactly once
27
D C B A
C A D B
B D A C
A B C D
Balanced Latin Squares address concernsabout first-order carryover effects
Orthogonal Latin Squares
VARIABLE 1 variable 2
28
D C B A
C A D B
B D A C
A B C D
a b c d
b a d c
c d a b
d c b a
Orthogonal Latin Squares
29
D C B A
C A D B
B D A C
A B C D
a b c d
b a d c
c d a b
d c b a
VARIABLE 1 x variable 2
Orthogonal Latin Squares
30
Da Cb Bc Ad
Bd Ac Db Ca
Ab Ba Cd Dc
Cc Dd Aa Bb
VARIABLE 1 x variable 2
Used when there are more than one variables to counterbalance and the pairings of conditions have to be unique.
Also known as Graeco-Roman or Euler Latin Squares
For more geeking out
31
http://www.cut-the-knot.org/arithmetic/latin_intro.shtmlandhttp
://pavlopoulos.wordpress.com/2010/10/01/magic-squares-and-r-p-lohses-concrete-art
/
THREATS TO INTERNAL VALIDITY
32
Threats to Internal Validity
• History– Events that occur between two measurements
• Maturation– Intrinsic changes in the participants
• Instrumentation– Intrinsic changes in the instrument
• Regression toward the mean– ???
33
Regression Towardthe Mean
34
Participant Real Value Error Observed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Average
St.Dev.
Participant Real Value Error Observed
1 92
2 112
3 118
4 91
5 99
6 97
7 130
8 95
9 112
10 93
11 128
12 91
13 123
14 99
15 116
16 104
17 115
18 104
19 123
20 120
21 128
22 119
23 93
24 105
25 111
Average 108.72
St.Dev. 12.93741
Participant Real Value Error Observed
1 92
2 112
3 118
4 91
5 99
6 97
7 130
8 95
9 112
10 93
11 128
12 91
13 123
14 99
15 116
16 104
17 115
18 104
19 123
20 120
21 128
22 119
23 93
24 105
25 111
Average 108.72
St.Dev. 12.93741
Participant Real Value Error Observed
1 92 -5
2 112 3
3 118 3
4 91 0
5 99 4
6 97 0
7 130 -1
8 95 4
9 112 5
10 93 -3
11 128 3
12 91 3
13 123 -2
14 99 -5
15 116 -1
16 104 -2
17 115 5
18 104 -4
19 123 -4
20 120 -3
21 128 -2
22 119 1
23 93 2
24 105 -5
25 111 5
Average 108.72 0.04
St.Dev. 12.93741 3.457841
Participant Real Value Error Observed
1 92 -5 87
2 112 3 115
3 118 3 121
4 91 0 91
5 99 4 103
6 97 0 97
7 130 -1 129
8 95 4 99
9 112 5 117
10 93 -3 90
11 128 3 131
12 91 3 94
13 123 -2 121
14 99 -5 94
15 116 -1 115
16 104 -2 102
17 115 5 120
18 104 -4 100
19 123 -4 119
20 120 -3 117
21 128 -2 126
22 119 1 120
23 93 2 95
24 105 -5 100
25 111 5 116
Average 108.72 0.04 108.76
St.Dev. 12.93741 3.457841 13.47306
Participant Real Value Error Observed
1 92 -5 87
2 112 3 115
3 118 3 121
4 91 0 91
5 99 4 103
6 97 0 97
7 130 -1 129
8 95 4 99
9 112 5 117
10 93 -3 90
11 128 3 131
12 91 3 94
13 123 -2 121
14 99 -5 94
15 116 -1 115
16 104 -2 102
17 115 5 120
18 104 -4 100
19 123 -4 119
20 120 -3 117
21 128 -2 126
22 119 1 120
23 93 2 95
24 105 -5 100
25 111 5 116
Average 108.72 0.04 108.76
St.Dev. 12.93741 3.457841 13.47306
Participant Real Value Error Observed
1 92 -5 87
10 93 -3 90
4 91 0 91
12 91 3 94
14 99 -5 94
23 93 2 95
6 97 0 97
8 95 4 99
18 104 -4 100
24 105 -5 100
16 104 -2 102
5 99 4 103
2 112 3 115
15 116 -1 115
25 111 5 116
9 112 5 117
20 120 -3 117
19 123 -4 119
17 115 5 120
22 119 1 120
3 118 3 121
13 123 -2 121
21 128 -2 126
7 130 -1 129
11 128 3 131
Average 108.72 0.04 108.76
St.Dev. 12.93741 3.457841 13.47306
Participant Real Value Error Observed
1 92 -5 87
10 93 -3 90
4 91 0 91
12 91 3 94
14 99 -5 94
23 93 2 95
6 97 0 97
8 95 4 99
18 104 -4 100
24 105 -5 100
16 104 -2 102
5 99 4 103
2 112 3 115
15 116 -1 115
25 111 5 116
9 112 5 117
20 120 -3 117
19 123 -4 119
17 115 5 120
22 119 1 120
3 118 3 121
13 123 -2 121
21 128 -2 126
7 130 -1 129
11 128 3 131
Average 108.72 0.04 108.76
St.Dev. 12.93741 3.457841 13.47306
Participant Real Value Error Observed
1 92 -5 87
10 93 -3 90
4 91 0 91
12 91 3 94
14 99 -5 94
23 93 2 95
6 97 0 97
8 95 4 99
18 104 -4 100
24 105 -5 100
16 104 -2 102
5 99 4 103
2 112 3 115
15 116 -1 115
25 111 5 116
9 112 5 117
20 120 -3 117
19 123 -4 119
17 115 5 120
22 119 1 120
3 118 3 121
13 123 -2 121
21 128 -2 126
7 130 -1 129
11 128 3 131
Average 108.72 0.04 108.76
St.Dev. 12.93741 3.457841 13.47306
Mlow = 91.2
Participant Real Value Error Observed
1 92 -5 87
10 93 -3 90
4 91 0 91
12 91 3 94
14 99 -5 94
23 93 2 95
6 97 0 97
8 95 4 99
18 104 -4 100
24 105 -5 100
16 104 -2 102
5 99 4 103
2 112 3 115
15 116 -1 115
25 111 5 116
9 112 5 117
20 120 -3 117
19 123 -4 119
17 115 5 120
22 119 1 120
3 118 3 121
13 123 -2 121
21 128 -2 126
7 130 -1 129
11 128 3 131
Average 108.72 0.04 108.76
St.Dev. 12.93741 3.457841 13.47306
Mlow = 91.2
Mhigh = 125.6
Participant Real Value Error Observed
1 92 -5 87
10 93 -3 90
4 91 0 91
12 91 3 94
14 99 -5 94
23 93 2 95
6 97 0 97
8 95 4 99
18 104 -4 100
24 105 -5 100
16 104 -2 102
5 99 4 103
2 112 3 115
15 116 -1 115
25 111 5 116
9 112 5 117
20 120 -3 117
19 123 -4 119
17 115 5 120
22 119 1 120
3 118 3 121
13 123 -2 121
21 128 -2 126
7 130 -1 129
11 128 3 131
Average 108.72 0.04 108.76
St.Dev. 12.93741 3.457841 13.47306
Mlow = 91.2
Mhigh = 125.6
Mall = 108.7
Participant Real Value Error 1 Observed 1
1 92 -5 87
10 93 -3 90
4 91 0 91
12 91 3 94
14 99 -5 94
23 93 2 95
6 97 0 97
8 95 4 99
18 104 -4 100
24 105 -5 100
16 104 -2 102
5 99 4 103
2 112 3 115
15 116 -1 115
25 111 5 116
9 112 5 117
20 120 -3 117
19 123 -4 119
17 115 5 120
22 119 1 120
3 118 3 121
13 123 -2 121
21 128 -2 126
7 130 -1 129
11 128 3 131
Average 108.72 0.04 108.76
St.Dev. 12.93741 3.457841 13.47306
Mlow = 91.2
Mhigh = 125.6
Mall = 108.7
First Observation
Participant Real Value Error 1 Observed 1 Error 2 Observed 2
1 92 -5 87 1 93
10 93 -3 90 -5 88
4 91 0 91 -5 86
12 91 3 94 1 92
14 99 -5 94 2 101
23 93 2 95 3 96
6 97 0 97 -2 95
8 95 4 99 -2 93
18 104 -4 100 1 105
24 105 -5 100 0 105
16 104 -2 102 -3 101
5 99 4 103 0 99
2 112 3 115 3 115
15 116 -1 115 -1 115
25 111 5 116 -5 106
9 112 5 117 -1 111
20 120 -3 117 0 120
19 123 -4 119 5 128
17 115 5 120 5 120
22 119 1 120 2 121
3 118 3 121 -1 117
13 123 -2 121 1 122
21 128 -2 126 -4 122
7 130 -1 129 1 131
11 128 3 131 -1 127
Average 108.72 0.04 108.76 -0.2 108.36
St.Dev. 12.93741 3.457841 13.47306 2.828427 13.58271
Mlow = 91.2
Mhigh = 125.6
Mall = 108.7
First Observation
Participant Real Value Error 1 Observed 1 Error 2 Observed 2
1 92 -5 87 1 93
10 93 -3 90 -5 88
4 91 0 91 -5 86
12 91 3 94 1 92
14 99 -5 94 2 101
23 93 2 95 3 96
6 97 0 97 -2 95
8 95 4 99 -2 93
18 104 -4 100 1 105
24 105 -5 100 0 105
16 104 -2 102 -3 101
5 99 4 103 0 99
2 112 3 115 3 115
15 116 -1 115 -1 115
25 111 5 116 -5 106
9 112 5 117 -1 111
20 120 -3 117 0 120
19 123 -4 119 5 128
17 115 5 120 5 120
22 119 1 120 2 121
3 118 3 121 -1 117
13 123 -2 121 1 122
21 128 -2 126 -4 122
7 130 -1 129 1 131
11 128 3 131 -1 127
Average 108.72 0.04 108.76 -0.2 108.36
St.Dev. 12.93741 3.457841 13.47306 2.828427 13.58271
Mlow = 91.2
Mhigh = 125.6
Mall = 108.7
First Observation
Mlow = 92
Mhigh = 123.8
Mall = 108.36
Second Observation
Participant Real Value Error 1 Observed 1 Error 2 Observed 2
1 92 15 107 5 97
2 112 0 112 -16 96
3 118 14 132 -10 108
4 91 18 109 -17 74
5 99 9 108 -20 79
6 97 -15 82 -10 87
7 130 -13 117 9 139
8 95 14 109 5 100
9 112 -2 110 8 120
10 93 6 99 -4 89
11 128 -19 109 -10 118
12 91 -20 71 -19 72
13 123 -2 121 -5 118
14 99 -6 93 1 100
15 116 -18 98 9 125
16 104 -16 88 9 113
17 115 -20 95 4 119
18 104 -15 89 9 113
19 123 -18 105 13 136
20 120 4 124 -17 103
21 128 18 146 8 136
22 119 6 125 13 132
23 93 -7 86 -20 73
24 105 18 123 18 123
25 111 19 130 -19 92
Average 108.72 -1.2 107.52 -2.24 106.48
St.Dev. 12.93741 14.23025 17.57868 12.50427 20.48398
Participant Real Value Error 1 Observed 1 Error 2 Observed 2
1 92 15 107 5 97
2 112 0 112 -16 96
3 118 14 132 -10 108
4 91 18 109 -17 74
5 99 9 108 -20 79
6 97 -15 82 -10 87
7 130 -13 117 9 139
8 95 14 109 5 100
9 112 -2 110 8 120
10 93 6 99 -4 89
11 128 -19 109 -10 118
12 91 -20 71 -19 72
13 123 -2 121 -5 118
14 99 -6 93 1 100
15 116 -18 98 9 125
16 104 -16 88 9 113
17 115 -20 95 4 119
18 104 -15 89 9 113
19 123 -18 105 13 136
20 120 4 124 -17 103
21 128 18 146 8 136
22 119 6 125 13 132
23 93 -7 86 -20 73
24 105 18 123 18 123
25 111 19 130 -19 92
Average 108.72 -1.2 107.52 -2.24 106.48
St.Dev. 12.93741 14.23025 17.57868 12.50427 20.48398
Participant Real Value Error 1 Observed 1 Error 2 Observed 2
1 92 15 107 5 97
2 112 0 112 -16 96
3 118 14 132 -10 108
4 91 18 109 -17 74
5 99 9 108 -20 79
6 97 -15 82 -10 87
7 130 -13 117 9 139
8 95 14 109 5 100
9 112 -2 110 8 120
10 93 6 99 -4 89
11 128 -19 109 -10 118
12 91 -20 71 -19 72
13 123 -2 121 -5 118
14 99 -6 93 1 100
15 116 -18 98 9 125
16 104 -16 88 9 113
17 115 -20 95 4 119
18 104 -15 89 9 113
19 123 -18 105 13 136
20 120 4 124 -17 103
21 128 18 146 8 136
22 119 6 125 13 132
23 93 -7 86 -20 73
24 105 18 123 18 123
25 111 19 130 -19 92
Average 108.72 -1.2 107.52 -2.24 106.48
St.Dev. 12.93741 14.23025 17.57868 12.50427 20.48398
Participant Real Value Error 1 Observed 1 Error 2 Observed 2
12 91 -20 71 -19 72
6 97 -15 82 -10 87
23 93 -7 86 -20 73
16 104 -16 88 9 113
18 104 -15 89 9 113
14 99 -6 93 1 100
17 115 -20 95 4 119
15 116 -18 98 9 125
10 93 6 99 -4 89
19 123 -18 105 13 136
1 92 15 107 5 97
5 99 9 108 -20 79
4 91 18 109 -17 74
8 95 14 109 5 100
11 128 -19 109 -10 118
9 112 -2 110 8 120
2 112 0 112 -16 96
7 130 -13 117 9 139
13 123 -2 121 -5 118
24 105 18 123 18 123
20 120 4 124 -17 103
22 119 6 125 13 132
25 111 19 130 -19 92
3 118 14 132 -10 108
21 128 18 146 8 136
Average 108.72 -1.2 107.52 -2.24 106.48
St.Dev. 12.93741 14.23025 17.57868 12.50427 20.48398
Participant Real Value Error 1 Observed 1 Error 2 Observed 2
12 91 -20 71 -19 72
6 97 -15 82 -10 87
23 93 -7 86 -20 73
16 104 -16 88 9 113
18 104 -15 89 9 113
14 99 -6 93 1 100
17 115 -20 95 4 119
15 116 -18 98 9 125
10 93 6 99 -4 89
19 123 -18 105 13 136
1 92 15 107 5 97
5 99 9 108 -20 79
4 91 18 109 -17 74
8 95 14 109 5 100
11 128 -19 109 -10 118
9 112 -2 110 8 120
2 112 0 112 -16 96
7 130 -13 117 9 139
13 123 -2 121 -5 118
24 105 18 123 18 123
20 120 4 124 -17 103
22 119 6 125 13 132
25 111 19 130 -19 92
3 118 14 132 -10 108
21 128 18 146 8 136
Average 108.72 -1.2 107.52 -2.24 106.48
St.Dev. 12.93741 14.23025 17.57868 12.50427 20.48398
Mlow = 83.2
Mhigh = 131.4
Mall = 107.5
First Observation
Participant Real Value Error 1 Observed 1 Error 2 Observed 2
12 91 -20 71 -19 72
6 97 -15 82 -10 87
23 93 -7 86 -20 73
16 104 -16 88 9 113
18 104 -15 89 9 113
14 99 -6 93 1 100
17 115 -20 95 4 119
15 116 -18 98 9 125
10 93 6 99 -4 89
19 123 -18 105 13 136
1 92 15 107 5 97
5 99 9 108 -20 79
4 91 18 109 -17 74
8 95 14 109 5 100
11 128 -19 109 -10 118
9 112 -2 110 8 120
2 112 0 112 -16 96
7 130 -13 117 9 139
13 123 -2 121 -5 118
24 105 18 123 18 123
20 120 4 124 -17 103
22 119 6 125 13 132
25 111 19 130 -19 92
3 118 14 132 -10 108
21 128 18 146 8 136
Average 108.72 -1.2 107.52 -2.24 106.48
St.Dev. 12.93741 14.23025 17.57868 12.50427 20.48398
Mlow = 83.2
Mhigh = 131.4
Mall = 107.5
First Observation
Mlow = 91.6
Mhigh = 114.2
Mall = 106.48
Second Observation
Thank you
55