Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 2
LECTURE 8
I. PARAMETRIC TESTS
1. ONE-WAY ANOVA (INDEPENDENT)
Imagine that I was interested in how different teaching methods affected students’ knowledge. I noticed that some lecturers were aloof and arrogant in their teaching style and humiliated anyone who asked them a question, while others were encouraging and supporting of questions and comments. I took three statistics courses where I taught the same material. For one group of students I wandered around with a large cane and beat anyone who asked daft questions or got questions wrong (punish). In the second group I used my normal teaching style, which is to encourage students to discuss things that they find difficult and to give anyone working hard a nice sweet (reward). The final group I remained indifferent to and neither punished nor rewarded students’ efforts (indifferent). As the dependent measure I took the students’ exam marks (percentage). Based on theories of operant conditioning, we expect punishment to be a very unsuccessful way of reinforcing learning, but we expect reward to be very successful. Therefore, one prediction is that reward will produce the best learning. A second hypothesis is that punishment should actually retard learning such that it is worse than an indifferent approach to learning. The data are in the file teach.sav.
(The example is from Field (2009))
Carry out a one-way ANOVA and use planned comparisons to test the hypotheses that
H1: reward results in better exam results than either punishment or indifferent; and H2: indifferent will lead to significantly better exam results than punishment.
In SPSS, Analyze > Compare Means > One-way ANOVAs
Move the variable exam into the dependent List and group to the Factor box.
PLANNED CONTRASTS
Click on Contrasts to access the Contrasts dialog box.
The function Polynominal helps us to detect the trend in the data. We have 3 groups, so it’s suggested to check this function, then choose quadratic in the dropdown list from the Degree tab.
The contrast section is where we will specify our planned comparisons to answer the hypotheses. To assign the weights for the comparison, we need to follow the following rules:
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 3
Rule 1: We should be careful in pair selection as if we exclude any group in one comparison, it will be excluded in subsequent comparison as well.
Rule 2: Groups coded with positive weights will be compared against groups coded with negative weights.
Rule 3: The sum of weights for a comparison should be zero. Rule 4: If a group is not involved in a comparison, automatically assign it a weight of 0. Rule 5: For a given contrast, the weights assigned to the group(s) not included in the
contrast should be equal to the number of groups included in the pair comparison.
(Field. 2009, p. 365)
*In addition to these rule, it should be noticed that the order in which we put the weights should be parallel with the values assigned to each group in the data.
For hypothesis 1, we want to compare the reward condition with the punishment and indifferent groups. In order to gain the sum of the weights to zero, we will assign the following weights for the groups:
1: punish 1: indifference -2: reward
So in SPSS, we will enter 1; 1; -2 because punish is given a value of 1, indifference 2, and reward 3.
For hypothesis 2, we do not include the reward group in the comparison, so we will assign 0 for this group. The weights in the second contrast will be as follows:
1: punish -1: indifference 0: reward
Click Continue to proceed, and access the Options dialog box. Then select the options as indicated below.
POST HOC TESTS
In case we do not have a priori hypothesis (no assumption as to which group differs from others), we can access the Post Hoc option to ask SPSS perform all the possible pairwise comparisons.
Click on the Post Hoc button to access the dialog box. There are a lot of different kinds of post hoc tests we can choose from. When group sizes are equal, Tukey and R-E-G-W-Q (Ryan-Elinot-Gabriel-
Welsch Range) tests are suggested.
When we already know which is the group in the study is the one that we’d like to compare with the others, we choose the Dunnett’s test and set the appropriate control category under this option. According to H1, reward (value 3) is compared against the others, so we will select Last for the Control Category.
The suggested equivalent test when Equal Variances Not Assumed is Games-Howell. So we will also select this option.
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 4
Click Continue to return to the main dialog box, and OK to run the analysis.
SPSS OUTPUT
The first important table we should look at is the Test of Homogeneity of Variances, which shows the result of the Levene’s test. The significance value p = .095 > .05, so the assumption of homogeneity of variances between the groups is met. For this reason, we will next examine the ANOVA table. However, if the assumption is violated, we should report and look at the result from the Robust Tests of Equality of Means.
Test of Homogeneity of Variances Exam Mark
Levene Statistic df1 df2 Sig.
2.569 2 27 .095
We should read the result in the row labeled Between Groups (Combined). The F statistic, which is the ratio between systematic variation (due to the teaching condition) and unsystematic variation within the data is significant: F = 21.008, p < .001, i.e. there is a significant difference in exam marks among different teaching conditions.
The Trend analysis can be found in the two rows labeled Linear and Quadratic Term. The result shows that the means across groups increase in a linear fashion, but not curvilinear.
ANOVA Exam Mark
Sum of Squares df Mean Square F Sig.
Between Groups (Combined) 1205.067 (SSM) 2 (dfM) 602.533 21.008 .000
Linear Term Contrast 1185.800 1 1185.800 41.344 .000
Deviation 19.267 1 19.267 .672 .420
Quadratic Term Contrast 19.267 1 19.267 .672 .420 Within Groups 774.400 27 28.681 (MSR) Total 1979.467 (SST) 29
Planned Contrast Results
The Contrast Coefficients table indicates the weights that we have assigned for each group in the first and second comparisons or contrast.
Contrast Coefficients
Contrast
Type of Teaching Method
Punish Indifferent Reward
1 1 1 -2 2 1 -1 0
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 5
For contrast 1 (comparing punishment with indifference and reward), t (27) = -5.978, p < .001 (significant).
For contrast 2 (comparing indifference and reward), the t value is also significant, so we can conclude that there is a difference in the exam marks in these two teaching conditions.
Contrast Tests
Contrast Value of Contrast Std. Error t df Sig. (2-tailed)
*. The mean difference is significant at the 0.05 level.
b. Dunnett t-tests treat one group as a control, and compare all other groups against it.
The Exam Mark table presents the results of the Tukey’s test and the REGWR tests, which show which subsets of groups have similar means. As can be seen, we have no such subsets and this is in accordance with the above-mentioned results.
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 6
Exam Mark
Type of Teaching Method N
Subset for alpha = 0.05
1 2 3
Tukey HSDa Punish 10 50.0000
Indifferent 10 56.0000
Reward 10 65.4000
Sig. 1.000 1.000 1.000
Ryan-Einot-Gabriel-Welsch Range Punish 10 50.0000
Indifferent 10 56.0000
Reward 10 65.4000
Sig. 1.000 1.000 1.000
Means for groups in homogeneous subsets are displayed. a. Uses Harmonic Mean Sample Size = 10.000.
EFFECT SIZE
ANOVAs are usually used in experimental research, so it’s also helpful to report the effect size. We have 2 ways to calculate (Field, 2009):
If we substitute the respective values found from the ANOVA table, we will obtain an r2 = 1205.067/1979.467 = 0.609. After taking the square root, r = .78. Doing the same for ω (omega), we will get ω = .76
In addition to the overall effect size, we are also interested in the effect sizes in the contrast. The contrasts are simply the t-tests, so the effect size is calculated as:
The values of t and df can be found in the Contrasts Test table. For contrast 1, with t = -5.978, df = 27, r = .75. For contrast 2, with t = -2.505, df = 27, r = .43.
REPORT THE RESULT
We just need to answer the two hypotheses, so we can write:
There was a significant effect of teaching conditions on exam marks, F (2, 27) = 21.01, p < .001, ω = .76. Planned contrasts revealed that reward produced significantly better exam grades than punishment and indifference, t (27) = -5.978, p < .01, r = .75 and that punishment produced significantly lower exam marks than indifference, t (27) = -2.51, p < .05, r = .43.
r (or η: eta) , ω (omega) : effect size
SSM: between-group effect SST: total amount of variance in the data MSR: within-subject effect dfM: degree of freedom, which is the number of the groups minus 1
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 7
2. ONE-WAY ANOVA (REPEATED MEASURE)
A group of students investigated the consistency of marking by submitting the same essays to four different lecturers. The mark given by each lecturer was recorded for each of the eight essays. It was important that the same essays were used for all lecturers because this eliminated any individual differences in the standard of work that each lecturer marked. This design is repeated-measures because every lecturer marked every essay. The independent variable was the lecturer who marked the esays and the dependent variable was the percentage mark given.
The data file is TutorMarks.sav
In SPSS, Analyze > General Linear Model > Repeated Measures
In the Within-Subject Factor Name, the given name is factor1. We will type the new name which matches our data, e.g. tutors. As we have 4 tutors, we will indicate 4 in the Number of Levels box. Click on Add to confirm.
Then click Define to continue.
In the main dialog box, move the four variables to the Within-Subjects Variables
Click on the Contrasts button to access the dialog box. As there is no meaningful difference in the order of the tutors, we will choose Repeated, which will conduct pairwise comparison between 2 adjacent tutors (e.g. 1 vs. 2, 2 vs. 3).
Click Change to confirm, and Continue to proceed. Then access the Options dialog box.
Move the repeated measures variable tutors to the Display Means for area and select Compare mean effects.
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 8
Under the Confidence interval adjustment, select Bonferroni, which is the most suggested correction method (Field, 2009).
Under the Display, select Descriptive Statistics and Transformation matrix.
Click Continue to return to the main dialog box, and OK to run the analysis.
SPSS OUTPUT
The Mauchly’s Test of Sphericity is significant (p < .05) so there are differences in variances across levels of the tutor variable. In other words, the assumption of sphericity has been violated.
Mauchly's Test of Sphericitya
Measure: MEASURE_1
Within Subjects Effect Mauchly's W Approx. Chi-Square df Sig.
Epsilonb
Greenhouse-Geisser Huynh-Feldt Lower-bound
tutor .131 11.628 5 .043 .558 .712 .333
To know if there is a difference in the marking among the tutors, we should look at the Tests of Within-Subjects Effects table. The F = 3.7, p = .028 < .05 means that there is a significant difference.
Tests of Within-Subjects Effects Measure: MEASURE_1
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 9
However, as the assumption of sphericity is violated, we should look at the corrected F values. Of the three, Greenhouse-Geisser applies stricter criteria (likely to make Type II error), and we can see that the p is non-significant. Huynh-Feldt corrected F-statistic gives us a significant value, p = .047. As Steven (2002) suggests, to be neutral, we can take the average of Greenhouse-Geisser and Huynh-Feldt’s estimates. This will give us a p value of (.063 + .047)/2 = .055, which is still non-significant.
Before making decision, we should also consult the Multivariate Tests table (MANOVA). As this test makes no assumption about sphericity, it serves as a good reference. Again, we find a non-significant difference. Hence, the conclusion is that there is no difference among tutors regarding essay marking.
Multivariate Testsa
Effect Value F Hypothesis df Error df Sig.
tutor Pillai's Trace .741 4.760b 3.000 5.000 .063
Wilks' Lambda .259 4.760b 3.000 5.000 .063
Hotelling's Trace 2.856 4.760b 3.000 5.000 .063
Roy's Largest Root 2.856 4.760b 3.000 5.000 .063
In the following part, we will look closely into which tutor marks differently from others. As we choose both Contrast and Post hoc test, we will look at them in turn.
CONTRAST
As we chose the Repeated contrast, SPSS will compare two tutors who are next to each other in terms of the values assigned in the data, e.g. the tutor with value 1 (Dr. Field) is compared against the tutor with value 2 (Dr. Smith).
tutora
Measure: MEASURE_1
Dependent Variable
tutor
Level 1 vs. Level 2 Level 2 vs. Level 3 Level 3 vs. Level 4
Dr. Field (1) 1 0 0 Dr. Smith (2) -1 1 0 Dr. Scrote (3) 0 -1 1 Dr. Death (4) 0 0 -1
According to the Tests of Within-Subjects Contrasts, Dr. Field and Dr. Smith significantly differ in their marking.
Tests of Within-Subjects Contrasts Measure: MEASURE_1
Source tutor Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared
tutor Level 1 vs. Level 2 (Dr. Field and Dr. Smith)
Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni.
REPORT THE RESULTS
We can write:
Mauchly’s test indicated that the assumption of sphericity had been violated, χ² (5) = 11.63, p < .05, therefore degrees of freedom were corrected using Greenhouse-Geisser estimates of sphericity (ε = .556). The results show that there was no significant difference in essay marking among the tutors, F (1.67, 11.71) = 3.7, p > .05
As the main ANOVA is non-significant, hence the results of contrast or post hoc tests are just for reference. However, in case the main ANOVA is significant, we should also report the effect size as the between-subject ANOVAs and the results of contrast or post hoc analysis.
The effect size for repeated measure ANOVAs is calculated differently from the between subject ANOVAs. For more detail about the equations, you can refer pages 480-481 in the book Discovering Statistics Using SPSS by Field (2009).
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 11
3. MANOVA (Multivariate analysis of variance)
A lecturer is interested to find out if students’ knowledge of different aspects of psychology improved throughout their degree.
He administered 5 tests (scored out of 15) including:
Exper (experimental psychology such as cognitive and neuropsychology etc.) Stats (statistics); Social (social psychology); Develop (developmental psychology); Person (personality).
on three cohorts of students: first, second, and third year.
Carry out the MANOVA to determine whether there are overall group differences along these five measures.
The data file is psychology.sav.
In SPSS, Analyze > General Linear Model > Multivariate
Move the five variables (exper, stats, social, develop, and person) to the Dependent Variables area, and group into the Fixed Factors (s) area.
Click on the Post Hoc option to access the dialog box. As we do not know if equal variances assumed or not and the group sizes are not equal, we will choose the Hochberg’s GT2, Gabriel and the Games–Howell.
Click Continue to proceed, then access the Option dialog box.
Move the group variable into the Display Means for to obtain means for the levels of group.
Under Display, select Descriptive Statistics, Estimates of effect size, and Homogeneity tests.
Click Continue to proceed, then click OK to run the analysis.
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 12
SPSS OUTPUT
The Box’s test of the assumption of equality of covariance matrices is non-significant indicating that the covariance matrices are equal across groups
Also, the Levene’s Test of equality of variances for each of the dependent variables is non-significant, therefore, the assumption of homogeneity of variance is met.
Box's Test of Equality of Covariance Matrices
a
Box's M 54.241 F 1.435 df1 30 df2 3587.253 Sig. .059
The important table we should look at is Multivariate Tests. All of the 4 tests are significant, so at this step, we can conclude that there is a significant difference across different cohorts in terms of the five knowledge tests.
Multivariate Testsa
Effect Value F Hypothesis df Error df Sig. Partial Eta Squared
Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a. Design: Intercept + group
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 13
The Tests of Between-Subjects Effects table presents the ANOVA tests for each of the 5 dependent variable. As can be seen, all of these separate tests are non-significant. This highlights the advantage of using MANOVA when we have many dependent variables, because MANOVA can reduce the error rate in case many ANOVAs are carried out.
Tests of Between-Subjects Effects (ANOVAs)
Source Dependent Variable Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared
group Experimental Psychology 18.430 2 9.215 2.461 .099 .117
Statistics 52.504 2 26.252 2.967 .064 .138
Social Psychology 23.584 2 11.792 1.941 .158 .095
Developmental 37.717 2 18.859 3.114 .056 .144
Personality 39.415 2 19.708 3.044 .060 .141
As the ANOVAs are non-significant, so we do not proceed with post hoc tests.
REPORT THE RESULT
We can choose one of the four tests to report the result of MANOVA:
Using Wilk’s Lamda, there was a significant difference in the scores on the five knowledge tests among the first, second, and third year students, Λ = .52, F (10, 66) = 2.53, p < .05.
In case one of the ANOVAs is significant, we should also need to report the significant post hoc tests.
II. NON-PARAMETRIC TESTS
1. THE KRUSKAL WALLIS TEST
Does physical exercise alleviate depression? We find some depressed people and check that they are all equivalently depressed to begin with. Then we allocate each person randomly to one of three groups:
no exercise 20 minutes of jogging perday or 60 minutes of jogging per day
At the end of a month, we ask the participant to complete a test that measures the degree of depression (scores from 0 to 100).
(This example is adapted from http://www.sussex.ac.uk/Users/grahamh/RM1web/Kruskal-Wallis%20Handoout2011.pdf)
As the data are not normally distributed, we will choose the non-parametric test, i.e. Kruskal Wallis.
In SPSS, Analyze > Nonparametric Tests > Independent Samples
Click on Fields to access the Fields dialog box. Move the depression (score) to the Test Fields, and group to the Groups area.
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 14
SPSS OUTPUT
The following tells us that the test is significant and we will reject the null hypothesis.
Hypothesis Test Summary
Null Hypothesis Test Sig. Decision
1 The distribution of score is the same across categories of group.
Independent-Samples Kruskal-Wallis Test
.026 Reject the null hypothesis.
Asymptotic significances are displayed. The significance level is .05.
At the bottom of the screen, if we click on the View option and select Pairwise Comparisons, we can identify which group has different level of depression than the other. The significance value should be read from the Adj.Sig. column (Adjusted significance). As can be seen, none of these are smaller than .05, but as the comparison between group 1 (no exercise) and group 3 (jogging for 60 minutes)
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 15
REPORT THE RESULT
We can write:
The Kruskal-Wallis test revealed there was a significant effect of exercise on depression levels, H (2) = 7.29, p < .05). Subsequent pairwise comparisons indicated that there was a significant difference in the scores of depression between group 1 (no-exercise) and group 3 (60-minute jogging), though the p value is marginal p = .051.
2. FRIEDMAN’S ANOVA
A researcher was interested in whether the growing café society in the UK has led to a change in preference for non-alcoholic beverages. She therefore asks a number of students to give their ratings (on a scale of 1 to 5) for their enjoyment of coffee, tea and hot chocolate. A rating of 5 signifies a greater enjoyment of the drink.
The data file is drinks_rating.sav
In SPSS, Analyze > Nonparametric Tests > Legacy Dialogs > K Related Samples
Move all the three variables into the Test Variables area. Under the Test Type, select Friedman.
Click on the Exact option to access the Exact Tests dialog box. As our sample is small, it’s suggested to select the Exact option.
Click Continue to return to the main dialog box, then click OK to run the analysis.
SPSS OUTPUT
The Friedman test is based on the ranks. The Ranks table shows the mean ranks for each type of drinks rated by the students.
The result of the test is found in the Test Statistics table, which in this case is significant.
Ranks
Mean Rank
Tea 2.40 Coffee 1.65 HotChocolate 1.95
Test Statisticsa
N 20 Chi-Square 7.475 df 2 Asymp. Sig. .024 Exact Sig. .022 Point Probability .001
a. Friedman Test
Introduction to Applied Statistics and Applied Statistical Methods Practical guidelines
Prof.Dr. Chang Zhu page 16
REPORT THE RESULT
We can write:
The Friedman test revealed that the students varied significantly in their preference of the three types on non-alcoholic beverages, χ²(2) = 7.48, p < .05.
POST HOC TESTS
Similar to ANOVAs, we can conduct post hoc tests to find out which groups are different by using the Mann-Whitney and Wilcoxon signed-rank tests after the Kruskal and Friedman tests, respectively. On word of notice is that we should re-set the critical value of significance. For example, if we do 3 comparison, the significance value should be .05/3 = .0167 for a test to be significant (Field, 2009).
ASSIGNMENT 8
(You can work alone or in group for this assignment. If you work in group, please stay in the same group of previous assignments and indicate the group members in the submission document).
You can choose one of the following options:
Option 1
A psychologist who is interested in aggression has devised an experimental paradigm in which participants play a computer game with an opponent. When the opponent makes an error the participant is invited to punish their opponent by exposing him to a blast of loud noise. The duration and volume of the noise blast are combined to give a measure of aggression. A total of 60 participants were tested using this procedure before being given feedback about their performance in the game. One third of the participants received negative feedback, one third received positive feedback and the remaining third received neutral feedback. Finally, the participants played the game again and their level of aggression was measured as before. The data from this study can be found in the file aggression.sav.
1. Conduct an ANOVA to determine whether there is a difference between levels of aggression among the participants measured before and after the feedback session.
2. Participants were randomly assigned to each of the three feedback conditions, and as a result the pre-test scores for these three groups should not differ. Test whether this is confirmed.
Option 2
Search for a research article that use one of the kinds of analysis of variances (independent/repeated ANOVA; MANOVA, Kruskal-Wallis, and Friedman ANOVA)
Briefly summarized the following:
- The variables measured in the study - The groups that the analysis were conducted for. - The study hypotheses - The tests that were used to test the hypotheses - The study’s conclusion (What has been found?)
![Lecture [8]](https://static.documents.pub/doc/80x56/577cc75c1a28aba711a0b3c6/lecture-8-578fec4e55e51.jpg)
