Non-parametric TestsThanks to Prof. Andy Field

Chapter 6

From correlation lecture in week11:Non-parametric

Parametric vs Non-parametric

Parametric Non-parametric

Measurement scale Interval or ratio Nominal or ordinal

Information used Parametric correlation uses information about the mean and deviation from the mean

Non-parametric correlation will use only the ordinal position of pairs of scores.

You have to look at the distribution of your data. Check that the distributions are approximately normal*.* The best way to do this is to check the skew and Kurtosis measures from

the frequency output from SPSS. For a relatively normal distribution:skew ~= 1.0kurtosis~=1.0If a distribution deviates markedly from normality then you take the risk that the statistic will be inaccurate. The safest thing to do is to use an equivalent non-parametric statistic.

• Non-parametric stat based on ranked data• Minimises the effects of• Extreme scores• Violations of the assumptions • Ranks the data, then applies Pearson’s r to the ranks.

Spearman’s r (Spearmans Rho) rs

Use rather than Spearman’s r when• Small data set• With large number of tied ranks(That is, if you rank all the scores and many have the same rank)

Kendall’s is less popular than Spearman’s but can be a better estimate.

Kendall’s tau (non-parametric)

We will learn the theory behind and how to analyse in SPSS 3 non-parametric tests.These tests are relevant for comparing 2 means.

When independent t-test assumptions are broken use:The Wilcoxon rank-sum (Ws) test OrMann–Whitney (U) test

When paired t-test assumptions are broken use:Wilcoxon signed-rank (T) test

Lecture Outline

Non-parametric tests are used when assumptions of parametric tests are not met.

It is not always possible to correct for problems with the distribution of a data set • In these cases we have to use non-parametric tests.• They make fewer assumptions about the type of data on which they can be used.• In general, you are better off trying to use a robust test but the range of robust

tests is limited in SPSS.

The non-parametric tests often rely on the calculation of ranks e.g.Wilcoxon rank-sum testKruskal–Wallis testWilcoxon signed-rank test

When to use nonparametric tests

The Wilcoxon rank-sum (Ws) test

and Mann–Whitney (U) test

Compares two means based on independent dataE.g., data from different groups of people

The Wilcoxon rank-sum (Ws) test and Mann–Whitney (U) test areIndependent tests

The Wilcoxon rank-sum (Ws) test and Mann–Whitney (U) test

These tests are the non-parametric equivalent of the independent t-test.

Use either to test differences between two conditions in which different participants have been used.

Frank Wilcoxon

Ranking DataThe tests in this lecture work on the principle of ranking the data for each group: • Lowest score = a rank of 1, • Next highest score = a rank of 2, and so on. • Tied ranks are given the same rank: the average of the

potential ranks.For an unequal group size• The test statistic (Ws) = sum of ranks in the group that contains the least people.

For an equal group size• Ws = the value of the smaller summed rank.

Add up the ranks for the two groups and take the lowest of these sums to be our test statistic.

The analysis is carried out on the ranks rather than the actual data.

TheoryA neurologist investigated the depressant effects of certain recreational drugs. • Tested 20 clubbers• 10 were given an ecstasy tablet to take on a Saturday night• 10 were allowed to drink only alcohol. • Levels of depression were measured using the Beck Depression

Inventory (BDI) the day after and midweek.File = drug.sav

Hypotheses• 1) Between those who took alcohol and those who took ecstasy,

depression levels will be different the day after.• 2) Between those who took alcohol and those who took ecstasy,

depression levels will be different midweek.

Ente

r dat

a in

to S

PSS

Provisional analysis using IBM SPSS

First enter the data into SPSS• Because the data are collected using different participants in each

group, we need to input the data using a coding variable. • For example, ‘Drug’ with the codes; 1 = ecstasy group and 2 =

alcohol group. • When you enter the data into SPSS remember to tell the computer

that a code of 1 represents the group that was given ecstasy and a code of 2 represents the group that was restricted to alcohol.

First, run some exploratory analyses on the data• Run these exploratory analyses for each group because we’re going to

be looking for group differences.

Exploratory Analysis Output

FIGURE 6.6 Normal Q-Q plots of depression scores after ecstasy and alcohol on Sunday and Wednesday

Exploratory Analysis Output

Interpreting OutputTesting for normality• Sunday data:• Ecstasy, D(10) = 0.28, p = .03 (non normal)• Alcohol, D(10) = 0.17, p = .20 (normal)

• Wednesday data:• Ecstasy, D(10) = 0.24, p = .13 (normal)• Alcohol, D(10) = 0.31, p = .01 (non normal)

• A non-parametric test should be used. Results of Levene’s test• The assumption of homogeneity has been met:• Sunday data, F(1, 18) = 3.64, p = .07.• Wednesday, F(1, 18) = 0.51, p = .49.

Rank the data ignoring the drug group to which a person belonged • A similar number of high and low ranks in each group suggests

depression levels do not differ between the groups.• A greater number of high ranks in the ecstasy group than the alcohol

group suggests the ecstasy group is more depressed than the alcohol group.

Rank the data

Ranking the Depression scores for Wednesday and Sunday

Running the Analysis

https://www.youtube.com/watch?v=esNb6RFIXvw

How to do non-parametric analysis

Calculating an Effect SizeThe equation to convert a z-score into the effect size estimate, r, is as follows (from Rosenthal, 1991: 19):

• z is the z-score that SPSS produces• N is the size of the study (i.e. the number of total

observations)• We had 10 ecstasy users and 10 alcohol users and so the

total number of observations was 20.

NZr

Reporting the ResultsFor the Mann–Whitney test:• Depression levels in ecstasy users (Mdn = 17.50) did not differ

significantly from alcohol users (Mdn = 16.00) the day after the drugs were taken, U = 35.50, z = −1.11, p = .280, r = −.25. However, by Wednesday, ecstasy users (Mdn = 33.50) were significantly more depressed than alcohol users (Mdn = 7.50), U = 4.00, z = −3.48, p < .001, r = −.78.

Report the median for non-parametric tests

Reporting the Results IIOr we could report Wilcoxon’s test:• Depression levels in ecstasy users (Mdn = 17.50) did not significantly

differ from alcohol users (Mdn = 16.00) the day after the drugs were taken, Ws = 90.50, z = −1.11, p = .280, r = −.25. However, by Wednesday, ecstasy users (Mdn = 33.50) were significantly more depressed than alcohol users (Mdn = 7.50), Ws = 59.00, z = −3.48, p < .001, r = −.78.

Wilcoxon signed-rank test

Wilcoxon signed-rank test is aPaired test…A pair of hands hold a sign up…

Comparing two related conditions: the Wilcoxon signed-rank test

Uses:• To compare two sets of scores, when these scores come from the

same participants. Imagine the experimenter was interested in the change in depression levels for each drug. • Non-parametric test because the distributions of scores for both drugs

were non-normal on one of the days.

Ranking data in the Wilcoxon signed-rank test

Running the Analysis

Output for the Ecstasy Group

Output for the alcohol group

Calculating an Effect sizeThe effect size can be calculated in the same way as for the Mann–Whitney test. In this case SPSS Output tells us that for the ecstasy group z is –2.53, and for the alcohol group is −1.99.

In both cases we had 20 observations• (although we only used 10 people and tested them

twice, it is the number of observations, not the number of people, that is important here).

The effect size is therefore:

Reporting the results of Wilcoxon signed-rank test

Reporting Test-Statistic:• For ecstasy users depression levels were significantly higher on

Wednesday (Mdn = 33.50) than on Sunday (Mdn = 17.50), T = 36, p = .012, r = .57. However, for alcohol users the opposite was true: depression levels were significantly lower on Wednesday (Mdn = 7.50) than on Sunday (Mdn = 16.0), T = 8, p = .047, r = −.44.

Reporting the values of z:• For ecstasy users, depression levels were significantly higher on

Wednesday (Mdn = 33.50) than on Sunday (Mdn = 17.50), z = 2.53, p = .012, r = .57. However, for alcohol users the opposite was true: depression levels were significantly lower on Wednesday (Mdn = 7.50) than on Sunday (Mdn = 16.0), z = −1.99, p = .047, r = −.44.

To sum up …When data violate the assumptions of parametric tests we can sometimes find a nonparametric equivalent• Usually based on analysing the ranked dataMann-Whitney/ Wilcoxon rank-sum Test• Compares two independent groups of scores• You should use the MW or rank-sum test when the data are not paired.Wilcoxon signed rank Test• Compares two dependent groups of scores • This is for when scores are paired

ParametricAnalysis of Variance. Comparing several mean. Also called general linear model. This will be easy for you guys because you have already learnt the linear model.

Non-parametricKrukal Wallis test (H). Several independent groups. It too ranks data.Joncheeere-Terpstra test. Looks for trends in data.Friedman’s ANOVA (Fr). Several related groups.

In the Future

Revision of the Stats part of VISN 2211

• Why do we need stats?• Evidence based practice- Appraisal

• Statistical models• The mean as a model• Sums of squares/fit/Variance

• Correlation• Graphs• Assumptions• Measuring Relationships

• Pearson r• R squared

• Non-parametric

Correlation Lecture outline

• There was no significant relationship between the number of adverts watched and the number of packets of toffee purchased, r = .87, p = .054.

• r = .87 is a large effect.• The sign of r is positive. As one variable increases, so

too does the other. Note that this doesn’t imply causation.

SPSS output

When interpreting a correlation coefficient there are 3 important things to consider.• The significance of r• The magnitude of r• The +/– sign of r

Understand linear regression with one predictorUnderstand how we assess the fit of a regression model• Total Sum of Squares•Model Sum of Squares• Residual Sum of Squares• F• R2

Know how to do Regression on IBM SPSSInterpret a regression modelSlide 40

Linear Regression Aims

Summary of Linear Regression• Simple regression is a way of predicting one variable from

another.• We do this by fitting a statistical model to the data in the

form of a straight line.• This line is the line that best summarises the pattern of

data.• We have to assess how well the line fits the data using:• R squared which tells us how much variance is explained by the

model compared to how much variance there is to explain in the first place. It is a proportion of variance in the outcome variable that is shared by the predictor variable.

• F, which tells us how much variability the model can explain relative to how much it can’t explain (i.e., it’s the ratio of how good the model is compared to how bad the model is).

• The b-value, which tells us the gradient of the regression line and the strength of the relationship between a predictor and the outcome variable. If its significant (Sig. < 0.05 in the SPSS table) then the predictor variable significantly predicts the outcome variable.

Comparing Means Lecture Outline• Hypothesis testing• Categorical predictors in the linear model.• Comparing means from a linear model perspectiveComparing Means. Rationale for the testsT-tests: Interpretation & Reporting results• Independent• Dependent (aka paired, matched)Calculating an Effect Size• Assumptions

Rationale to the t-test continued

t =

observed differencebetween sample

means−

expected differencebetween population means(if null hypothesis is true)

estimate of the standard error of the difference between two sample means

Compares two means based on independent dataE.g., data from different groups of people

Independent t-testOn average, participants given a cloak of invisibility engaged in more acts of mischief (M = 5, SE = 0.48), than those not given a cloak (M = 3.75, SE = 0.55). This difference, was not significant t(22) = −1.71, p = .101; however, it did represent a medium-sized effect d = .65.

Paired t-testOn average, participants given a cloak of invisibility engaged in more acts of mischief (M = 5, SE = 0.48), than those not given a cloak (M = 3.75, SE = 0.55). This difference was significant t(11) = −3.80, p = .003 and represented a medium-sized effect d = .65.

Slide 46

Chi Squared LectureOutlineStatistical• Categorical Data• Contingency Tables• Chi-Square test• Likelihood Ratio Statistic• Odds Ratio

Diagnostic• Sensitivity • Specificity

Likelihood ratios in diagnostic testing.

Slide 47

To Sum Up …We approach categorical data in much the same way as any other kind of data:• we fit a model, we calculate the deviation between our model and the

observed data, and we use that to evaluate the model we’ve fitted.• We fit a linear model.

Two categorical variables• Pearson’s chi-square test• Likelihood ratio test

Effect Sizes• The odds ratio is a useful measure of the size of effect for categorical

data.

We will learn the theory behind and how to analyse in SPSS 3 non-parametric tests.These tests are relevant for comparing 2 means.

When independent t-test assumptions are broken use:The Wilcoxon rank-sum (Ws) test OrMann–Whitney (U) test

When paired t-test assumptions are broken use:Wilcoxon signed-rank (T) test

Non-Parametric testsLecture Outline

Videohttps://www.youtube.com/watch?feature=player_embedded&v=sOnqjkJTMaA9:40