+ All Categories
Home > Documents > Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as...

Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as...

Date post: 22-Dec-2015
Category:
View: 221 times
Download: 1 times
Share this document with a friend
68
Nonparametric tests II as randomisation tests
Transcript
Page 1: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Nonparametric tests II

as randomisation tests

Page 2: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Lecture Outline

• Background: Nonparametric tests as randomisation tests– The sign test– The Wilcoxon signed ranks test– The Mann-Whitney test

• General remarks on randomisation tests

• Brief Review of the course so far

Page 3: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

after before640.0 1050.0 70.0 84.0 83.0 77.0 64.0 110.0420.0 440.0 6.4 4.8 26.0 48.0 2.2 16.0 75.0 340.0 16.0 430.0

Page 4: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

after before change640.0 1050.0 -410.0 70.0 84.0 -14.0 83.0 77.0 6.0 64.0 110.0 -46.0420.0 440.0 -20.0 6.4 4.8 1.6 26.0 48.0 -22.0 2.2 16.0 -13.8 75.0 340.0 -265.0 16.0 430.0 -414.0

Page 5: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

after before change640.0 1050.0 -410.0 70.0 84.0 -14.0 83.0 77.0 6.0 64.0 110.0 -46.0420.0 440.0 -20.0 6.4 4.8 1.6 26.0 48.0 -22.0 2.2 16.0 -13.8 75.0 340.0 -265.0 16.0 430.0 -414.0

schange-414.0-410.0-265.0-46.0-22.0-20.0-14.0-13.81.66.0

Page 6: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > stest 'change'

Sign Test for Median

Sign test of median=0.000 versus N.E. 0.000

N BELOW EQUAL ABOVE P-VALUE MEDIANchange 10 8 0 2 0.1094 -21.00MTB >

Page 7: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign

WilcoxonSignedRanks

Mann-Whitney

Page 8: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

WilcoxonSignedRanks

Mann-Whitney

Page 9: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

WilcoxonSignedRanks

Mann-Whitney

Page 10: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

WilcoxonSignedRanks

Mann-Whitney

Page 11: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s

WilcoxonSignedRanks

Mann-Whitney

Page 12: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Mann-Whitney

Page 13: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > stest 'change'

Sign Test for Median

Sign test of median=0.000 versus N.E. 0.000

N BELOW EQUAL ABOVE P-VALUE MEDIANchange 10 8 0 2 0.1094 -21.00MTB >

Page 14: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > stest 'change'

Sign Test for Median

Sign test of median=0.000 versus N.E. 0.000

N BELOW EQUAL ABOVE P-VALUE MEDIANchange 10 8 0 2 0.1094 -21.00MTB >

which added = number of non-zero datapoints (in this case there are no zeroes)

Page 15: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

So if we take ten items that might be plus or minus,

Page 16: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

So if we take ten items that might be plus or minus,and randomly choose them, we get the set of relevant comparisons for our dataset of 8 minus and 2 plus. This is the randomisation part of the test.

Page 17: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

So if we take ten items that might be plus or minus,and randomly choose them, we get the set of relevant comparisons for our dataset of 8 minus and 2 plus. This is the randomisation part of the test.

To decide whether our actual dataset is extreme in the distribution, we calculate the test statistic in each case - just the number of plusses. We count in what fraction of cases, the relevant comparison has a more extreme number of plusses, that is, either 2 or fewer, or 8 or more.

Page 18: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > stest 'change'

Sign Test for Median

Sign test of median=0.000 versus N.E. 0.000

N BELOW EQUAL ABOVE P-VALUE MEDIANchange 10 8 0 2 0.1094 -21.00MTB >

Page 19: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

109876543210

200

100

0

C3

Page 20: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

109876543210

200

100

0

C3

Page 21: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

The truth about confidence intervals

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

Page 22: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > stest 'change'

Sign Test for Median

Sign test of median=0.000 versus N.E. 0.000

N BELOW EQUAL ABOVE P-VALUE MEDIANchange 10 8 0 2 0.1094 -21.00MTB >

Page 23: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > stest 'change'

Sign Test for Median

Sign test of median=0.000 versus N.E. 0.000

N BELOW EQUAL ABOVE P-VALUE MEDIANchange 10 8 0 2 0.1094 -21.00MTB >

Page 24: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > stest 0 c3

Sign Test for Median: C3

Sign test of median = 0.00000 versus not = 0.00000

N Below Equal Above P MedianC3 10 8 0 2 0.1094 -21.00

MTB > stest 10 c3

Sign Test for Median: C3

Sign test of median = 10.00 versus not = 10.00

N Below Equal Above P MedianC3 10 10 0 0 0.0020 -21.00

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

Page 25: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0median N Below Equal Above P Median

-50 10 3 0 7 0.3438 -21.00-40 10 4 0 6 0.7539 -21.00-35 10 4 0 6 0.7539 -21.00-30 10 4 0 6 0.7539 -21.00-25 10 4 0 6 0.7539 -21.00-20 10 5 1 4 1.0000 -21.00-15 10 6 0 4 0.7539 -21.00-10 10 8 0 2 0.1094 -21.00-5 10 8 0 2 0.1094 -21.000 10 8 0 2 0.1094 -21.005 10 9 0 1 0.0215 -21.0010 10 10 0 0 0.0020 -21.0015 10 10 0 0 0.0020 -21.00

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

Page 26: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0median N Below Equal Above P Median

-50 10 3 0 7 0.3438 -21.00-40 10 4 0 6 0.7539 -21.00-35 10 4 0 6 0.7539 -21.00-30 10 4 0 6 0.7539 -21.00-25 10 4 0 6 0.7539 -21.00-20 10 5 1 4 1.0000 -21.00-15 10 6 0 4 0.7539 -21.00-10 10 8 0 2 0.1094 -21.00-5 10 8 0 2 0.1094 -21.000 10 8 0 2 0.1094 -21.005 10 9 0 1 0.0215 -21.0010 10 10 0 0 0.0020 -21.0015 10 10 0 0 0.0020 -21.00

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

Page 27: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0median N Below Equal Above P Median

-50 10 3 0 7 0.3438 -21.00-40 10 4 0 6 0.7539 -21.00-35 10 4 0 6 0.7539 -21.00-30 10 4 0 6 0.7539 -21.00-25 10 4 0 6 0.7539 -21.00-20 10 5 1 4 1.0000 -21.00-15 10 6 0 4 0.7539 -21.00-10 10 8 0 2 0.1094 -21.00-5 10 8 0 2 0.1094 -21.000 10 8 0 2 0.1094 -21.005 10 9 0 1 0.0215 -21.0010 10 10 0 0 0.0020 -21.0015 10 10 0 0 0.0020 -21.00

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

Page 28: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0median N Below Equal Above P Median

-50 10 3 0 7 0.3438 -21.00-40 10 4 0 6 0.7539 -21.00-35 10 4 0 6 0.7539 -21.00-30 10 4 0 6 0.7539 -21.00-25 10 4 0 6 0.7539 -21.00-20 10 5 1 4 1.0000 -21.00-15 10 6 0 4 0.7539 -21.00-10 10 8 0 2 0.1094 -21.00-5 10 8 0 2 0.1094 -21.000 10 8 0 2 0.1094 -21.005 10 9 0 1 0.0215 -21.0010 10 10 0 0 0.0020 -21.0015 10 10 0 0 0.0020 -21.00

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

Page 29: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0median N Below Equal Above P Median

-50 10 3 0 7 0.3438 -21.00-40 10 4 0 6 0.7539 -21.00-35 10 4 0 6 0.7539 -21.00-30 10 4 0 6 0.7539 -21.00-25 10 4 0 6 0.7539 -21.00-20 10 5 1 4 1.0000 -21.00-15 10 6 0 4 0.7539 -21.00-10 10 8 0 2 0.1094 -21.00-5 10 8 0 2 0.1094 -21.000 10 8 0 2 0.1094 -21.005 10 9 0 1 0.0215 -21.0010 10 10 0 0 0.0020 -21.0015 10 10 0 0 0.0020 -21.00

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

Page 30: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0median N Below Equal Above P Median

-50 10 3 0 7 0.3438 -21.00-40 10 4 0 6 0.7539 -21.00-35 10 4 0 6 0.7539 -21.00-30 10 4 0 6 0.7539 -21.00-25 10 4 0 6 0.7539 -21.00-20 10 5 1 4 1.0000 -21.00-15 10 6 0 4 0.7539 -21.00-10 10 8 0 2 0.1094 -21.00-5 10 8 0 2 0.1094 -21.000 10 8 0 2 0.1094 -21.005 10 9 0 1 0.0215 -21.0010 10 10 0 0 0.0020 -21.0015 10 10 0 0 0.0020 -21.00

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

Page 31: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

Page 32: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

The green values cannot be rejected at the 5% level, while the red values can.

Page 33: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

The green values cannot be rejected at the 5% level, while the red values can.

The range of green values is therefore the 95% confidence interval for the median based on the sign test.

Page 34: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

The real definition of 95% confidence interval

• is “the set of values of a parameter that cannot be rejected at the 5% level”

• is therefore not “the set of values that the parameter has a 95% chance of belonging to”, as many textbooks claim. (This is called a “fiducial interval”.)

Page 35: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > sinterval 'change'

Sign Confidence Interval

Sign confidence interval for median

ACHIEVED POSI N MEDIAN CONFIDENCE CONFIDENCE INTERVAL TIONchange 10 -21.000 0.8906 (-265.000, -13.800) 3 0.9500 (-314.640, -8.528) NLI 0.9785 (-410.000, 1.600) 2

MTB >

Page 36: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0median N Below Equal Above P Median

-50 10 3 0 7 0.3438 -21.00-40 10 4 0 6 0.7539 -21.00-35 10 4 0 6 0.7539 -21.00-30 10 4 0 6 0.7539 -21.00-25 10 4 0 6 0.7539 -21.00-20 10 5 1 4 1.0000 -21.00-15 10 6 0 4 0.7539 -21.00-10 10 8 0 2 0.1094 -21.00-5 10 8 0 2 0.1094 -21.000 10 8 0 2 0.1094 -21.005 10 9 0 1 0.0215 -21.0010 10 10 0 0 0.0020 -21.0015 10 10 0 0 0.0020 -21.00

. .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

Page 37: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

after before change640.0 1050.0 -410.0 70.0 84.0 -14.0 83.0 77.0 6.0 64.0 110.0 -46.0420.0 440.0 -20.0 6.4 4.8 1.6 26.0 48.0 -22.0 2.2 16.0 -13.8 75.0 340.0 -265.0 16.0 430.0 -414.0

schange-414.0-410.0-265.0-46.0-22.0-20.0-14.0-13.81.66.0

97.85%89.06%

97.85%89.06%

Page 38: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > sinterval 'change'

Sign Confidence Interval

Sign confidence interval for median

ACHIEVED POSI N MEDIAN CONFIDENCE CONFIDENCE INTERVAL TIONchange 10 -21.000 0.8906 (-265.000, -13.800) 3 0.9500 (-314.640, -8.528) NLI 0.9785 (-410.000, 1.600) 2

MTB > . .. . . .: ..---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

Page 39: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Why does Minitab give three confidence intervals for the sign test?

• the p-value for rejecting a value changes in a step function at observed values

• so exact confidence intervals are given between observed values, at whatever level of confidence is attained

• the NLI (Non-Linear Interpolation) confidence interval is a confidence trick

Page 40: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Lecture Outline

• Background: Nonparametric tests as randomisation tests– The sign test– The Wilcoxon signed ranks test– The Mann-Whitney test

• General remarks on randomisation tests

• Brief Review of the course so far

Page 41: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

after before change640.0 1050.0 -410.0 70.0 84.0 -14.0 83.0 77.0 6.0 64.0 110.0 -46.0420.0 440.0 -20.0 6.4 4.8 1.6 26.0 48.0 -22.0 2.2 16.0 -13.8 75.0 340.0 -265.0 16.0 430.0 -414.0

schange-414.0-410.0-265.0-46.0-22.0-20.0-14.0-13.81.66.0

Page 42: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Mann-Whitney

Page 43: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Symmetryabout zero

Mann-Whitney

Page 44: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Symmetryabout zero

Absolutevalues ofdifferences

Mann-Whitney

Page 45: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Symmetryabout zero

Absolutevalues ofdifferences

Sign of eachdifference

Mann-Whitney

Page 46: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Symmetryabout zero

Absolutevalues ofdifferences

Sign of eachdifference

Sum of ranks ofnegativedifferences

Mann-Whitney

Page 47: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Symmetryabout zero

Absolutevalues ofdifferences

Sign of eachdifference

Sum of ranks ofnegativedifferences

Numberedcoin tossing

Mann-Whitney

Page 48: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > wtest 'change'

Wilcoxon Signed Rank Test

TEST OF MEDIAN = 0.000 VERSUS MEDIAN N.E. 0.000

N FOR WILCOXON ESTIMATED N TEST STATISTIC P-VALUE MEDIANchange 10 10 3.0 0.014 -46.00MTB >

MTB > stest 'change'

Sign Test for Median

Sign test of median=0.000 versus N.E. 0.000

N BELOW EQUAL ABOVE P-VALUE MEDIANchange 10 8 0 2 0.1094 -21.00

Page 49: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > wtest 'change'

Wilcoxon Signed Rank Test

TEST OF MEDIAN = 0.000 VERSUS MEDIAN N.E. 0.000

N FOR WILCOXON ESTIMATED N TEST STATISTIC P-VALUE MEDIANchange 10 10 3.0 0.014 -46.00MTB >

MTB > stest 'change'

Sign Test for Median

Sign test of median=0.000 versus N.E. 0.000

N BELOW EQUAL ABOVE P-VALUE MEDIANchange 10 8 0 2 0.1094 -21.00

Page 50: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > wtest 'change'

Wilcoxon Signed Rank Test

TEST OF MEDIAN = 0.000 VERSUS MEDIAN N.E. 0.000

N FOR WILCOXON ESTIMATED N TEST STATISTIC P-VALUE MEDIANchange 10 10 3.0 0.014 -46.00MTB >

MTB > stest 'change'

Sign Test for Median

Sign test of median=0.000 versus N.E. 0.000

N BELOW EQUAL ABOVE P-VALUE MEDIANchange 10 8 0 2 0.1094 -21.00

The Wilcoxon test is more powerful than the Sign Test

Page 51: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > sinterval 'change'

Sign Confidence Interval

Sign confidence interval for median ACHIEVED POSI N MEDIAN CONFIDENCE CONFIDENCE INTERVAL TIONchange 10 -21.000 0.8906 (-265.000, -13.800) 3 0.9500 (-314.640, -8.528) NLI 0.9785 (-410.000, 1.600) 2

MTB > winterval 'change'

Wilcoxon Signed Rank Confidence Interval

ESTIMATED ACHIEVED N MEDIAN CONFIDENCE CONFIDENCE INTERVALchange 10 -46 94.7 ( -218, -8)MTB >

Page 52: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

MTB > sinterval 'change'

Sign Confidence Interval

Sign confidence interval for median ACHIEVED POSI N MEDIAN CONFIDENCE CONFIDENCE INTERVAL TIONchange 10 -21.000 0.8906 (-265.000, -13.800) 3 0.9500 (-314.640, -8.528) NLI 0.9785 (-410.000, 1.600) 2

MTB > winterval 'change'

Wilcoxon Signed Rank Confidence Interval

ESTIMATED ACHIEVED N MEDIAN CONFIDENCE CONFIDENCE INTERVALchange 10 -46 94.7 ( -218, -8)MTB >

The Wilcoxon confidence interval is narrower

Page 53: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Sign vs Wilcoxon Signed Ranks

Page 54: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Sign vs Wilcoxon Signed Ranks

• Less powerful • More powerful

Page 55: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Sign vs Wilcoxon Signed Ranks

• Less powerful– Less sensitive– Wider confidence

intervals

• More powerful– More sensitive– Narrower confidence

intervals

Page 56: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Sign vs Wilcoxon Signed Ranks

• Less powerful– Less sensitive– Wider confidence

intervals

• Uses less information– only sign of

difference

• More powerful– More sensitive– Narrower confidence

intervals

• Uses more information– also size of

difference

Page 57: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

after before change640.0 1050.0 -410.0 70.0 84.0 -14.0 83.0 77.0 6.0 64.0 110.0 -46.0420.0 440.0 -20.0 6.4 4.8 1.6 26.0 48.0 -22.0 2.2 16.0 -13.8 75.0 340.0 -265.0 16.0 430.0 -414.0

schange-414.0-410.0-265.0-46.0-22.0-20.0-14.0-13.81.66.0

Page 58: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Lecture Outline

• Background: Nonparametric tests as randomisation tests– The sign test– The Wilcoxon signed ranks test– The Mann-Whitney test

• General remarks on randomisation tests

• Brief Review of the course so far

Page 59: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Symmetryabout zero

Absolutevalues ofdifferences

Sign of eachdifference

Sum of ranks ofnegativedifferences

Numberedcoin tossing

Mann-Whitney

Page 60: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Symmetryabout zero

Absolutevalues ofdifferences

Sign of eachdifference

Sum of ranks ofnegativedifferences

Numberedcoin tossing

Mann-Whitney

Two groupsfrom samedistribution

Page 61: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Symmetryabout zero

Absolutevalues ofdifferences

Sign of eachdifference

Sum of ranks ofnegativedifferences

Numberedcoin tossing

Mann-Whitney

Two groupsfrom samedistribution

Set of ranks;numbers ineach group

Page 62: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Symmetryabout zero

Absolutevalues ofdifferences

Sign of eachdifference

Sum of ranks ofnegativedifferences

Numberedcoin tossing

Mann-Whitney

Two groupsfrom samedistribution

Set of ranks;numbers ineach group

Groupmembershipof datapoints

Page 63: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Symmetryabout zero

Absolutevalues ofdifferences

Sign of eachdifference

Sum of ranks ofnegativedifferences

Numberedcoin tossing

Mann-Whitney

Two groupsfrom samedistribution

Set of ranks;numbers ineach group

Groupmembershipof datapoints

Sum of ranks infirst group

Page 64: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

H0 FixedInformation

RandomisedInformation

Test Statistic Notes

Sign Prob of +equalsProb of -

Number ofnon-zerodifferences

Sign of eachdifference

Number of +’s BinomialDistributionin Theory

WilcoxonSignedRanks

Symmetryabout zero

Absolutevalues ofdifferences

Sign of eachdifference

Sum of ranks ofnegativedifferences

Numberedcoin tossing

Mann-Whitney

Two groupsfrom samedistribution

Set of ranks;numbers ineach group

Groupmembershipof datapoints

Sum of ranks infirst group

Randomassignmentto groups

Page 65: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Lecture Outline

• Background: Nonparametric tests as randomisation tests– The sign test– The Wilcoxon signed ranks test– The Mann-Whitney test

• General remarks on randomisation tests

• Brief Review of the course so far

Page 66: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

In these randomisation tests,

• there is a simple direct connection between the null hypothesis and the randomisation procedure

• there is freedom of choice of test statistic

• estimation relies on scales of measurement and so is not as ‘principled’ as hypothesis tests

Page 67: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Lecture Outline

• Background: Nonparametric tests as randomisation tests– The sign test– The Wilcoxon signed ranks test– The Mann-Whitney test

• General remarks on randomisation tests

• Brief Review of the course so far

Page 68: Nonparametric tests II as randomisation tests. Lecture Outline Background: Nonparametric tests as randomisation tests – The sign test – The Wilcoxon signed.

Last remarks• Randomisation tests are powerful tools

• All parametric and nonparametric tests can be understood as randomisation tests

• Nowadays they are used when no others can be used.

• NEXT WEEK: Conclusion to course and some exam questions. READ Chapter 14 of textbook.


Recommended