T-tests continued

Outline

Review the One-sample case Independent samples

Basics Unequal N Example Issues

Paired samples Basics Examples

Comparing formulas Conclusion

Two samples

Previously compared sample mean to a known population mean

Now want to compare two samples Null hypothesis: the mean of the population

of scores from which one set of data is drawn is equal to the mean of the population of the second data set H0: 1=2 or 1 - 2 = 0

Independent samples

Consider the original case

Now want to consider not just 1 mean but the difference between 2 means

The ‘nil’ hypothesis, as before, states there will be no difference H0: 1 - 2 = 0

Xs

Xt

Which leads to...

Now statistic of interest is not a single mean, but the difference between means:

Mean of the ‘sampling distribution of the differences between means’ is:

21 XX

1 2

Variability

Standard error of the difference between means

Since there are two independent variables, variance of the difference between means equals sum of their variances

X 1 X 2

X1

2 X 2

2 12

n1

22

n2

Same problem, same solution

Usually we do not know population variance (standard deviation)

Again use sample to estimate it Result is distributed as t (rather than z)

Formula

All of which leads to:

2

22

1

21

21212121

21

ns

ns

XX

s

XXt

XX

But...

If we are dealing with a ‘nil’ null hypothesis:

So the formula reduces to:

Across the 2 samples we have (n1-1) and (n2-1) degrees of freedom1

df = (n1-1) + (n2-1) = n1 + n2 - 2

tX1 X 2 sX1 X 2

X1 X 2 s12

n1 s2

2

n2

1 2 0

Unequal sample sizes

Assumption: independent samples t test requires samples come from populations with equal variances

Two estimates of variance (one from each sample) Generate an overall estimate that reflects the fact

that bigger samples offer better estimates Oftentimes the sample sizes will be unequal

Weighted average

Which gives us:

Final result is:

spooled2

(n1 1)s12 (n2 1)s2

2

df1 df2

spooled2

(n1 1)s12 (n2 1)s2

2

n1 n2 2

Example

New drug MemoPlus is supposed to aid memory. 5 people take the drug and we compare them to a control group of 10 people who take a placebo.Look at their free recall of 10 items on a word list.Control: mean = 4.5, SD = 2.0Drug: mean = 6.0, SD = 3.0

Start with variance

Using

Get:

[note pooled variance nearer to group’s with larger n]

spooled2

(n1 1)s12 (n2 1)s2

2

n1 n2 2

2 22 (5 1)(2 ) (10 1)(3 )

7.465 10 2pooleds

1 2

2

1 2

1 1 1 17.46 1.50

5 10pX Xs s

n n

Calculate t

Enter the values:

Critical value approach t.05(13) = 2.16

Specific p p = .17

1 2

1 2 4.5 6.0 1.51.00

1.50 1.50X X

X Xt

s

Conclusion?

Our result does not give us a strong case for rejecting the null. MemoPlus, like Ginko Biloba1, does nothing special for memory.

More issues with the t-test

In the two-sample case we have an additional assumption (along with normal distribution of sample scores and independent observations)

We assume that there are equal variances in the groups Recall our homoscedasticity discussion, it is the exact

same assumption1

Often this assumption is untenable, and the results, like other violations result in using calculated probabilities that are inaccurate Can use a correction, e.g. Welch’s t

More issues with the t-test

It is one thing to say that they are unequal, but what might that mean?

Consider a control and treatment group, treatment group variance is significantly greater

While we can do a correction, the unequal variances may suggest that those in the treatment group vary widely in how they respond to the treatment (e.g. half benefit, ¼ unchanged, ¼ got worse)

Another reason for heterogeneity of variance may be related to an unreliable measure being used

No version of the t-test takes either into consideration Other techniques, assuming enough information has been

gathered, may be more appropriate (e.g. a hierarchical approach), and more reliable measures may be attainable

T-test for dependent samples

Paired sample t-test Comparing two distributions

that are likely to be correlated in some way

Gist: some sort of meaningful relationship between our two samples.

When are two distributions likely to be correlated?

Natural Pairs: Comparing the scores of two

subjects who are related naturally E.g. twins.

Matched Pairs: Pairs two subjects on some

characteristic that is likely to be important to the variable that you are studying. Members of the pair are randomly assigned to the two conditions. E.g. married couples

Repeated Measures: Same person at two different

times.

Difference scores

While we technically have two samples of data, the test regards the single sample of difference scores from one pair to another

Transform the paired scores into a single set of scores Find the difference for each case

Now are we have a single set of scores and a known population mean, a familiar situation Use the one sample t test concept

If the null hypothesis is true then on average there should be no difference Thus mean difference is 0 ( D = 0 )

Paired t test

Where = Mean of the difference scores

sD = Standard deviation of the difference scoresn = Number of difference scores (# pairs)df = n-1

tD DSD

D 0SDn

DSDn

D

Example

We want to know if dog owners affection for their dog changes after they take a course in dog psychology

We measure their responses on the the Dog Affection Inventory both before and after the course

Data

Before After Difference8 22 -146 10 -4

12 21 -96 20 -14

10 17 -712 15 -316 14 214 16 -27 12 -59 13 -4

Average 10 16 -6Sd 3.431877 4 5.120764

t (9)= 3.71, p = .005 2

Conclusion ?

71.3

1012.5

00.61

t

ts and Zs

1 2

1 2 1 2( ) ( )

X

X

X

X

D

D

X X

X Xz

Xz

Xt

s

Dt

s

X Xt

s

The good and the bad regarding t-tests The good If assumptions are met, t-test is fine When assumptions aren’t met, t-test may still be robust with

regard to type I error in some situations With equal n and normal populations HoV violations won’t

increase type I much With non-normal distributions with equal variances, type I

error rate is maintained also The bad Even small departures from the assumptions result in power

taking a noticeable hit (type II error is not maintained) t-statistic, CIs will be biased