Intro to Statistics for the Intro to Statistics for the Behavioral SciencesBehavioral Sciences

PSYC 1900PSYC 1900

Lecture 10: Hypothesis Tests for Two Lecture 10: Hypothesis Tests for Two Means: Related & Independent SamplesMeans: Related & Independent Samples

Clarification of Estimating Standard Clarification of Estimating Standard ErrorsErrors

The sample sd is an unbiased estimator of the population sd, but The sample sd is an unbiased estimator of the population sd, but any single sample sd is likely to underestimate the population sd.any single sample sd is likely to underestimate the population sd. Standard error calculations using the sample sd will usually Standard error calculations using the sample sd will usually

produce probability values that are too low (i.e., z scores that produce probability values that are too low (i.e., z scores that are too high).are too high).

Consequently, we use the t distribution, as opposed to the Consequently, we use the t distribution, as opposed to the normal, to adjust for this bias.normal, to adjust for this bias.

Sample variance

800.0750.0

700.0650.0

600.0550.0

500.0450.0

400.0350.0

300.0250.0

200.0150.0

100.050.0

0.0

Fre

qu

en

cy

1400

1200

1000

800

600

400

200

0

X N

One More Example for When One More Example for When Population Mean is KnownPopulation Mean is Known

One case where this is quite common is One case where this is quite common is testing whether participants’ responses testing whether participants’ responses are greater than chance.are greater than chance. For example, can participants identify For example, can participants identify

subliminally-presented stimuli.subliminally-presented stimuli. The comparison mean would be .50The comparison mean would be .50

Number of trials where responses are scored 0 or 1 Number of trials where responses are scored 0 or 1 for incorrect or correct.for incorrect or correct.

0 1: .50; : .50X XH H

One More Example for When One More Example for When Population Mean is KnownPopulation Mean is Known

We do not however know what the We do not however know what the population variance is.population variance is. We must estimate it using the sample We must estimate it using the sample

variance.variance. When we do this, we underestimate it, When we do this, we underestimate it,

resulting in lower standard errors and higher z-resulting in lower standard errors and higher z-scores (i.e. Type I errors).scores (i.e. Type I errors).

Therefore, we will use the t distribution.Therefore, we will use the t distribution.

X

ss

N

.50 .50

X

X Xt

ssN

One More Example for When One More Example for When Population Mean is KnownPopulation Mean is Known

Let’s assume the sample is 25 people, the Let’s assume the sample is 25 people, the mean accuracy =.56, and the sample mean accuracy =.56, and the sample sd=.09.sd=.09.

.09.018

25Xs

.56 .50(24) 3.33; .01

.018t p

Two-Tailed Significance Level

df .10 .05 .02 .01 10 1.812 2.228 2.764 3.169 15 1.753 2.131 2.602 2.947 20 1.725 2.086 2.528 2.845 25 1.708 2.060 2.485 2.787 30 1.697 2.042 2.457 2.750 100 1.660 1.984 2.364 2.626

Confidence LimitsConfidence Limits What is the 95% confidence interval for What is the 95% confidence interval for

accuracy?accuracy?

95%

2.06(.018) .56 .56 .04

.60; .52

.52 .60

upper lower

CI

Comparing Means from Comparing Means from Related SamplesRelated Samples

A more frequent case found in behavioral A more frequent case found in behavioral research is the comparison of two sets of research is the comparison of two sets of scores that are related (i.e., not scores that are related (i.e., not independent).independent). Pre-test / post-test designsPre-test / post-test designs Dyads Dyads Dependence implies that knowing a score in Dependence implies that knowing a score in

one distribution allows you better than chance one distribution allows you better than chance prediction about the related score in the other prediction about the related score in the other distribution.distribution.

Comparing Means from Comparing Means from Related SamplesRelated Samples

The null hypothesis in all cases is:The null hypothesis in all cases is:

This can be recast using difference scores.This can be recast using difference scores. Difference scores are calculated as the Difference scores are calculated as the

difference between the subjects’ performance difference between the subjects’ performance on two occasions (or the difference between on two occasions (or the difference between related data points)related data points)

0 1 2 1 1 2: ; :H H

0 1: 0DH

Comparing Means from Comparing Means from Related SamplesRelated Samples

Once we do this, we are again working with a Once we do this, we are again working with a “single” sample with a known prediction for the “single” sample with a known prediction for the mean.mean.

Thus, we can use a t test as we did previously, with Thus, we can use a t test as we did previously, with minor modifications.minor modifications.

We simply calculate the sd of the distribution of We simply calculate the sd of the distribution of difference scores and then use it to estimate the difference scores and then use it to estimate the associated standard error.associated standard error. Note that df’s again = N-1.Note that df’s again = N-1.

0 0

DD

D Dt

ssN

Advantages and Advantages and Disadvantages of Using Disadvantages of Using

Related SamplesRelated Samples Greatly reduces variabilityGreatly reduces variability

Variability is only with respect to change in dvVariability is only with respect to change in dv Provides perfect control for extraneous Provides perfect control for extraneous

variablesvariables Control group is perfectControl group is perfect

Require fewer participants Require fewer participants

Problems of order and carry-over effectsProblems of order and carry-over effects Experience at time 1 may alter scores at time 2 Experience at time 1 may alter scores at time 2

irrespective of any manipulationsirrespective of any manipulations

Effect SizeEffect Size

Can we use p-values to quantify the Can we use p-values to quantify the magnitude of an effect?magnitude of an effect?

No, as any given difference between means No, as any given difference between means will be more or less significant as a function will be more or less significant as a function of sample size (all else being equal).of sample size (all else being equal).

We need a measure of the magnitude of the We need a measure of the magnitude of the differences that is separate from sample size.differences that is separate from sample size.

Effect SizeEffect Size

Cohen’s d is a common effect size Cohen’s d is a common effect size measure for comparing two means.measure for comparing two means.

By convention: d=.2 small, d=.5 By convention: d=.2 small, d=.5 medium, d=.8 largemedium, d=.8 large Can be interpreted as “non-overlap” of Can be interpreted as “non-overlap” of

distributions.distributions.

1

1 2

X

X Xd

s

Comparing Means from Comparing Means from Independent SamplesIndependent Samples

This represents one of the most frequent This represents one of the most frequent cases encountered in behavioral research.cases encountered in behavioral research. No specific information about the population No specific information about the population

mean or variance is know.mean or variance is know. We randomly sample two groups and provide We randomly sample two groups and provide

one with a relevant manipulation.one with a relevant manipulation. We then wish to determine whether any We then wish to determine whether any

differences in group means is more likely differences in group means is more likely attributable to the manipulation or to sampling attributable to the manipulation or to sampling error.error.

Comparing Means from Comparing Means from Independent SamplesIndependent Samples

In this case, we have two independent In this case, we have two independent distributions, each with its own mean and distributions, each with its own mean and variance.variance.

We can easily determine what the We can easily determine what the difference is between the two means, but difference is between the two means, but we will need a measure of sampling error we will need a measure of sampling error with which to compare it.with which to compare it.

Unlike previous examples, we will need a Unlike previous examples, we will need a standard error for the difference between standard error for the difference between two means.two means.

Standard Errors for Mean Standard Errors for Mean Differences Between Independent Differences Between Independent

SamplesSamples The logic is similar to what we have done The logic is similar to what we have done

before.before. Assume two distinct population distributions. Assume two distinct population distributions.

Then, sample pairs of means from each. Then, sample pairs of means from each. The distribution of the mean differences The distribution of the mean differences

constitutes the appropriate sampling constitutes the appropriate sampling distribution.distribution. Its sd is the standard error for the t test.Its sd is the standard error for the t test. The variance sum law dictates that the variance The variance sum law dictates that the variance

of the sum (or difference) of two independent of the sum (or difference) of two independent variables is equal to the sum of their variances.variables is equal to the sum of their variances.

1 2 1 2

1 2 1 2

2 21 1

: , ,

' : , ,

means

sd sn n n n

The means and sd’s for the distributions and their differences are calculated as at right.

We know from the central limit theorem that the resulting sampling distributions will be normal.

But, the problem of not knowing what the true population sd is arises.

To deal with this problem, we must again use the t as opposed to normal distribution to calculate standard errors.

t Tests for Independent t Tests for Independent SamplesSamples

The formula is a generalization of the previous The formula is a generalization of the previous formula. The null is that the mean difference formula. The null is that the mean difference between the samples is zero.between the samples is zero. df’s = (ndf’s = (n11-1)+(n-1)+(n22-1)=n-1)=n11+n+n22-2.-2.

1 2

1 2 1 2 1 2 1 2

2 21 2

1 2

1 2

2 21 2

1 2

( ) ( )

( )

X X

X X X Xt

s s sn n

X Xt

s sn n

t Tests for Independent t Tests for Independent Samples with Unequal n’sSamples with Unequal n’s

In the previous formula, we assumed equal In the previous formula, we assumed equal condition n’s. Sometimes, however, the n of one condition n’s. Sometimes, however, the n of one sample exceeds the other, in which case its sample exceeds the other, in which case its variance is a better approximation of the variance is a better approximation of the population variance. In such cases, we pool the population variance. In such cases, we pool the variances using a weighted average.variances using a weighted average.2 2

1 1 2 2

1 2

1 2 1 2

2 22

1 21 2

( 1) ( 1)

2

( ) ( )

1 1

p

p pp

n s n ss

n n

X X X Xt

s ssn nn n

Assumptions for t TestsAssumptions for t Tests

Homogeneity of VarianceHomogeneity of Variance The population variances of the two The population variances of the two

distributions are equaldistributions are equal Implies that the variance of the two samples Implies that the variance of the two samples

should be relatively equalshould be relatively equal Heterogeneity is usually not a problem unless Heterogeneity is usually not a problem unless

the variance of one sample is greater than 3 the variance of one sample is greater than 3 times that of the other.times that of the other.

If this occurs, SPSS and other programs will If this occurs, SPSS and other programs will provide an both a normal and adjusted t value.provide an both a normal and adjusted t value.

The adjustment lowers the df’s which reduces The adjustment lowers the df’s which reduces chances for a type I error.chances for a type I error.

Assumptions for t TestsAssumptions for t Tests

Normality of DistributionsNormality of Distributions We assume that the sampled data are We assume that the sampled data are

normally distributed.normally distributed. They need not be exactly normal, but They need not be exactly normal, but

should be unimodal and symmetric.should be unimodal and symmetric. Really only a problem for small samples, Really only a problem for small samples,

as the CLT applies everywhere for large as the CLT applies everywhere for large samples.samples.

Effect SizeEffect Size

Cohen’s d is also used for independent Cohen’s d is also used for independent samples.samples.

The only difference is that we use the The only difference is that we use the pooled sd term.pooled sd term.

1 2

p

X Xd

s

Confidence LimitsConfidence Limits What is the 95% confidence interval for What is the 95% confidence interval for

accuracy?accuracy?

1 295% 1 2 .05 X XCI X X t s