Confidence Intervals with small samples
What is small sample size?Less than 30 (one of those special numbers in stats)Sampling distribution tends to spread outStandard normal curve not adequate
Use the t-distributionsTheoretical sampling distributionsFlatter peak, wider at the tailsApproaches normal curve as sample size increasesUse values of t instead of zDescribed by degrees of freedom (n-1 for confidence
intervals)
Hypothesis testing• Are the differences– Representative of “real” effects?– Just by chance?
• Null hypothesis ( HO)– Means are not different– Stated in terms of population parameter– μA=μB
• Alternative hypothesis (H1)– Difference too large to be by chance– μA≠μB
– May be directional or non-directional
TruthD
ecis
ion
Ho is True
Do Not Reject Ho
α
Correct
β
Reject Ho
Ho is False
CorrectType I error
Type II error
Type I Error• Significance Level (Alpha level , α level)• Your choice of how much risk you are willing to
take of saying there is a difference when there really is no difference
• Set this before the study– Conventionally is 0.05• This is arbitrary, but almost always what we choose
– Choose the level based on the Type I error concern
Type I Error
• Probability Values (Evaluated after the study)– Probability of finding this big a difference by chance
• p = .07 of this big a difference by chance
– You are not stating the probability of the inverse• p = .93 that it is real difference is not appropriate
– Compare p-value to alpha level if greater
– Compare your p-value (calculated after the study) with your α level (set before the study)
– If the p-value is less than α, reject the null– If the p-value is greater than α, fail to reject the null
Type II ErrorStatistical Power
• Beta (β)– Probability of failing to reject a false HO(null hypotheses)
– Β of 0.20 is 20% chance we will make a Type II error• Statistical Power– Complement of β (not compliment)– In this example 0.80 (1.00 – 0.20 = 0.80)– 80% probability of correctly rejecting the null
• Before: a priori - power used to determine sample size• After: post hoc – power reported if HO not rejected
“If there was a difference, could we have found it?”
Determinants ofStatistical Power
• Significance Criterion– As α increases, power increases(As α increases from 0.05 to 0.10, power increases)
• Variance– As variance decreases, power increases
• Sample size– As sample size increases, power increases
• Effect size (difference b/w the group means)
– As effect size increases, power increases
z-• z - score represents the distance between:– A sample score and– Sample mean– Divided by the standard deviation
You will see this in osteoporosis scores(+2 for z- score is 2 SD away from a healthily woman mean)
• z - ratio represents the distance between:– A sample mean and– Population mean– Divided by the standard error of the mean
Critical Region That portion of the curve above and below z
If calculated z > critical z, reject HO
One or two tailed test? Non directional hypothesis– two tailed
z of 2.00 encompasses 95.44 % (non-critical) 4.66% is the critical area
z of 1.96 encompass 95%, Critical region is 5%, 2.5% in each tail of a non-directional test
Directional hypothesis– one tailed z of 1.645 encompasses 95%
Critical region is 5%, all in one tail of a directional test, while NON-direction will be 2.5% Practically, you are disregarding everything in the other tail
Table A.1 back of P & W
Figure A: Intervention 1 is different than Intervention 2
Figure B: Intervention 1 is less effective than Intervention 2
Parametric Statistics• Used to estimate population parameters• Based on assumptions– Randomly drawn from a normally distributed population– Variances in the samples equal (at least roughly)– Interval or ratio scale
• Classically, if assumptions violated, use non-parametric tests
• Many view parametric stats as Robust enough to withstand even major violation
t-test Examines two means
Two groups Two conditions/two performances
Statistical significance based onDifference in the means
Between the groups The effect size
Variance Within the groups How variable are the scores
Fig 19.1
t-test Based on a ratio
Difference between group means/Variability within groups
Difference between means Treatment effect and error variance
One mean- second mean and variability between the groups. In both the numerator and denominator of t-test ratio, so holds it to zero.
Variability within groups Error variance alone
Equal and unequal variances affect t-ratio SPSS and most other packages automatically test for this. Where?
Ratio can be written: Treatment effect and error variance/Error variance
NOTE:Error variance Not mistakes Is anything that is not due to the independent variable
t-test If the null is true
Ratio reduces to: Error/Error
The bigger the difference - The bigger the ratioHow does the ratio get bigger?
This ratio is compared to the critical Determines significance
Is the ratio significant? Based on critical value (but now t instead of z) Entering arguments (Table A.2)
Alpha level (almost always 0.05 Degrees of freedom ( one or a few less than n )
CI can be constructed for where the true mean difference lies
t-test Independent t-test
Usually random assignmentCan be convenience assignmentNo inherent relationship between the groupsDegrees of freedom = total sample size – 2
Paired t-testAn inherent relationship between the groups
Self (repeated measure test) Twins
Difference scores for each pair comparedDegrees of freedom = number of paired scores – 1
Find this in P & W or in an SPSS output table – don’t calculate for this class!
ANOVA Examines three (or more) means
Three (or more) groups Three (or more) conditions/ three (or more) performances
Statistical significance based on Difference in the means
Between the groups The effect size
Variance Within the groups How variable are the scores
This should sound familiar
ANOVA Based on ratio
Treatment effect and error variance/Error variance
For ANOVA it is the F-ratio Derived from the Sum of Squares (SS) Larger the SS the larger the ______________? Calculate SS
(each score minus sample mean, square each result, sum them)
Then determine the Mean Square (MS) MSb = SSb/dfb (dfb = one less than the number of groups) MSe = SSe/df e (df e = total N – number of groups)
F statistic is the ratio = MSb/Mse
Ratio of the between groups variance to error variance
Find this in P & W or in an SPSS output table – don’t calculate for this class!