Date post: | 24-Dec-2015 |
Category: |
Documents |
Upload: | elmer-fleming |
View: | 213 times |
Download: | 0 times |
ANOVA3/19/12
• Mini Review of simulation versus formulas and theoretical distributions
• Analysis of Variance (ANOVA) to compare means:testing for a difference in means between multiple groups
Section 8.1 Professor Kari Lock MorganDuke University
• Anonymous Midterm Evaluation (due TODAY, 5pm)
• Project 1 (due Thursday, 3/22, 5pm)
• Homework 7 (due Monday, 3/26)NO LATE HOMEWORK ACCEPTED!Turn in by Friday, 3/23, 5pm to get it graded
before Exam 2.
To Do
Two Options for p-valuesWe have learned two ways of calculating p-values:
The only difference is how to create a distribution of the statistic, assuming the null is true:
1)Simulation (Randomization Test): • Directly simulate what would happen, just by
random chance, if the null were true
2)Formulas and Theoretical Distributions: • Use a formula to create a test statistic for which
we know the theoretical distribution when the null is true, if sample sizes are large enough
Two Options for IntervalsWe have learned two ways of calculating intervals:
1)Simulation (Bootstrap): • Assess the variability in the statistic by
creating many bootstrap statistics
2)Formulas and Theoretical Distributions: • Use a formula to calculate the standard error
of the statistic, and use the normal or t-distribution to find z* or t*, if sample sizes are large enough
Pros and Cons1)Simulation Methods
PROS:• Methods tied directly to concepts, emphasizing conceptual
understanding• Same procedure for every statistic• No formulas or theoretical distributions to learn and
distinguish between• Works for any sample size• Minimal math needed
CONS:• Need entire dataset (if quantitative variables)• Need a computer• Newer approach, so different from the way most people do
statistics
Pros and Cons2)Formulas and Theoretical Distributions
PROS:• Only need summary statistics• Only need a calculator• The approach most people take
CONS:• Plugging numbers into formulas does little for conceptual
understanding• Many different formulas and distributions to learn and
distinguish between• Harder to see the big picture when the details are different for
each statistic• Doesn’t work for small sample sizes• Requires more math and background knowledge
Two Options
• If the sample size is small, you have to use simulation methods
• If the sample size is large, you can use whichever method you prefer
• It is redundant to use both methods, unless you want to check your answers
Accuracy• The accuracy of simulation methods depends on the number of simulations (more simulations = more accurate)
• The accuracy of formulas and theoretical distributions depends on the sample size (larger sample size = more accurate)
• If the sample size is large and you have generated many simulations, the two methods should give essentially the same answer
Multiple Categories
•So far, we’ve learned how to do inference for a difference in means IF the categorical variable has only two categories
•Today, we’ll learn how to do hypothesis tests for a difference in means across multiple categories
Hypothesis Testing
1.State Hypotheses
2.Calculate a statistic, based on your sample data
3.Create a distribution of this statistic, as it would be observed if the null hypothesis were true
4.Measure how extreme your test statistic from (2) is, as compared to the distribution generated in (3)
test statistic
Hypotheses
To test for a difference in means across k groups:
0 1 2: ...: At least one
k
a i j
HH
Test Statistic
Why can’t use the familiar formula
to get the test statistic?
•More than one sample statistic•More than one null value
We need something a bit more complicated…
sample statistic null value
SE
Difference in Means
Whether or not two means are significantly different depends on
• How far apart the means are
• How much variability there is within each group
Difference in Means
group1 group2
02
46
810
group1 group2
02
46
810
1214
group1 group2
4.5
5.0
5.5
6.0
6.5
group1 group2
02
46
810
group1 group2
02
46
810
1214
group1 group2
4.5
5.0
5.5
6.0
6.5
group1 group2
02
46
810
group1 group2
02
46
810
1214
group1 group2
4.5
5.0
5.5
6.0
6.5
1
2
1 2
65
2
X
ssX
1
2
1 2
95
2
X
ssX
1
2
1 2
5
0.6
2s
XX
s
Analysis of Variance
•Analysis of Variance (ANOVA) compares the variability between groups to the variability within groups
Total Variability
VariabilityBetween Groups
VariabilityWithin Groups
Analysis of Variance
If the groups are actually different, then
a) the variability between groups should be higher than the variability within groups
b) the variability within groups should be higher than the variability between groups
Discoveries for Today
•How to measure variability between groups?
•How to measure variability within groups?
•How to compare the two measures?
•How to determine significance?
Notation•k = number of groups
•nj = number of units in group j
•n = overall number of units = n1 + n2 + … + nk
Discoveries for Today
•How to measure variability between groups?
•How to measure variability within groups?
•How to compare the two measures?
•How to determine significance?
Sums of Squares
•We will measure variability as sums of squared deviations (aka sums of squares)
•familiar?
Sums of Squares
2
1
n
ii
X X
Total Variability
VariabilityBetween Groups
VariabilityWithin Groups
2
1
k
j jj
n X X
2
,11
jnk
i j jij
X X
overall mean
data value i
overall mean
mean in group j mean in
group j
ith data value in group j
Sum over all data values Sum over all groups Sum over all data values
Deviations
Group 1
Group 2
X
Total iX X
Overall Mean
1X
Group 1 Mean
,
Within i j jX X
1
BetweenX X
Sums of Squares
2
1
n
ii
X X
Total Variability
VariabilityBetween Groups
VariabilityWithin Groups
2
1
k
j jj
n X X
2
,11
jnk
i j jij
X X
SST (Total sum of squares)
SSG(sum of squares due to groups)
SSE(“Error” sum of squares)
Source
Groups
Error
Total
df
k-1
n-k
n-1
Sum ofSquares
SSG
SSE
SST
MeanSquareMSG =
SSG/(k-1)MSE =
SSE/(n-k)
ANOVA TableThe “mean square” is the
sum of squares divided by the degrees of freedom
variability
average variability
Discoveries for Today
•How to measure variability between groups?
•How to measure variability within groups?
•How to compare the two measures?
•How to determine significance?
F-Statistic
•The F-statistic is a ratio of the average variability between groups to the average variability within groups
•The F-statistic is the test statistic for testing for a difference in means across more than 2 groups
average between group variability
average within group variability
MSGF
MSE
Source
Groups
Error
Total
df
k-1
n-k
n-1
Sum ofSquares
SSG
SSE
SST
MeanSquareMSG =
SSG/(k-1)MSE =
SSE/(n-k)
FStatistic
MSGMSE
ANOVA Table
Jumping and Bone DensityDoes jumping improve bone density?30 rats were randomized to three treatment groups:
No jumping (10 rats - group 1) 30 cm jump (10 rats - group 2)60 cm jump (10 rats - group 3)
Rats performed 10 jumps per day, 5 days per week. Bone density was measured after 8 weeks.
Jumping and Bone Density
Control Highjump Lowjump
560
580
600
620
640
660
mean sd nControl 601.1 27.36360 10Lowjump 612.5 19.32902 10Highjump 638.7 16.59351 10
0 :: At least one
NJ LJ HJ
a i j
HH
Jumping and Bone Density
2
1
20013.4n
ii
SST X X
2
1
7433.9k
j jj
SSG n X X
2
,11
12579.5jnk
i j jij
X XSSE
Source
Groups
Error
Total
df
k-1 = 2
n-k =27
n-1=29
Sum ofSquares7433.9
12579.5
20013.4
MeanSquare
7433.9/2 =3716.9
12579.5/27 = 465.9
FStatistic
3716.9465.9
=7.98
ANOVA Table
Discoveries for Today
•How to measure variability between groups?
•How to measure variability within groups?
•How to compare the two measures?
•How to determine significance?
How to determine significance?
We have a test statistic. What else do we need to perform the hypothesis test?
A distribution of the test statistic assuming H0 is true
How do we get this? Two options:1) Simulation2) Distributional Theory
F-statisticIf there really is a difference between the groups, we would expect the F-statistic to be
a) Higher than we would observe by random chance
b) Lower than we would observe by random chance
If the null hypothesis is true, what kind of F-statistics would we observe just by random chance?
Simulation•Rerandomize (reallocate) the rats to treatment groups, keeping response values fixed
•Calculate the F-statistic
•Repeat this simulation many times to form a randomization distribution
•Calculate the p-value as the proportion as extreme or more extreme than the observed F-statistic
F-distributionRandomization Distribution
F-statistic
Frequency
0 2 4 6 8 10
0100
200
300
400
500
600
p-value = 0.002
Because a difference in groups would make the F-statistic higher, calculate probability in the upper tail
F-distributionRandomization Distribution
F-statistic
Frequency
0 2 4 6 8 10
0100
200
300
400
500
600
Randomization Distribution
F-statistic
Frequency
0 2 4 6 8 10
0100
200
300
400
500
600
F-distribution
F-DistributionIf the following conditions hold,
1.Sample sizes in each group are large (each nj ≥ 30) OR the data are relatively normally distributed
2.Variability is similar in all groups
3.The null hypothesis is true
then the F-statistic follows an F-distribution
•The F-distribution has two degrees of freedom, one for the numerator of the ratio (k – 1) and one for the denominator (n – k)http://www.capdm.com/demos/software/html/capdm/qm/fdist/usage.html
Equal Variance•The F-distribution assumes equal within group variability for each group
•As a rough rule of thumb, this assumption is violated if the standard deviation of one group is more than double the standard deviation of another group
F-distributionCan we use the F-distribution to calculate the p-value for the jumping and bone density F-statistic?
a) Yesb) Noc) I need more information
mean sd nControl 601.1 27.36360 10Lowjump 612.5 19.32902 10Highjump 638.7 16.59351 10
Jumping and Bone Density
Control Highjump Lowjump
560
580
600
620
640
660
F-distribution p-values1)Online applet: http://www.danielsoper.com/statcalc3/calc.aspx?id=7
2)RStudio:
> pf(7.98,2,27,lower.tail=FALSE) [1] 0.001892532
3) TI-83:2nd DISTR 9:Fcdf( 7.98, 9999, 2, 27
Source
Groups
Error
Total
df
k-1
n-k
n-1
Sum ofSquares
SSG
SSE
SST
MeanSquareMSG =
SSG/(k-1)MSE =
SSE/(n-k)
FStatistic
MSGMSE
p-value
Use Fk-1,n-k
ANOVA Table
Source
Groups
ErrorTotal
df
2
27
29
Sum ofSquares7433.9
12579.5
20013.4
MeanSquare3716.9
465.9
F-Statistic
7.98
ANOVA Table
p-value
0.0019
We have strong evidence that jumping does increase bone density, at least in rats.
Study Hours by Class YearIs there a difference in the average hours spent studying per week by class year at Duke?
(a) Yes(b) No(c) Cannot tell from this data(d) I didn’t finish
318
24984
SSG
SSE
Source
Groups
ErrorTotal
df
2
195
197
Sum ofSquares
318
24984
20013.4
MeanSquare
159
128.1
F-Statistic
1.24
ANOVA Table
p-value
0.29
Summary• Analysis of variance is used to test for a difference in means between groups by comparing the variability between groups to the variability within groups
• Sums of squares are used to measure variability
• The F-statistic is the ratio of average variability between groups to average variability within groups
• The F-statistic follows an F-distribution, if sample sizes are large (or data is normal), variability is equal across groups, and the null hypothesis is true