Chapter 11: Multiple Comparisons & Analysis of Variance

One population, Two population, ...

• Chapter 9: Inference (confidence intervals, hypothesis testing) for mean for one group/one population

• Chapter 9: Inference (confidence intervals, hypothesis testing) to compare the means of two groups/two populations

• To review... briefly look at a few of those one and two-mean inference procedures/situations

Ho: μ = 1 Ha: μ > 1

where μ = mean heat conductivity transmitted per square meter of surface per degree Celsius difference on the two sides of the glass

Is there evidence that the conductivity of this type of glass is greater than 1? Carry out an appropriate test.

Does logging significantly change the mean number of species in a plot after 8 years? Give appropriate statistical evidence to support your conclusion. Assume both populations are Normally distributed.

We want to test Ho: μU = μL OR μU – μL = 0

Ha: μU ≠ μL OR μU – μL ≠ 0

where μU & μL are the mean number of species in unlogged and logged plots, respectfully

• Is there good evidence that red wine drinkers’ mean polyphenol levels were different from white wine drinkers’ mean polyphenol levels? Assume both populations are approximately Normal.

• We want to test:

Ho: μR = μW or μR – μW = 0

Ha: μR ≠ μW or μR – μW ≠ 0

where μR & μW are the mean percent change in polyphenols for men who drink red and white wine, respectfully.

Nothing magical about the numbers one or two...

• Sometimes there is a need to compare three, four, five, or more groups with each other.

• ANOVA (Analysis of Variance) is a method for doing that; tests whether there is an association between a categorical variable that identifies different groups, and a numerical variable.

• The phrase “Analysis of Variance” can be misleading; the procedure really looks at means/compares means.

ANOVA example...

• We may want to know which diet (Weight Watchers, Jenny Craig, Atkins, Slim-Fast, etc.), the categorical variable, is best for losing particular amounts of weight (3 pounds, 10 pounds, 25 pounds), the numerical variable.

• ANOVA accurately performs multiple comparisons

• If we chose to conduct several individual comparisons (several two-sample/two-population) procedures (two-sample t procedure, confidence interval, hypothesis test), life gets very messy (more on this in a moment)

How does anova work?

• Basic idea: Accurately calculates variation within a given group as well as between several groups

• This leads to a test statistic (like a t score/value) that accurately compares several different groups without the problem of multiple comparisons.

• Like all procedures we have discussed, ANOVA works best if certain conditions are met; more on this later...

...popcorn & kernels popped & problems with multiple comparison ...

In chapter 9, we would have...

Ho: mean no oil = mean medium oil

Ha: mean no oil ≠ mean medium oil

Ho: mean no oil = mean maximum oil

Ha: mean no oil ≠ mean maximum oil

Ho: mean medium oil = mean maximum oil

Ha: mean medium oil ≠ mean maximum oil

Three different hypothesis tests... This is called multiple comparison... Comparing multiple pairs of means

Remember α...

• Rejection zone (when conducting an hypothesis test); significance level; usually 5% (0.05)

• α is also the probability of committing a type I error (rejecting the null hypothesis when it really is true)

• Basic problem with multiple comparisons is that even though the probability of something going wrong (making an incorrect decision; committing an error) on one occasion is small (5%), if we keep repeating the experiment, eventually something will go wrong.

Big chances to make big mistakes...

• Essentially, by doing multiple tests, we are creating more opportunities to mistakenly reject the null hypothesis.

• The more tests we do, the greater the probability that we will mistakenly reject the null hypothesis at least one.

• For our three hypothesis tests, each with α= 0.05, the overall significance level (or probability that we will conclude that at least one amount of oil is more effective than another, when the truth is that all amounts are equally effective is 14%.

Big chances to make big mistakes...

• 14% doesn’t seem too high... But we were shooting for 5%... And this is just with 3 procedures.

• DETOUR...by the way, how did I calculate that 14%?

• Remember binomial distributions? Let’s take a moment and review how I calculated that 14%. Pull up Minitab...

So, anova to the rescue...

• ANOVA tests whether a categorical variable is associated with a numerical variable. This is the same as testing whether the mean value of a numerical variable is different in different groups

• ANOVA looks at the variation within each group and between all groups; then creates a ratio comparing these numbers called the F-statistic

• F =

€

Variation .Between.Groups

Variation .Within .Groups

ANOVA looks at variation within & between

• Look at variation within each group

• Look at variation between all groups group

ANOVA looks at variation within & between

• Look at variation within each group

• Look at variation between all groups group

ANOVA looks at variation within & between

Look at variation within each group

Look at variation between all groups group

Like all other procedures...

• We have conditions that must be checked and met

• Random Sample & Independent Measurements

• Independent Groups

• Same Variance

Let’s practice a few...

• Does the amount of oil used affect the number of kernels popped when one is making popcorn? Researchers randomly assigned bags with 50 unpopped kernels to be popped with no oil, a medium amount of oil (1/2 tsp), or the maximum amount of oil (1 tsp). Thirty-six bags were assigned to each group. After 75 seconds, the popped kernels were counted.

• Using significance level of 5%, fan ANOVA test was carried out.

• Hypothesize:

• Ho: μnone = μmedium = μmax

• Ha: The mean number of kernels popped differs by amount of oil used.

Ho: μnone = μmedium = μmax

Ha: The mean number of kernels popped differs by amount of oil used.

• Does the amount of oil used affect the number of kernels popped when one is making popcorn? Using significance level of 5%, carry out an ANOVA test.

• Check Conditions:

• Random Sample & Independent Measurements

• Independent Groups

• Same Variance

Ho: μnone = μmedium = μmax

Ha: The mean number of kernels popped differs by amount of oil used.

• Does the amount of oil used affect the number of kernels popped when one is making popcorn? Using significance level of 5%, carry out an ANOVA test.

Ho: μnone = μmedium = μmax

Ha: The mean number of kernels popped differs by amount of oil used.

• Reject null hypothesis. With anαof 5% and a p-value of about 2%, we reject the null hypothesis that the mean number of kernels popped are all equal.

Now let’s use minitab...

Test weather the mean number of tv’s differs among these three schools. Use a 5% significance level. Assume conditions have been checked and met.

Ho: All populations are =ha: the population means are ≠

• Minitab; enter data into three columns with labels for each (OC, VA, MC)

• Go to ANOVA, one-way ANOVA, choose ‘responses are in separate columns for each factor level’

• In responses box, insert your three columns of data (OC, VA, MC); then OK

• F-Value = 0.48; p-value = 0.6205

• Interpretation: Fail to reject. With an α level of 5% and a p-value over 62%, we do not have enough evidence to show that the population means are ≠

ANOVA HW & Test questions...

• No need to do the HW in this section

• Just be familiar with the general PP concepts

• Test questions on ANOVA? Minimal. Will be on final, but not a main focus (probably just 1-2 simple questions about ANOVA)