Basic conceptBasic concept

Measures of central tendencyMeasures of central tendency

Measures of dispersion & variability

Measures of tendency centralMeasures of tendency central

Arithmetic mean (= simple averageArithmetic mean (= simple average))

summationmeasurement in population

index of measurement

• Best estimate of population mean is the sample mean, X

n

XX

n

ii

1

sample size

Measures of variabilityMeasures of variability All describe how “spread out” the All describe how “spread out” the

data are.data are.

1. Sum of squares,sum of squared deviations from the mean

• For a sample,

2)( XXSS i

2.2. Average or mean sum of Average or mean sum of squares = variance, squares = variance, ss22::

• For a sample,

1

22

n

XXs i )(

Why?

nn – 1 represents the – 1 represents the degrees of degrees of freedomfreedom, , , or number of independent , or number of independent quantities in the estimate quantities in the estimate ss22..

1

22

n

XXs i )(

• therefore, once n – 1 of all deviations are specified, the last deviation is already determined.

01

n

ii XX )(Greek

letter “nu”

3.3. Standard deviation, Standard deviation, ss

• For a sample,1

2

n

XXs i )(

• Variance has squared measurement units – to regain original units, take the square root …

4.4. Standard error of the meanStandard error of the mean

• For a sample,ns

sX

2

Standard error of the mean is a Standard error of the mean is a measure of variability among the measure of variability among the means of repeated samples from a means of repeated samples from a population.population.

Means of repeated random samples, each Means of repeated random samples, each with sample size, with sample size, nn = 5 values = 5 values … …

14

14

14

14

1414

14

1414

14

1515

14

1213

13

1313

13

1313

16

16 16

16

12

14

15

14X 15X 13X

A Population of Values

For a large enough number of large For a large enough number of large samples, the frequency distribution samples, the frequency distribution of the sample means (= sampling of the sample means (= sampling distribution), approaches a normal distribution), approaches a normal distribution.distribution.

Sample mean

Frequency

Normal distribution: bell-shaped curveNormal distribution: bell-shaped curve

Testing Testing statistical hypotheses between 2 meansstatistical hypotheses between 2 means

1.1. State the research question in State the research question in terms of statistical hypotheses.terms of statistical hypotheses.

• We always start with a statement that hypothesizes “no difference”, called the null hypothesis = H0.

E.g., H0: Mean bill length of female hummingbirds is equal to mean bill length of male hummingbirds,

µ=µ .

Then we formulate a statement Then we formulate a statement that must be true if the null that must be true if the null hypothesis is false, called the hypothesis is false, called the alternate hypothesisalternate hypothesis = = HHAA . .

E.g., HA: Mean bill length of female hummingbirds is not equal to mean bill length of male hummingbirds,

µµ .

• If we reject H0 as a result of sample evidence, then we conclude that HA is true.

2. Choose an appropriate statistical test that would allow you to reject H0 if H0 were false.

E.g., Student’s E.g., Student’s tt test for test for hypotheses about meanshypotheses about means

William Sealey Gosset (a.k.a. “Student”)

Is the difference between sample

means bigger than we would expect,

given the variability in the

sampled populations?

21

21

XXs

XXt

Standard error of the difference between the sample means

• To estimate s(X1—X2), we must first

know the relation between both

populations.

Mean of sample 2

Mean of sample 1

t Statistic,

Relation between populationsRelation between populations

Dependent populationDependent population Independent populationIndependent population

1. Identical (homogenous ) variance

2. Not identical (heterogeneous) variance

Pooled variance:Pooled variance:21

212

SSSS

sp

• Then,

2

2

1

2

21 n

s

n

ss pp

XX

Independent Population with homogenous variances

3. Select the level of significance for the statistical test.

• Level of significance = alpha value = = the probability of incorrectly rejecting the null hypothesis when it is, in fact, true.

Traditionally, researchers choose Traditionally, researchers choose = 0.05.= 0.05.

• 5 percent of the time, or 1 time out of 20, the statistical test will reject H0 when it is true.

• Note: the choice of 0.05 is arbitrary!

4.4. Determine the Determine the critical valuecritical value the the test statistic must attain to be test statistic must attain to be declared significant.declared significant.

• Most test statistics have a frequency distribution …

Test statistic

Freq

uen

cy

When sample sizes are small, the When sample sizes are small, the sampling distribution is described sampling distribution is described better by the better by the t t distribution than by distribution than by the standard normal (the standard normal (ZZ) ) distribution. distribution.

• Shape of t distribution depends on degrees of freedom, = n – 1.

Z = t(=)

t(=25)

t(=1)t(=5)

t

The distribution of a test statistic is The distribution of a test statistic is divided into an area of acceptance divided into an area of acceptance and an area of rejection.and an area of rejection.

t

Area of Rejection

Area of Acceptance

Area of Rejection

Lower critical value

Upper critical value

0

0.95 0.0250.025For = 0.05

5.5. Perform the statistical test.Perform the statistical test.

003

5025147515

21

21.

.

..

XXs

XXt

5021

. XXs

E.g., Mean bill length from a sample of 5 female hummingbirds, X1 = 15.75;

Mean bill length from a sample of 5 male hummingbirds, X2 = 14.25;

6.6. Draw and state the conclusions.Draw and state the conclusions.

• Compare the calculated test statistic with the critical test statistic at the chosen .

• Obtain the P-value = probability for the test statistic.

• Reject or fail to reject H0.

• Critical t for a test about equality = t(2),

E.g., to test H0: µ = µ, HA: µ µ

at = 0.05 using n = 5, n = 5,

, if |t| 2.306, reject H0.

t(2), = t0.05(2),8 = 2.306.

Since calculated Since calculated tt > > tt0.05(2),80.05(2),8 (because 3.000 > 2.306), reject (because 3.000 > 2.306), reject HH00..

Conclude that hummingbird bill length is sexually size-dimorphic.

What is the probability, P, of observing by chance a difference as large as we saw between female and male hummingbird bill lengths?

0.01 < P < 0.02

Analysis of VarianceAnalysis of Variance

(ANOVA)(ANOVA)

What is ANOVA?What is ANOVA? ANOVA (Analysis of Variance) is a procedure designed to ANOVA (Analysis of Variance) is a procedure designed to

determine if the manipulation of one or more determine if the manipulation of one or more independent variables in an experiment has a independent variables in an experiment has a statistically significant influence on the value of the statistically significant influence on the value of the dependent variable.dependent variable.

It is assumedIt is assumed• Each independent variable is categorical (nominal scale). Each independent variable is categorical (nominal scale).

Independent variables are called Independent variables are called FactorsFactors and their values are and their values are called called levelslevels..

• The dependent variable is numerical (ratio scale)The dependent variable is numerical (ratio scale) The basic idea is that the “variance” of the dependent The basic idea is that the “variance” of the dependent

variable given the influence of one or more independent variable given the influence of one or more independent variables {Expected Sum of Squares for a Factor} is variables {Expected Sum of Squares for a Factor} is checked to see if it is significantly greater than the checked to see if it is significantly greater than the “variance” of the dependent variable (assuming no “variance” of the dependent variable (assuming no influence of the independent variables) {also known as influence of the independent variables) {also known as the Mean-Square-Error(MSE)}.the Mean-Square-Error(MSE)}.

Analysis of VarianceAnalysis of Variance (ANOVA) can be used to test for the (ANOVA) can be used to test for the equality of three or more population means using data equality of three or more population means using data obtained from observational or experimental studies.obtained from observational or experimental studies.

We want to use the sample results to test the following We want to use the sample results to test the following hypotheses.hypotheses.

HH00: : 11==22==33==. . . . . . = = kk

HHaa: Not all population means are equal: Not all population means are equal

If If HH00 is rejected, we cannot conclude that all population is rejected, we cannot conclude that all population means are different.means are different.

Rejecting Rejecting HH00 means that at least two population means means that at least two population means have different values.have different values.

Analysis of VarianceAnalysis of Variance

Assumptions for Analysis of VarianceAssumptions for Analysis of Variance

For each population, the response variable is For each population, the response variable is normally distributed.normally distributed.

The variance of the response variable, denotedThe variance of the response variable, denoted 22, , is the same for all of the populations.is the same for all of the populations.

The effect of independent variable is additiveThe effect of independent variable is additive The observations must be independentThe observations must be independent..

Analysis of Variance:Analysis of Variance:Testing for the Equality of K Population Testing for the Equality of K Population

MeansMeans

Between-Treatments Estimate of Between-Treatments Estimate of Population VariancePopulation Variance

Within-Treatments Estimate of Within-Treatments Estimate of Population VariancePopulation Variance

Comparing the Variance Estimates: Comparing the Variance Estimates: The The FF Test Test

ANOVA TableANOVA Table

A between-treatments estimate of A between-treatments estimate of σσ2 2 is called the is called the mean square due to treatmentsmean square due to treatments (MSTR).(MSTR).

The numerator of MSTR is called the The numerator of MSTR is called the sum of sum of squares due to treatmentssquares due to treatments (SSTR).(SSTR).

The denominator of MSTR represents the The denominator of MSTR represents the degrees degrees of freedomof freedom associated with SSTR. associated with SSTR.

Between-Treatments Estimate Between-Treatments Estimate of Population Varianceof Population Variance

2

1

( )

MSTR1

k

j jj

n x x

k

2

1

( )

MSTR1

k

j jj

n x x

k

The estimate ofThe estimate of 22 based on the variation of the based on the variation of the sample observations within each treatment is sample observations within each treatment is called the called the mean square due to errormean square due to error (MSE).(MSE).

The numerator of MSE is called the The numerator of MSE is called the sum of sum of squares due to errorsquares due to error (SSE).(SSE).

The denominator of MSE represents the degrees The denominator of MSE represents the degrees of freedom associated with SSE.of freedom associated with SSE.

Within-Treatments Estimate Within-Treatments Estimate of Population Varianceof Population Variance

2

1

( 1)

MSE

k

j jj

T

n s

n k

2

1

( 1)

MSE

k

j jj

T

n s

n k

Comparing the Variance Estimates: Comparing the Variance Estimates: The The F F Test Test

If the null hypothesis is true and the ANOVA If the null hypothesis is true and the ANOVA assumptions are valid, the sampling distribution assumptions are valid, the sampling distribution of MSTR/MSE is an of MSTR/MSE is an FF distribution with MSTR d.f. distribution with MSTR d.f. equal to equal to kk - 1 and MSE d.f. equal to - 1 and MSE d.f. equal to nnTT - - kk..

If the means of the If the means of the kk populations are not equal, populations are not equal, the value of MSTR/MSE will be inflated because the value of MSTR/MSE will be inflated because MSTR overestimates MSTR overestimates σσ22..

Hence, we will reject Hence, we will reject HH00 if the resulting value of if the resulting value of

MSTR/MSEMSTR/MSE appears to be too large to have been appears to be too large to have been selected at random from the appropriate selected at random from the appropriate FF

distributiondistribution..

Test for the Equality of Test for the Equality of kk Population Population MeansMeans

HypothesesHypotheses

HH00: : 11==22==33==. . . . . . = = kk

HHaa: Not all population means are equal: Not all population means are equal

Test StatisticTest StatisticFF = MSTR/MSE = MSTR/MSE

Test for the Equality of Test for the Equality of kk Population Population MeansMeans

Rejection RuleRejection Rule

Using test statistic: Reject Using test statistic: Reject HH00 if if FF > > FFaa

Using Using pp-value: Reject -value: Reject HH00 if if pp-value < -value < aa

where the value of where the value of FFa a is based on an is based on an FF distribution with distribution with kk - 1 numerator degrees of - 1 numerator degrees of freedom and freedom and nnTT - - kk denominator degrees of denominator degrees of freedomfreedom

The figure below shows the rejection region The figure below shows the rejection region associated with a level of significance equal to associated with a level of significance equal to where where FF denotes the critical value. denotes the critical value.

Sampling Distribution of MSTR/MSESampling Distribution of MSTR/MSE

Do Not Reject H0Do Not Reject H0 Reject H0Reject H0

MSTR/MSEMSTR/MSE

Critical ValueCritical ValueFF

ANOVA TableANOVA TableSource of Sum of Degrees of MeanSource of Sum of Degrees of Mean

Variation Squares Freedom Squares FVariation Squares Freedom Squares F

TreatmentTreatment SSTRSSTR kk - 1 - 1 MSTR MSTR/MSE MSTR MSTR/MSE

ErrorError SSESSE nnT T - - kk MSE MSE

TotalTotal SSTSST nnTT - 1 - 1

SST divided by its degrees of freedom SST divided by its degrees of freedom nnTT - 1 is simply - 1 is simply the overall sample variance that would be obtained if the overall sample variance that would be obtained if we treated the entire we treated the entire nnTT observations as one data set. observations as one data set.

k

j

n

iij

j

xx1 1

2 SSESSTR)(SST

k

j

n

iij

j

xx1 1

2 SSESSTR)(SST

Example: Reed ManufacturingExample: Reed Manufacturing Analysis of VarianceAnalysis of Variance

J. R. Reed would like to know if the mean number J. R. Reed would like to know if the mean number of hours worked per week is the same for the of hours worked per week is the same for the department managers at her three manufacturing department managers at her three manufacturing plants (Buffalo, Pittsburgh, and Detroit). plants (Buffalo, Pittsburgh, and Detroit).

A simple random sample of 5 managers from each A simple random sample of 5 managers from each ofof

the three plants was taken and the number of hoursthe three plants was taken and the number of hours

worked by each manager for the previous week isworked by each manager for the previous week is

shown on the next slide.shown on the next slide.

Sample DataSample Data

Plant 1Plant 1 Plant 2Plant 2 Plant 3Plant 3

ObservationObservation Buffalo Pittsburgh Buffalo Pittsburgh DetroitDetroit

11 48 48 73 73 51 51

22 54 54 63 63 63 63

33 57 57 66 66 61 61

44 54 54 64 64 54 54

55 62 62 74 74 56 56

Sample MeanSample Mean 55 55 68 68 57 57 Sample VarianceSample Variance 26.026.0 26.5 26.5

24.524.5

Example: Reed ManufacturingExample: Reed Manufacturing

HypothesesHypotheses

HH00: : 11==22==33

HHaa: Not all the means are equal: Not all the means are equal

where: where:

1 1 = mean number of hours worked per = mean number of hours worked per week by the managers at Plant 1 week by the managers at Plant 1

2 2 = mean number of hours worked per = mean number of hours worked per week by the managers at Plant 2week by the managers at Plant 2

3 3 = mean number of hours worked per = mean number of hours worked per week by the managers at Plant 3week by the managers at Plant 3

Example: Reed ManufacturingExample: Reed Manufacturing

Mean Square Due to TreatmentsMean Square Due to Treatments

Since the sample sizes are all equalSince the sample sizes are all equal

μμ= (55 + 68 + 57)/3 = 60= (55 + 68 + 57)/3 = 60 SSTR = 5(55 - 60)SSTR = 5(55 - 60)22 + 5(68 - 60) + 5(68 - 60)22 + 5(57 - 60) + 5(57 - 60)22 = 490 = 490

MSTR = 490/(3 - 1) = 245MSTR = 490/(3 - 1) = 245 Mean Square Due to ErrorMean Square Due to Error

SSE = 4(26.0) + 4(26.5) + 4(24.5) = 308SSE = 4(26.0) + 4(26.5) + 4(24.5) = 308

MSE = 308/(15 - 3) = 25.667MSE = 308/(15 - 3) = 25.667

==

Example: Reed ManufacturingExample: Reed Manufacturing

FF - Test - Test

If If HH00 is true, the ratio MSTR/MSE should be is true, the ratio MSTR/MSE should be

near 1 because both MSTR and MSE are near 1 because both MSTR and MSE are estimating estimating 22. .

If If HHaa is true, the ratio should be significantly is true, the ratio should be significantly larger than 1 because MSTR tends to larger than 1 because MSTR tends to

overestimate overestimate 22..

Example: Reed ManufacturingExample: Reed Manufacturing

Example: Reed ManufacturingExample: Reed Manufacturing

Rejection RuleRejection Rule

Using test statistic: Reject Using test statistic: Reject HH00 if if FF > 3.89 > 3.89

Using Using pp-value: Reject -value: Reject HH00 if if pp-value < .05-value < .05

where where FF.05.05 = 3.89 is based on an = 3.89 is based on an FF distribution distribution with 2 numerator degrees of freedom and 12 with 2 numerator degrees of freedom and 12 denominator degrees of freedomdenominator degrees of freedom

Example: Reed ManufacturingExample: Reed Manufacturing

Test StatisticTest Statistic

FF = MSTR/MSE = 245/25.667 = 9.55 = MSTR/MSE = 245/25.667 = 9.55 ConclusionConclusion

FF = 9.55 > = 9.55 > FF.05.05 = 3.89, so we reject = 3.89, so we reject HH00. The mean . The mean number of hours worked per week by department number of hours worked per week by department managers is not the same at each plant. managers is not the same at each plant.

ANOVA TableANOVA Table

Source of Sum of Degrees of MeanSource of Sum of Degrees of Mean

Variation Squares Freedom Square FVariation Squares Freedom Square F

Treatments Treatments 490490 2 245 9.55 2 245 9.55 Error Error 308308 12 12 25.667 25.667

Total Total 798 798 1414

Example: Reed ManufacturingExample: Reed Manufacturing

Step 1Step 1 Select the Select the ToolsTools pull-down menu pull-down menu Step 2Step 2 Choose the Choose the Data AnalysisData Analysis option option Step 3Step 3 Choose Choose Anova: Single FactorAnova: Single Factor

from the list of Analysis Toolsfrom the list of Analysis Tools

… … continuedcontinued

Using Excel’s Anova: Using Excel’s Anova: Single Factor Tool Single Factor Tool

Step 4Step 4 When the Anova: Single Factor dialog When the Anova: Single Factor dialog

box appears:box appears:

Enter B1:D6 in the Enter B1:D6 in the Input RangeInput Range box box

Select Grouped By Select Grouped By ColumnsColumns

Select Select Labels in First RowLabels in First Row

Enter .05 in the Enter .05 in the AlphaAlpha box box

Select Select Output RangeOutput Range

Enter A8 (your choice) in the Enter A8 (your choice) in the Output Output RangeRange box box

Click Click OKOK

Using Excel’s Anova: Using Excel’s Anova: Single Factor ToolSingle Factor Tool

Value Worksheet (top portion)Value Worksheet (top portion)

A B C D E1 Observation Buffalo Pittsburgh Detroit2 1 48 73 51 3 2 54 63 634 3 57 66 615 4 54 64 54 6 5 62 74 56

Using Excel’s Anova:Using Excel’s Anova: Single Factor Tool Single Factor Tool

Value Worksheet (bottom portion)Value Worksheet (bottom portion)

Using Excel’s Anova: Using Excel’s Anova: Single Factor ToolSingle Factor Tool

A B C D E F G8 Anova: Single Factor9

10 SUMMARY11 Groups Count Sum Average Variance12 Buffalo 5 275 55 2613 Pittsburgh 5 340 68 26.514 Detroit 5 285 57 24.5151617 ANOVA18 Source of Variation SS df MS F P-value F crit19 Between Groups 490 2 245 9.54545 0.00331 3.8852920 Within Groups 308 12 25.66672122 Total 798 14

Using the Using the pp-Value-Value• The value worksheet shows that the The value worksheet shows that the pp-value -value

is .00331is .00331

• The rejection rule is “The rejection rule is “Reject Reject HH00 if if pp-value < .05”-value < .05”

• Thus, we reject Thus, we reject HH00 because the because the pp-value -value = .00331 < = .00331 < = .05 = .05

• We conclude that the mean number of hours We conclude that the mean number of hours worked per week by the managers differ worked per week by the managers differ among the three plantsamong the three plants

Using Excel’s Anova: Using Excel’s Anova: Single Factor ToolSingle Factor Tool