PS 601 Notes Part II Statistical TestsNotes Version - March 8, 2005

Statistical testsWe can use the properties of probability density functions to make probability statements about the likelihood of events occurring.The standard normal curve provides us with a scale or benchmark for the likelihood of being at (or above or below) any point on the scale

Standard normal valuesNote for instance that if we look at the value 1.5 under the standard normal table, we find the value .4332.This means that the probability of having a standard normal value greater than 1.5 is .5 - .4332 = .0668

In Applied TermsIf IQ has a mean of 100, and a standard deviation of 20, what is the probability that any given individuals IQ will be greater than or equal to 130.Standardize the score of 130

Look up 1.5 in the standard normal table

Two-tailed hypothesesIn general our hypothesis is:Did the sample come from some particular population?If the sample mean is too high or too low, we suspect that it did not.Thus, we must check to see if the sample mean is either significantly higher, or significantly lower.This is called a two-tailed test.When in doubt, most tests are best done as two-tailed ones

The One Tailed HypothesisSometimes we suspect, or hypothesize, directione.g. The average income for West Virginia will be significantly lower than the country as a whole.HA: Xbar < This is a one-tailed testWe ignore the tail in the direction not hypothesized

The Z-testThe z-test is based upon the standard normal distribution.

In this case we are making statements about the sample mean, instead of the actual data values

The Z-test (cont.)Note that the Z-test is based upon two parts.The standard normal transformation

The standard deviation of the sampling distribution.

The Z-test an exampleSuppose that you took a sample of 25 people off the street in Morgantown and found that their personal income is $24,379And you have information that the national average for personal income per capita is $31,632 in 2003.Is the Morgantown significantly different from the National Average

Sources: (1) Economagic(2) US Bureau of Economic Analysis

What to conclude?Should you conclude that West Virginia is lower than the national average? Is it significantly lower?Could it simple be a randomly bad sampleAssume that it is not a poor sampling techniqueHow do you decide?

Example (cont.)We will hypothesize that WV income is lower than the national average.HA: WVInc < USInc (Alternate Hypothesis)H0: WVInc = USInc (Null Hypothesis)

Since we know the national average ($31,632), and standard deviation (15000), OK I made this up to make the problem simplerwe can use the z-test to make decide if WV is indeed statistically significantly lower than the nation

Example (cont.)Using the z-test, we get

OK so what?

The Probability of a Type I errorWe would like to infer that WV had a lower income than the national average, but we must examine whether we simply got these numbers by chance in a random sampleWe would like to not make mistakes when we make statistical decisions.We know we will.With statistical inference, we have the ability to decide how often we find it acceptable to be wrong by random chance.Thus we set the probability of making a Type I error.P(Type I error) = = ?By convention =.05

The Critical Value of Z (cont)Ok, now we know zWe know that we can make probability statements about z, since it is from the standard normal distributionWe know that if z =1.96 then the area out in the tail past 1.96 is equal to .025This means that the likelihood of obtaining a value of z > 1.96 by random chance in any given sample is less than .025.

The Critical Values of Z to memorizeTwo tailed hypothesisReject the null (H0) if z 1.96, or z -1.96One tailed hypothesisIf HA is Xbar > , then reject H0 if z 1.645If HA is Xbar < , then reject H0 if z -1.645

Z test example (cont.)Suppose we decided to look at a different state, say Oregon. Would you try a 1-tailed test?Which way? HA: Xbar > or HA: Xbar <

Without an a priori reason to hypothesize higher oir lower, use the 2-tailed testAssume Oregon has a mean of 29,340, and that we collected a somewhat larger sample, say 100.Using the z-test, we get

What would we conclude? What if n=25? 1000?

The applicability of the z-testWe frequently run into a problem with trying to do a z test.The sample size may be below the number needed for the CLT to apply (N~30)While the population mean () may be frequently available, the population standard deviation () frequently is not.Thus we use our best estimate of the population standard deviation the sample standard deviation (s).

The t testWhen we cannot use the population standard deviation, we must employ a different statistical testThink of it this way:The sample standard deviation is biased a little low, but we know that as the sample size gets larger, it becomes closer to the true value.As a result, we need a sampling distribution that makes small sample estimates conservative, but gets closer to the normal distribution as the sample size gets larger, and the sample standard deviation more closely resembles the population standard deviation.Thus we need the Students t

The t-test (cont.)The t-test is a very similar formula.

Note the two differencesusing s instead of The resultant is a value that has a t-distribution instead of a standard normal one.

The t distributionThe t distribution is a statistical distribution that varies according to the number of degrees of freedom (Sample size 1)As df gets larger, the t approximates the normal distribution.For practical purposes, the t-distribution with samples greater than 100 can be viewed as a normal distribution.

Selecting the critical value t-distSelecting the critical value of the t-distribution requires these steps.Determine whether one- or two-tailed test.Select level (=.05)Determine degrees of freedom (n-1)Find value for t in appropriate column (table if one- & two-tailed tests are separate tables)Critical value of t is at intersection of df row and -level column.

Interpreting t-valueThe t-test formula gives you a value that you can compare to the critical value.If:Conducting a two tailed test, if the calculated t-value is outside the range of t to +t, we conclude that the sample is significantly different that the population.Note that a t-value that exceeds the critical value means that the probability of that t is less than the selected -level.Hence if t > C.V . of t, then p(t) < (say .05)

Interpreting t-value one tailed testThe t-test formula gives you a value that you can compare to the critical value.If:Conducting a one-tailed test, if the calculated t-value is greater that the critical value of t, or less than (critical value of t), we conclude that the sample is significantly different that the population. Choice of t or t is determined by the one-tailed test direction.Note that a t-value that exceeds the critical value means that the probability of that t is less than the selected -level.Hence if t > C.V . of t, then p(t) < (say .05)

T-test exampleSuppose we decided to look at Oregon, but do not know the population standard deviationWould you try a 1-tailed test?Which way? HA: Xbar > or HA: Xbar <

Like the z-test, without an a priori reason to hypothesize higher or lower, use the 2-tailed testAssume Oregon has a mean of 29,340, and that we collected a sample of 169.Using the t-test, we get

What would we conclude? What if n=25? 1000?

Two-sample t-testFrequently we need to compare the means of two different samples.Is one group higher/lower than some other group?e.g. is the Income of blacks significantly lower than whites?The two-sample t difference of means test is the typical way to address this question.

ExamplesIs the income of blacks lower than whites?Are teachers salaries in West Virginia and Mississippi alike?Is there any difference between the background well and the monitoring well of a landfill?

The Difference of means Test Frequently we wish to ask questions that compare two groups.Is the mean of A larger (smaller) than B?Are As different (or treated differently) than Bs?Are A and B from the same population?To answer these common types of questions we use the standard two-sample t-test

The Difference of means Test The standard two-sample t-test is:

The standard two sample t-testIn order to conduct the two sample t-test, we only need the two samplesPopulation data is not required.We are not asking whether the two samples are from some large population.We are asking whether they are from the same population, whatever it may be.

Assumptions about the varianceThe standard two-sample t-test makes no assumptions about the variances of the underlying populations.Hence we refer to the standard test as the unequal variance test.If we can assume that the variances of the tow populations are the same, then we can use a more powerful test the equal variance t-test.

The Equal Variance testIf the variances from the two samples are the same we may use a more powerful variation

Where

Which test to Use?In order to choose the appropriate two-sample t-test, we must decide if we think the variances are the same.Hence we perform a preliminary statistical test the equal variance F-test.

The Equal Variance F-testOne of the fortunate properties on statistics is that the ratio of two variances will have an F distribution.Thus with this knowledge, we can perform a simple test.

Interpretation of F-testIf we find that P(F) > .05, we conclude that the variances are equal.If we find that P(F) .05, we conclude that the variances are unequal.We then select the equal or unequal-variance t-test accordingly.The F distribution

Degrees of freedomNote that the degrees of freedom is different across the two testsEqual variance testDf = n1 +n2-2Unequal variance testDf = complicated real number not integer

Contingency TablesOften we have limited measurement of our data.Contingency Tables are a means of looking at the impact of nominal and ordinal measures on each other.They are called contingency tables because one variables value is contingent upon the other.Also called cross-tabulation or crosstabs.

Contingency TablesThe procedure is quite simple and intuitively appealingConstruct a table with the independent variable across the top and the dependent variable on the sideThis works fairly well for low numbers of categories (r,c < 6 or so)

Contingency Tables An examplePresidents are often suspected of using military force to enhance their popularity.What do you suppose the data actually look like? Any conjecturesLets categorize presidents as using force,or not, and as having popularity above and below 50%Are there definition problems here?Which is independent and which is dependent?

Contingency Tables

Presidential ApprovalUse of Military Force < 50% > 50%Not Used1670%2841%4448%Used730%4059%4752%Total23100%68100%91100%

Measures of IndependenceAre the variables actually contingent upon each other?Is the use of force contingent upon the presidents level of popularity?We would like to know if these variables are independent of each other, or does the use of force actually depend upon the level of approval that the president have at that time?

2 Test of IndependenceThe 2 Test of Independence gives us a test of statistical significance.It is accomplished by comparing the actual observed values to those you would expect to see if the two variables are independent.

2 Test of IndependenceFormula

Where

Chi-Square Table (2)

Interpreting the 2The Table gives us a 2 of 5.55 with 1 degree of freedom [d.f. = (r-1)*(c-1)]The critical value of 2 with 1 degree of freedom is 3.84 (see 2 Table)We therefore conclude that Presidential popularity and use of force are related.We technically reject the null hypothesis that Presidential popularity and use of force are independent.Note: 2 is influenced by sample size. It ranges from 0.0 to .

Corrected 2 measuresSmall tables have slightly biased measures of 2 If there are cell frequencies that are low, then there are some adjustments to make that correct the probability estimates that 2 provides.

Yates Corrected 2 For use with a 2x2 table with low cell frequencies (5

Measures of AssociationNot only do we want to see whether the variables of a cross-tabulation are independent, we often want to see if the relationship is a strong or weak one.To do this, we use what are referred to as measures of association.The level of measurement determines what measure of association we might use.

Measures of AssociationWe group them according to whether the variables are nominal or ordinal.If one variable is nominal, use nominal measures.If both are ordinal, use an ordinal measure.If either is interval, generally we use a different statistical design.

Measures based on 2Contingency CoefficientKramers V

Yules QMay be used on any 2x2 table, nominal or ordinalIf we define out table with cell counts as

Yules Q is calculated as:

Q ranges from 0 to 1.0Q compares concordant pairs to discordant pairs

Phi

GammaWill equal 1.0 if any cell is empty

Lambda Asymmetric measure of associationCalculation depends on whether the column variable or the row variable is independent

Ordinal MeasuresGoodman & Kruskals GammaFor Ordinal x Ordinal tablesMay also be used if one of the variables is a nominal dichotomy

LambdaAsymmetrioc

Tau-b & Tau-cSimilar to GammaIf r=c, use tau-b; if rc, use tau-c