Observations vs. ExperimentsObservations vs. Experiments

Target Goals:Target Goals:I can distinguish between an observational study and I can distinguish between an observational study and an experiment. an experiment. I can explain how a lurking variable in an I can explain how a lurking variable in an observational study can lead to confounding.observational study can lead to confounding.I can identify experimental units, treatments, factors, I can identify experimental units, treatments, factors, explanatory and response variables in an experiment .explanatory and response variables in an experiment .

4.2 a4.2 a

h.w: p. 230: 37-42; p 253: 45, 47, 49, 55. 57h.w: p. 230: 37-42; p 253: 45, 47, 49, 55. 57

Review SRSReview SRS

Recall:Recall: Individuals are selected so that Individuals are selected so that allall

possible combinations (every sample of possible combinations (every sample of size n) size n) of individuals have an equal of individuals have an equal chancechance to be in the sample.to be in the sample.

Types of Simple Random Samples:Types of Simple Random Samples:

Systematic Random SampleSystematic Random Sample Stratified Random SampleStratified Random Sample Multistage SampleMultistage Sample Cluster SampleCluster Sample

Types of BiasTypes of Bias

UndercoverageUndercoverage NonresponseNonresponse Response BiasResponse Bias Wording of QuestionsWording of Questions

Observational StudyObservational Study

Observes individuals and measures Observes individuals and measures variables of interestvariables of interest

to influence the response.to influence the response. Poor way to gauge effect.Poor way to gauge effect. Usually fails because ofUsually fails because of

confounding by lurking variableconfounding by lurking variable..

but does not attemptbut does not attempt

Pg. 213 Stratified vs. ClusterPg. 213 Stratified vs. Cluster

Clusters chosen for convenience and for Clusters chosen for convenience and for practical reasons but don’t offer the statistical practical reasons but don’t offer the statistical advantage that stratified does.advantage that stratified does.

Cluster: should represent the whole populationCluster: should represent the whole population

Strata: contain similar individuals and large Strata: contain similar individuals and large differences between the strata.differences between the strata.

Experiment Experiment

Deliberately Deliberately imposes some treatment imposes some treatment on on individuals in order to individuals in order to observe their observe their response.response.

Only source of fully convincing data.Only source of fully convincing data.

Confounded Confounded Two variables (explanatory or lurking variables) Two variables (explanatory or lurking variables)

are said to be are said to be confounded confounded

(mixed up) (mixed up)

when their effects on a response variable when their effects on a response variable cannot be distinguished cannot be distinguished from each other.from each other.

Example:Example: If we observe welfare mothers, the If we observe welfare mothers, the effect of job training programs effect of job training programs on finding on finding workwork is confounded with is confounded with the characteristics of the characteristics of mothers who seek out training on their own.mothers who seek out training on their own.

Exercise: Teaching ReadingExercise: Teaching Reading

An educator wants to compare the effectiveness of An educator wants to compare the effectiveness of computer software that teaches reading with that of a computer software that teaches reading with that of a standard reading curriculum. He tests the reading standard reading curriculum. He tests the reading ability of each student in a class of fourth graders, ability of each student in a class of fourth graders, and then divides them into two groups. One group and then divides them into two groups. One group uses the computer regularly, while the other studies a uses the computer regularly, while the other studies a standard curriculum. At the end of the year, he retests standard curriculum. At the end of the year, he retests all the students and compares the increase in reading all the students and compares the increase in reading ability in the two groups. Is this an experiment?ability in the two groups. Is this an experiment?

Yes, this is an experiment. Yes, this is an experiment.

Why or why not? Why or why not?

A treatment is imposed.A treatment is imposed.

What are theWhat are the explanatoryexplanatory and and response response variables?variables?

Explanatory variable:

The teaching methodThe teaching method Response variable:Response variable:

The increase in reading ability on pre and post The increase in reading ability on pre and post tests.tests.

Exercise: The Effects of PropagandaExercise: The Effects of Propaganda

In 1940, a psychologist conducted an In 1940, a psychologist conducted an experiment to study the effect of propaganda experiment to study the effect of propaganda on attitude toward a foreign government. He on attitude toward a foreign government. He administered a test of attitude toward the administered a test of attitude toward the German government to a group of American German government to a group of American students. After the students read German students. After the students read German propaganda for several months he tested them propaganda for several months he tested them again to see if their attitudes had changed.again to see if their attitudes had changed.

Unfortunately, Germany attacked and Unfortunately, Germany attacked and conquered France while the experiment was in conquered France while the experiment was in progress. progress.

Explain clearly why confounding makes it Explain clearly why confounding makes it impossible to determine the effect of reading impossible to determine the effect of reading the propaganda.the propaganda.

We can never know how much of the We can never know how much of the change in attitude is due to the change in attitude is due to the explanatory variable (reading explanatory variable (reading propaganda) propaganda)

And how much is due to the historical And how much is due to the historical events of that time.events of that time.

The data gave no information about the The data gave no information about the effect of reading propaganda.effect of reading propaganda.

If we want to observe individuals and If we want to observe individuals and record data record data without interventionwithout intervention, we , we conduct an conduct an observational study.observational study.

If we want to examine a If we want to examine a cause and effectcause and effect relationship, we conduct an relationship, we conduct an experimentexperiment..

TheThe individualsindividuals on which the on which the experiment is doneexperiment is done are calledare called

If the units are If the units are peoplepeople, they are called, they are called

experimental units.experimental units.

subjectssubjects . .

The The experimental conditionexperimental condition we apply to we apply to the units is called the the units is called the

The The explanatory variablesexplanatory variables (causing a (causing a

change in the other variables)change in the other variables) are called are called

The factors may be applied in differentThe factors may be applied in different

treatment.treatment.

factors.factors.

levels.levels.

Example :Example : The Physician’s Health StudyThe Physician’s Health Study Does regularly taking aspirin help protect Does regularly taking aspirin help protect

people against heart attacks?people against heart attacks? experimental unitsexperimental units:: subjects; 21,996 male subjects; 21,996 male

Dr.’sDr.’s factors:factors: aspirin (y or n), beta carotene (y or n) aspirin (y or n), beta carotene (y or n)

assigned randomlyassigned randomly response variable:response variable: subjects who suffered heart subjects who suffered heart

attacks attacks

After several years: After several years: 239 of placebo group had heart attacks239 of placebo group had heart attacks 139 of aspirin group had heart attacks139 of aspirin group had heart attacks

ConclusionConclusion Beta carotene had no effectBeta carotene had no effect : CONTROL : CONTROL There is evidence that taking asprin reduces There is evidence that taking asprin reduces

heart attacks.heart attacks.

In principle, experiments can give good In principle, experiments can give good evidence for causation.evidence for causation.

A placebo is a “dummy pill” or inactive treatment that is A placebo is a “dummy pill” or inactive treatment that is indistinguishable from the real treatment.indistinguishable from the real treatment.

Exercise:Exercise:Resisting DroughtResisting Drought

For the experimental situation described, For the experimental situation described, identify the experimental units or subjects, the identify the experimental units or subjects, the factors (explanatory variables), the treatments, factors (explanatory variables), the treatments, and the response variables.and the response variables.

The ability to grow in the shade may help pines found The ability to grow in the shade may help pines found in the dry forests of Arizona to resist drought. How in the dry forests of Arizona to resist drought. How well do these pines grow in shade? Investigators well do these pines grow in shade? Investigators planted pine seedlings in a greenhouse in either full planted pine seedlings in a greenhouse in either full light or light reduced to 5% of normal by a shade light or light reduced to 5% of normal by a shade cloth. At the end of the study, they dried the young cloth. At the end of the study, they dried the young trees and weighed them.trees and weighed them.

Units:Units:

The individual treesThe individual trees

Factors:Factors:

The amount of lightThe amount of light

Treatments:Treatments:Full light and reduced Full light and reduced lightlightResponse variables:Response variables:

The weight of the treesThe weight of the trees

Exercise: Exercise: Improving Response RateImproving Response Rate

How can we reduce the rate of refusals in telephone How can we reduce the rate of refusals in telephone surveys? Most people who answer at all listen to the surveys? Most people who answer at all listen to the interviewer’s introductory remarks and then decide interviewer’s introductory remarks and then decide whether to continue. One study made telephone calls whether to continue. One study made telephone calls to randomly selected households to ask opinions to randomly selected households to ask opinions about the next election. In some calls, the interviewer about the next election. In some calls, the interviewer gave her name, in others she identified the university gave her name, in others she identified the university she was representing, and in still some others she she was representing, and in still some others she identified both herself and the university. identified both herself and the university.

For each type of call, the interviewer either did For each type of call, the interviewer either did or did not offer to send a copy of the final or did not offer to send a copy of the final survey results to the person interviewed. Do survey results to the person interviewed. Do these differences in the introduction affect these differences in the introduction affect whether the interview is completed? whether the interview is completed?

Units:Units: The individuals who were called.The individuals who were called.

Factors:Factors: One factor is what information is offered.One factor is what information is offered.

Treatments are:Treatments are: Giving name, identifying university, or both of Giving name, identifying university, or both of

these.these.

Second Factor:Second Factor: Offering to send a copy of the results.Offering to send a copy of the results.

Treatments areTreatments are Either offering or not offeringEither offering or not offering

Response variables:Response variables: Whether the interview is completedWhether the interview is completed..