Stats: Modeling the World

Chapter 13Experiments

Observational Studies vs. Experiments

Observational study –

Experiment -

A study in which the researcher merely observes what is happening or what has happened in the past

A study in which the researcher imposes a treatment in order to measure a response

Language of ExperimentsOnly an experiment can prove causal relationships

Explanatory variable

Response variable

Observational Study or Experiment?Study: Medicine Mix-ups hurt about 1 in 15 hospitalized kids

Study: Painkillers don’t help older brains

Healthy eating adds up on grocery bills

Ginger helps relieve nausea in chemo patients

Elements of an ExperimentExperimental units:

Subjects or participants:

Factors:

Levels:

Treatments:

Ch 13

ExampleA farm products manufacturer wants to determine if the yield of crop is different when the soil is treated with three different types of fertilizer. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yields from the plots will be compared.a) What are the experimental units?

b) What is the explanatory variable(s)?

c) How many factors are there? How many levels for each factor?

d) What is the response variable?

2004B #2A

The Four Principles of Design

All experimental designs should follow some basic rules. The four major principles are:

1) Control2) Randomization3) Replication4) Blocking **Not essential, but useful (we will discuss it later)

Control through Comparison

A control group serves as a reference mark for an actual treatment to be compared.

Experiments are comparative in nature: We compare the response to a treatment versus to:

another treatmentno treatmenta placeboor any combination of the above

Placebos placebo - A “fake” treatment that looks just like the treatments being tested.

BlindingThere are two main classes of individuals who can affect the outcome of an experiment: (1) those who could influence the results and (2) those who evaluate the results.

single-blind:

double-blind:

RandomizationRandomization allows us to spread out the effects of unknown or uncontrollable sources of variation among the treatments, which helps reduce bias.

**Note: Without randomization, we cannot use the methods of Statistics to draw conclusions from the study!

Experiments vs. SamplesHow does randomization differ for experiments and samples?

- Samples

- Experiments

ReplicationTwo types of replication show up in experimental designs:

Replication within the study

Replication of the study

Explaining with a Diagram

Completely Randomized Design

Article Analysis

POD - Article

With your partner, read through the “Placebo Effect” article

Discuss the main points of the article with your partner. What did you find interesting?

Write a brief summary of this article in your POD folder, with the idea that someone NOT in this class would understand what the article was about.

About the Placebo EffectThe “placebo effect” is an improvement in health due not to any treatment but only to the patient’s belief that he or she will improve.

The placebo effect is not understood, but it is believed to have therapeutic results on up to a whopping 35% of patients.

Other Important Vocab for Experiments

Whenever you are designing an experiment, there are a few items you might keep in mind….

Do you have ESP?

Statistical Significance

Statistical SignificanceLet’s say Mr. Waddell correctly identified the hidden card 14 out of 25 times. Do you think this level of success is statistically significant? Explain?

How many correct identifications out of 25 would Mr. Waddellhave to make to convince you that she has ESP? Explain.

Drawing a Conclusion…If the goal of an experiment is to compare two treatments, then how can we tell if a treatment really works?

What does “statistically significant” really mean? “a difference larger than what we would have expected due to chance alone.”

Experimental Units?

Factors? (Exp Variable)

Treatments?

Response Variable?

Let’s do a Completely Randomized Design…

ConfoundingConfounding variables are two variables (explanatory or lurking variables) that are confounded when their effects on a response variable cannot be distinguished from each other.

studying

intelligence

Good grade on testCAUSE?

Confounding?

Well-designed experiments take steps to defeat confounding.

POD:

How did

YOU assign

the old/ne

w foods

w respec

t to the

temp?

Draw your design in diagram form…

Blocking – The 4th elementWhen we have a situation in which groups of subjects are similar, it is often a good idea to gather them together into blocks.

By blocking, we isolate the variability due to the differences between the blocks so that we can see the differences due to the treatments more clearly.

BlockingWhat does it mean to “isolate the

variability”?

Randomized Block DesignWe wish to run an experiment on the effects of TV ads on adults. Since we think that males and females may react differently, we will separate the genders first, then run the experiment.

Matched Pairs DesignChoose pairs of subjects that are closely matched

— e.g., same sex, height, weight, age, and race. Within each pair, randomly assign who will receive which treatment. The most closely matched

pairs studies use identical twins.

Matched Pairs DesignIt is also possible to just use a single person

and give the two treatments to this person over time in random order. In this case, the “matched pair” is just the same person at different points in time.

Try it!

All Three Designs…Ms. Curtis makes soap as a hobby… She wants to compare her Shea Butter soap to her Goat’s Milk soap to see which one is better as a facial soap.

Completely Randomized Design…

Ms. Curtis makes soap as a hobby… She wants to compare her Shea Butter soap to her Goat’s Milk soap to see which one is better as a facial soap.

Randomized Block Design…

Matched Pairs Design…