34
Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 1 of 34 Chapter 1 Section 5 The Design of Experiments
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 1 of 34
Chapter 1Section 5
The Design ofExperiments
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 2 of 34
Chapter 1 – Section 5
● Learning objectives Define designed experiment Understand the steps in designing an experiment Understand the completely randomized design Understand the matched-pairs design
1
2
3
4
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 3 of 34
Chapter 1 – Section 5
● Learning objectives Define designed experiment Understand the steps in designing an experiment Understand the completely randomized design Understand the matched-pairs design
1
2
3
4
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 4 of 34
Chapter 1 – Section 5
● Data can be collected in two main ways Through sample surveys Through designed experiments
● Sample surveys lead to observational studies● Designed experiments enable researchers to
control variables, leading to additional conclusions
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 5 of 34
Chapter 1 – Section 5
● A designed experiment is a controlled study● The purpose of designed experiments is to
control as many factors as possible to isolate the effects of a particular factor
● Designed experiments must be carefully set up to achieve their purposes
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 6 of 34
Chapter 1 – Section 5
● Some variables in a designed experiment are controlled, those are the explanatory variables
● These variables are also sometimes called the factors
● Factors Are part of a controlled environment Has values that can be changed by the researcher Are considered as possible causes
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 7 of 34
Chapter 1 – Section 5
● Examples of factors are The dosage of a drug in a medical experiment The type of teaching method in an education
experiment One drug by itself compared to that drug used in
conjunction with another
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 8 of 34
Chapter 1 – Section 5
● The designed experiment analyzes the affects of the factors on the response variable
● Response variables Are not part of a controlled environment Has values that are measured by the researcher Measure the effects
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 9 of 34
Chapter 1 – Section 5
● Examples of response variables are The blood pressures of the patients The test scores for a class The sizes of a cancerous tumor for patients
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 10 of 34
Chapter 1 – Section 5
● A treatment is a combination of the values of the factors
● Examples of treatments Giving one medication to one group of patients and a
different medication to another Using one type of fertilizer on a set of plots of corn
and a different type of fertilizer on a different set of plots
Playing country music to one group of mice and rap music to another
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 11 of 34
Chapter 1 – Section 5
● The treatment is applied to experimental units (people, plants, materials, other objects, …)
● When the experimental units are people, we refer to them as subjects
● Subjects in an experiment correspond to individuals in a survey
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 12 of 34
Chapter 1 – Section 5
● An example of a designed experiment is to determine whether a new drug, Drug N, is more effective at treating high blood pressure than the existing drug, Drug E
● Patients with high blood pressure are given either Drug N or Drug E
● The blood pressures are measured one month later
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 13 of 34
Chapter 1 – Section 5
● For this experiment Factor – the type of drug Response variable – blood pressure Treatment – given Drug N or Drug E Experimental units / subjects – the patients
● For this experiment Factor – the type of drug Response variable – blood pressure Treatment – given Drug N or Drug E Experimental units / subjects – the patients
● If patients given Drug N have significantly lower blood pressures than patients given Drug E, we would wish to conclude that Drug N is more effective
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 14 of 34
Chapter 1 – Section 5
● Changes in behavior of subjects● Changes in behavior of subjects● For an experiment comparing a new drug to no
treatment at all If the subject knows that he or she is given a drug, he
or she may feel better (the placebo effect)
● Changes in behavior of subjects● For an experiment comparing a new drug to no
treatment at all If the subject knows that he or she is given a drug, he
or she may feel better (the placebo effect)● For an experiment comparing a new drug to an
existing drug If the subject knows which drug he or she is given,
that may change his or her behavior
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 15 of 34
Chapter 1 – Section 5
● To avoid the effects of subject behavior Subjects not given any medication are often given a
placebo such as a sugar tablet The subjects will not know which treatment they get
● To avoid the effects of subject behavior Subjects not given any medication are often given a
placebo such as a sugar tablet The subjects will not know which treatment they get
● To avoid the effects of researcher behavior The researchers are not told which drug they are
administering
● To avoid the effects of subject behavior Subjects not given any medication are often given a
placebo such as a sugar tablet The subjects will not know which treatment they get
● To avoid the effects of researcher behavior The researchers are not told which drug they are
administering● When both the subjects and the researchers do
not know which treatment, this is called double-blind
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 16 of 34
Chapter 1 – Section 5
● Learning objectives Define designed experiment Understand the steps in designing an experiment Understand the completely randomized design Understand the matched-pairs design
1
2
3
4
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 17 of 34
Chapter 1 – Section 5
● Conducting an experiment involves considerable planning
● Conducting an experiment involves considerable planning
● Planning steps
● Conducting an experiment involves considerable planning
● Planning steps Identify the problem
● Conducting an experiment involves considerable planning
● Planning steps Identify the problem Determine the factors
● Conducting an experiment involves considerable planning
● Planning steps Identify the problem Determine the factors Determine the number of experimental units
● Conducting an experiment involves considerable planning
● Planning steps Identify the problem Determine the factors Determine the number of experimental units Determine the level of each factor
● Conducting an experiment involves considerable planning
● Planning steps Identify the problem Determine the factors Determine the number of experimental units Determine the level of each factor
● Implementation steps
● Conducting an experiment involves considerable planning
● Planning steps Identify the problem Determine the factors Determine the number of experimental units Determine the level of each factor
● Implementation steps Conduct the experiment
● Conducting an experiment involves considerable planning
● Planning steps Identify the problem Determine the factors Determine the number of experimental units Determine the level of each factor
● Implementation steps Conduct the experiment Test the claim
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 18 of 34
Chapter 1 – Section 5
● Identify the problem● The first step in planning an experiment (or in
most any project at all) is to identify the problem● The identification would include
The general purpose of the experiment The response variable The population
● This is also referred to as the claim
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 19 of 34
Chapter 1 – Section 5
● Determine the factors● The second step in planning an experiment is to
determine the factors to be studied● The factors could be identified
By subject matter experts By the purpose of the experiment Using results from previous studies
● Factors must be identified as either fixed, controlled, or uncontrolled
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 20 of 34
Chapter 1 – Section 5
● Determine the number of experimental units● In general, the more the experiment units, the
more effective the experiment● The number of experimental units
Could be limited by time Could be limited by money
● There are techniques to calculate the number of experimental units (to be covered later)
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 21 of 34
Chapter 1 – Section 5
● Determine the level of each factor● Three ways to deal with the factors● Determine the level of each factor● Three ways to deal with the factors
Control – fix the levels at a constant level (for factors not of interest)
● Determine the level of each factor● Three ways to deal with the factors
Control – fix the levels at a constant level (for factors not of interest)
Manipulate – set the levels at predetermined levels (for factors of interest)
● Determine the level of each factor● Three ways to deal with the factors
Control – fix the levels at a constant level (for factors not of interest)
Manipulate – set the levels at predetermined levels (for factors of interest)
Randomize – randomize the experimental units (for uncontrolled factors not of interest)
● Determine the level of each factor● Three ways to deal with the factors
Control – fix the levels at a constant level (for factors not of interest)
Manipulate – set the levels at predetermined levels (for factors of interest)
Randomize – randomize the experimental units (for uncontrolled factors not of interest)
● Randomization decreases the effects of uncontrolled factors, even ones not identified
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 22 of 34
Chapter 1 – Section 5
● Conduct the experiment● The subjects are assigned at random to the
treatments When a treatment is applied to more than one
experimental unit, this is called replication Replication is useful for accuracy, to further decrease
the effects of uncontrolled factors● Collect and process the data
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 23 of 34
Chapter 1 – Section 5
● Test the claim● This is inferential statistics● Techniques of inferential statistics are studied in
chapters 9 through 14
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 24 of 34
Chapter 1 – Section 5
● Learning objectives Define designed experiment Understand the steps in designing an experiment Understand the completely randomized design Understand the matched-pairs design
1
2
3
4
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 25 of 34
Chapter 1 – Section 5
● A completely randomized design is when each experimental unit is assigned to a treatment completely at random
● A completely randomized design is when each experimental unit is assigned to a treatment completely at random
● An example A farmer wants to test the effects of a fertilizer We choose a set of plants to receive the treatment We randomly assign plants to receive different levels
of fertilizer
● A completely randomized design is when each experimental unit is assigned to a treatment completely at random
● An example A farmer wants to test the effects of a fertilizer We choose a set of plants to receive the treatment We randomly assign plants to receive different levels
of fertilizer● This has similarities to completely random
sampling
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 26 of 34
Chapter 1 – Section 5
● We control as many factors as we can Amount of watering Method of tilling Soil acidity
● We control as many factors as we can Amount of watering Method of tilling Soil acidity
● Randomization decreases the effects of uncontrolled factors Rainfall Sunlight Temperature
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 27 of 34
Chapter 1 – Section 5
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 28 of 34
Chapter 1 – Section 5
● Learning objectives Define designed experiment Understand the steps in designing an experiment Understand the completely randomized design Understand the matched-pairs design Understand the randomized block design
1
2
3
5
4
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 29 of 34
Chapter 1 – Section 5
● A matched-pair design is when the experimental units are paired up and each of the pair is assigned to a different treatment
● A matched-pair design is when the experimental units are paired up and each of the pair is assigned to a different treatment
● A matched pair design requires Units that are paired (twins, the same person before
and after the treatment, …) Only two levels of treatment (one for each of the pair)
● A matched-pair design is when the experimental units are paired up and each of the pair is assigned to a different treatment
● A matched pair design requires Units that are paired (twins, the same person before
and after the treatment, …) Only two levels of treatment (one for each of the pair)
● An example A subject before receiving the medication The same subject after receiving the medication
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 30 of 34
Chapter 1 – Section 5
● Test whether students learn better while listening to music or not Match students by IQ and gender (to control those
factors) Randomly choose one of each pair (to decrease the
effects of other uncontrolled factors Assign that one to a quiet room and the other to a
room with music (the treatment) Administer the test and analyze the test scores
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 31 of 34
Chapter 1 – Section 5
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 32 of 34
Chapter 1 – Section 5
● An example We are testing the effects of treatments A, B, and C
on soybean plants Assume that group 1 is treated with A and group 2 is
treated with B Assume that Chemgro plants have higher yields than
Pioneer plants Assume that group 1 has more Chemgro plants
(happens because of randomization) than group 2
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 33 of 34
Chapter 1 – Section 5
● If group 1 (treatment A) has higher yields than group 2 (treatment B) Is this because treatment A is more effective than B? Is this because there are more Chemgro plants in
group 1?
● If group 1 (treatment A) has higher yields than group 2 (treatment B) Is this because treatment A is more effective than B? Is this because there are more Chemgro plants in
group 1?● It is not possible to distinguish
The effects of Treatment A versus B The effects of Chemgro versus Pioneer
● If group 1 (treatment A) has higher yields than group 2 (treatment B) Is this because treatment A is more effective than B? Is this because there are more Chemgro plants in
group 1?● It is not possible to distinguish
The effects of Treatment A versus B The effects of Chemgro versus Pioneer
● When two effects cannot be distinguished, this is called confounding
Sullivan – Fundamentals of Statistics – 2nd Edition – Chapter 1 Section 5 – Slide 34 of 34
Summary: Chapter 1 – Section 5
● The planning for designed experiments is crucial to the success of the experiment
● A double-blind implementation of experiments reduces the amount of changes in behavior
● There are different good methods for assigning treatments to experimental units Completely random Matched-pairs Randomized blocks
