Experimental Design: Single factor designs

Psych 231: Research Methods in Psychology

Announcements

Reminder: your group project experiment method section is due in labs this week

Remember to download, print and READ the class exp articles

Methods of Controlling Variability

Comparison Production Constancy/Randomization

Methods of Controlling Variability

Comparison – An experiment always makes a comparison, so it

must have at least two groups• Sometimes there are control groups

– This is typically the absence of the treatment» Without control groups if is harder to see what is really

happening in the experiment» it is easier to be swayed by plausibility or inappropriate

comparisons• Sometimes there are just a range of values of the IV

Methods of Controlling Variability

Production– The experimenter selects the specific values of the

Independent Variables • Need to do this carefully

– Suppose that you don’t find a difference in the DV across your different groups

» Is this because the IV and DV aren’t related?» Or is it because your levels of IV weren’t different

enough

Methods of Controlling Variability

Constancy/Randomization– If there is a variable that may be related to the

DV that you can’t (or don’t want to) manipulate• Control variable: hold it constant• Random variable: let it vary randomly across all of the

experimental conditions

– But beware confounds, variables that are related to both the IV and DV but aren’t controlled

Experimental designs

So far we’ve covered a lot of the about details experiments generally

Now let’s consider some specific experimental designs.– 1 Factor, two levels– 1 Factor, multi-levels– Factorial (more than 1 factor)– Between & within factors

Poorly designed experiments

Example: Does standing close to somebody cause them to move?– So you stand closely to people and see how long before

they move

– Problem: no control group to establish the comparison group (this design is sometimes called “one-shot case study design”)

Single variable – One Factor designs

1 Factor (Independent variable), two levels– Basically you want to compare two treatments

(conditions)– The statistics are pretty easy, a t-test

T-test = Observed difference btwn conditions

Difference expected by chance

1 factor - 2 levels

Example– How does anxiety level affect test performance?

• Two groups take the same test– Grp1 (moderate anxiety group): 5 min lecture on the

importance of good grades for success– Grp2 (low anxiety group): 5 min lecture on how good

grades don’t matter, just trying is good enough

1 factor - 2 levels

participants

Low

Moderate Test

Test

Random Assignment

Anxiety Dependent Variable

Single variable – one Factor

anxiety

low moderate

8060

low moderatete

st p

erfo

rman

ce

anxiety

One factor

Two levels

Use a t-test to see if these points are statistically different

Single variable – one Factor

Advantages:– Simple, relatively easy to interpret the results– Is the independent variable worth studying?

• If no effect, then usually don’t bother with a more complex design

– Sometimes two levels is all you need• One theory predicts one pattern and another predicts a

different pattern

Single variable – one Factor

Disadvantages:– “True” shape of the function is hard to see

• interpolation and extrapolation are not a good idea

Interpolation

low moderate

test

per

form

ance

anxiety

What happens within of the ranges that you test?

Extrapolation

low moderate

test

per

form

ance

anxiety

What happens outside of the ranges that you test?

high

Poorly designed experiments

Example 1: – Testing the effectiveness of a stop smoking

relaxation program– The subjects choose which group (relaxation or no

program) to be in

Poorly designed experiments Non-equivalent control groups

participants

Traininggroup

No training (Control) group

Measure

Measure

Self Assignment

Independent Variable

Dependent Variable

RandomAssignment

– Problem: selection bias for the two groups, need to do random assignment to groups

Poorly designed experiments

Example 2: Does a relaxation program decrease the urge to smoke?– Pretest desire level – give relaxation program – posttest

desire to smoke

Poorly designed experiments One group pretest-posttest design

participants Pre-test Training group

Post-testMeasure

Independent Variable

Dependent Variable

Dependent Variable

– Problems include: history, maturation, testing, and more

1 Factor - multilevel experiments

For more complex theories you will typically need more complex designs (more than two levels of one IV)

1 factor - more than two levels– Basically you want to compare more than two

conditions– The statistics are a little more difficult, an ANOVA

(analysis of variance)

1 Factor - multilevel experiments

Example (same as earlier with one more group)– How does anxiety level affect test performance?

• Three groups take the same test– Grp1 (moderate anxiety group): 5 min lecture on the

importance of good grades for success– Grp2 (low anxiety group): 5 min lecture on how good

grades don’t matter, just trying is good enough– Grp3 (high anxiety group): 5 min lecture on how the

students must pass this test to pass the course

1 factor - 3 levels

participants

Low

Moderate Test

Test

Random Assignment

Anxiety Dependent Variable

High Test

1 Factor - multilevel experiments

anxiety

low mod high

8060 60

low modte

st p

erfo

rman

ceanxiety

high

1 Factor - multilevel experiments

Advantages– Gives a better picture of the relationship (function)– Generally, the more levels you have, the less you have

to worry about your range of the independent variable

Relationship between Anxiety and Performance

low moderate

test

per

form

ance

anxiety

2 levels

highlow modte

st p

erfo

rman

ce

anxiety

3 levels

1 Factor - multilevel experiments

Disadvantages– Needs more resources (participants and/or stimuli)– Requires more complex statistical analysis (analysis of

variance and pair-wise comparisons)

Pair-wise comparisons

The ANOVA just tells you that not all of the groups are equal.

If this is your conclusion (you get a “significant ANOVA”) then you should do further tests to see where the differences are– High vs. Low– High vs. Moderate– Low vs. Moderate

Next time

Adding a wrinkle: between-groups versus within-groups factors

Read chapter 11