Research Design

Research Design

“The best way to escape a problem is to solve it.”

-- Brendan Francis

research design

a methodology – to establish a sequence for

measurement,– to reduce confounds, and – to increase control

indicates the basic structure of the study

A research plan is

a detailed description of a proposed study designated to investigate a specific problem.

• (Gay, 83).

As used in research, the term “design” refers to the researcher’s plan for how to proceed.

We said previously the distinction between good and poor research, more often than any other factor, is the distinction between asking a good or poor research question.

The success, however, is often a function of the

extent to which the methods of endeavor have

been subjected to quantification through the use of measurement and

mathematics.

Measurement

systematic representation of the data by numbers

objectives– objectivity– assignment of numbers to data

according to well-established rules

Mathematics

the systematic representation of the relationships among the measurements with mathematical expressions

a universal language that any science or technology may use with power and convenience.

the vocabulary of mathematics is – unlimited– defined with rigorous accuracy

Contributors to the nature of the design

nature of the hypothesis the variables involved

– constructs constraints of the “real world”

Constructs

Traits such as intelligence, learning, anxiety, motivation, are not directly observable.

Referred to as constructs -- implying constructs of the scientist’s imagination

Constructs cannot be seen heard or felt. Can only be inferred by phenomenon such

as test scores, pulse rate, observed aggressive acts, etc.

• Best, J. & Kahn, J. (1998). Research in Education . Boston: Allyn and Bacon.

Constraints of the Real World: Problems in Conducting Research With Humans

No two persons are alike. No one person is completely

consistent from moment to moment. Humans are influenced by the

research process itself. The behavioral sciences have been

limited by the use of constructs.• Best, J. & Kahn, J. (1998). Research in Education . Boston: Allyn and Bacon.

Controlling the threats of internal and external validity is the art of research design.

Whether conducting research or evaluating the reporting of research findings, we want to be reasonably sure of two things:

to understand what influence, if any, the independent variable has on the dependent variable, and

to feel comfortable that the experimental environment sufficiently mirrors the real world to allow for generalizability of findings.

Internal and External

Internal– The differences are a result of the

manipulation of the independent variable.

External– The results of the study can be

expected to confirmed with similar studies of other groups

Threats to Internal Validity-- Campbell and Stanley

history maturation testing instrumentation statistical regression differential selection of subjects mortality selection-maturation interaction

Threats to External Validity

Pretest-Treatment Interactions Multiple Treatment Interference Selection-Treatment Interaction Specificity of Variables Experimenter Effects Reactive Arrangements

Controlling the threats of internal and external validity is the art of research design.

Methods for controlling Methods for controlling effects of variableseffects of variables

Hold the variable constant Randomize the effects of the

variable Match subjects with respect to the

variable Control the variable statistically

Design questions

what will be done to whom when

Must perform two related functions

must provide a logical structure that enables the researcher to pinpoint the effects of the independent variable on the dependent variable and thus answer the research question

help the researcher to rule out confounds as an alternative explanation for our findings

One focus of control is through participation

assignment selection

random selection

– selection of the entire group of participants for an experiment

– Larger samples are generally better– use of a random number generator or

random number table random assignment

– after participants have been selected, they can be assigned to experimental and control groups

Randomization

Dewey wins! All behavioral research should use the

method of randomization. It is the most powerful technique we

have for eliminating unintended participant-assignment confounds from the design of experiments.

Randomization randomizes everything.

Selecting participants

who how many how selected how assigned to treatments

selection

sampling– see text

• procedure• types

– simple random sample– stratified random sample– cluster sampling– two-stage cluster sampling

matching

Stratified samples Use of matched pairs Subject participates in each phase

of the experiment. (Could be source of additional threats.)

controlled statistically

If two variables are correlated, one can be used to predict the other:– GPA and SAT

Design has answered the who -- Now --

what will be done when

A Way of Looking at Designto Maximize Internal Validity

Between Subjects design

Within Subjects design

Between Subjects Design

separate groups of subject receive different levels of the independent variable.

to control for individual differences:– randomly assigned subjects to groups,

or– match subjects characteristics in each

group

Within Subjects Design

all subjects receive all levels of the independent variable.

to try to minimize carry over effects:– randomize the order of treatments, or– counterbalance the order of

treatments.• each treatment occurs in each time period

of the experiment

Pre-experimental designs

The One Shot Case Study The One Group Pretest-Posttest

Design The Static Group Comparison

Experimental

as determined by Campbell and Stanley participants are randomly assigned to

groups there are at least two levels of the

independent variable the design can control for the major

threats to internal validity the experiment compares two alternative

theoretical positions (optional)

Experimental Designs

the Pretest-Posttest Control Group design

the Posttest Only Control Group design

the Solomon Four Group design

The Solomon Four Group Design

R O1 X O2

R O3 O R X O5

R O6

Quasi-Experimental designs

The nonequivalent control group design

The time series design Counterbalanced design Factorial design

Descriptive Research Methods

Survey Interview Observational

Discuss

A newspaper article suggested that people who eat breakfast are healthier than those who do not. When this research was criticized, the researcher replied, “What’s the problem? All of our participants were randomly selected.”

Respond to these researchers.

Question 8aExplain the difference in

pre-experimental, experimental and quasi-experimental research

designs. Why is randomization such an

important factor?

Question 8bExplain the Solomon Four Group Design. What are

its strengths and weaknesses?