KNR 405Intro & ValiditySlide 1

KNR 405

Applied Motor Learning

KNR 405Critiquing Research

Slide 2Applied Motor Learning

What’s it about The web site

http://www.cast.ilstu.edu/smith/405/405_home.htm

The general structure... Get a syllabus and read it Look particularly at the course

schedule 3 or 4 bits to it

KNR 405Critiquing Research

Slide 3Applied Motor Learning

The general structure... 3 or 4 bits to it

Research critiquing Introduction (motor control) (mostly) Motor Learning & (some) Sport

Psychology research readings Summary – overall message

KNR 405Critiquing Research

Slide 4Applied Motor Learning

So, now the first bit... Research critiquing

KNR 405Critiquing Research

Slide 5Elements of research design

Operationalization What do you want to measure? Whatever it is, you’ve got to choose a way to

measure it When you do, you will operationalize it

Whether or not you’ve made a good choice is determined by measurement validity

If you operationalize well, you should have good construct validity

KNR 405Critiquing Research

Slide 6Elements of research design

Independent variable What you or nature manipulates in some way

E.g. 1: What happens when you get older? Age is the independent variable (nature is the

manipulator) E.g. 2: What happens when you drink?

Blood alcohol level is the IV (you are the manipulator)

Critiquing IVs: Exhaustive? Mutually exclusive attributes? See also construct validity…

KNR 405Critiquing Research

Slide 7Elements of research design

Dependent variable The thing that is influenced (changed) by your

independent variable E.g. 1 (IV = Age): Skin sag, baldness, frequency of

urine expulsion, memory strength E.g. 2 (IV = Alcohol consumption): Balance, inhibition,

frequency of urine expulsion Can you think of any others?

Critiquing DV’s: see operationalization, reliability, measurement validity (all later)

KNR 405Critiquing Research

Slide 8Elements of research design

Hypothesis A specific statement of prediction

Inductive vs. deductive research Deductive has ‘em, inductive often doesn’t

Types One-tailed vs. two-tailed Directional vs. non-directional Association vs. difference

Hypothetical-deductive model

KNR 405Critiquing Research

Slide 9Validity as ‘aspects of truth’

Validity: the best available approximation to the truth* of a given proposition, inference, or conclusion

* This allows for criticism – which is where we come in

Conclusion Internal External Construct

KNR 405Critiquing ResearchSlide 10

Validity - General

Principles of validity

KNR 405Critiquing ResearchSlide 11

Conclusion Validity

Principles: Conclusion validity

“Conclusion validity is the degree to which conclusions we reach about relationships in our data are reasonable”

Two possible problems: conclude that there is no relationship when in fact there is

(you missed the relationship or didn't see it) conclude that there is a relationship when in fact there is

not (you're seeing things that aren't there!)

KNR 405Critiquing ResearchSlide 12

Conclusion Validity

Principles: Conclusion validity

conclude that there is no relationship when in fact there is (you missed the relationship or didn't see it) Low reliability

poor reliability of treatment implementation random irrelevancies in the setting random heterogeneity of respondents

Low statistical power Sample size, effect size, alpha level, power

KNR 405Critiquing ResearchSlide 13

Conclusion Validity

Principles: Conclusion validity

conclude that there is a relationship when in fact there is not (you're seeing things that aren't there!) fishing and the error rate problem

Too many analyses conducted at an inappropriate alpha level

KNR 405Critiquing ResearchSlide 14

Conclusion Validity

Principles: Conclusion validity

Using stats the wrong way can lead to either problem – violate statistical assumptions and the tests don’t work properly (so you can’t have faith in your findings)

KNR 405Critiquing ResearchSlide 15

Conclusion Validity

Principles: Improving Conclusion validity

Good statistical power Good reliability Good implementation

KNR 405Critiquing ResearchSlide 16

Internal Validity

KNR 405Critiquing ResearchSlide 17

Internal Validity

Single-group threats – taken care of by adding control group

Multiple-group threats – taken care of by random assignment

Social interaction threats

KNR 405Critiquing ResearchSlide 18

Internal Validity

Principles: Internal validity

First the design – what type of threats should we be looking for? See handout

Use the internal validity of the design to guide your discussion of the likelihood of alternative plausible explanations of the relationship examined in the study

KNR 405Critiquing ResearchSlide 19

Internal Validity

Principles: Internal validity

Use the internal validity of the design to guide your discussion of the likelihood of alternative plausible explanations of the relationship examined in the study Note – the key word is plausible Also, note that you are trying to suggest an alternative

reason why the relationship being studied might come about

KNR 405Critiquing ResearchSlide 20

Construct Validity: Critiquing

Determined by Operationalization

KNR 405Critiquing ResearchSlide 21

Construct Validity

Principles: Construct validity

Here the dependent and independent variables must be considered

State what they are first, and what they are purporting to measure, then proceed to critique whether they do the job

KNR 405Critiquing ResearchSlide 22

External Validity

Principles: External validity

Think of the goals for generalization of the study, and try to evaluate whether there are exceptions (important instances in which the expected relationship might not be found)

KNR 405Critiquing ResearchSlide 23

Validity

Proximal similarity model (Campbell, 1963)