+ All Categories
Home > Documents > Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Date post: 15-Jan-2016
Category:
Upload: landon-brayshaw
View: 216 times
Download: 0 times
Share this document with a friend
13
Experimental design ITS class December 2, 2004
Transcript
Page 1: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Experimental design

Experimental design

ITS classDecember 2, 2004

ITS classDecember 2, 2004

Page 2: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Quantitative researchQuantitative research

Quantify relationships between variables

Measured on a sample of subjects

Relationships expressed through effect statistics Correlations, relative

frequencies, diff. between means

Quantify relationships between variables

Measured on a sample of subjects

Relationships expressed through effect statistics Correlations, relative

frequencies, diff. between means

Page 3: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Types of StudyTypes of Study

Descriptive Measure things as they are

Experimental Take measurements Try an intervention

(treatment) Take measurements

Descriptive Measure things as they are

Experimental Take measurements Try an intervention

(treatment) Take measurements

Page 4: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Descriptive studiesDescriptive studies

Case Case series Cross-sectional Cohort or prospective or

longitudinal Case-control or

retrospective

Case Case series Cross-sectional Cohort or prospective or

longitudinal Case-control or

retrospective

Page 5: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Experimental studiesExperimental studies

AKA longitudinal, repeated-measures, interventions

Without a control group Time series Crossover

With a control group

AKA longitudinal, repeated-measures, interventions

Without a control group Time series Crossover

With a control group

Page 6: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Control group studiesControl group studies

Necessary if treatment effect cannot be “washed out”

Random assignment: min. chance that either group is not typical of population

Single-blind: subjects Double-blind: experimenter

too Best studies: data analyzed

blind

Necessary if treatment effect cannot be “washed out”

Random assignment: min. chance that either group is not typical of population

Single-blind: subjects Double-blind: experimenter

too Best studies: data analyzed

blind

Page 7: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Ethical issuesEthical issues

Randomized controlled study may not be ethical

E.g.: Heart disease study over 10 years

Access to treatment Inform subjects of

randomization

Randomized controlled study may not be ethical

E.g.: Heart disease study over 10 years

Access to treatment Inform subjects of

randomization

Page 8: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Quality of designsQuality of designs

Quality of evidence for causal relationships

Least for case, case-series Cross-sectional, case-

control Prospective Experimental best

evidence

Quality of evidence for causal relationships

Least for case, case-series Cross-sectional, case-

control Prospective Experimental best

evidence

Page 9: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

ConfoundingConfounding

Two variables may be related for other reasons

Control for potential confounding factors

Two variables may be related for other reasons

Control for potential confounding factors

Page 10: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

SamplesSamples

Generalizability Representative Safest: Random sample

Balance: Stratified random (%s) Balance on pretest!

Selection bias: Not representative Sources: age, socioeconomic

status Self-selection into groups High compliance

Generalizability Representative Safest: Random sample

Balance: Stratified random (%s) Balance on pretest!

Selection bias: Not representative Sources: age, socioeconomic

status Self-selection into groups High compliance

Page 11: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Sample sizeSample size

Statistical significance Big enough to be sure you will

detect the smallest worthwhile effect To be sure =

detect 80% of time Detect =

95% of time, expect a smaller value if there is no effect, p<0.05

Smallest worthwhile effect = Smallest to make a diff. to subjects’

lives

Statistical significance Big enough to be sure you will

detect the smallest worthwhile effect To be sure =

detect 80% of time Detect =

95% of time, expect a smaller value if there is no effect, p<0.05

Smallest worthwhile effect = Smallest to make a diff. to subjects’

lives

Page 12: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Statistical significanceStatistical significance

P-value: the probability of getting

something more extreme than your result, when there is no effect in the population

P-value: the probability of getting

something more extreme than your result, when there is no effect in the population

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Page 13: Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

SourcesSources

http://www.sportsci.org/jour/0001/wghdesign.html: Good resource for basic stats

http://www.softwareevaluation.de/ : Website for EASy-D: a database of evaluation of adaptive systems, also links to research-based usability guidelines

http://www.sportsci.org/jour/0001/wghdesign.html: Good resource for basic stats

http://www.softwareevaluation.de/ : Website for EASy-D: a database of evaluation of adaptive systems, also links to research-based usability guidelines


Recommended