CLINICAL RESEARCH METHODS AND STUDY DESIGN

Gregory W. Heath, D.H.Sc., M.P.H.Department of Medicine

University of Tennessee College of Medicine, Chattanooga

Adapted from Jack M. Guralnik, M.D., PhD – NIA/NIH, 2004

Epidemiology

The study of the distribution and determinants of disease and injury in human populations

Fundamental Assumptions of Epidemiology

1. Human disease does not occur at random

2. Human disease has causal and preventivefactors that can be identified through systematicinvestigation of different populations or subgroupsof individuals within a population

Hennekens and Buring, 1987

Study Designs - Overview

Descriptive studies Populations: correlation (ecological) studies Individuals

Case reportsCase seriesCross-sectional surveys

Analytic studies Observational studies

Case-control studiesProspective cohort studies – historical and longitudinal

Intervention studies (clinical trials)

Case Reports and Case SeriesCase Reports and Case Series   Objective: To make observations about Objective: To make observations about patients patients with defined clinical with defined clinical characteristics (e.g., patients with a characteristics (e.g., patients with a certain disease or cluster of symptoms)certain disease or cluster of symptoms)  Design: Design: - - simple description of clinical data simple description of clinical data -- data derived from a well-defined data derived from a well-defined

group of individualsgroup of individuals

Observations should be comprehensive and Observations should be comprehensive and adequately detailedadequately detailed

1.1. Clear definitions of phenomenon being Clear definitions of phenomenon being studiedstudied

  2.2. Definitions should be applied equally to Definitions should be applied equally to

all all individuals in the seriesindividuals in the series  

3.3. Observations should be reliable and Observations should be reliable and reproduciblereproducible

  

Interpretation/Conclusion:Interpretation/Conclusion:1.1. What observations have been made prior to What observations have been made prior to

this report?this report?  

2.2. What new phenomenon is illustrated?What new phenomenon is illustrated?  

3.3. What further studies should be done?What further studies should be done?  

4.4. Is the study group representative of all Is the study group representative of all patientspatients with this disorder-- can conclusions be with this disorder-- can conclusions be

generalized?generalized?  

AdvantagesAdvantages  

1.1. Useful in hypothesis formation, natural Useful in hypothesis formation, natural history history studies, describing "clinical experience"studies, describing "clinical experience"  

2.2. Easy and inexpensive to do in hospital settingsEasy and inexpensive to do in hospital settings

Disadvantages: Biased selection of study patients Disadvantages: Biased selection of study patients may lead to inability to generalize study resultsmay lead to inability to generalize study results  

1.1. Were only sickest or most typical patients Were only sickest or most typical patients included in the study?included in the study?

  2.2. Were the findings a chance happening or a Were the findings a chance happening or a

characteristic of the disease?characteristic of the disease?

    

CDC. Pneumocystis pneumonia-- Los Angeles. CDC. Pneumocystis pneumonia-- Los Angeles. MMWRMMWR 1981; 30:250-2. 1981; 30:250-2.  

Initial report of five cases of pneumocystis pneumonia Initial report of five cases of pneumocystis pneumonia in previously healthy, homosexual men.in previously healthy, homosexual men.  

Case report/Case series - ExampleCase report/Case series - Example

Cross-Sectional Studies (Prevalence Surveys)Cross-Sectional Studies (Prevalence Surveys)   

A.A. Object: To make observations concerning Object: To make observations concerning the the prevalence and characteristics of a disease prevalence and characteristics of a disease and and other participant characteristics in a well-other participant characteristics in a well-defined population  defined population  

Prevalence – number with disease at one point in timenumber at risk at that point

Incidence – number of new cases of disease over a period of timenumber of people at risk during that period

Prevalence vs. Incidence

Design Design   

1.1. Define the population under studyDefine the population under study  

2.2. Derive a sample of the populationDerive a sample of the population  

3.3. Define the characteristics being studiedDefine the characteristics being studied

ObservationsObservations  

1.1. Should be standardized and clearly definedShould be standardized and clearly defined  

2.2. Methods of data collection should be Methods of data collection should be consistently applied to all study participantsconsistently applied to all study participants

Presentation of findingsPresentation of findings  

1.1. Prevalence (%, cases per 10Prevalence (%, cases per 1055, etc.) for the , etc.) for the observation in the populationobservation in the population

  2.2. Mean or median levels of relevant factors in Mean or median levels of relevant factors in

the populationthe population  

3.3. Important subgroups may need separate data Important subgroups may need separate data presentation (e.g., age, race, sex)presentation (e.g., age, race, sex)

  

ConclusionsConclusions  1.1. Descriptive: Descriptive: a.a. How common is the factor in How common is the factor in

the study population? the study population?   

b.b. What are the characteristics of the What are the characteristics of the group of interest (those with group of interest (those with

disease, disease, of given age, etc.) in the of given age, etc.) in the population? population?   

c.c. What are the distributions of What are the distributions of factors factors of interest (age, blood of interest (age, blood pressure, vital pressure, vital capacity, etc.) in the capacity, etc.) in the study study population?population?

Conclusions (con’t)Conclusions (con’t)

2.2. Associative: Associative:   

a.a. What are the relationships of the factors of What are the relationships of the factors of interest to other factors in the study interest to other factors in the study population?population?

  b.b. How do persons with the factor of interest How do persons with the factor of interest

differ from those without it?differ from those without it?

AdvantagesAdvantages  

1.1. Inexpensive for common diseasesInexpensive for common diseases  

2.2. Provide more representative cases than do case Provide more representative cases than do case series series

3.3. Relatively short duration of the studyRelatively short duration of the study  

4.4. Can be addressed to specific populations of Can be addressed to specific populations of interestinterest

  5.5. Can examine wide variety of factors Can examine wide variety of factors

simultaneouslysimultaneously

  

   DisadvantagesDisadvantages  

1.1. Unsuitable for rare diseasesUnsuitable for rare diseases

2.2. Bias may be operative Bias may be operative

3.3. High refusal rate / low response rateHigh refusal rate / low response rate  

4.4. Generally more expensive and time-Generally more expensive and time-consuming than case-control studies consuming than case-control studies

  5.5. The disease process may alter measurements The disease process may alter measurements

6.6. No data are collected regarding temporal No data are collected regarding temporal relationship relationship

    

          Hedley AA, Ogden CL, Johnson CL, Carroll MD, Curtin LR, Flegal KM. Prevalence of overweight and obesity among US children, adolescents, and adults, 1999-2002. JAMA 2004;291:2847-50.

Prevalence data on overweight and obesity using measured height and weight in National Health and Nutrition Examination  Survey (NHANES)

Cross Sectional Study - ExampleCross Sectional Study - Example

Case-Control Study Case-Control Study   

Object: To make observations Object: To make observations regarding regarding possible associations possible associations between a disease between a disease and one or more and one or more hypothesized risk hypothesized risk factors factors

Design 

1. General strategy: To compare the prevalence or level of the possible risk factor between a representative group of disease subjects (cases) and a representative group of disease-free subjects (controls) derived from the same population

Exposed Exposed Non-Exposed

Disease No Disease

Non-Exposed

Basic assumptions

  a. Cases are representative of all patients who develop the disease

  b. Controls are representative of the general "healthy" population who do not develop the disease

  c. Information is collected from cases and controls in the same way

Selection of casesSelection of cases  

a.a. Should have standardized selection criteria Should have standardized selection criteria from a well-defined populationfrom a well-defined population

  b.b. Sources: case registries, admission records, Sources: case registries, admission records,

pathology logspathology logs  

c.c. Aim for as high a participation rate as possibleAim for as high a participation rate as possible  

Selection of controls-- the most difficult issueSelection of controls-- the most difficult issue  

a.a. The perfect control group probably doesn't The perfect control group probably doesn't existexist

  b.b. Must have standard selection criteria from a Must have standard selection criteria from a

well-defined populationwell-defined population  

c.c. Sources: sample of general population, Sources: sample of general population, neighborhood, families neighborhood, families

d.d. Cost and accessibility should be considered in the Cost and accessibility should be considered in the selection of controlsselection of controls

  e.e. Multiple control groups are considered to be Multiple control groups are considered to be

methodologically superior  methodologically superior  

  

Observations:Observations: Data are collected "looking back" for Data are collected "looking back" for possible exposurespossible exposures

     1. 1. All observations should be made using the same All observations should be made using the same

methods in cases and controlsmethods in cases and controls   2. 2. Validity of measurement techniques should be Validity of measurement techniques should be

establishedestablished

Potential sources of bias: Selection and observationPotential sources of bias: Selection and observation  

1.1. Types of selection biasTypes of selection bias  

a.a. Prevalence-incidence bias Prevalence-incidence bias 

b.b. Non-respondent bias:Non-respondent bias:

Types of observational or interviewer biasTypes of observational or interviewer bias

a.a. Diagnostic suspicion biasDiagnostic suspicion bias  

b.b. Exposure suspicion biasExposure suspicion bias

Types of observational or interviewer bias Types of observational or interviewer bias (continued)(continued)

  c.c. Recall bias Recall bias

  d.d. Family information biasFamily information bias

Presentation of findings: The 2 X 2 tablePresentation of findings: The 2 X 2 table

Characteristic/ Characteristic/ ExposureExposure

Presence of DiseasePresence of Disease

TotalTotalNumber with Number with

DiseaseDiseaseNumber Number

without Diseasewithout Disease

PresentPresent aa bb a + ba + b

AbsentAbsent cc dd c + dc + d

TotalTotal a + ca + c b + db + d NN

Advantages of a case-control studyAdvantages of a case-control study  

1.1. May be the only way to study the etiology of rare May be the only way to study the etiology of rare diseasesdiseases

  2.2. Can study multiple etiologic factors simultaneouslyCan study multiple etiologic factors simultaneously

  3.3. Less time-consuming and expensiveLess time-consuming and expensive

  4.4. If assumptions are met, associations and risk If assumptions are met, associations and risk

estimates are consistent with other types of studiesestimates are consistent with other types of studies

Disadvantages of a case-control studyDisadvantages of a case-control study  

1.1. Does not estimate incidence or prevalenceDoes not estimate incidence or prevalence  

2.2. Relative risk is only indirectly measuredRelative risk is only indirectly measured

3.3. Both selection and information biases may give Both selection and information biases may give potentially spurious evidence of association between potentially spurious evidence of association between a factor and a diseasea factor and a disease

  4.4. Usually cannot study rare exposuresUsually cannot study rare exposures

  5.5. Temporal relationship between exposure and disease Temporal relationship between exposure and disease

can be difficult to documentcan be difficult to document

    

          Herbst AL, Ulfelder H, Poskaner DC. Adenocarcinoma of the vagina: Association of maternal stilbesterol therapy with tumor appearance in young women. N Engl J Med 1974;284:878-881.

Case-control design was able to identify relationship of exposure to stilbestrol during mother’s pregnancy with occurrence of rare tumor in female offspring many years later

Case-Control Study - ExampleCase-Control Study - Example

Prospective or Longitudinal Cohort Studies . Prospective or Longitudinal Cohort Studies .     Object: Object: To make observations concerning the To make observations concerning the

association between a given exposure (risk association between a given exposure (risk factor) and subsequent development of a factor) and subsequent development of a disease/outcome disease/outcome 

Study Design: To identify a group of persons Study Design: To identify a group of persons

exposed to a purported risk factor exposed to a purported risk factor and aand a second similar group non- second similar group non- exposed to the risk factor and follow exposed to the risk factor and follow forward to compare incidence forward to compare incidence rates between groups.rates between groups.

Types of Prospective StudiesTypes of Prospective Studies  

Concurrent prospective study (longitudinal study): a Concurrent prospective study (longitudinal study): a defined population at present time is surveyed to defined population at present time is surveyed to identify exposed and non-exposed individuals who identify exposed and non-exposed individuals who are followed forward in time (e.g., several years) to are followed forward in time (e.g., several years) to define incidence ratesdefine incidence rates

  Non-concurrent prospective study (retrospective or Non-concurrent prospective study (retrospective or historical cohort study): a defined population has had historical cohort study): a defined population has had presence presence or absence of exposure ascertained in an or absence of exposure ascertained in an accurate, accurate, objective fashion in the past and is objective fashion in the past and is surveyed at present for occurrence of disease to surveyed at present for occurrence of disease to allow definition of incidence rates in exposed and allow definition of incidence rates in exposed and non exposednon exposed

TIMEFRAMES FOR HYPOTHETICAL CONCURRENT AND NON-CONCURRENT PROSPECTIVE STUDIES

CONDUCTED IN 2003

2003

2013

2023

1983

1993

2003

Concurrent: Non-Concurrent:

Defined Population

Exposed Non-Exposed

Disease No Disease

DiseaseNo

Disease

Longitudinal Historical

3.3. AssumptionsAssumptions  

a.a. Exposed and non-exposed groups are Exposed and non-exposed groups are representative samples of a well-defined general representative samples of a well-defined general populationpopulation

  b.b. Absence of "exposure" also well defined and Absence of "exposure" also well defined and

assumed to be maintained in the non-exposed assumed to be maintained in the non-exposed group during the course of the studygroup during the course of the study

  ObservationsObservations  

1.1. Definitions of disease outcome should be well Definitions of disease outcome should be well determined prior to the study's inception and should determined prior to the study's inception and should not be changed during the course of the studynot be changed during the course of the study

a.a. Endpoints may vary in "hardness", e.g., from death Endpoints may vary in "hardness", e.g., from death to subjective symptomsto subjective symptoms

  b.b. Standard criteria should be applied to both exposed Standard criteria should be applied to both exposed

and non-exposed groups, i.e., there should be no bias and non-exposed groups, i.e., there should be no bias in determining outcomes in exposed vs. non exposedin determining outcomes in exposed vs. non exposed

2.2. Definitions of disease should be reliable and reproducibleDefinitions of disease should be reliable and reproducible  3.3. Every effort should be made to minimize the "lost to Every effort should be made to minimize the "lost to follow-up" rate since large non-response rates (> 20%) follow-up" rate since large non-response rates (> 20%) raise raise questions as to the accuracy of incidence rates in questions as to the accuracy of incidence rates in exposed exposed and non-exposed groupsand non-exposed groups

Presentation of findings: The 2 X 2 tablePresentation of findings: The 2 X 2 table

Presence Of Disease

CharacteristicNumber With

DiseaseNumber Without

Disease Total

Present a b a + b

Absent c d c + d

Total a + c b + d N

ConclusionsConclusions  

1.1. To what larger groups can the results be To what larger groups can the results be generalized?generalized?

  2.2. Is the association significant? Is the Is the association significant? Is the

association strong?association strong?

Advantages of the prospective studyAdvantages of the prospective study  

1.1. Cases are incident cases and may be more Cases are incident cases and may be more representative of cases than in case-control studiesrepresentative of cases than in case-control studies

  2.2. Design provides more information about the natural Design provides more information about the natural

history of the diseasehistory of the disease  

3. 3. Incidence rates are availableIncidence rates are available  

4. 4. Relative risk is directly estimatedRelative risk is directly estimated  

5. 5. Fewer sources of bias than retrospective studiesFewer sources of bias than retrospective studies

6.6. Many diseases can be studied with regard to their Many diseases can be studied with regard to their relationship to the exposurerelationship to the exposure

  7.7. Temporal relationships between exposure and Temporal relationships between exposure and

disease firmly establisheddisease firmly established  

8.8. Best to study effects of rare exposure with frequent Best to study effects of rare exposure with frequent cases among the exposedcases among the exposed

  DisadvantagesDisadvantages  

1.1. Duration of the study may be exceedingly long, Duration of the study may be exceedingly long, making difficult the maintenance of consistent study making difficult the maintenance of consistent study methods and enthusiasm of the staffmethods and enthusiasm of the staff

Disadvantages (continued) Disadvantages (continued) 

2.2. Follow-up of free-living populations may be very Follow-up of free-living populations may be very expensiveexpensive

  3.3. Large populations often requiredLarge populations often required  4.4. Exposures can be studied only if baseline data are Exposures can be studied only if baseline data are

availableavailable  5.5. Rare diseases cannot be studiedRare diseases cannot be studied

6.6. Several types of bias may produce spurious Several types of bias may produce spurious association (bias of assessment, loss to follow-up)association (bias of assessment, loss to follow-up)

    

          Doll R, Hill AB. The mortality of doctors in relation to their smoking habits: A preliminary report. Br Med J 1954;228(i):1451-1455.

Doll R, Peto R, Boreham J, Sutherland I. Mortality in relation to smoing: 50 years observations on mae British doctors. Br Med J 2004;328:1519-1533.

Prospective cohort study that showed early increase in risk of lung cancer and heart disease mortality and confirmed this over 50 years of follow-up.

Prospective Longitudinal Cohort Study - ExampleProspective Longitudinal Cohort Study - Example

    

          Plassman BL, Havlik RJ, Steffens DC, et al. Documented head injury in early adulthood and risk of Alzheimer's disease and other dementias. Neurology 2000;55:1158-1166.

Military medical records used to identify WW II head trauma exposure group and non-trauma

comparison group who were traced and evaluated for dementia 50 years later.

Prospective Historical Cohort Study - ExampleProspective Historical Cohort Study - Example

    A.A. Statistical significanceStatistical significance

  B.  B. Strength of association (odds ratio, relative risk)Strength of association (odds ratio, relative risk)

  C.  C. Dose-response relationshipsDose-response relationships

  D.  D. Temporal sequenceTemporal sequence

  E.  E. Consistency of the association (internal "validity")Consistency of the association (internal "validity")

  F.  F. Replication of results (external validity)Replication of results (external validity)

  G.  G. Biological plausibility Biological plausibility

  H.  H. Experimental evidenceExperimental evidence

Causal Inference in Observational Studies: Epidemiologic Criteria

Case reportsCase reports

Case seriesCase series

Cross-sectional surveysCross-sectional surveys

Case-control studiesCase-control studies

Prospective cohort studiesProspective cohort studies

Clinical trialsClinical trials

Hierarchy of Study DesignHierarchy of Study Design