Vanderbilt Sports Medicine

SELECTING LITERATUREAn Evidence-Based Medicine

Approach

SELECTING LITERATUREAn Evidence-Based Medicine

ApproachKurt P. Spindler, MD

Professor & Vice Chair, OrthopaedicsDirector, Vanderbilt Sports Medicine & Ortho PCC

Head Team Physician, Vanderbilt University

B+JD Young Investigators InitiativeMay 13-15, 2005

Why EBM Select Literature?

1. Identify clinically relevant problem!2. Limit selection BIAS in peer- review

paper3. Learn hierarchy clinical studies4. Develop hypothesis from best study

designs5. Caveat: still need to play to study

sections bias for innovation and “clinical importance”

Is There Evidence in Literature Supporting EBM Approach for

ORTHOPAEDICS?1. JBJS-A, Jan 2003 Editorial (Heckman)

• Introducing levels of evidence• Five levels• Four study types

2. AJSM, 2002 Abstract Format (Reider)• Background• Hypothesis• STUDY DESIGN: list• Methods / Results / Conclusion• Clinical relevance

GOAL: Reach for Peaks!EBM approach review clinical literature

Concepts apply basic science

Application template

Basics

Title

Author

Reference

HYPOTHESIS– PRIMARY– SECONDARY

Type of Study

Treatment

Diagnosis

Screening

Prognosis

Causation

CLINICAL PRACTICE

In vitro• cell/matrix• gene biomechanic

THOUSANDS

In vivo• relevant animal models• biology • healing• biomechanic• safety

TENS OF THOUSANDS

Controlled Clinical Trials• outcomes• complications• risk/benefit• cost/benefit

HUNDREDS OF THOUSANDS

StudyDesign:

Cost:

ExtremeExtremeCAUTIONCAUTION

MAYBEMAYBE

YES: EBMYES: EBM

Clinical QUESTIONDetermines Study Design

Research ExamplePreferred

Study Design

Therapy Autograft choice RCT

Diagnosis Labral tearCross-sectional survey

ScreeningRole flexibility as injury risk

Cross-sectional survey

PrognosisPredictors OA after ACLR

Longitudinal cohort study

Risk FactorRisk OA Div I Athlete

Cohort or case control

1° HYPOTHESIS OR PREFERRED

TOPIC RESEARCH [EXAMPLES] RESEARCH DESIGN

TREATMENT [DRUG, PREVEN- RCT

TION, SURG]

Diagnosis [dx test] Cross-sect survey

Screening [value of test] Cross-sect survey

PROGNOSIS [DISEASE, INJURY, LONGITUDINAL CONDITION] COHORT

Causation [exposure to . . . ] Cohort or case-control

Study Type with Preferred Design

Sports Medicine Question

Ho: Anterior knee pain after ACL reconstruction is dependent on

autograft choice between Ham vs PTWhat do you believe?Approach to literature review:– Select articles that support your bias?–What is research topic?– Treatment choice– Focus review -- RCTs

Traditional Hierarchy ofClinical Treatment Studies

1. RCT (randomized controlled trials) = only computer or random # table acceptable

2. Cohort: two or more groups selected basis differences exposure to “agent” and f/u

3. Case control: pts particular disease/condition identified + “matched” control

4. Cross-sectional: data collected single timepoint5. Case reports/series: medical hxs one or more

patients with condition/tx reported on

Why do Treatment Studies Need Control Group?

1. Basics Scientific Method!

2. If no control group: tx is same, better, or worse than what?

3. Quality of “control” group one measure of validity of results

4. Unfortunately majority orthopaedic literature lack control group -- case series

Anterior Knee Pain S/P ACL Recon

Ho: Autograft choice Ham vs PTEBM Review: Systematic review nine RCTsRef: Spindler AJSM 2004Answer: NO DIFFERENCE 8/9 studies!Caveat:

a. Kneeling pain > PT 4/4 studies!b. Bynum PT ACL Recon PF pain

Preop = 40%, Postop = 20%, p < 0.05

Basic Science Grant

1. FOCUS LITERATURE EBM2. HYPOTHESIS: focus EBM key

clinical problem3. BEST STUDY DESIGN TOPIC– Clinical relevance—systematic

reviews– Background—related topics– Prelim data—review similar

studies– Design—metrics, techniques,

alternatives– Stats—method, sample size or

power

How to Identify Bias Study BIAS Example

Allocation groups Selection Fail randomize

Intervention Performance Fail control confounding variables

Follow-up Exclusion Not uniform or

(or Transfer) inadequate (<70%)

Outcomes Detection Dissimilar evaluation

independent examiner?

Validated question-

naire?

Definition of Bias1. SELECTION or SUSCEPTIBILITY = difference in

comparison groups secondary to incomplete randomization

2. PERFORMANCE = differences in care provided apart from intervention being evaluated

3. EXCLUSION or TRANSFER = differences in withdrawal from trial

4. DETECTION = different evaluation for outcomes best independent examiner or blinding examiner or validated outcome questionnaire self-administered

Sports Medicine Examples Bias

1. SELECTION:• ACL tr pt self-select OR vs Nonop tx = evaluate OA• Soccer teams self-select ACL inj prevention training, then

report difference incidence ACL tr

2. PERFORMANCE:• Report outcome of meniscal allograft or autologous

chondrocytes fail control concomitant ACL recon or HTO!

3. EXCLUSION OR TRANSFER:• Report conclusions based <70% f/u outcome variable

Statistical and Clinical Significance Outcomes

Absolute If ns power = ( ) ClinicallyAbsolute If ns power = ( ) Clinically

Outcome/Result Difference P for ( ) diff significantOutcome/Result Difference P for ( ) diff significant

a.a.

b.b.

c.c.

d.d.

Examples Statistical Significance vs Clinical Significance

1. Primary Ho and each AIM determine sample size by choosing a clinically meaningful difference in a single result or outcome measure chosen

2. Instrumented Laxity (KT 1000) ACL Recon Graft Choice

• Literature studies powered detect 1 mm side to side difference (n ≈ 70)

• How many surgeons would change practice if results 1 mm (few) vs 2 mm (some) vs 3 mm (many)

3. Thus clinical significance is based on both individual and “consensus” scientific community

4. Power or sample size set at 80% avoid Type II () error

Ideal vs Reality in Study Section

NIAMS has no study section for clinical research/outcomes.If your systematic review does not support perceived bias think twice. Recommend refocus support bias.Clinical significance vs statistical significance not well understood by basic science study sections. Plethora funded NIAMS studies without clinical significance but positive statistical results.Seek expert funded opinion on your grant.

Pearls

1. Develop ideas methods, results, statistics from best EBM in literature review based on realities previously discussed.

2. Retrospective review “your” cases!• Establish sample size• Timelines to complete• Generate methods

3. Consult statistician BEFORE begin study!

Summary1. EBM review literature2. Generate hypothesis (Ho)3. Construct preliminary AIMS4. Review literature modify Ho

and AIMS5. Develop TEAM6. CONSULT STATISTICIAN7. Clinical retrospective

reviews clinical pts, variation, outcomes

ReferencesWright JG: JBJS-Am 2000

Hurwitz SR: JBJS-Am 2000

McLeod RS: Surgery 1996

Greenhalgh T: How to Read a Paper. Br Med J 2001

Lang TA and Secic M: How to Report Statistics in Medicine. ACP 1997

Spindler K, Johnson R, Reider B: ICL AOSSM 2002