Medical Literature:
Reading, Interpreting, and Writing
Muhammad Mamdani, PharmD, MA, MPHDirector, Applied Health Research Centre
LKSKI of St. Michael’s HospitalAssociate Professor – University of Toronto
Adjunct Faculty – King Saud UniversityFebruary 2010
Objectives
To better understand the basic purpose of research and the context under which research is conducted
To review basic clinical research study designs
To review two standard study review tools
To review basic principles in writing clinical research manuscripts
Clinical Research: Basic Purpose and Context
The Common Goal of Research
‘The’ GoalAttempt to estimate the ‘truth’
TRUTH
Study 3Study 1 Study 2
Perception: Reality vs Belief
Variability in Perceptions
The Trouble with ResearchExample: H. pylori eradication and non-ulcer dyspepsia
McColl et al (NEJM, 1998)Treatment (n=154): Omeprazole 20 mg bid + amoxicillin 500 mg tid + 400 mg metronidazole tid x 2 weeks
Comparison (n=154): omeprazole 20 mg bid x 2 weeksSymptom resolution at 1 year: Tx = 21% vs. Comparison = 7% (p<0.001)
Blum et al (NEJM, 1998)Treatment (n=164): Omeprazole 20 mg bid + amoxicillin 1000 mg bid + 500 mg clarithromycin bid x 1 week
Comparison (n=164): omeprazole 20 mg bid x 1 weekSymptom resolution at 1 year: Tx = 27% vs. Comparison = 21% (p=0.17)
The Evolving Nature of Research
The Evolving Nature of Research
Analyzed 115 articles published in 1990-2003 in the 3 major general medical journals (NEJM, JAMA, Lancet) and specialty journals that had received over 1000 citations each by August 2004
49 reported evaluations of health care interventions; 45 claimed that the interventions were effective.
By 2004 5/6 non randomised studies and 9/39 randomised trials were already contradicted or found to be exaggerated
Ioannidis JP. JAMA 2005
But How Do I Know About the Quality and Validity
of the Study?
Understanding Clinical Research: What do I Need to Fully Interpret a
Clinical Study?
Understand Clinical Research Methodology: Epidemiology
Analytical ExpertiseQuantitative: BiostatisticsQualitative: Qualitative Analytics
Clinical SenseMedical / Healthcare Training
The Medical Research Spectrum(Sung, JAMA, 2003)
The Medical Research Spectrum(Sung, JAMA, 2003)
Basic Clinical Research Methodology
Comparative Clinical Research: A Simplistic Overview(Melot, CCM, 2009)
Approaches to Clinical Research
Two Major ApproachesExperimental Studies: Randomized Trials
Randomly allocate subjects to different treatment strategies and follow them up to assess outcomes
Operationally complex, analytically simple
Observational Studies Operationally simple, analytically complex
Randomized Clinical Trial: Design(Melot, CCM, 2009)
Validity and Biases in Clinical Trials(Melot, CCM, 2009)
Major Observational Study Types
Cross-sectional Study
Cohort Study
Case-Control Study
Case-Cross-Over Study
TIME
Basic Schematic for Cross-Sectional StudyBasic Schematic for Cross-Sectional Study
Data for analysis
Major Issues:1)Temporal Sequencing2)Selection Bias3)Confounding
TIME
Look-back Window
Index Entry Date
for Exposure
End of Follow-up Date
Observation Window
Basic Schematic for Cohort StudyBasic Schematic for Cohort Study
Major Issues:1)Selection Bias2)Confounding
Basic Schematic for Case-Control StudyBasic Schematic for Case-Control Study
Time Period
Event
No Event
Cases
Controls
Time Period
Major Issues:1)Selection Bias2)Confounding3)Only indirect estimates of time effects
Basic Schematic for Case-Crossover StudyBasic Schematic for Case-Crossover Study
Time Period A Time Period B
Event
Compare exposure in Time Period A vs. Time Period B
only among patients with an event and exposure in either period
Major Issues:1)Largely used for exposures with immediate effects2)Small number3)‘Reverse protopathic’ bias
How Do I Know Which Study Design is Best?
Level of EvidenceLevel of Evidence Study TypeStudy Type
Level 1Level 1 RCTsRCTs
Level 2Level 2 Cohort StudiesCohort Studies
Level 3Level 3 Case-Control Case-Control StudiesStudies
Level 4Level 4 Case SeriesCase Series
Level 5Level 5 Expert OpinionExpert OpinionOxford Centre for Evidence-Based Medicine, 2002
Analytical Aspects
Numeric Literacy(Horton and Switzer, NEJM, 2005)
Sophistication of statistical methods or articles published in the NEJM has been increasing over time
Some statistical tests T-tests Contingency tables Non-parametric tests Epidemiologic statistics Pearson’s correlation Nonparametric correlation (e.g. spearman’s correlation) Simple linear regreasion Analysis of variance Transformations
Only 21% of articles published between January 2004 – June 2005 would be considered accessible by those with basic training in biostatistics
Numerical Methods
Risks: Proportions of Patients Absolute Risk (AR) Number Needed to Treat (NNT)
Ratio of Risks Relative Risk (RR) Hazard Ratio (HR) Odds Ratio (OR)
Measures of Association: An Example
PlaceboPlacebo TreatmentTreatment
MIMI 100100 3030
No MINo MI 1990019900 1997019970
Total n = 40,000
N per study group = 20,000
Absolute Risks
AR (Placebo) = 0.5%
AR (Treatment) = 0.15%
ARR = 0.5% - 0.15% = 0.35%
NNT = 1/0.35% = 296
Relative Risks
RR = 0.15% / 0.5% = 0.30
RRR = 1/RR = 70%
OR (CC: Logistic Regression)
HR (Cohort: Cox PH Models)
Issues with Numerical Approaches
Lacy et al, Am J Cardiol, 2001
Assessed 400 health professionals for their willingness to prescribe a drug based on different measures for reporting the same likelihoods
Literature Evaluation Tools
Standards for Assessing Quality of Research
Clinical Trials CONSORT statement http://www.consort-statement.org/
Observational Research STROBE Statement http://www.strobe-statement.org/
Consort Statement
STROBE
Group Activity: Pitt et al
Group Exercise20 Minutes Group Review
One person from each group to present for up to 2 minutes
Use CONSORT
GroupsGroup 1: Abstract and IntroductionGroup 2: MethodsGroup 3: ResultsGroup 4: Comment – make recommendation
Pitt et al: Study Overview
RALES: Randomized ALdactonE StudyPitt et al, N Engl J Med, 1999
Double-blind randomized controlled trial in patients with congestive heart failure
Primary endpoint: all-cause mortality
Screened (n=?)
Randomize
Spironolactone (n=822)
25 mg po qd
- Increase to 50 mg po qd where tolerated
Placebo (n=841)
Inclusion Exclusion
RALES Criteria
Inclusion / ExclusionNYHA class III or IV at time of enrolmentLVEF < 35% within 6 monthsExclude patients with serum creatinine 2.5 mg/dL or serum potassium > 5 mmol/L
Follow-upLab and clinic follow-up at 4 weeks and 3 and 6 months
Appropriate use of ACE inhibitors and beta-blockers
D/c K-sparing diuretics and K+ supplementsHolding spironolactone for hyperkalemia or creatinine > 4 mg/dL
RALES - Results
OutcomeOutcome Absolute RiskAbsolute Risk Relative RiskRelative Risk
DeathDeath Pl=46% Pl=46% Spir=35%Spir=35%
p<0.001p<0.001
0.70 0.70
(0.60-0.82)(0.60-0.82)
ReadmissioReadmission for HFn for HF
Pl=36% Pl=36% Spir=26%Spir=26%
p<0.001p<0.001
0.650.65
(0.54-0.77)(0.54-0.77)
Serious Serious HyperkalemHyperkalem
iaia
Pl=1.2% Pl=1.2% Spir=1.7%Spir=1.7%
p=0.42p=0.42
NSNS
What Happened in Actual Practice?
Juurlink et al, NEJM, 2004
Spironolactone Prescription Uptake1994-2001
0
20
40
60
80
100
120
140
160
Year
Pre
scri
bin
g r
ate
(p
er
1000 p
ati
en
ts)
RALES
Hospital Admission Associated with K+
0
2
4
6
8
10
12
14
Year
Ad
mis
sio
n r
ate
(per
100
0 p
atie
nts
)
RALES
Admission Associated with K+ Ending in Death
0
0.5
1
1.5
2
2.5
Year
Death
rate
(p
er
1000 p
ati
en
ts)
RALES
What about the expected benefits?
Readmission for Heart Failure
0
50
100
150
200
250
Year
Ad
mis
sio
n r
ate
(p
er
10
00
p
ati
en
ts)
RALES
All-Cause Mortality
0
10
20
30
40
50
60
70
Year
Rat
e o
f dea
th fr
om
an
y ca
use
(per
100
0 p
atie
nts
) RALES
What Happened???
In the ‘Real World’….Different patients may get the drug
> 50% of population that would normally use the drugs in clinical practice often do not meet inclusion / exclusion criteria of RCTs (e.g. Gill et al, CJCP, 2004)
Patients may not be monitored as carefully
Patients may not take their drug as they should
Patients may take interacting drugs that they shouldn’t be on
Patients may not adhere to their drugs optimally
What’s the Problem?
Anton et alJ Clin Pharm Ther 2003; 28: 285-7
Anton et al. J Clin Pharm Ther 2003;28:285-7
Retrospective cohortSingle hospital110 patients Rx spironolactone + ACEI
Mean age 71 years, half with DM
OutcomesCessation of spironolactoneHyperkalemia
Findings
24% developed K+ (> 5.5 mEq/L) at 1 year
Many stopped spironolactone
J Clin Pharm Ther 2003;28:285-7
Bozkurt et alJ Am Coll Cardiol 2003;41:211-4
CriteriaCriteria Bozkurt et Bozkurt et alal
(n=104)(n=104)
RALESRALES
(n=822)(n=822)
NHYA (% patients)NHYA (% patients)
II
IIII
IIIIII
IVIV
UndocumentedUndocumented
4.5%4.5%
4.5%4.5%
15.3%15.3%
10.3%10.3%
65.4%65.4%
0%0%
0.5%0.5%
72%72%
27%27%
0%0%
% Patients with LVEF < % Patients with LVEF < 35%35%
54.8%54.8% 100%100%
% Patients with renal % Patients with renal insufficiency at insufficiency at baselinebaseline
30.7%30.7% ExcludedExcluded
Use of beta-blockersUse of beta-blockers 34.6%34.6% 11%11%
In a highly selected group of patients with severe heart failure who are largely free of other risk factors for hyperkalemia and who can be monitored closely, the addition of spironolactone to standard therapy (as defined in 1994) decreases hospitalization for heart failure and saves lives, with no significant risk of hyperkalemia.
What RALES said…
What we heard…• In patients with heart failure, spironolactone saves lives
Interpreting Evidence: Key Points
Interpretation of evidence depends on:
Availability of evidence
Ability to understand, assimilate, and interpret evidence and its limitations
Putting It All Together
What Drives Interpretation?
Study design / methodologyAppropriate design Inclusiveness / generalizability
Statistical analysis and reporting
Clinical relevance and ‘sense’
Where Does Evidence Fit Into the Decision-Making Process?
BELIEFS
BEHAVIOUR
Evidence Personal Values / Experience
Societal Values ‘Other’ Factors
Clinical Research:Writing a Scientific
Manuscript
What Do Journal Editors Look For?
Characteristic Implications
Clinical Relevance
Research question should be relevant:Ask other clinicians about question’s clinical importance
Originality Research question should ideally be unique:Perform a PubMed search
Clarity Research question should be simpleStudy design should be well explained with ONE primary endpointBefore beginning study – know which 4 or 5 figures and/or tables will be produced; make mock figures/tablesKnow your audience: simple language
Good Science Assemble a team: study lead, methodologist, statistician, clinician, at least one senior researcherMust have good design, sufficient sample size, and analysis plan
Brevity Keep the manuscript short: 2,000 – 3,000 wordsConsider different formats: brief reports, research letters
What will maximize the journal’s IMPACT FACTOR?
Fogg, BJ, et al, What makes Websites Credible? Stanford Persuasive Technology Lab
Author Photo Results
believable
trustworthy
competent
credible
unbiased
expert
COMPOSITE
casual photo
no photo
formal photo
Fogg, BJ, et al, What makes Websites Credible? Stanford Persuasive Technology Lab
The Anatomy of a Clinical Research Question
Is the question simple?The question must be easy to understand and the results must be easy to convey
Start with thinking about the 4 or 5 Tables or Figures that would result from your research
Is the question important?It should be important to others besides the researcher
Dependent on:Funding sourceIntended audience Expected impact
The Anatomy of a Clinical Research Question
Is it the question unique?Must have a solid background / contextExtensive literature search to assess novelty of the research
Is the question specific?ExposureOutcome: reflects only ONE well-defined primary endpoint
Nature of assessment (e.g. relationship vs. difference – need to state intervention)
Research ImpactHigh Impact
Great questions Great questionsGreat findings Poor findings
Low Impact
Poor questions
Great findings
No ImpactNo Impact
Poor questions Poor questions
Poor findingsPoor findings
Other ConsiderationsBefore beginning your study, know which journal(s) you would like to targetCarefully review their format requirements (usually on website)
Review sample studies from that journal
Focus on a good abstractMany journals screen articles for review based on the abstract
Review past article’s abstracts for format and wording style
Make cover letter brief but relevantProvide clinical context and rationale for the studyBriefly summarize study findingsBriefly summarize implications of study findings on clinical practice and/or public health
Try to keep it brief – maximum 1 page
Research and Communications: An Example
BackgroundA particular group of antibiotics called ‘fluoroquinolone’ antibiotics have become the most widely prescribed group of antibiotics About 22 million prescriptions dispensed annually in US for fluoroquinolone
antibiotics 16 individual fluoroquinolone antibiotics have been available on the
market; In 2005, gatifloxacin (Tequin) was the fluoroquinolone antibiotic of choice for US public health system
Some of these drugs have problems: serious adverse events have led to the withdrawal or restriction of several fluoroquinolones in recent years Temafloxacin (blood sugar and kidney problems) Grepafloxacin and sparfloxacin (heart problems) Trovafloxacin (liver problems)
In 2005, a student noticed major changes in blood glucose levels in patients receiving Tequin and approached researchers to investigate this observation
So We Conducted a Study (Park-Wyllie et al, NEJM, 2006)
PopulationExamined records of over 1.4 million elderly residents of Ontario age 66 years and older between 2002-2004
Analysis limited to individuals who were using one of several selected antibiotics
OutcomesHospital admission related to severe changes in blood glucose levels
What Were the Results?
Hospital admissions related to severe drops in blood glucose levelsGatifloxacin associated with 4x the risk of hospital admission compared to other antibiotics
Hospital admissions related to severe increases in blood glucose levelsGatifloxacin associated with almost 17x the risk of hospital admission compared to other antibiotics
The Basic Communications Plan
Raise the profile of the issue
Make the numbers tangible and meaningful
Target the right journal
Identify the appropriate audiences
Making the Numbers Tangible
If we consider the 1.4 million elderly residents of Ontario during the study timeframe:
There were nearly 17,000 courses of gatifloxacin treatment administered
For every 100 courses of gatifloxacin, we may expect 1 hospital admission for dysglycemia
On average, it was estimated that at least 1 elderly person in Ontario was hospitalized every week as a result of dysglycemia that was likely associated with gatifloxacin
ImpactPublished early online (March 1st, 2006) in the New England Journal of Medicine given its clinical relevance
Recognized nationally as a significant research contribution:Canadian Society of Clinical Pharmacology
Best Publication Award for 2006
Gatifloxacin withdrawn from market in May 2006
