The Platinum Journal
How to Write a Manuscript and Get It Published in European Urology
The Importance of Statistical Design and Analysis
Richard Sylvester, ScDEORTC Headquarters
Brussels, Belgium
The Platinum Journal
Interpreting Published Results
• Urologists need precise information to be able to judge whether the conclusions of a publication are relevant to their practice.
• The confidence that a reader has in the conclusions is influenced by the way results are reported.
The Platinum Journal
Report Results in Accordance With Published Guidelines
Randomized Clinical Trials (RCTs)
• CONSORT is an evidence-based, minimum set of recommendations for reporting RCTs
– CONSORT Statement: 2010• 25 item checklist focuses on reporting how
the trial was designed, analyzed, and interpreted. http://www.consort-statement.org/
The Platinum Journal
Statistical Design:– item 3: trial design– item 6: definition of outcomes– item 7: sample size – items 8 – 11: randomization and blinding
CONSORT Statement
The Platinum Journal
Statistical Analysis:– item 12: statistical methods– item 14: recruitment– item 15: baseline data– item 16: numbers analyzed– item 17: outcomes and estimation– item 18: ancillary analyses– item 20: limitations
CONSORT Statement
The Platinum Journal
Statistical Design (1)
• Objectives and Hypotheses– Type of study (epidemiological, clinical trial,…)– Trial design
• Case control• Non randomized phase 2 study• Randomized phase 3 trial…
– Objectives• Superiority, Non Inferiority, or…..?
The Platinum Journal
Statistical Design (2)
• Outcome Measures (Endpoints)– Provide a clear definition of endpoints
• Primary endpoint• Secondary endpoint(s)• How and when were they measured?
– Are the endpoints appropriate for the study?• Short-term (surrogate) versus long-term
endpoints
The Platinum Journal
Statistical Design (3)
• Sample Size– Based on primary endpoint– Provide assumptions
• Event rate in the control group• Size of difference to be detected• Type 1 error rate (alpha) and power
– Interim analyses• Duration of follow up
– Specify number of patients and events required
The Platinum Journal
Statistical Design (4)• Randomization
– Method used– Allocation concealment (central randomization?)– Blinding
• Possible biases related to the randomization procedure can invalidate the study results
Note: a poorly designed study might not allow any conclusions to be drawn!!
The Platinum Journal
Statistical Methods (1)• Describe the statistical methodology used to
estimate and compare treatment groups for the endpoints– Response rate– Time to event: progression, death– Competing risks
• Describe any multivariate (adjustment) techniques that will be used
The Platinum Journal
Statistical Methods (2)
• Define the primary analysis population (intent to treat: all randomized)
• Indicate if– interim analyses– early stopping rules– subgroup analyses– multiple comparisons were foreseen and carried out
The Platinum Journal
Statistical Analysis (1)
Provide by treatment group• Number of patients
– randomized– included in the efficacy and safety analyses:
• All randomized (ITT: intent to treat)• All eligible (exams prior to randomization)• Per protocol (may be biased!)
The Platinum Journal
Statistical Analysis (2)
Provide by treatment group• Distribution of baseline patient characteristics
– Number and percent of patientsAre there any imbalances in important prognostic factors?
Note: do not compare patient characteristics using p values
The Platinum Journal
Statistical Analysis (3)
• Outcomes– Duration of follow up (median, maximum)– Summary of results for each treatment group
• Number and percent of patients with the event• Kaplan-Meier time to event curves
– Treatment comparison• Estimate of size of treatment effect or difference
– Scale: absolute or relative difference (OR, HR)?• Confidence interval for difference• Two sided p values (not simply non significant (NS))
The Platinum Journal
Statistical Analysis (4)• Ancillary Analyses
– Adjustment for prognostic factors• Pre-specified in protocol• Because of imbalances in prognostic factors
– Subgroup analyses• A priori specified in protocol• Exploratory: data dredging and fishing expeditions
– Multiple comparisons and interim analyses• Effect on error levels
The Platinum Journal
Statistical Analysis (5)
• Adverse Events (by treatment group)– Include all patients who started protocol
treatment– Toxicity grading scale used– Amount of treatment received– Stop treatment due to toxicity– Toxic deaths– Medical significance is more important than
statistical significance!
The Platinum Journal
Discussion• Interpretation
– Evidence for hypotheses studied? • Absence of evidence ≠ evidence of absence
– Non significant does not imply equivalent!– Statistical significance does not imply medical
significance!– Limitations and sources of potential bias– Dangers associated with
• Small sample sizes• Multiplicity of analyses
– Generalizability of results
The Platinum Journal
Summary of Statistical Issues• Provide details concerning
– Trial design: endpoints, sample size and method of randomization
– Statistical analysis methods used– Analysis population
• Report outcomes in accordance with CONSORT • Check consistency of results in Abstract, Results,
Discussion, Tables and Figures• Are the conclusions supported by the data?
The Platinum Journal
Take Home Message• The correct statistical design and analysis of a
clinical research project is a prerequisite for ensuring the statistical validity of the conclusions.
• A statistician should be involved in the project– upfront at the design stage– do the statistical analysis – take part in the writing of the manuscript
The Platinum Journal
Suggested Reading: Reporting Results• European Urology:
http://www.europeanurology.com/about-the-journal/for-authors
• International Committee of Medical Journal Editors: http://www.icmje.org/
• Equator Network: http://www.equator-network.org/; http://www.equator-network.org/about-equator/equator-publications0/equator-network-publications-2010/
• I. Simera, D. Moher, J. Hoey et al. A catalogue of reporting guidelines for health research. Eur J Clin Invest, 40, 35–53, 2010.
The Platinum Journal
Suggested Reading: Reporting Results• Randomized Clinical Trials: CONSORT Statement.
http://www.consort-statement.org/consort-statement/• Non Randomized Studies: TREND Statement.
http://www.cdc.gov/trendstatement/• Meta-analyses: PRISMA Statement.
http://www.prisma-statement.org/• Diagnostic Accuracy Studies: STARD Statement.
http://www.stard-statement.org/• Epidemiology Studies: STROBE Statement.
http://www.strobe-statement.org/• Molecular Studies: see European Urology guidelines
The Platinum Journal
Suggested Reading: Reporting Results• Tumor Marker Prognostic Studies: REMARK: L.M.
McShane, D.G. Altman, W. Sauerbrei et al. Reporting recommendations for tumor marker prognostic studies. Journal of Clinical Oncology, 23, No. 36, 9067 – 72, December 20, 2005.
• C.D. Scales, R.D. Norris, S.A. Keitz et al. A critical assessment of the quality of reporting of randomized, controlled trials in the urology literature. J Urol, 177, 1090 – 95, March 2007.
• V.M. Montori, R. Jaeschke, H.J. Schunemann et al. User’s guide to detecting misleading claims in clinical research reports. BMJ, 329, 1093 – 96, November 6, 2004.
The Platinum Journal
Suggested Reading: Statistical Guidelines• British Medical Journal: Statistical Notes
http://www-users.york.ac.uk/~mb55/pubs/pbstnote.htm
• Greenhalgh T. How to read a paper: Statistics for the non-statistician. I: Different types of data need different statistical tests. BMJ, 315, 364-366,1997http://www.bmj.com/content/315/7104/364.full
• Greenhalgh T. How to read a paper: Statistics for the non-statistician. II: Significant relations and their pitfalls. BMJ, 315, 422-425,1997http://www.bmj.com/content/315/7105/422.full
The Platinum Journal
Suggested Reading: Statistical Guidelines• Journal of Clinical Oncology.
http://jco.ascopubs.org/site/ifc/stats.xhtml• Halabi S. Statistical considerations for the design and
analysis of phase III clinical trials in prostate cancer. Urol Oncol: Sem and Orig Investig, 26, 300-307, 2008