Date post: | 29-May-2015 |
Category: |
Documents |
Upload: | jonas-ranstam |
View: | 178 times |
Download: | 2 times |
The unity of all science consists alone in the method, not its material.
Pearson K. The grammar of science. London, Black, 1892.
Statistics is the study of uncertainty.
Savage LJ. The foundations of statistics. New York, Wiley, 1954.
The aim of statistics reviewing
Accurate and transparent description of the uncertainty in presented findings.
“Statisticians are experts in handling uncertainty”.
Lindley DV. The philosophy of statistics. The Statistician 2000;49:293-337.
Experi-mental
Study Study design design
Obser-vational
Few ethics concernsBias adjustmentsExternal validityLong follow up
Few sample size restrictions
Ethics concernsRandomization
Controlled conditionsInternal validityShort follow up
Sample size restrictions
Medical research methodology
Experi-mental
Study Study design design
Obser-vational
Internal validity by design (blocking of known risk factors and randomization of
other)
Potential for confounding: none
Internal validity by statistical analysis (confounding adjustment for known and
measured risk factors)
Potential for confounding: massive
Statistical aspects – internal validity
Confounder (or case-mix) adjustmentHow much of the variation in endpoints can be explained by known factors, and how much has unknown causes?
Variation with unknown origin
95%-99% Arthroplasty revision
85%-95% EQ-5D, SF36
70%-80% Coronary heart disease
Risk factors, confounding, and the illusion of statistical control'...it is essential to remember that “statistical control” is nothing more than a highly fallible process filled with judgment calls that often go unnoticed in practice.'
Christenfeld NJS, Sloan RP, Carrol D, Greenland S.Psychosomatic Medicine 2004;66:868–875
Simple modelMultiple model
(or multivariable, but not multivariate)
Linear regression analysis
Stepwise regression
Statistics
We calculated odds ratios by logistic regression analysis, to estimate the relationship between failure of the osteotomy and possible preoperative risk factors. We performed multivariate, stepwise (backward) logistic regression and entered variables with a p-value of ≤ 0.05 into the model.
Unified theory of bias Bias can be reduced to or explained by 3 structures
1. Reverse causation
Outcome precedes exposure measurement or outcome can have effect on exposure. Measurement error or Information bias.
2. Common cause
Confounding by association, confounding by indication.
3. Conditioning on common effects
Collider, selection bias, time varying confounding.
Covariate selectionAdequate Background Knowledge
Confounder identification must be based on understanding of the causal structure linking the variables being studied (treatment and disease).
Condition on the minimal set of variables necessary to remove confounding.
Inadequate Background Knowledge
Remove known instrumental variables, colliders, intermediates (variables with post treatment measurement.
ConfoundingUnder-adjustment
occurs when a confounder is not adjusted for.
Over-adjustment
can occur from adjusting instrumental variables, intermediate variables, colliders, variables caused by outcome.
Confounder
Common cause, i.e., confounder
Confounder L distort the effect of treatment A on disease Y
Always adjust for confounders, unless small data set and confounder has strong association with treatment and week association with outcome
Confounder example
A = treatment1: statin alone0: niacin alone
L = Baseline Cholesterol1: LDL ≥ 160 mg/dL0: LDL < 160 mg/dL
Y = Myocardial infarction1: Yes0: No
Intermediate variableAdjusting for intermediate variable I in a fixed covariate model will remove the effect of treatment A on disease/outcome Y
In a fixed covariate model we do not want to include variables influenced by A or Y
Intermediate exampleA = treatment1: statin alone0: niacin alone
I = Post-treatment Cholesterol1: LDL ≥ 160 mg/dL0: LDL < 160 mg/dL
Y = Myocardial infarction1: Yes0: No
ColliderAdjusting for a collider can produce bias
Conditioning on common effect F without adjustment of U1 or U2 will induce an association between U1 and U2, which will confound the association between A and Y
Collider example
Variables associated with treatment or disease onlyInclusion of variables associated with treatment only can cause bias and imprecision
Variables associated with disease but not treatment (risk factors) can be included in models. They are expected to decrease variance of treatment effect without increasing bias
Including variables associated with disease reduces the chance of missing important confounders
Reality is complicated
http://www.dagitty.net
http://www.dagitty.net
Any claim coming from an observational study is most likely to be wrong
12 randomised trials have tested 52 observational claims (about the effects of vitamine B6, B12, C, D, E, beta carotene, hormone replace- ment therapy, folic acid and selenium).
“They all confirmed no claims in the direction of the observational claim. We repeat that figure: 0 out of 52. To put it in another way, 100% of the observational claims failed to replicate. In fact, five claims (9.6%) are statistically significant in the opposite direction to the observational claim.”
Young S, Karr A. Deming, data and observational studies. Significance, September 2011.
Aetiology Study scope Study scope Treatment
Pre-specified hypotheses
Confirmation
Legislation, regulatory guidelines
Uncertainty intolerance
Hypothesis generation
Exploration
Academic analysis freedom
Uncertainty tolerance
Medical research methodology
Aetiology Study scope Study scope Treatment
Randomized clinical trials
Patient registerstudies
Epidemiologicalstudies
Laboratory experiments
Medical research methodology
Experi-mental
Study Study design design
Obser-vational
Aetiology Study scope Study scope Treatment
Protected type-1 error rate for specified endpoints
Sample size based on the type-2 error rate
Specified type-1 and -2 error uncertainty
(confidence intervals)
No multiplicity consideration for safety endpoints
Multiplicity issuesnot addressed
Sample size not based on type-2 error rate
Bonferroni correction within endpoints
Few type-2 error considerations
Statistical aspects - precision
Experi-mental
Study Study design design
Obser-vational
Aetiology Study scope Study scope Treatment
Experi-mental
Study Study design design
Obser-vational
Drug development
Phase 1Discovery(Phase 0)
Phase 2Phase 3
Phase 4
PMS(Phase 5)
Aetiology Study orientation Study orientation Treatment
Device development
Randomizedperformance
trials
Safetyfollow-up
in registries
Biomechanicsin vitro, etc.
Experi-mental
Study Study design design
Obser-vational
It is impossible to do clinical research so badly that it cannot be published
“There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.”
Drummond Rennie 1986 (editor of NEJM and JAMA)
Arthroplasty registry analyses
Crucial issues
- Fulfillment of methodological assumptions (Gaussian distr, homogeneity of variance, proportionality, linearity, etc.)- Confounding adjustment (risk factors, causality, linearity, etc.) - Clinical significance and estimation uncertainty (95%CI).
Should be avoided
- P-value culture- Bonferroni correction- Post-hoc power- Predictions
Thank you for your attention!
Indicators for statistical review- Randomized trials
- Patient registry (safety) studies
- Analyses of knees, hips, elbows... (bilateral observations)
- Pseudo-replicates (esp. in laboratory experiments)
- “No difference” manuscripts
- Stepwise regression
- ???