Don’t be the fool on the hill!The role of a biostatistician in the medical research processGeorg Heinze
Medical University of Vienna
Center for Medical Statistics, Informatics, and Intelligent Systems
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
1
The fool on the hill (P. McCartney 1968)
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
2
Lyrics Analogy
Day after dayAlone on a hillThe man with the foolish grinIs keeping perfectly stillBut nobody wants to know himThey can see that he's just a foolAnd he never gives an answer
The statistician working in the academic ivory tower
…doing strange things…
Researchers do not want to work with the statistican
He does not provide useful answers
But the fool on the hillSees the sun going downAnd the eyes in his headSee the world spinning round
Statistician knows what would be right
The fool on the hill (P. McCartney 1968)
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
3
Lyrics Analogy
Well on the wayHead in a cloudThe man of a thousand voicesTalking perfectly loud
But nobody ever hears himOr the sound he appears to makeAnd he never seems to notice
It needs ‚speaking in the language of the medicalresearchers‘
And nobody seems to like himThey can tell what he wants to doAnd he never shows his feelings
Empathy for the medical research project!
He never listens to themHe knows that they're the foolsThey don't like himBut the fool on the hill…
What is the medical research question?
Don‘t be the fool on the hill!
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
4
• The fool on the hill does not provide consulting services
• … loses connection to medical science
• … has no empathy for the medical research project
• Though he has a good knowledge on what would be right
• He does not take the effort to translate his knowledge to the medical researcher
• He does some research, but this is mostly ignored by the medical community
Personal statement
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
5
• Research:
• Clinical collaborative papers
• Methodological papers
• Interdisciplinary research projects
• Methodological research projects
• Teaching activities
• Advice to undergraduate and graduate students
• Practical ‚Statistical Consulting‘ for students of statistics
• Regular advising activities
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
6
A roadmap: the role of a biostatistician
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
7
• 1. Creative and technical aspects
• 2. Personal aspects – about our and their expectations
• 3. Infrastructure: possible models
• 4. Experience reports
• 5. Final remarks
On the role of the biostatistician
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
8
• The biostatistician‘s input to medical research needs some skills:
• A creative aspect:
• A technical aspect:
Experience with data
Common sense
Mathe-matics
Communication FormalizationTranslation
Examples of a biostatistician‘s input: design phase
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
9
• Creative aspect:
• The ideal clinical endpoint is often binary, but resources do not allow for a large study
• Finding a suitable continuous endpoint
• Is a control group always needed?
• Technical aspect:
• Translates the meaning of the expected effect size as ‚clinically relevant difference‘
• Estimates SD from literature data or from logical reasoning
• Performs sample size calculation
• Describes sample size calculation concisely for study protocol
Examples of a biostatistician‘s input: data analysis
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
10
• Creative aspect:
• Knows about properties of methods (repeated-sampling properties)
• Selects a method with good properties
• Tries to avoid common mistakes(dichotomania, no proper validation, no adjustments for multiple testing, …)
• Technical aspect:
• Is able to use computer code
• Can perform ‚looped‘ analyses (e.g. bootstrap, CV)
• Can implement complex analyses
• Sets up R project with markdown for reproducibility
Examples of a biostatistician‘s input: inference
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
11
• Creative aspect:
• Data-driven decisions can severely impact validity
• E.g. reporting only ‚most significant‘ outcome, variable selection, missing data
handling, …
• Has a feeling of which approach is optimal (without checking with data)
(uses background knowledge efficiently)
• Technical aspect:
• Accurate description of assumption, limitations, data-driven decisions etc.
• Methods, results, tables, figures are prepared such that misinterpretation is avoided
Examples of a biostatistician‘s input: discussion
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
12
• Creative aspect:
• Recognizes limitations, dependency on assumptions etc.
• Can make conclusions on generalizability of the results
• Technical aspect:
• Can perform sensitivity analysis to check robustness
• Can perform simulation studies to evaluate the dependency of main results on
assumptions
• ‚Translates‘ limitations etc. to medical researcher
I am a medical researcher. I need …
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
13
• Support in study design, sample size justification
• Support in writing statistical methods parts for my study protocol
• Support in revising my study protocol after IRB evaluation
• Support in setting up randomization
• Support in writing statistical analysis plans
• Support in data analysis
• Support in writing my manuscript (methods&results)
I am a biostatistician. I want …
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
14
• to do research on study design and data analysis
• to derive new mathematical procedures
• understand the asymptotic and small-sample properties of my method
• some data as example to apply my theory
• a tenure track position in academia
• some funding for PhDs
• to learn about the relevance of methodology from real studies
A conflict of interest
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
15
Medical research
Service
♥ Methodologicalresearch! ♥
Service Methods research
• ‚Fame‘ among statisticians
• My model is used by many others
• Mathematical research, a pure pleasure
• Defining my own real world
• Produces credits for tenure-tracks
• Tedious, often with no success
• Difficult to raise funding
• ‚Fame‘ among medical researchers
• Often with quick success
• Hop on projects
• Coping with the real world
• Reproducing others‘ models
• Medical research, I don‘t understand it anyway
• Does not produce credits for tenure-tracks
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
16
Possible models I
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
17
• Biostatistician is service provider
• No academic career possible, only ‚staff scientist‘
• Still, this can be a safe and attractive position, if some conditions are given:
Medical research
Service
Expectations: the ideal case
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
18
• What a consulting statistician expects
• A relevant research question
• Finding the ‚best‘ methodology: a plausible model, meaningful assumptions,
estimation with high accuracy
• Focusing on a few, not many projects
• A clear research plan and interpretation of results according to it
• A clean data base to work on
• A junior statistical programmer doing the hard work
Expectations: the no-gos
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
19
• What a consulting statistician doesn‘t expect
• Searching a data base for a publishable research question
• Having to find methodology
that gives the ‚most significant‘ result no matter which assumptions
• Having to use outdated methodology
because ‚the reviewers won‘t understand new stuff‘
• Stupid cross-tabulating and 100‘s of t-tests
• Fine-formatting graphics in 1200 dpi
• Solving all kinds of data base and computer problems
Possible models II
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
20
• Biostatistician only performing methods research
• If successful, biostatistician will pursue academic career
• Difficult to realize in a medical school
Medical research
Methodologicalresearch
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
21
• Relevance of position will be questioned
• Risk of becoming a ‚fool on the hill‘
• Or better to contribute to a properly conducted medical study?
Medical research
What‘s he doing?
Hooray! I finallypublished the
10th iteration ofthe extendedversion of …
After some years …
…or should I better collaborate
with him?
Possible models III
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
22
• Joining forces for both medical and methodological research projects
Medical research
Methodologicalresearch
Possible models III
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
23
• Joining forces for both medical and methodological research projects
Medical research
Methodologicalresearch
stimulate
enhance
Experience report 1: partnership in an interdisciplinary research project
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
24
• ‚Syskid‘ was a EU-funded project involving 25 partners receiving 16M€ funding
• I was involved in setting up the consortium and the steering group
• I led one of two statistical partner groups (~400k€)
• It was a huge effort
• Although the goal of the project was biomarker discovery for kidney disease,
we could publish 17 papers with a statistician as 1st author
• A temporary position was turned into a tenure track, finally permanent position
• Several follow-up projects: e.g. PROVALID, BeatDKD, CaReSyAn
Rainer Oberbauer, MD
Experience report 2:Prediction models
CeMSIIS - Section for Clinical Biometrics cemsiis.meduniwien.ac.at/kb
Georg Heinze
25
• What I have predicted in my career:
• Probability of preterm birth
• Survival after:
• Cervical cancer diagnosis
• Kidney donation
• Kidney transplantation
• Ovarian cancer diagnosis
• Cardiovascular events
• Progression of kidney disaeas
• Monitoring of kidney function in diabetics
• Adenomae in coloscopies
• …
• Recurrence of deep vein thrombosis
Goscinny & Uderzo (1975): Asterix and the soothsayer, Orion paperback
Experience report 2:Prediction of recurrence of venous thromboembolism
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
26
• Goal: identify patients at low risk
for recurrence, as they might
unneccessarily suffer from
the side effects of
anticoagulation
• A large prospective cohort was
available
Dr. Sabine Eichinger
The Viennaprediction model
CeMSIIS - Section for Clinical Biometrics cemsiis.meduniwien.ac.at/kb
Georg Heinze
27
Points0 10 20 30 40 50 60 70 80 90 100
Sexfemale
male
Localizationdistal pulmonary embolism
proximal
DDimer100 150 200 250 400 500 750 1000 1500 2000
Total Points0 50 100 150 200 250 300 350
12 months cumulative recurrence rate0.02 0.04 0.06 0.08 0.1 0.12 0.15
60 months cumulative recurrence rate0.1 0.2 0.3 0.4 0.5
Three variablesselected from ~8
Vienna Prediction Model
CeMSIIS - Section for Clinical Biometrics cemsiis.meduniwien.ac.at/kb
Georg Heinze
28
2008 2010 2012 2014 2016 2017
First trials with regression trees and simple Cox models.Model development: only three variables selected.
Publication in Circulation. Editor‘s Pick.
Vienna Prediction Model
CeMSIIS - Section for Clinical Biometrics cemsiis.meduniwien.ac.at/kb
Georg Heinze
29
CeMSIIS - Section for Clinical Biometrics cemsiis.meduniwien.ac.at/kb
Georg Heinze
30
Vienna Prediction Model
CeMSIIS - Section for Clinical Biometrics cemsiis.meduniwien.ac.at/kb
Georg Heinze
31
2008 2010 2012 2014 2016 2017
First trials with regression trees and simple Cox models.Model development: only three variables selected.
Publication in Circulation. Editor‘s Pick.Web implementation.Press echo.
Production of an App.First external validation studies.
Alternative models published.Started VALID validation study.Update: Dynamic VPM (JAHA).
Systematic Review.NL-Validation study.
CeMSIIS - Section for Clinical Biometrics cemsiis.meduniwien.ac.at/kb
Georg Heinze
32
Validating the clinical benefit
CeMSIIS - Section for Clinical Biometrics cemsiis.meduniwien.ac.at/kb
Georg Heinze
33
First unprovoked venous thromboembolism
OAC for >3 and <6months
Stop of OAC Patient information/informed consent
3 weeks after stop of OAC D-Dimer testing and calculation of recurrence risk according to VPM
VPM > 180 points VPM < 180 points
Treatment accordingto current guidelinesleft at the discretion
of the treatingphysician
No furtheranticoagulant
treatment
Prospective follow-upClinic visits after 3, 12, and 24 months
Figure 3: Flowchart of the study plan
Recurrence rate at 12 months <5% (??)
Lessons from Vienna Prediction Model project
CeMSIIS - Section for Clinical Biometrics cemsiis.meduniwien.ac.at/kb
Georg Heinze
34
• ‚Stimulating Environment‘: partnership with mutual respect
• Publications: Quality > Quantity
• „Simple“ model better than „complex“ model:
• Clinical application
• Stability vs. flexibility
• Further collaboration, funding a statistical PhD
• Model was discussed among the experts (Moons, Harrell)
• DVPM stimulated further research on shrinkage factors (Dunkler JStatSoft 2016)
• International cooperation, R-Software, …
Uexküll‘s notion of ‚Umwelt‘
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
35
• How do you train a guide dog (for the blind)?
• Umwelt = ‚Self-centered world‘
Jakob von Uexküll(1864-1944)
Recipe for successful partnership
CeMSIIS - Section for Clinical Biometrics
Georg Heinze
36
• Statisticians recognize the relevance of the research project
• Medical researchers recognize the added value that statisticians may give to their
project, but also their expectations
• Both extend their ‚Umwelt‘ to include also their partner