STATISTICAL ANALYSIS, INTERPRETATION AND PRESENTATION ...Panum_ March2013).pdf · STATISTICAL...

STATISTICAL ANALYSIS, INTERPRETATION AND

PRESENTATION

(issues in medical statistics!)

1,2Robin Christensen, BSc, MSc, PhD

1Senior Biostatistician;

Head of Musculoskeletal Statistics Unit

The Parker Institute

2Associate Professor of Statistics in Medicine;

Institute of Sports Science and Clinical Biomechanics,

University of Southern Denmark

How likely is it, that…… (”Perceiving Probability”)

•  You will experience an ”Aeroplane Crash”?

•  You will win in the Danish ”Lotto”?

•  You will have a myocardial infarction?

•  You will have a fracture within the next year?

•  You will have Diarrhoea when holidaying in Greece?

•  You will fall asleep during this stat sessions? 2

Probability: P =

•  From a set of playing cards → I can predict a specific card! (?)

(P = 1/52 = 0.019)

•  We would say the null hypothesis (H0) is: ”of course you cannot (stupid idiot)”

•  Alternatively (HA): ”Wow – it really works (for you)”

•  Thus, if we reject the null hypothesis (H0) – we are likely to believe that ”this guy is efficacious”

•  However, more trials are needed to confirm this finding! 3

P < 0.05 (5%)

- Likely to be trustworthy? •  Removing all the black cards (same set of playing cards) →

”I can predict a specific card”! •  We would say the null hypothesis (H0) is: ”of course you cannot (stupid idiot)”

•  HA: ”Wow – He really did it” (having implications for public health) •  Happy to reject the null hypothesis (H0) – we believe you

(P < 0.05) •  More trials are needed to confirm this finding!

(P = 1/26 = 0.038)

4

WHICH COMPARISONS SHOULD BE MADE?

Hypothesis:

”the number of responders (e.g. ”Staying Alive”) will be higher on the new experimental drug compared to control”

i.e.

the ”chance” that a patient responds will be higher on the new experimental drug (pE) compared to control (pC)

The statistical hypothesis:

H0: pE = pC

If P<0.05 (two-sided) then we might assume

HA: pE ≠ pC

Sampling and probability Estimating the number of obese individuals in a sample!

Obs# BMI 1 18.2 2 20.1 3 22.1 4 24.0 5 27.6 6 28.4 7 30.1 8 32.8 9 38.0

10 42.0

Category Count %

Normal 4 40%

Overweight 2 20%

Obese:class I 2 20%

Obese:class II 1 10%

Obese:class III 1 10% 6

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Normal Overweight Obese, class I Obese, class II Obese, class III

Category

Perc

enta

ge in

sam

ple

Sampling and probability

Median = 28 kg/m2

4/10 are obese

(40%)

7

Sampling and probability •  We infer about a population based on a small subset of the

population (= the sample)

•  The sample should be representative of the population from which it is drawn AND for which the determination is being made

•  If the sample is not representative of the population, there will be bias in the statistical results - leading to misleading conclusions

•  We would expect 40% of individuals in the bigger population to be obese! (?)

•  NO – we would NOT!!! (the sample was on patients awaiting Total Knee Alloplasty) 8

Steering group:

Dr Doug Altman

Dr David Moher

Dr Kenneth F. Schulz

Dr John Hoey

http://www.equator-network.org

WHATEVER YOU DO – REMEMBER TO: Include the study design in the title

Explicit design in title #1:

I DON’T HAVE A CLUE HOW TO TREAT MY PATIENT:

A Randomised Controlled Trial


THE HEAVIER - THE SMARTER: A Cross-Sectional Study


MALE PHYSICIANS EARN MORE MONEY DURING THEIR CAREER THAN

FEMALE PHYSICIANS: A Cohort Study


WHY DO SOME OF MY PATIENTS DIE?: A Case-Control Study


DO I KNOW WHAT I AM DOING?: A Reliability Study


ASK FOR A SECOND OPINION: An Agreement Study


DO I KNOW WHICH PATIENT IS REALLY ILL?: A Diagnostic Test Accuracy Study


I HAVE READ ALL THESE TRIALS ON THE TOPIC - WHAT’S THE OVERALL EVIDENCE?:

A Meta-Analysis of Randomised Controlled Trials

Three aspects of clinical practice Diagnosis

Therapy

Monitoring: outcomes

Inadequate = wrong diagnosis

Unclear (e.g., no patient impact)

Adequate = important

Explicit evaluation of the importance of outcomes (e.g. OA)

•  It is essential to differentiate the critical outcomes from the important ones.

1 ---

3 --- 4 --- 5 --- 6 --- 7 --- 8 ---

2 ---

9 --- Importance of end points Critical for decision making

Important but not critical for decision making

Not important for decision making – of lower importance to patients

Mortality Disablement Need for TKA/THA Pain, Disability, HRQoL

X-Ray (JSW/JSN)

http://www.gradeworkinggroup.org/

ROM/MRI/US/CRP Biomarkers…..

Different types of outcome variables (NOCS)

•  Nominal data

•  Continuous data

•  Ordinal data

Responder / Non-Responder; Death / Alive

Body weight (kg); Muscle strength (Nm);

LDL Cholesterol (mmol/L); Diastolic BP (mmHg)

•  Survival data Visual Analogue Scale (0-100 mm VAS)

Likert scales (e.g., No pain, Some pain, Moderate pain, Severe pain)

Time to drop-out (still-in-study: yes/no)

Essential Outcome Measures: Trials

•  Nominal data

•  Continuous data

•  Ordinal data

Responder: yes / no

Weight change: Δkg

Health Related Quality of Life: SF-36 (PCS/MCS 0-100)

Change in KOOS-Pain: ΔScore (-100 to 100)

No pain, Some pain, Moderate pain, Severe pain No pain vs Pain

Statistical Analysis

•  Descriptive statistics

•  Statistical inference

-  Describing the entire sample from statistical estimates

-  Presenting ”sufficient” estimates capturing the data

-  Unbiased estimates from Unbiased samples

-  Point estimation

-  Establishing the precision (e.g., 95% CIs)

-  Hypothesis test(s)

Statistical models in epidemiology

•  Causal models: involves a randomisation technique

(excluding selection bias and confounding by indication)

Causal vs. Empirical models

•  Empirical models: estimation from observed data

(includes selection bias and confounders)

•  Case-Control studies

•  Cross-Sectional studies

•  Cohort (longitudinal) studies

Statistical models in epidemiology Case-Control studies

•  Compares exposures between Disease (+) vs Disease (-)

•  Cases (+) and Controls (-) are representative of a population of interest

•  Controls (-) should represent the population from which the Cases (+) arose

Statistical models in epidemiology Cross-Sectional studies

•  Assess all individuals at the same point in time

•  Prevalence of exposures, risk factors, or disease symptoms

•  Can indentify potential causal associations; e.g. correlations between variables

Statistical models in epidemiology Cohort (longitudinal) studies

•  Follow people over time

•  Obtain information about people and their exposures, let time pass, and then assess the occurrence or status of the outcome

•  Common: Make contrast between individuals who are exposed and not exposed

•  Prospective cohort studies are more reliable than retrospective cohort studies. . . . .

Illustrating Cross-Sectional studies

Time (years)

Outcome measure

µ (t = 0)

π (t = 0)

Illustrating a Cohort study

Time (years)

Outcome measure

µ (t = 0)

π (t = 0)

µ (t = after)

π (t = after)

Illustrating Person-Years

Time (years)

Number of Persons in Cohorts (no.)

Incidents/Person-Years

Time (years)

Number of Persons in Cohort (no.)

x xx

x

x

x xx

x

x x x

x

x xx

x

Reader’s guide to critical appraisal of cohort studies •  What comparison is being made? •  Does the comparison make clinical sense? •  What are the potential selection biases? •  Has there been a systematic effort to identify and measure potential confounders? •  Is there information on how the potential confounders are distributed between the comparison groups? •  What methods are used to assess differences in the distribution of potential confounders? •  Are the analytical strategies clearly described? •  Do different analytical strategies used yield consistent results? •  Are the results plausible?

What is bias?

•  A systematic error, or deviation from the truth, in results or inferences

http://www.cochrane-handbook.org/

•  Biases can operate in either direction: different biases can lead to underestimation or overestimation of the true intervention effect

•  Some are small (and trivial compared with the observed effect) and some are substantial (so that an apparent finding may be entirely due to bias)

•  More rigorous studies are more likely to yield results that are closer to the truth.

Bias?

-20 -10 0 10 20

Effect of Intervention

High-Risk of Bias Low-Risk of BiasCombined High-Risk of Bias Combined Low-Risk of Bias

Favors Intervention Favors Placebo

Bias?

-20 -10 0 10 20

Effect of Intervention

High-Risk of Bias Low-Risk of BiasCombined High-Risk of Bias Combined Low-Risk of Bias

Favors Intervention Favors Placebo

X2

X11 X1

X8

X7 X18

X4

X3 X16 X22

X12

X9 X10

X21

X20 X19

X17

X15 X5 X6

X25

X14

X23 X13

X24

Randomized Controlled Trial

X2

X11 X1

X8

X7 X18

X4

X3 X16 X22

X12

X9 X10

X21

X20 X19

X17

X15 X5 X6

X25

X14

X23 X13

X24

EXPERIMENTAL

CONTROL


X2

X11 X1

X8

X7 X18

X4

X3 X16 X22

X12

X9 X10

X21

X20 X19

X17

X15 X5 X6

X25

X14

X23 X13

X24

EXPERIMENTAL

CONTROL

NO!

– THIS IS NOT AN RCT!


•  PROTOCOL (i.e. Objective & primary outcome) •  www.CLINICALTRIALS.gov

•  INFORMED CONSENT

•  (BASELINE MEASUREMENTs)


Evidence Synthesis

’The PICO framework’

•  Patients

•  Intervention(s)

•  Comparison(s)

•  Outcome(s)

Osteoarthritis (i.e. ACR crit.)

Exercise (add-on: Concomitant med.)

Nothing (add-on: Concomitant med.)

Patient important outcome?

Clinician important outcome?

X1

X8

X7 X18

X4

X3 X16 X22

X12

X9 X10

X21

X20 X19

X17

X15 X5 X6

X25 X24

ELIGIBLE PATIENTS Randomized Controlled Trial

X1

X4

X6 X7

X11

X12 X13 X14

X2

X5 X3

X8

X10 X9

X15

X16 X17 X18

X19 X20

ELIGIBLE PATIENTS - Included Randomized Controlled Trial

X1

X4

X6 X7

X11

X12 X13 X14

X2

X5 X3

X8

X10 X9

X15

X16 X17 X18

X19 X20

ELIGIBLE PATIENTS - Randomized Randomized Controlled Trial

Randomized Controlled Trial (RCT)


Net benefit = 0.75 – 0.25 Net Benefit = 0.50 point

(ie, 50%point benefit compared to placebo; NNT=2)


Statistical inference –

Unbiased samples….

CO

NSO

RT 2010 checklist of inform

ation to include w

hen reporting a randomised trial

INTRODUCTION

METHODS

RESULTS

DISCUSSION

Risk of bias: RCTs http://www.cochrane-handbook.org/

(1) Sequence generation

(2) Allocation concealment

(3) Blinding of participants, personnel and outcome assessors

(4) Incomplete outcome data

(5) Selective outcome reporting

(6) Other sources of bias Christensen and Bliddal Arthritis Research & Therapy 2010, 12:105

Risk of bias http://www.cochrane-handbook.org/

Sequence generation:

“Describe the method used to generate the allocation sequence in sufficient detail to allow an assessment of whether it should produce comparable groups”

Was the allocation sequence adequately generated?

Adequate

Unclear

Inadequate

INCLUDED PATIENTS - Randomized

We generated the two comparison groups using simple randomization,

with an equal allocation ratio (1:1), by referring to a table of random numbers

Random Code: {0, 1, 1, 1, 1,

0, 1, 0, 0, 0,

1, 0, 0, 1, 1,

0, 1, 0, 0, 1}



Allocation concealment:

“Describe the method used to conceal the allocation sequence in sufficient detail to determine whether intervention allocations could have been foreseen in advance of, or during, enrolment.”

Was allocation adequately concealed?

Adequate

Unclear

Inadequate

Allocation sequence concealment

•  Central randomization

•  Sequentially numbered drug containers •  Sequentially numbered, opaque, sealed envelopes


Til afdelingens sekretærer!

Vedr. Projekt KF-13-11-08

Hvis du ser en mulig deltager i konsultationen - der vil udfylde ’informed consent’ – da allokér patienten til gruppen som angivet t.h. for kaffemaskinen. Tak!

MVH

Dr. Kokren

Randomized Controlled Trial:

The Parker Institute:

Musculoskeletal Statistics Unit

Til afdelingens sekretærer!

Vedr. Projekt KF-13-11-08

Hvis du ser en mulig deltager i konsultationen - der vil udfylde ’informed consent’ – da allokér patienten til gruppen som angivet t.h. for kaffemaskinen. Tak!

MVH

Dr. Kokren Concealed allocation?


Sequence generation:

Was the allocation sequence adequately generated?

Allocation concealment: Was allocation adequately concealed?

Adequate

Unclear

Inadequate

Adequate

Unclear

Inadequate

[email protected]

Blinding of participants, personnel and outcome assessors

When considering the risk of bias from lack of blinding it is important to consider specifically:

1.  who was and was not blinded;

2.  risk of bias in actual outcomes due to lack of blinding during the study (e.g. due to co-intervention or differential behaviour);

3.  risk of bias in outcome assessments (considering how subjective or objective an outcome is)


Blinding of participants, personnel and outcome assessors:

“Describe all measures used, if any, to blind study participants and personnel from knowledge of which intervention a participant received. Provide any information relating to whether the intended blinding was effective.”

Was knowledge of the allocated intervention adequately prevented during the study?

Adequate

Unclear

Inadequate

Handling: Incomplete outcome data

•  All envelopes opened ~ Intention-to-treat population

•  All randomized included in the analyses

•  Use an Intention-to-Treat analysis (Non-responder analysis most appropriate on average)


Statistical inference - applied

100 100

92 80

Handling: Incomplete outcome data

n[E] n[C] N[E] N[C] p[E] p[C]ITT 25 25 100 100 25.0% 25.0%PP #1 25 25 67 90 37.3% 27.8%PP#2 25 25 90 67 27.8% 37.3%Modified ITT 25 25 95 95 26.3% 26.3%

i.e., the results may vary according to something not being the treatment


Incomplete outcome data

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

0.0% 10.0% 20.0% 30.0% 40.0% 50.0%

Attrition (drop-out) Rate [%]

Res

pons

e R

ate

(%)

True effect!

Biased estimate!

Bias

[email protected]

LUNDEX ( L ) simulation scene

0.000

0.100

0.200

0.300

0.400

0.500

0.600

0.700

0.800

0.900

0 12 24 36 48

Duration (Months)

Prop

ortio

n re

spon

ding

After 6 months~ 50% responds and sustains (ITT)

N = 1000

n = 950 n = 900 n = 800

n = 700 n = 600

LUNDEX

[email protected]


Incomplete outcome data:

“Describe the completeness of outcome data for each main outcome, including attrition and exclusions from the analysis. State whether attrition and exclusions were reported, the numbers in each intervention group (compared with total randomized participants), reasons for attrition/exclusions where reported, and any re-inclusions in analyses performed by the review authors.”

Were incomplete outcome data adequately addressed?

Adequate

Unclear

Inadequate

Selective outcome reporting •  Global pain score for index joint

•  Pain on walking for index joint

•  Western Ontario McMaster Universities (WOMAC) Pain subscale

•  Lequesne index

•  Pain in index joint during activities other than walking

Selective outcome reporting •  Global pain score for index joint

•  Pain on walking for index joint

•  Western Ontario McMaster Universities (WOMAC) Pain subscale

•  Lequesne index

•  Pain in index joint during activities other than walking

P = 0.04

P = 0.01

P = 0.09

P = 0.17

P = 0.06


Selective outcome reporting:

“State how the possibility of selective outcome reporting was examined by the review authors, and what was found.”

Are reports of the study free of suggestion of selective outcome reporting?

Adequate

Unclear

Inadequate

http://www.equator-network.org

http://www.lean.org/

Principles of Lean Thinking (Five steps guiding the implementation of lean)

Specify value from the standpoint of the end customer by product family

Identify all the steps in the value stream for each product family, eliminating whenever possible those steps that do not create value

Make the value-creating steps occur in tight sequence so the product will flow smoothly toward the customer

MAPPING THE VALUE STREAM: Reporting Health Research TITLE (Identify value)

INTRODUCTION (Setting the scene) •  Background (What we know!)

•  Rationale (Why this is important!)

•  Objective (Specific aim!) METHODS (What we anticipate)

•  Participants (incl/excl criteria)

•  Interventions (comparison(s))

•  Outcomes (primary/secondary)

•  Sample size (how many)

•  Statistical methods (hypotheses)

PROTOCOL (Identify ”the customer”)

RESULTS !!?

•  Participant flow

•  Recruitment

•  Baseline data (ITT population)

•  Outcomes and estimation

•  Ancillary analyses (post hoc?)

DISCUSSION (What happened?)

•  Interpretation

•  Generalizability

•  Overall evidence

•  Implications for practice

•  Implications for research

SUBMIT? (Customer/Consumer)

TITLE

INTRODUCTION

METHODS

PROTOCOL

RESULTS

DISCUSSION

SUBMIT

MAPPING THE VALUE STREAM

PUBLISH

(Register)

Population

Stikprøve (N) Stikprøve (N)

Diagram over det teoretiske udfaldsrum for et statistisk velfunderet design (modificeret efter Lund & Røgind, 2004)

Overblik over Stikprøven (N)

Her kan man med rette lave RCT

Intervention (I) Kontrol (K)

Hypotese test:

I = K

Estimation (konfidensinterval)

(I – K ) ± 95% KI

π µ

Date post:	16-Mar-2019
Category:	Documents
Upload:	truongquynh
View:	227 times
Download:	0 times

STATISTICAL ANALYSIS, INTERPRETATION AND PRESENTATION ...Panum_ March2013).pdf · STATISTICAL...

Documents