Drug Induced Injury

The contribution of statistics to establishing ‘cause’

Statistics in Law

Statistics is about ‘description’, ‘estimation’ and ‘probability / likelihood’.

Statistics play a part: ‘Sally Clark’ case Probability of two cot deaths 1 in 73

million Epidemiological evidence Sally Clark case, Gregg v Scott,

McTear v Imperial Tobacco Prosecutor’s fallacy

Probability of observing evidence given innocence Probability of innocence given observed evidence

e.g. fire alarm if major fire ‘balance of probabilities’

Motivation

To discuss issues relating to the use of statistics in law, with particular attention to the law in relation to medicine. Establishing causality

Systematic experimentation Interpreting ‘statistics’ Population versus Individual risk Examples of questionable ‘statistics’

Vioxx (rofecoxib) Gregg v Scott Oral contraceptives case

Causality and Risk

“I would rather discover one causal law than be King of Persia” Democritus (460-370 B.C.)

How can causality be established? Observation Induction (Observation Observation) Correlation? Deduction (Theorem Proof)

Attributing causality

Even significant correlation does not imply causation (Fairchild v Glenhaven Funeral Services)

Even ’significant correlation’ is not sufficient Folkes v Chadd, Hill v Metro. Asylum Board

e.g. Number of divorces versus importation of tobacco

spurious correlation

Attributing causality

Even plausible relationships are not necessarily causal e.g. socioeconomic status and heart disease

Confounded by (at least) smoking MMR and Autism confounded by time /

improvements in diagnoses? “Confounded” other plausible explanations Confounding is often an intractable problem in both

‘observational’ and ‘individual’ data How can causality be established avoiding problems

such as confounding?

Systematic Experimentation

“Development of Western Science is based on two great achievements: the invention of the formal logical system (in Euclidean geometry) … and the discovery of the possibility to find out causal relationships by systematic experimentation”

(Albert Einstein, 1953)

Systematic Experimentation

A clinical trial is a systematic experiment of a medical intervention in human subjects

In many instances the optimal clinical trial is controlled, adequately powered, fully pre-specified, randomised and double-blind. The idea being to create groups of patients almost identical (in reality and in perception) except for the intervention of interest.

“adequately powered” sufficient number of patients to estimate the quantity of interest with desired precision.

Importance of control Northwick Park

Systematic Experimentation

If an event of interest occurs with greater frequency in the treated group of patients, it might be argued that the treatment causes the event.

This cannot be said with certainty, but a probability is attached to the likelihood that two interventions differ.

Statistics quantifies this likelihood Probability of observing data given a null hypothesis

If that probability is less than 5% it is common to assume that the effect is ‘established’. Compare with ‘balance of probabilities’

Interpreting clinical trial data Data are often presented as an estimated

effect plus a confidence interval Essentially, all statistics based on samples are estimates. Confidence Interval If the experiment were repeated 100 times,

95 percent of such intervals would contain the true value. In lay terms:

The estimated effect is our best guess at the difference in effect between two treatments

The confidence interval is a measure of uncertainty around that effect, the wider the interval the less certain the estimate.

If the confidence interval excludes the point of no difference, the difference is said to be statistically significant.

Interpreting clinical trial data – an example Vioxx (rofecoxib) and the risk of Myocardial Infarction

Relative risk 2.24, 95% Confidence Intervals (1.24 – 4.02), P=0.007 < 5%

In lay terms: Vioxx was estimated as more than doubling the risk

of MI compared to ‘controls’ The probability that the risks for Vioxx and ‘controls’

were the same is 0.7%. This is ‘statistically significant’.

Interpreting clinical trial data - warnings Even carefully controlled experiments can mislead. Lack of external validity “Lies, damn lies and statistics” – retrospective analysis

and exploration of subgroups can prove anything….. Subgroup analyses

Aspirin is highly effective in reducing the odds of vascular death after acute MI…

…but not in Geminis or Libras! Bias – a systematic deviation from the truth – can be

introduced by carefully chosen statistical methodology. Lack of statistical significance does not automatically

imply similarity

Interpreting clinical trial data – an example (continued) An example: Vioxx (rofecoxib) and the risk of Myocardial Infarction

Relative risk versus all controls 2.24, 95% Confidence Intervals (1.24 – 4.02), P=0.007

The controls were a mixture of placebos, non-naproxen NSAIDs and naproxen, thought to potentially have a cardioprotective effect.

Relative risk versus placebo 1.04 (0.34, 3.12) Relative risk versus non-naproxen NSAIDs 1.55 (0.55, 4.36) Relative risk versus naproxen 2.93 (1.36, 6.33)

Can we say from these data alone whether rofecoxib causes an increase in incidence of MI?

Major implications for drug regulation, continual assessment of Risk: benefit even in the post-marketing setting?

An aside – Product liability for Medicines and Medicinal Devices Established duties of care

‘reasonable care in researching the properties of a product’

‘liability … for unknown risks will generally be assessed on whether sufficient research or testing was undertaken’

What is reasonable / sufficient? N = ???? Time = ???? Dose = ????

Observational experiment

Not all experiments can be conducted optimally i.e. can’t randomise to being male / female

Use observational experiments Case / control studies, cohort studies,

epidemiological database studies These are useful, sometimes necessary, but

arguably, less reliable because sources of confounding are harder to control.

Individual risk

“Even Jonny Wilkinson has a 3 in 4 chance of getting high cholesterol when he’s older” paraphrased from a commercial for Zocor Heart Pro

(simvastatin) sponsored by Boots This is drawn from the population risk of a male over

55 having ‘high’ cholesterol. However, simply because 3 in every 4 males

experience the event does not imply that the risk for an individual male is 3/4.

Similarly if risk of MI is doubled for population taking rofecoxib, what does this imply for the individual? We cannot accurately say from the population data alone.

Estimating individual risk

Individual risk may be attributable to many factors, including, but not limited to, gender, family history, genetics, socioeconomic status, smoking, exercise, diet…..

We can model this risk by estimating the weight to be given to each relevant factor according to its relationship to outcome.

Use epidemiological evidence to model e.g. Framingham to relate LDL to risk of cardiac event. Probability of high cholesterol = (age) + (gender) +

(history) + (smoking) + (smoking) + …… However, we can rarely specify the model with sufficient

precision to ‘prove’ merely to ‘inform’

Gregg v Scott

On balance of probability was 10-yr survival affected?

Delay in treatment estimated as reducing survival from 42% to 25%

Therefore, on balance of probabilities, he would have died anyway.

Issue 1: Does this really measure the ‘loss’ to the patient? E.g. 99% chance of survival 51% chance of

survival = no loss

Gregg v Scott

Issue 2: Why 10-year survival? Probability of 5-year survival would perhaps have

been estimated as being over 50% before, but not after, the delay.

Probability of 1-year survival almost certainly over 50% both before and after the delay

Has the outcome been determined by the (arbitrary) choice of ‘statistical’ cut-off?

Issue 3: Was the epidemiological model applicable to Mr Gregg?

Oral Contraceptives

How was the test derived? Was ‘balance of probabilities’ used as the basis for agreeing to use a doubling of risk (relative risk=2) to allow the judge to reach a decision?

Is it appreciated that probability > 50% and relative risk > 2 are not related?

There is a difference between saying: The risk is increased two-fold by 3G OCs compared to 2G OCs The probability that 3G OCs increase risk compared to 2G OCs

is greater than 50% The probability that 3G OCs increase risk by two-fold compared

to 2G OCs is greater than 50% What was the most important question to answer? Is relative risk of 2 also arbitrary? Aside: BMJ criticisms

Summary

Systematic experimentation on a large population is arguably the most reliable way to establish ‘cause’, but one cannot necessarily draw inferences for a given individual because of confounding factors.

Even optimally designed experiments can give rise to misleading statistics and should be expertly interpreted

A clear understanding of statistical principles would appear to be necessary for those relying on statistics in law.

It is not argued that statistics should be used as the final arbiter of causation, but that the subject should be sufficiently well understood to be able to weigh the statistical evidence appropriately when deliberating.