Teaching Probability and Statistics to Law Students

Philip Dawid

University College London

Outline

1. Pitfalls of Statistical Evidence

2. Conceptions of Probability

3. The Mathematics of Pascalian Probability

4. Probability and Statistics in Court

Main Point:

P (A j B) 6= P (B j A)

1. Pitfalls of Statistical Evidence

Example

• Data: A survey has shown that 80% of all alcoholics were bottle‑fed as babies.

• Inference: A bottle‑fed baby has a high chance of becoming alcoholic.

Example Table

Alcoholic

Non-alcoholic

Total

Bottle

240

1360

1600

Breast 60

140

200

Total

300

1500

1800

P (bottlej alcoholic) = 240=300 = 0:80

P (alcoholic j bottle) = 240=1600 = 0:15P (alcoholic j breast) = 60=200 = 0:30

2. Conceptions of Probability

Statistical Probability

• One of the most straightforward conceptions of probability is as a simple proportion in a suitable population or subpopulation

Smoker Non-smoker Total Male 80 40 120 Female 60 40 100 Total 140 80 220

P( )M = 120/220 = 6/11

P( )F = 40/220 = 5/11

P( )S = 140/220 = 7/11

P( )N = 80/220 = 4/11

Marginal Probabilities

P( | )M S = 80/140 = 4/7

P( | )F S = 60/140 = 3/7

P( | )M N = 40/80 = 1/2

P( | )F N = 40/80 = 1/2

Smoker Non-smoker Total Male 80 40 120 Female 60 40 100 Total 140 80 220

Conditional Probabilities

P( | )S M = 80/120 = 2/3

P( | )N M = 40/120 = 1/3

P( | )S F = 60/100 = 3/5

P( | )N F = 40/100 = 2/5

P( & )M S = 80/220 = 4/11

P( & )M N = 40/220 = 2/11 P( & )F S = 60/220 = 3/11

P( & )F N = 40/220 = 2/11

Smoker Non-smoker Total Male 80 40 120 Female 60 40 100 Total 140 80 220

Joint Probabilities

We easily verify the following properties:

P( ) P( ) 1

P( ) P( ) 1

P( & ) P( & ) P( )

P( & ) P( & ) P( )

P( & ) P( & ) P( )

P( & ) P( & ) P( )

P( & ) P( | ) P( )

M F

S N

M S M N M

F S F N F

M S F S S

M N F N N

M S M S S

• These properties may easily be seen to hold no matter what values are used as data in the above table.

• For any attributes A and B that may be possessed by the individuals in a population, statistical probabilities will always obey the following rules:

1.0 P(A) 1.2. If A is possessed by all members of the

population, P(A) = 1.3. If A and B cannot occur together, P(A or B) =

P(A) + P(B).4.P(A and B) = P(AB) P(B).

• These rules form the basis of the formal mathematical theory of probability.

• They apply to probabilities based on other interpretations than the purely statistical.

• To manipulate probabilities, no matter what their origin or interpretation, we can pretend that they have been formed as proportions in a population and its subpopulations.

Other “Proportion-Based” Conceptions

• Classical Probability (cards, dice,…)– proportions of “equally possible” cases

• Empirical Probability– (limiting) proportions in a sequence of

outcomes

• Metaphysical Probability– proportions in hypothetical parallel universes

Epistemological Conceptions

• Subjective Probability– Bayes, Laplace,…

• Logical Probability– Carnap, Keynes, Jeffreys

• Non-Pascalian Probability– Belief functions– Fuzzy logic– Baconian logic

3. The Mathematics of Pascalian Probability

Example

• L = “accused left town after the murder”

• G = “accused is guilty of the murder.”

• Assess

• Want – also need– say

P( | ) 0.9

P( | ) 0.2

L G

L G

P( | )G L

P( )G

P( ) 0.6G

Tabular Method

1000

L L

G

G

Tabular Method

600

400

1000

L L

G

G

P( ) 0.6G

Tabular Method

540 60 600

400

1000

L L

G

G

P( | ) 0.9L G

Tabular Method

540 60 600

80 320 400

1000

L L

G

G

P( | ) 0.2L G

Tabular Method

540 60 600

80 320 400

620 380 1000

L L

G

G

Tabular Method

540 60 600

80 320 400

620 380 1000

L L

G

G

So P( | ) 540/620 = 0.871G L

Simplification

540 600

80 400

620 1000

L L

G

G

• Only need probabilities (under the various hypotheses considered) for the ACTUAL evidence presented– LIKELIHOOD

Bayes’s Theorem

P( | ) P( ) P( | )

P( | ) P( ) P( | )

G L G L G

G L G L G

POSTERIOR ODDS = PRIOR ODDS LIKELIHOOD RATIO

(540 / 600)540 60080 400 (80 / 400)

540 600

80 400

620 1000

L L

G

G

4. Probability and Statistics in Court

Identification Evidence• DNA match M

• “Random match probability” P– e.g. 1 in 10 million

• Assume suspect would match if guilty

• What is the strength of the evidence?

Prosecutor’s Argument

• The probability of the observed match between the sample at the scene of the crime and that of the suspect having arisen by innocent means is 1 in 10 million

• With a probability overwhelmingly close to 1, the suspect is guilty.

Defence Argument

• There are ~ 30 million individuals who might possibly have committed this crime.

• One of these is the true culprit. • Of the remaining 30 million innocent individuals, each

has a probability of 1 in 10 million of providing a match to the crime trace.

• So we expect about 3 innocent individuals to match. • In the whole population we expect 4 matching

individuals: 1 guilty, and 3 innocent. • We know that the suspect matches, but that only tells us

is that he is one of these 4. • So the probability that he is guilty is only 0.25.

Bayesian ArgumentP( | ) P( ) P( | )

P( | ) P( ) P( | )

P( ) 1P( )

G M G M G

G M G M G

GPG

Posterior odds = Prior odds £ 10 million

Prior odds 1 ) Prosecution

Prior odds 1 in 3 million ) Defence

Adams Case

• Rape

• M: DNA match1/(2 million)P

• B1: Victim did not recognise suspect• B2: Suspect had alibi

• ~ 200,000 potential perpetrators

Illustrative Analysis

prior odds =1

20000

LR1 =:10:90

=19

LR1 =:25:50

=12

LRM = 2million

posterior odds =59

Legal Process

• Convicted• Appeal:

– The task of the jury is to “evaluate evidence and reach a conclusion not by means of a formula, mathematical or otherwise, but by the joint application of their individual common sense and knowledge of the world to the evidence before them”

• Appeal granted on the basis that the trial judge had not adequately dealt with the question of what the jury should do if they did not want to use Bayes’s theorem.

• Retrial– “Bearing in mind that Mr Adams differed from Ms Marley’s

description of her assailant and the fact that she failed to pick him out at the identity parade, what is the probability that she would say at the magistrates’ hearing that he did not look like her assailant, if he had not been her assailant?”

• Reconvicted• Second Appeal

– It might be appropriate for the jury “to ask themselves whether they were satisfied that only X white European men in the UK would have a DNA profile matching that of the rapist who left the crime stain… It would be a matter for the jury, having heard the evidence, to give a value of X.”

• Appeal dismissed

Legal Process

What is X?

• Expected number of innocent individuals matching the crime DNA is 1/10.

• Likelihood ratio from the other evidence is 1/18.• So count Adams as only 1/18 of an individual.• Probability Adams, rather than anyone else, is

the guilty party is

(1 18) {(1 18) (1 10)} 0 36

• OK for jury ???

Current (UK) Position

• Bayesian arguments, while not formally banned, need to be presented with a good deal of circumspection.

• Reexamination of role and use of expert evidence.