1

The design of animal experiments

Michael FW Festingc/o Understanding Animal Research, 25 Shaftsbury

Av. London, UK. michaelfesting@aol.com

2

Replacement e.g. in-vitro methods, less sentient animals

Refinement e.g. anaesthesia and analgesia, environmental

enrichment Reduction

Research strategy Controlling variability Experimental design and statistics

Principles of Humane Experimental Technique

(Russell and Burch 1959)

3

A well designed experiment Absence of bias

Experimental unit, randomisation, blinding High power

Low noise (uniform material, blocking, covariance) High signal (sensitive subjects, high dose) Large sample size

Wide range of applicability Replicate over other factors (e.g. sex, strain): factorial

designs Simplicity Amenable to a statistical analysis

4

The animal as the experimental unit

Animals individually treated. May be individually housed or grouped

N=8n=4

5

A cage as the Experimental Unit.

Treatment in water or diet.

N=4n=2

Treated TreatedControl Control

6

An animal for a period of time: repeated measures or crossover design

Animal

1

2

3

Treatment 1

Treatment 2

N

4

4

4

N=12n= 6

7

Teratology: mother treated, young measured

Mother is the experimental unit.

N=2n=1

8

Failure to identify the experimental unit correctly in a 2(strains) x 3(treatments) x 6(times) factorial design

ELD groupELD group

Single cage of 8 mice killed at each time point (288 mice in total)

9

Experimental units must be randomised to treatments

Physical: numbers on cards. Shuffle and take one

Tables of random numbers in most text books

Use computer. e.g. EXCEL or a statistical package such as MINITAB

10

Randomisation

Original Randomised1 21 31 31 12 22 12 22 13 33 23 33 1

NB Randomisation should include housing and order in which observations are made

11

Failure to randomise and/or blind leads to more “positive” results

Blind/not blind odds ratio 3.4 (95% CI 1.7-6.9)

Random/not random odds ratio 3.2 (95% CI 1.3-7.7)

Blind Random/ odds ratio 5.2 (95% CI 2.0-13.5)not blind random

290 animal studies scored for blinding, randomisation and positive/negative outcome, as defined by authors

Babasta et al 2003 Acad. emerg. med. 10:684-687

12

Some factors (e.g. strain, sex) can not be randomised so special care is needed to ensure comparability

Outbred TO (8-12 weeks commercial)

Inbred CBA (12-16 weeks Home bred)

Six cages of 7-9 mice of each strain: error bars are SEMs

"CBA mice showed greater variability in body weights than TO mice..."

13

A well designed experiment Absence of bias

Experimental unit, randomisation, blinding High power

Low noise (uniform material, blocking, covariance) High signal (sensitive subjects, high dose) Large sample size

Wide range of applicability Replicate over other factors (e.g. sex, strain): factorial

designs Simplicity Amenable to a statistical analysis

14

High power: (good chance of detecting the effect of a treatment, if there is one)

High Signal/Noise ratio= High Standardized effect size= High |1-2|/= HighDifference between means)/SD

Student’s t =( X1-X2)/Sqrt (2S2/n)

15

Power Analysis for sample size and effects of variation

A mathematical relationship between six variables Needs subjective estimate of effect size to be detected

(signal) Has to be done separately for each character Not easy to apply to complex designs Essential for expensive, simple, large experiments

(clinical trials) Useful for exploring effect of variability

A second method “The Resource Equation” is described later

16

Power analysis: the variables

Sample size

Signala) Effect size of scientific interest

or b) actual response

Chance of a false positive result.

Significance level (0.05)

Sidedness of statistical test (usually 2-sided)

Power of theExperiment (80-90%?)

NoiseVariability of the

experimental material

17

Group size and Signal/noise ratio

0

20

40

60

80

100

120

140

0 0.5 1 1.5 2 2.5 3

Effect size (Std. Devs.)

Gro

up

siz

e

90%

80%

Assuming 2-sample, 2 sided t-test and 5% significance level

Signal/noise ratio

Power

Neutral

Bad

Good

18

Comparison of two anaesthetics for dogs under clinical conditions (Vet. Anaesthes. Analges.)

Unsexed healthy clinic dogs,• Weight 3.8 to 42.6 kg. • Systolic BP 141 (SD 36) mm Hg

Assume: • a 20 mmHg difference between anaesthetics is of clinical importance, • a significance level of =0.05• a power=90% • a 2-sided t-test

Signal/Noise ratio 20/36 = 0.56Required sample size 68/group

19

Power and sample size calculations using nQuery Advisor

20

A second paper described:

• Male Beagles weight 17-23 kg• mean BP 108 (SD 9) mm Hg.• Want to detect 20mm difference between groups (as before)With the same assumptions as previous slide:

Signal/noise ratio = 20/9 = 2.22

Required sample size 6/group

21

Summary for two sources of dogs: aim is to be able to detect a 20mmHg change in blood pressure

Type of dog SDev Signal/noise Sample %Power (n=8)

size/gp(1) (2) Random dogs 36 0.56 68 18Male beagles 9 2.22 6 98

(1)Sample size: 90% power(2)Power, Sample size 8/group

Assumes =5%, 2-sided t-test and effect size 20mmHg

The scientific dilemma: With small sample sizes we can not detect an important effect in genetically heterogeneous animals.

We can detect the effect in genetically homogeneous animals, but are they representative?

22

Variation in kidney weight in 58 groups of rats

0

10

20

30

40

50

60

70

80

90

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57

Sample number

Va

ria

bil

ity

Mycoplasma

Outbred

F1

F2

Gartner,K. (1990), Laboratory Animals, 24:71-77.

23

Required sample sizes

Factor Type Std.Dev Signal/ noise*

Sample size

Power**

Genetics F1 hybrid 13.5 0.74 30 80

F2 hybrid 18.4 0.54 55 53

Outbred 20.1 0.49 67 46

Disease Mycoplasma free

18.6 0.54 55 53

With Mycoplasma

43.3 0.23 298 14

*signal is 10 units, two sided t-test,=0.05, power = 80%** Assuming fixed sample size of 30/group

24

The randomised block design: another method of controlling noise

B C A

A C B

B A C

A C B

B C A B1

B2

B3

B4

B5

Treaments A, B & C

• Randomisation is within-block

• Can be multiple differences between blocks

• Heterogeneous age/weight

• Different shelves/rooms• Natural structure (litters)• Split experiment in time

25

A randomised block experiment

050

100150200250300350400450500

1 2 3

Week

Ap

op

tosis

sco

re

Control

CGP

STAU

365 398 421 423 432 459 308 320 329

Treatment effect p=0.023(2-way ANOVA)

26

Analysis of apoptosis data

Analysis of Variance for Score

Source DF SS MS F PBlock 2 21764.2 10882.1 114.82 0.000Treatmen 2 2129.6 1064.8 11.23 0.023Error 4 379.1 94.8Total 8 24272.9

27

-10.0 -7.5 -5.0 -2.5 0.0 2.5 5.0 7.5

0

1

2

3

Residual

Fre

quen

cy

Histogram of Residuals

0 1 2 3 4 5 6 7 8 9

-20

-10

0

10

20

Observation Number

Res

idua

l

I Chart of Residuals

Mean=3.16E-14

UCL=20.17

LCL=-20.17

300 350 400 450

-10

0

10

Fit

Res

idua

l

Residuals vs. Fits

-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5

-10

0

10

Normal Plot of Residuals

Normal Score

Re

sid

ual

Residual Model Diagnostics

28

Another method of determining sample size: The Resource Equation

Depends on the law of diminishing returns Simple. No subjective parameters Useful for complex designs and/or multiple outcomes

(characters) Does not require estimate of Standard Deviation Crude compared with Power Analysis

E= (Total number of animals)-(number of groups)

10<E<20 (but give some tolerance)

29

0 5 10 15 20 25 30 35

2.0

4.5

7.0

9.5

12.0

Degrees of freedom

Stu

dent

's t

, 5%

crit

ical

val

ue

E= (total numbers)-(number of groups)

10<E<20

The Resource Equation & Sample Size

But if experimental subjects are cheap (e.g. multi-well plates, E can be much higher

30

A well designed experiment Absence of bias

Experimental unit, randomisation, blinding High power

Low noise (uniform material, blocking, covariance) High signal (sensitive subjects, high dose) Large sample size

Wide range of applicability Replicate over other factors to (e.g. sex, strain) to increase

generality: factorial designs Simplicity Amenable to a statistical analysis

31

Factorial designs

Single factor design

Treated Control

E=16-2 = 14

One variable at a time (OVAT)

Treated ControlTreated Control

E=16-2 = 14 E=16-2 = 14

Factorial design

Treated Control

E=16-4 = 12

32

Factorial designs

(By using a factorial design)”.... an experimental investigation, at the same time as it is made more comprehensive, may also be made more efficient if by more efficient we mean that more knowledge and a higher degree of precision are obtainable by the same number of observations.”

R.A. Fisher, 1960

33

A 4x2 factorial design

Analysed with Student’s t-test: This is not appropriate because:1. Each test is based on too few animals (n=3-4), so lacks power2. It does not indicate whether there are strain differences in protein thiol status3. It does not indicate whether dose/response differs between strains4. A two-way design should be analysed using a 2-way ANOVA

34

Incorrect statistical analysis leading to excessive numbers of animals

8 mice per group8 groups = 64 mice. E= 64-8 =56

Alternative3 mice per group:8 groupsE=24-8 = 16

Saving:40 miceFormal test of interaction

One experiment or4 separate experiments?

35

2 (strains) x 4 (Animal units) factorial

36

Effect of chloramphenicol (2000mg/kg) on RBC count

Strain Control TreatedC3H 7.85 7.81

8.77 7.218.48 6.968.22 7.10

CD-1 9.01 9.187.76 8.318.42 8.478.83 8.67

Tests: Use a two-way ANOVA with interaction

1. Do the treatment means averaged across strains differ?

2. Do the strains differ, averaged across treatments

3. Do the two strains respond to the same extent?

Should not be analysedusing two t-tests1. Each test lacks power due to small sample size2. Will not give a test of whether strains differ in response

37

A 2x2 factorial design with interaction

Source DF SS MS F Pstrain 1 2.4414 2.4414 13.13 0.003Treatment 1 0.8236 0.8236 4.43 0.057strain*treat. 1 1.4702 1.4702 7.91 0.016Error 12 2.2308 0.1859Total 15 6.9659

6.5

7

7.5

8

8.5

9

Control Treated Control Treated

Strain and treatment

Red

blo

od

cell

cou

nt

C3H CD-1

Pooled variance

38

Use of several inbred strains to reduce noise, increase signal and explore generality

500 1000 1500 2000 2500

CD-1 8 8 8 8 8 8

CBA 2 2 2 2 2

C3H 2 2 2 2 2

BALB/c

2

2 2 2 2

C57BL

2

2 2 2 2

2

2

2

2

Inbred

0 Outbred

Dose of chloramphenicol (mg/kg)

Festing et al (2001) Fd. Chem.Tox. 39:375

Effect of chloramphenicol on mouse haematology

39

WBC Strain Control TreatedCBA 1.90 0.40CBA 2.60 0.20C3H 2.10 0.40C3H 2.20 0.40BALB/c 1.60 1.30BALB/c 0.50 1.40C57BL 2.30 0.80C57BL 2.20 1.10

CD-1 3.00 1.90CD-1 1.70 1.90CD-1 1.50 3.50CD-1 2.00 1.20CD-1 3.80 2.30CD-1 0.90 1.00CD-1 2.60 1.30CD-1 2.30 1.60

Example of a factorial compared with a single factor design

Four inbred strains

One outbred stock

40

Signal NoiseStrain N 0 2500 (Difference) (SD) Signal/noise pCBA 4 2.25 0.30 1.95 0.34 5.73C3H 4 2.15 0.40 1.85 0.34 5.44BALB/c 4 1.05 1.35 (-0.30) 0.34 (-0.88)C57BL 4 2.25 0.95 1.30 0.34 3.82Mean 16 1.93 1.20 0.73 0.34 2.15 <0.001Dose * strain <0.001

WBC counts following chloramphenicol at 2500mg/kg

Signal NoiseStrain N 0 2500 (Difference) (SD) Signal/noise pCD-1 16 2.23 1.83 0.40 0.86 0.47 0.38

White blood cell counts

41

Genetics is important: Twenty two Nobel Prizes since 1960 for work depending on inbred strains

CancermmTV

Transmissableencephalopathacies/prionsPruisner

Retroviruses, Oncogenes & growth factorsCohen, Levi-montalcini, Varmus, Bishop, Baltimore, Temin

Humoral immunity/antibodiesT-cell receptorTonegawa, Jerne

Cell mediated immunityImmunological toleranceH2 restriction, immune responsesMedawar, Burnet, Doherty, ZinkanagelBenacerraf (G.pigs)

GeneticsSnell C.C. Little, DBA, 1909

Inbred Strains and derivativesJackson Laboratory

monoclonal antibodiesBALB/c miceKohler and Millstein

SmellAxel & Buck

ES cells & “knockouts”Evans, Capecchi, Smithies

42

18th Annual Short Course on Experimental

Models of Human Cancer

August 21-30, 2009

Bar Harbor, ME

courses.jax.org

43

Conclusions

Five requirements for a good design Unbiased (randomisation, blinding) Powerful (signal/noise ratio: control variability) Wide range of applicability (factorial designs, common but

frequently analysed incorrectly) Simple Amenable to statistical analysis

Mistakes in design and analysis are common Better training in experimental design would improve

the quality of research, save money, time and animals

44

45