Shelley Hurwitz MedicReS World Congress 2014

Biostatistics and Ethics

Shelley Hurwitz, PhD

Brigham and Women’s Hospital Harvard Medical School

Fellow, American Statistical Association

Advisory Board on Ethics, International Statistical Institute

MedicReS World Congress on Good Medical Research

New York October 17, 2014

1

Outline

• Reputation of statistics

• Statisticians respond

• Ethical guidelines

• Reproducibility

2

• “Figures will not lie, but liars will figure.”

• “There are three kinds of lies: Lies, damned lies, and statistics"

• “Facts are stubborn things, but statistics are pliable” • “How to lie with statistics”

• "Statistics are no substitute for judgment.”

Reputation of Statistics

3

• “Figures will not lie, but liars will figure.” (1889)


• “Facts are stubborn things, but statistics are pliable”

• “How to lie with statistics”



4


• “There are three kinds of lies: Lies, damned lies, and statistics” (1906)





5







6




• “How to lie with statistics” (1954)



7







8

Outline




• Reproducibility

9

Series of articles in British Medical Journal, 1980-1

“Statistics and Ethics in Medical Research” by Douglas G. Altman

• Misuse of statistics is unethical • Study design • How large a sample? • Collecting and screening data • Analysing data • Presentation of results • Interpreting results • Improving the quality of statistics in medical journals

10

John C Bailar III "Science, Statistics, and Deception"

Annals of Internal Medicine 1986;104:259-60.

• Summarizes important role of statistics in scientific research

• Practices that can distort scientific inferences:

– Failure to deal honestly with readers about non-random error (bias) – Post hoc hypothesis – Inappropriate statistical tests and other statistical procedures – Fragmentation of reports – Low statistical power – Suppressing, trimming, or “adjusting” data – Undisclosed repetition of “unsatisfactory” experiments – Selective reporting of findings

11

David L. DeMets “Statistics and Ethics in Medical Research”

Science and Engineering Ethics. 1999; 5(1) 97-117.

• Statistical methods are powerful tools but can easily be

misused. • Pressures to get “right” results can be difficult, especially for

junior statisticians. • Community must be vigilant; misuse is hard to detect. • Just as no one may know if you disobey a stop sign late at

night, rarely will anyone be able to double check in detail our statistical analyses and design.

• Ultimately, proper use of statistics in medical research rests with the integrity of the individual.

12

John S. Gardenier and David B. Resnik “The Misuse of Statistics: Concepts, Tools, and a Research Agenda”

Accountability in Research: Policies and Quality Assurance. 2002 (9) 65-74.

• Misconduct

• Incompetence

• Negligence

• Honest error

• Deliberate deception

– Pressures to publish, produce results, or obtain grants

– Career ambitions or aspirations

– Conflicts of interest and economic motives

– Inadequate supervision, education, or training

13

Outline



• Guidelines for Ethical Statistical Practice

• Reproducibility

14

The Statistical Society of Australia Inc. Code of Conduct (1998)

American Statistical Association

Ethical Guidelines for Statistical Practice (1999)

Statistical Society of Canada Code of Ethical Statistical Practice (2004)

International Statistical Institute

Declaration on Professional Ethics (2010)

Royal Statistical Society Code of Conduct (2014)

15

American Statistical Association Ethical Guidelines for Statistical Practice

1949: Commission on Statistical Standards to develop

a code of ethical practices 1954: 2 surveys 1956: Interest was low. Idea was tabled. 1977: Interest increased, new committee formed. 1980: Interim Code of Conduct published, requesting

comments. There was extensive input. 1983: Final version published. 1996: The Committee on Professional Ethics started

drafting a revision. 1998: Draft published. Invited comments. Open

session at annual meeting. 1999: Current version approved by Board of Directors.

16

Domains of Ethical Responsibility in ASA’s and other societies’ codes of conduct

• Society

• Public interest

• The profession

• Funders

• Clients

• Employers

• Research subjects

• Research team colleagues

• Other statisticians

17

15 years later New ethical concerns for statisticians

• Massive amounts of data

• Complex data structure

• High dimensionality

• De-identification

• Security & encryption

• Increasingly multidisciplinary studies

18

Outline




• Reproducibility

19

“How Science Goes Wrong” The Economist, 10/19/2013

• Half of published research cannot be replicated.

• 2000-10 roughly 80,000 patients took part in clinical trials based on research that was later retracted because of mistakes or improprieties

• Lost opportunity, lost money

• Obligation to publish or perish

• Honest confusion “sweet signal of a genuine discovery” and “freak of the statistical noise”

• Spurious correlations in newspapers, morning TV

• Failure to report failure leads to more researchers exploring blind alleys already explored.

20

“Fraud in Medical Research: An International Survey of Biostatisticians”

Ranstam, Buyse, George, Evans, Geller, Scherrer, Lesaffre, Murray, Edler, Hutton, Colton, and Lachenbruch,

for the ISCB Subcommittee on Fraud Controlled Clinical Trials 21:415–427 (2000)

• 51% of biostatisticians know of at least one fraudulent project.

• 31% have been involved in fraudulent projects.

• 13% have been directly asked to support fraud.

– Fabrication or falsification of data

– Suppression or selective deletion of data

– Deceptive design or analysis

– Deceptive reporting of results

21

The Duke Saga

2006: Nature Medicine “Genomic signatures to guide the use of chemotherapeutics”

2009: The Annals of Applied Statistics “Deriving chemosensitivity from cell lines: Forensic

bioinformatics and reproducible research in high- throughput biology”

2011: Nature Medicine “Retraction: Genomic signatures to guide the use of

chemotherapeutics”

22

Nature Medicine 2006, Vol. 12, 1294 – 1300

“Genomic Signatures to Guide the use of Chemotherapeutics”

By

Anil Potti, Holly K Dressman, Andrea Bild, Richard F Riedel, Gina Chan, Robyn Sayer, Janiel Cragun, Hope Cottrill, Michael J Kelley,

Rebecca Petersen, David Harpole, Jeffrey Marks, Andrew Berchuck, Geoffrey S Ginsburg, Phillip Febbo, Johnathan

Lancaster & Joseph R Nevins

23

Corrigenda: Genomic signatures to guide the use of chemotherapeutics

By Anil Potti, Holly K Dressman, Andrea Bild, Richard F Riedel,

Gina Chan, Robyn Sayer, Janiel Cragun, Hope Cottrill, Michael J Kelley, Rebecca Petersen, David Harpole, Jeffrey Marks, Andrew Berchuck, Geoffrey S Ginsburg, Phillip Febbo,

Johnathan Lancaster & Joseph R Nevins

• Nature Medicine 13, 1388 (2007)

• Nature Medicine 14, 889 (2008)

24

The Annals of Applied Statistics

2009, Vol. 3, 1309 – 1334

“Deriving Chemosensitivity From Cell Lines: Forensic Bioinformatics and Reproducible

Research in High-throughput Biology”

By Keith A. Baggerly

and Kevin R. Coombes

25

Nature Medicine 2011, Vol.17, 135

“Retraction: Genomic Signatures to Guide the use of Chemotherapeutics”

By

Anil Potti, Holly K Dressman, Andrea Bild, Richard F Riedel, Gina Chan, Robyn Sayer, Janiel Cragun, Hope Cottrill, Michael J Kelley,

Rebecca Petersen, David Harpole, Jeffrey Marks, Andrew Berchuck, Geoffrey S Ginsburg, Phillip Febbo, Johnathan

Lancaster & Joseph R Nevins

26

Uninteresting Work

• Replication of positive results is not interesting work. • Negative studies are not that interesting. • Open access to all data • Clinical trial registration • Journals should set aside room for

– Replication studies – Negative studies – Expanded methods sections

• Funders should set aside funding for this uninteresting work.

27

“Lies, Damned Lies, and Medical Science”

by

David H. Freedman October 4, 2010

The Atlantic

28

“Why Most Published Research Findings are False”

by

Professor John P. A. Ioannidis

2005, PLOS Medicine

1,138,461 on-line views

1,413 citations 10,464 shares on Facebook and Twitter

29

What reduces the likelihood a scientific finding is true?

1. Studies conducted in the scientific field are small. 2. Effect sizes in the field are small. 3. Number of studies is high, pre-tested relationships are not

selected for study. 4. Highly flexible designs, definitions, outcomes, analyses. 5. Greater financial and other interests and prejudices in a

scientific field. 6. Greater “hot-ness” of the field, with more teams involved.

Ioannidis, JPA (2005) “Why Most Published Research Findings are False”

30

Biostatistics and Ethics Conclusion

Updating Wright, 1889: It is our duty, as practical statisticians, to prevent the investigator from figuring;

in other words, to prevent him or her from perverting the truth, in the interest of some theory he or she wishes to establish.

DeMets, 1999: Ultimately proper use of statistical methods must rest with the integrity of the

individual statistician.

• Integrity • Transparency • Reproducibility • Accountability • Guidelines

31

Thank you

Contact information [email protected]

32

Date post:	01-Dec-2014
Category:	Health & Medicine
Upload:	medicres
View:	657 times
Download:	0 times

Shelley Hurwitz MedicReS World Congress 2014

Health & Medicine