Information Classification: General
What should scientific journalsdo to foster research integrity?
Lex Bouter
Information Classification: General
▪ Research integrity
▪ Research climate and publication pressure
▪ Selective reporting
▪ What can scientific journals do?
Content
2
Research Integrity
Research Ethics
ResponsibleResearch & Innovation
Information Classification: General
4
Research Integrity (RI) concerns behaviors of researchersthat hamper validity (truth) of research or trust in scienceand between scientists.
Research Ethics (RE) concerns the ethical considerations ofresearch with humans and animals.
Responsible Research & Innovation (RRI) concerns thebenefits and harms of research for society and theenvironment.
Information Classification: General
Spectrum of research practices
How it should be done:
Relevant, Valid, Reproducible, Efficient
Sloppy science:
Ignorance, honest error or dubious integrity
Scientific fraud:
Fabrication, Falsification, Plagiarism3
ResponsibleConduct ofResearch
QuestionableResearchPractices
ResearchMisconduct2%
34%
Information Classification: General
6
What is good for the validity and reliability of
science is not always good for your professional career
Information Classification: General
Functioning of moral compass depends on:
▪Individual virtuousness
▪Research climate
▪Perverse incentives
Information Classification: General
▪ Research integrity
▪ Research climate and publication pressure
▪ Selective reporting
▪ What can scientific journals do?
Content
8
Information Classification: General
9
Funded by:Templeton Foundation
Vrije UniversiteitUniversity of Amsterdam
Amsterdam UMC
Information Classification: General
10
Information Classification: General
Research Integrity Climate
11
▪ junior researchers perceive the research integrity climate more negatively than senior researchers
▪ junior researchers note that their supervisors are too little committed to talk about key research integrity principles
▪ PhD students perceive more competition and suspicion among colleagues than associate and full professors
▪ researchers from the natural sciences have a more positive perception of the research integrity climate
▪ Researchers from social sciences as well as from the humanitiesperceive less fairness of their departments’ expectations in terms of publishing and acquiring funding
Information Classification: General
12
Information Classification: General
Perceived Publication Pressure
13
▪ Publication pressure is a particularly detrimental stressor for postdocs and assistant professors
▪ Publication pressure concerns researchers from all disciplinary fields but is highest for researchers in the humanities
▪ Our findings emphasize the need to move the debate forward towards a healthy publication climate, where researchers are incentivised to optimize quality and integrity of their publications
Information Classification: General
▪ Research integrity
▪ Research climate and publication pressure
▪ Selective reporting
▪ What can scientific journals do?
Content
14
Information Classification: General
15
We need to talk about theelephants in
the room
Information Classification: General
personal
interests
sponsor
interests
QRP & RM
(false)
positive
results
citations
publications
media
attention
grants
&
tenure
How things can go wrong
16
Information Classification: General
17
How negative results disappear from the published literature
Information Classification: General
18
Replicability of studies is only 10-40 %
Information Classification: General
Important causes of ‘replicability crisis’
▪ Selective reporting▪ Low power ▪ P-hacking▪ HARKing
19
Hypothesizing AfterResults are Known
Information Classification: General
34 Researcher Degrees of Freedom that can be usedto get Positive Results
20
Information Classification: General
Study Protocol
Analysis Plan
Amendments
Data Sets → Open Data
Reports → Open Access
Transparency of
Always prospectively
Publicly – if possible
21
Information Classification: General
22
Information Classification: General
23
Information Classification: General
▪ Weighted random sample from Scopus (N=835)
▪ 19 topics: RI and transparency in reporting
▪ Absence or presence of topic in ItA
▪ Machine reading with manual checks
Instructions to Authors
24
Information Classification: General
25
Information Classification: General
26
Information Classification: General
▪ Research integrity
▪ Research climate and publication pressure
▪ Selective reporting
▪ What can scientific journals do?
Content
27
Information Classification: General
▪ Researchers
▪ Research institutes
▪ Funders of research
▪ Scientific and scholarly journals
No quick fixes and many stakeholders
28
Information Classification: General
What can scientific journals do?
29
1. follow the TOP guidelinesand be as transparant as possible
2. demand preregistration or introduce registered reportsto avoid selective reporting
3. use reporting guidelinesto make publications more clear and informative
4. promote preprintswith a view to avoid delay and improve quality
Information Classification: General
What can scientific journals do?
30
5. have open peer review and give reviews a DOIand be as transparant as possible
6. check for spin, reporting of study limitations, text recycling and image duplication
to improve quality and trustworthyness
7. promote post-publication peer reviewto allow scholarly debate after publication
8. allow updated versionsto enable later correction of errors and omissions
Information Classification: General
31
Registered Reports
Information Classification: General
32
N = 413
Information Classification: General
33
Preprints andPost-publication Peer Review
Information Classification: General
34
Information Classification: General
www.nrin.nl@NRIN_Integrity
35
Information Classification: General
www.wcrif.org
Information Classification: General
www.reward-equator-conference-2020.com