1 Mutual Learning Exercise (MLE) on Open Science: Altmetrics and Rewards Background/Challenge Paper No. 3 Incentives and Rewards to Engage in Open Science Activities September 2017 Contents 1 Introduction .....................................................................................................................................3 2 Background: Advantages and Challenges of Making Research Open .............................................3 2.1 Evaluation and Credit Systems ............................................................................................... 7 2.2 Diversity in Research Cultures and Quality Assurance Criteria..............................................8 2.3 Costs, Accountabilities and Long-Term Sustainability ........................................................... 9 2.4 Skills and Training ...................................................................................................................9 2.5 Intellectual Property Concerns.............................................................................................11 2.6 Semantic Ambiguity .............................................................................................................12 2.7 Ethical and Societal Concerns ..............................................................................................13 2.8 High Resource Bias ...............................................................................................................13 3 Incentives and Rewards for Open Science: Researchers ...............................................................13 3.1 Assessment and Promotion Criteria.....................................................................................14 3.2 Training on OA Guidelines and Implementation Tools ........................................................16 3.3 Citation and Authorship Cultures .........................................................................................17 3.4 Guarantees of the International and Sustainable Nature of OS Initiatives and Related Infrastructures ...................................................................................................................................18 3.5 Open Science Prizes: Establishing Champions and Role Models .........................................18 4 Incentives and Rewards for Open Science: Research Organisations and Funding Bodies ............19 4.1 Fostering Interdisciplinary and Translational Research .......................................................21 4.2 Promoting Social Engagement and Responsible Innovation ...............................................22 4.3 Enhancing Educational Resources ........................................................................................22 4.4 Improving Management Practice .........................................................................................23 4.5 Improving Transparency and External Accountability .........................................................24 4.6 Enhancing International Visibility and Reputation ..............................................................25 5 Incentives and Rewards for Open Science: National Governments ..............................................25 5.1 Improving Transparency and External Accountability .........................................................25 5.2 Promoting Social Engagement and Responsible Innovation ...............................................26
Transcript
1
Mutual Learning Exercise (MLE) on Open Science: Altmetrics and Rewards
Background/Challenge Paper No. 3
Incentives and Rewards to Engage in Open Science Activities
1 Introduction This Background/Challenge Paper has been developed to help Mutual Learning Exercise (MLE)
participants prepare for the 3rd Working Meeting in Dubrovnik on 12 and 13 September 2017.
The topic for this Challenge/Background Paper and for the 3rd Working Meeting is Incentives
and Rewards to Engage in Open Science Activities. This Challenge/Background Paper gives
an overview and a preliminary assessment of the different practices that are currently being
used and/or investigated for the purpose of incentivising and rewarding researchers and their
institutions to engage with open science activities. This document also aims at gathering
experiences (good and bad) about the use of rewards and incentives for Open Science in the
member states (MS) participating in the MLE. It is based on a review of relevant background
academic literature and policy documents, discussions at the previous meetings of the MLE
(particularly the one on alternative metrics for Open Science, which took place in May 2017 in
Helsinki) and on answers to open ended questions sent to the participants of the MLE. Some
data have also been sourced from the European Open Science Monitor, which constitutes the
most updated data source on Open Science implementation policies across European member
states to date (http://ec.europa.eu/research/openscience).
This challenge paper aims to stimulate discussion among participants of the MLE over credit
activities which are important to Open Science, such as open review and evaluation, as well as
citation, curation and management of research data. It also aims to generate consensus around
what incentives could be put in place in order to provide relevant training and support by
research institutions. The challenge paper focuses specifically on the incentives and rewards
that could be put in place for three types of stakeholders: (1) researchers, (2) research
performing organisations and funding bodies, and (3) national governments. After the 3rd
Working Meeting a Report on Incentives and Rewards to Engage in Open Science Activities will
be produced and it will be one of the main Deliverables of this MLE.
2 Background: Advantages and Challenges of Making Research Open The revolutionary potential of Open Science to enhance research quality, reliability, integrity
and societal impact has been widely discussed in academia and policy (Hey et al 2009, Royal
Society 2012, Kitchin 2014, Leonelli 2016, McKiernan et al 2016). In the 2015 book Open
Innovation, Open Science, Open to the World, which outlines the European Commission’s
commitment to Open Science and its vision for its development and impact, Open Science is
defined as:
“a new approach to the scientific process based on cooperative work and new ways of
diffusing knowledge by using digital technologies and new collaborative tools. The idea
captures a systemic change to the way science and research have been carried out for
the last fifty years: shifting from the standard practices of publishing research results in
scientific publications towards sharing and using all available knowledge at an earlier
stage in the research process” (p. 33)
The movement towards Open Science runs parallel to a broad international debate about the
current state and social function of research. It has been widely recognized that research,
4
particularly as institutionalized within academia, has developed into a social system dominated
by systemic drivers that cause it to be self-referential in the incentives applied to academic
career advancement of the researchers and institutional reputation. This generated
hypercompetitive research behaviors, a systematic devaluation of the quality and
reproducibility of research outputs in favor of high volume and prestige, and the dominance of
publication in high impact factors journals over other, more desirable research goals (such as
addressing socially relevant topics, undertaking high-risk and innovative projects, and enacting
a research agenda that aligns to the values and goals of democratic societies). The current
system of research assessment and institutionalization has encouraged a disconnection
between knowledge production and the role that research can and should play to help achieve
key societal aims (Kleinman 2000, Radder 2010, Nordmann et al 2011, Miedema 2012, Wilsdon
et al 2015). Open Science, with its recognition of the importance of societal engagement (for
instance in the form of citizen science involvement in the design and ongoing development of
projects) and of the diversity of outputs and resourced developed by any one research group
(including models, data, code, workflows and non-academic writings, all of which can be made
available for repurposing by other groups around the globe), provides an important opportunity
as well as an effective strategy to remedy this situation.
There is ongoing debate on whether Open Science is better defined by the use of new digital
tools, a specific set of values, and/or practices of collaboration and sharing (Grubb and
Easterbook 2011, Royal Society 2012, Mauthner and Parry 2013, Fecher and Friesike 2014,
Levin and Leonelli 2016). This report does not commit to any one of these definitions, all of
which capture important aspects of the movement to Open Science. What matters is that the
very existence of a debate around what Open Science can be provides policy-makers, research
institutions, funding bodies and researchers themselves with an opportunity to critically
consider what does and should count as high-quality research, what goals researchers should
pursue, how research results should be evaluated and disseminated, and how research should
be supported and embedded within society (Leonelli 2016). At the same time, there are three
crucial points of consensus among Open Science experts, practitioners and policy-makers:
(1) Open Science involves a systemic shift in current practices of research, publishing
and evaluation
(2) Open Science affects all stages of the research process, including data gathering,
analysis, review and conceptualization (figure 1), and its implementation involves a
wide set of governance structures ranging from technical mechanisms of review to
broad policy guidelines (figure 2).
(3) The implications of Open Science for any one research systems (at the international,
national, regional and local levels, as well as across disciplinary domains and
otherwise defined areas of research interest) need to be considered with reference
to its specific characteristics, and thus the mechanisms through which Open Science
is implemented are likely to vary.
5
Figure 1: Open Science components and their role in the research process (source:
• the fight against fraud, lack of integrity and duplication of efforts
• public engagement and the involvement of a broad range of stakeholders in the design and
evaluation of research
• the social, ethical and political legitimacy of science, resulting in an increase of public trust
and participation in research, thus countering the current trends towards “alternative facts”
and “fake news”.
Thus the implementation of Open Science principles and methods provides a way to increase
the scientific, economic and social value of research, while also reducing the waste of resources
created by narrow-minded competition and by the lack of sharing of data, models, methods and
materials.
It should be noted that viewing openness as a key norm for the conduct of science is by no
means a novel idea. Since at least the scientific revolution, the principles of public scrutiny,
transparency and the reproducibility of results have been employed to demarcate science from
other knowledge-making activities and to define how research should be conducted and what
counts as an output (Merton 1942, Popper 1945, Longino 2002, Kitcher 2011). Some research
communities (including in fields such as astronomy, metereology, natural history and, more
recently, genomics) have long advocated the open sharing of data and ideas, grounded in the
belief that collaboration and public debate can and do engender reliable and insightful
knowledge. This raises the question of why this approach has not taken hold in contemporary
mainstream research. As argued above, the practice of science has undergone critical
developments over the last century which have limited researchers’ ability to enact openness in
their daily decision-making. Open Science is therefore defined by the European Commission as a
radical departure from the ways in which research is normally conducted and assessed.
It is the main contention of this challenge paper that the main explanation for the current lack of
uptake lies in the lack of incentives and rewards for Open Science practices. The remainder of
this section discusses eight areas in which contemporary modes of research support,
publication and assessment pose obstacles and disincentives to making research open: (1)
evaluation and credit systems; (2) diversity in research cultures; (3) costs and accountabilities;
(4) skills and training; (5) intellectual property regimes; (6) semantic ambiguity; (7) ethical and
social concerns; and (8) high resource bias. Understanding this landscape is a crucial
precondition to the remaining sections of the challenge paper, which examines the incentives
and rewards that could counter this trend and stimulate the adoption and implementation of
Open Science among different stakeholders.
2.1 Evaluation and Credit Systems
Openness is strongly constrained by the evaluation measures used to assess the outputs of
research, which typically rely on quantitative measures of the number of articles produced, the
venue in which they were published, and the number of citations garnered. As indicated by
previous MLE reports, these measures take no account of efforts made by researchers to share
their work, propose highly creative and innovative solutions and tackle real world problems,
thus putting researchers who engage in Open Science at a strong disadvantage with respect to
colleagues who only care about publishing papers in top-ranking journals (Benedictus et al
2016). This leads to a risk-averse mentality and reinforces a “publish or perish” model of
academic success. The situation is made worse by the fierce competition for limited jobs and
resources characterizing most research fields, which leads researchers to privilege speed over
8
quality of discoveries, and acts as a strong disincentive to the sharing of results and methods.
The pressure to publish leads to information control. Within such a climate, “increasing
transparency in research practices can have unintended consequences. Anything that is open to
public scrutiny can be used to assess the practices in question, which may be premature for
ongoing projects that need time to yield clear and widely intelligible results. It may also
compound researchers’ fears of being scooped. It is not hard to imagine that researchers forced
to render lab or field notes, protocols or software freely accessible to others will feel the need to
create shadow procedures and infrastructures for those parts of their practice that they do not
want, or cannot share” (Leonelli et al 2015, 12; see also Tenopir et al. 2011 and 2015; Poline et
al. 2012; Schäfer et al. 2011). The Open Science practice that is perhaps worst affected is Open
Data sharing, given the considerable labor involved in preparing data for donation in a public
repository, the risks that researchers associate with such a procedure, and the complete lack of
rewards associated to data publication (Edwards et al 2011).
This situation has a strong impact on the choices and behaviors of junior researchers, who are
most vulnerable to assessment requirements since their employment and research directions
depend entirely on those. Conservatism in assessment is partly driven by the metrics and
evaluation criteria endorsed by research performing institutions and funding bodies, as detailed
in the previous MLE report. Conservatism is typically also supported by senior academics who
inform promotion decisions and the allocation of funding. These individuals often reproduce the
assessment cultures through which they have proved successful. It is therefore crucial to
provide incentives for senior academics to embrace and reward Open Science activities in their
evaluation work. With open science being incorporated into incentive and reward structures
young researchers can establish and adapt themselves to new systems of merit and recognition.
It is thus particularly important to regard Open Science as part of a more comprehensive
systemic change in research processes, management, administration and evaluation, so as to
engender the cultural and structural changes necessary to enable Open Science.
2.2 Diversity in Research Cultures and Quality Assurance Criteria Different disciplines have very different criteria for assessing research excellence and quality,
which are rooted in the history of each field, their subject matter and methods, and their role in
society. This diversity is most pronounced between the humanities and qualitative social
sciences, and more quantitative fields in the social and natural sciences. At the same time,
attitudes towards openness vary enormously also within disciplines and epistemic communities
(Fecher et al 2015, Levin et al 2016). Different branches of biology, for instance, have
completely different attitudes to competition and sharing, with fields such as animal biology
remaining fiercely competitive while genomics and model organism biology endorsed what they
call a “share and survive” ethos (Rhee 2004). Fields more heavily invested in commercial
partnerships and applications (such as pharmacology and biomedicine) also tend to share less
than fields focused primarily on fundamental research.
Perhaps the most variable, and yet the most important methodological concern for researchers
looking to share results is the interpretation of research quality, controls and validation criteria.
These are closely tuned to the specific methods and materials used in each research group, and
it is therefore no surprise that there is such high variability in the criteria used to assess the
reliability of outputs. It is impossible to establish detailed benchmarks for what counts as “good
quality” data and metadata for each research fields and objectives, as these depend on the
9
nature of the questions, phenomena and methods at hand. This has significant consequences for
the implementation of Open Science.
Worries around quality assurance regularly figure among the highest source of concern for
natural science researchers in relation to Open Science (Borgman 2012, EU Survey on Science
2.0, Digital Science Report 2016, Fecher et al 2016). For instance, researchers are concerned
about the reliability of data shared via digital repositories, particularly given that many such
data have not undergone peer review. By the same tokens, many researchers are reluctant to
share data and materials that they themselves consider to be of lower quality. In the humanities
and social sciences, similar concerns are raised about quality controls relating to the publication
of articles and monographs, since the reputation and reviewing mechanisms used by particular
publishers plays a significant role in quality assurance, and Open Access publications are still
perceived as of being of lower quality (Laakso et al 2011). The extent to which researchers trust
each other’s work matters enormously to the success of large research projects and to the
efficient repurposing of available research outputs. Fostering such trust requires relevant
training and skills, and needs to be underpinned by credible mechanisms for quality
assessment.
2.3 Costs, Accountabilities and Long-Term Sustainability
The set-up of infrastructures, assessment mechanisms and publication venues that serve Open
Science is not quick nor cheap, and needs to be implemented in a coordinated way by a large set
of stakeholders. Those benefitting from Open Science tools range from researchers to research
institutions, funders, industrial partners and society at large, thus making it very hard to settle
on how to divide up labour and who should take financial responsibility for supporting their
development and maintenance. To make matters even more complex, Open Science is an
international phenomenon that requires coordination among nation states and different
systems of research evaluation, funding and support. Given the complexities of allocating
accountabilities and coordinating roles in this complex landscape, the business models and
long-term funding strategies underpinning many Open Science initiatives remain unclear. Major
uncertainty is linked to the short-term nature of current funding of archives and data
infrastructures and the choice of who should be responsible for supporting, developing and
updating these tools in the future, a particularly burdensome task given the constant shifts in
related technology and knowledge base (Nelson 2009, Bastow and Leonelli 2010, Royal Society
2012, Eschenfelder and Shankar 2016). Many researchers view this uncertainty as a source of
risk for the integrity and accessibility of the material placed in digital repositories, and thus as a
major disincentive to engage in Open Science practices (Tenopir 2011 and 2015, Borgman 2012,
Leonelli et al 2016, Digital Science 2016). Trust in the reliability of archival services for pre-
prints and data is low, for instance, particularly when they are funded by individual institutions
(who may decide to interrupt the service) or short-term projects (after which further funding
may not be found).
2.4 Skills and Training
There is considerable confusion among researchers about what openness means in practice,
what options are available to implement it, what is legal, what is recommended by funders,
learned societies, publishers, research institutions and governmental bodies, and whether such
recommendations are compatible with each other. As illustrated by several recent surveys and
highlighted by MLE participants, many researchers know very little about the variety of formats
10
and practices of Open Science, how to choose and implement them, and what they could
contribute to their work. A survey carried out for a recent report of the EU Working Group on
Education and Skills under Open Science (2017) highlights that while European researchers
have some understanding of Open Access, Open Source and Open Data activities, they are less
aware of Open Peer Review, Open Education, Citizen Science and Open Notebooks. Over 40% of
respondents reported being unaware of any of the current international initiatives and policies
supporting Open Science, such as OpenAIRE, the European Open Science Cloud and the OA
Button (see figure 3).1
Figure 3. Researchers’ awareness of open access activities and related mandates, as measured
through a survey conducted in 2016-2017 by the EU Working Group on Education and Skills under
Open Science (2017). N refers to the number of respondents to the survey. In the first figure,
researchers were asked to rate their level of awareness of the key Open Science components. In the
second figure, researchers were asked whether they were acquainted with specific Open Science
projects and tools.
The Report of concludes that “researchers are largely unaware of Open Science policies and
practices, require more skills training and support to practise Open Science, and need to be
incentivised to begin, and continue to practise, Open Science.” This is striking given the complex
and sophisticated practices and skills involved in disseminating and reproducing data, software,
techniques, methods, protocols and materials, vis-à-vis research articles. These practices are
typically different depending on the field of study, and need to be aligned with existing methods
and instruments used by researchers in their everyday work. At the same time, minimal
standards need to be agreed and implemented internationally to guarantee interoperability and
1 OpenAIRE is a European project aiming to promote Open Science practices and related tools (https://www.openaire.eu/); the European Open Science Cloud is the infrastructure coordinated at European level to serve open research and innovation (https://ec.europa.eu/research/openscience/index.cfm?pg=open-science-cloud); and the OA Button, an initiative supporting legal access to publications and data beyond paywalls (https://openaccessbutton.org/).
effective linkages among local resources (Sansone et al 2012).2 See for example figure 4, which
lists the many types of tools required to successfully manage and openly share data, and the
variety of resources available to researchers in specific fields (in this case, plant science) to cope
with such demands. To deal with this new source of complexity, researchers are in severe need
of Open Science training and expert assistance, a point which will be expanded upon in what
follows.
Table 1. General Tools for Data Management (source: Leonelli et al 2017).
2.5 Intellectual Property Concerns While evidence exists of the advantages that Open Science could yield for translational research
and commercial applications, the push towards free sharing of research outputs is not easily
aligned with the multiple intellectual property regimes currently underpinning the production
and application of knowledge in both public and private research organisations (Evans 2010,
Haeussler 2011, Holmes 2016, Minssen and Pierce 2017). Researchers are generally confused
about which modes of intellectual property apply to their outputs (Borgman 2012, Levin et al
2016). The confusion is compounded by the multiple layers of accountability to which any
individual researcher is exposed, which range from accountability to research groups, home
institutions, research sponsors, research partners, governmental agencies and assessment
exercises. Given the international nature of research and funding networks, any one project
2 One of the concrete projects related to data sharing is GO FAIR, aiming to standardize and make available a tool box for researchers how to make data ready for sharing (https://www.dtls.nl/fair-data/go-fair/).
(and related outputs) can also be liable to various national legislations, and thus to different and
sometimes contradictory licensing terms. In its Open Science Agenda, the European Commission
has pledged to address the issue through “better taking into account of public benefits, social
interest and the situation of academics and innovative industries when reviewing the European
copyright legislation” (Directorate-General for Research and Innovation, 2016).
Another set of concerns with intellectual property emerges in relation to publication of written
outputs. First, there are issues with adjudicating authorship claims, and distinguishing authors
from other types of contributors. This is particularly difficult in cases where much of the
intellectual work underpinning a publication has been done by emerging professionals such as
data scientists and data curators, whose contributions are not always adequately recognised; or
where published claims are heavily based on data, techniques or materials openly shared by
other research groups, thus prompting questions around whether acknowledgement should
come in the form of authorship, citation, or in other ways. Second, there are issues with
understanding licensing conditions associated to publication. Despite the availability of clear
and legible licensing models (such as the Creative Commons), many publishing companies rely
on customised contracts whose implications for Open Science dissemination are not
straightforward for authors to understand, and which authors do not have access to at the
moment of choosing their publication venue. In an attempt to address this issue, the Working
Group on Open Science Publishing of the Open Science Policy Platform has recently
recommended that “it is essential that sufficient information about OA requirements, mandates
and modalities of compliance are made available by publishing outlets, in a clear and intelligible
manner, to researchers and research institutions needing to take decisions around how to
disseminate their outputs” (2017).
2.6 Semantic Ambiguity Openness can mean different things to different stakeholders, and is associated to ideas as
different as “free of license”, “free of ownership”, “under CC-BY license”, “common good”, “good
enough to share”, “unrestricted access and/or use”, or “accessible without payment”, to name
but a few existing interpretations (Grubb and Easterbrook 2011). This is true even of
researchers working within the same institution and discipline (Levin et al 2016). Alongside the
above-mentioned confusion surrounding intellectual property regimes of relevance to Open
Science, most researchers also lack understanding of whether and how Open Science policies
align with existing metrics of excellence and impact deployed by research institutions, research
funders and governmental agencies. In the UK, for instance, impact is often associated to public-
private research partnerships operating under closed regimes of intellectual property, an
arrangement strongly encouraged both by individual universities and by the government
(Research Council UK 2014). At the same time, the UK Higher Education Authority has strongly
endorsed Open Science guidelines, including mandatory Open Access for all publicly funded
research and an in principle commitment to Open Data (HEFCE 2014). As documented by
historians of science, secrecy has long played a strategic role in various branches of research,
particularly when dealing with projects of considerable military, commercial or ethical
sensitivity such as the development of widely applicable surveillance systems or techniques that
could be used to create weapons (e.g. Royal Society 2012, Balmer 2015, Rappert 2007). This
makes decisions around what to keep close and make open particularly delicate, and provides a
disincentive to researchers and institutions who are not sure about how to assess the
advantages of Open Science and how to apply Open Science guidelines to their specific case.
13
2.7 Ethical and Societal Concerns
It is important to think broadly about who is a relevant stakeholder for open science, and how
open science practices can help address the tensions characterizing the role and perception of
scientific expertise within Western media and society. Students of science and technology have
lamented the large discrepancy between evaluation criteria used to assess scientific excellence
and the assessment of the social, cultural and economic functions of science (a phenomenon
that Paul Wouters has called “e/valuation gap”, Wouters 2017).
Many ethical issues are emerging specifically in relation to Open Science implementation. For
instance, there are concerns with privacy and ownership of data, which are particularly evident
in the case of personal data used in biomedical or social science research (Nuffield Council of
Bioethics 2015, Mittelstadt and Floridi 2016, Prainsack and Buyx 2016). The European
Commission has listed the relation between data protection regulations and open data policies
as an urgent issue to tackle within the Open Science Agenda (Directorate-General for Research
and Innovation, 2016). There are also ethical and social concerns around big data mining and
the social functions and uses of surveillance technologies and artificial intelligence built on
access to publicly available research data. A recent report by the Royal Society and the British
Academy has highlighted the importance of data governance structures supporting human
flourishing, and the relation between such governance and the implementation of Open Science
policies (Royal Society and British Academy 2017).
2.8 High Resource Bias
Open Science implementation tends to focus on high-resource, internationally well-recognised
research environments. For example, open data repositories mostly display outputs of English-
speaking labs in prominent research institutions, which have: funds to curate contents and
participate in the adoption and development of expensive equipment and software; visibility to
determine dissemination formats/procedures; and resources and confidence to build on data
donated by others (Science International 2015, Bezuidenhout et al 2017). Low-resource
research environments, defined as having access to fewer technological, organizational and
human resources with respect to their needs, tend to invest much less effort in developing and
implementing Open Science tools. Furthermore, few provisions have hitherto been made for
situation of systematic disadvantage (where researchers lack infrastructures and online access)
and vulnerability (where access to a particular source of type of materials or data is what gives
competitive edge, as in archaeology, botany), resulting in researchers fearing that sharing their
results will undermine rather than increase their international credibility, and increase their
disadvantage with respect to better resourced colleagues (Bezuidenout et al 2017). There is a
risk that the implementation of Open Science practices will result in an increased digital divide
between research conducted in well-established research institutions and research carried out
by small institutions, citizen scientists or small enterprises.
3 Incentives and Rewards for Open Science: Researchers A recent review of the existing research on the ways in open research activities may benefit
researchers concludes that “open research is associated with increases in citations, media
attention, potential collaborators, job opportunities, and funding opportunities” (Morey et al
2016). However, the vast majority of researchers are subject to requirements made by their
peers, disciplinary communities, home institutions and funding bodies, which act as barriers to
14
the adoption and implementation of Open Science behaviors. Furthermore, the highly
competitive nature of contemporary academia makes many researchers nervous about sharing
their data and materials, for fear of being “scooped” and losing their competitive advantage.
Shifting these perceptions, moving away from the current “publish or perish” culture and
making sure that researchers are rewarded for Open Science behaviours are among the most
important goals of Open Science policies, as a truly Open Science landscape can only emerge
through the participation and input of those who carry out research activities (Neylon 2012).
In position statements such as the Bratislava Declaration of Young Researchers (2016) and the
Position Statements on Open Access and Data of the European and Global Young Academies
(2016), junior and mid-career European researchers have signaled their willingness to engage
in Open Science activities. They have also pointed out the obstacles that stand in their way,
which include publication-oriented systems of assessment as well as conservatism among
senior members of their research communities. Taking this feedback on board, the EU Report of
the Working Group on Skills for Open Science proposed a distinction between the roles played
by researchers at different career stages in Open Science implementation. Researchers at early
career and employment stages are encouraged to recognize “the need for [Open Science] skills
as part of their learning process as well as the need to link to recognition/rewards and the
impact of acquiring and using OS skills”. Senior researchers in established positions are invited
to recognise “the need to take leadership and ensure that their mentees acquire the skills as
well as the need to demonstrate to them the positive effects of sharing data and
information.” This section discusses some of the main incentives and rewards that could be used
to foster engagement in Open Science activities by researchers at all levels of career
development, taking into account the role played by senior researchers in research evaluation
and the allocation of rewards.
3.1 Assessment and Promotion Criteria
The survey carried out by the EU Report on Rewards (2017) made it clear that “evaluation
criteria are still most often based on scholarly publications and their number is the most
widespread indicator of performance. Other criteria such as measuring the impact of the
scientific production on the academic community (citations, h index, etc.) are much less
assessed and the least used are the purely qualitative evaluations that require critical reading of
the publications and assessment of other achievements than scientific production such as
openness, sharing, support to the community, team spirit, participation in citizen science and
information of the lay public.” In contrast to this, the DORA declaration, the Metric Tide report,
the Leiden Manifesto and the EU Expert Report on Alternative Research Metrics have
recommended the use of multiple indicators, including both quantitative and qualitative
metrics, to assess and support the multiplicity of career paths across the research system. This
involves a shift in citation cultures, and a move away from prestige-led assessment grounded on
the reputation of the publication venue in which articles appear or the research location where
the work is being conducted.
It is crucial to avoid assessment mechanisms that regard Open Access publishing and Open
Science practices such as data sharing as “academic career suicide”. Open Science behaviours
are time-consuming and resource-intensive (Tenopir et al 2011 and 2015, Borgman 2012,
Acord and Hartley 2013, Wallis et al 2013, Ankeny and Leonelli 2015), and need to be rewarded
both by funding bodies and by research institutions (Fecher et al 2015). Open Access
publication needs to be encouraged by research communities and learned societies (thus
15
countering the widespread belief that Open Access publishing is damaging to the credibility of
researchers and constitutes an indicator of low research quality). Furthermore, it is crucial that
assessment encompasses all levels of seniority, and take account of the diverse backgrounds
and competencies of individual researchers. Because of such complexity and sensitivity to
context, the EU Expert Group Report on next-generation metrics concluded that: “Evaluating a
researcher cannot be reduced to a number because their merits, achievements, usefulness are a
complex set of different variables, impossible to be summarised by a single figure” (Wilsdon et
al. 2017).
The Open Science Career Assessment Matrix (OS-CAM) has been proposed by the EU Working
Group on Open Science Rewards as a platform for a fair and transparent system of assessment
for all researchers, regardless of where they come from, work or publish (figure 5).3
Table 2. Open Science Career Assessment Matrix scheme from EU High Level Report (September 2017; source: ADD URL)
3 This work builds on the outputs of ACUMEN, a large-scale collaboration across nine European research institutes to investigate alternative research assessment practices (http://research-acumen.eu/).
16
Implementing the OS-CAM system aims to acknowledge the invisible labour carried out as part
of research, particularly the efforts involved in teamwork, management, public engagement and
professional service such as refereeing, teaching and data management. This system
encompasses researchers at all career levels, ranging from early career to established senior
researchers. It benefits particularly young researchers, since it encourages them to acquire
professional skills and experience in several aspects of research work, and not to sacrifice
research quality and public engagement activities in the name of producing more publications.
Thus, the system is geared to valuing the content and reach of research activities. It also has the
potential to facilitate the movement of researchers between academia and other sectors, as
early career researchers are explicitly assessed in the exercise of skills that are valued also
outside academia. The EU Report notes that the OS-CAM is compatible with the European
Charter for Researchers and Code of Conduct for their Recruitment (which was published in
2005) and can also be used as a reference point to take more explicit account of the roles,
responsibilities and entitlements of researchers, employers and funders of researchers.
3.2 Training on OA Guidelines and Implementation Tools
Research institutions should provide systematic training to scientific researchers on practices
such as self-archiving, on different formats of data sharing and its advantages and potential
downsides, and on how to make information intelligible for specific user groups. Moreover, as
an increasing number of institutions run research or teaching initiatives around ‘big data’, it is
important that these are not narrowly focused on technical skills such as predictive analytics or
data cleaning, but they deal with big data comprehensively, including its societal, ethical,
philosophical and regulatory aspects. This will lead to higher levels of awareness of the
potential benefits and drawbacks of Open Science among scientific researchers as well as among
wider publics, which in turn facilitates more meaningful and targeted support of Open Science.
The Report of the EU Working Group on Skills for Open Science identifies four broad categories
of Open Science skills and expertise, which include (1) those necessary for open access
17
publishing, such as how to choose a publishing venue and related licensing; (2) those that
concern data sharing and re-use, including standards for the formatting and curation of data
and metadata; (3) those that ground participation in and beyond one’s scholarly community,
such as needed to manage research to preserve its integrity and abide to the law (a difficult
challenge given the potential tensions between Open Data directives and Data Protection
legislations); and (4) those needed to engage the general public in research planning and
4.6 Enhancing International Visibility and Reputation
Given the evidence that openly shared research garners more visibility and citations (Morey et
al 2016) and the increasing tendency for research assessment to take Open Science behaviors
into account, Open Science activities can be expected to foster the performance of research
institutions within Research and University Rankings. More broadly and more significantly,
given the current dependence of such rankings on traditional assessments such as impact
factors, Open Science activities foster more collaboration within and beyond each institution,
and create a rich landscape of outputs which research institutions can claim as their own. They
also enhance the ability of institutions to track who adopts and repurposes the knowledge and
tools that they create, and to which effect.
Furthermore, the implementation of OS-CAM assessment can enhance the reputation of
research institutions among prospective employee, thus helping to attract and retain talent, and
increasing the international visibility of research efforts and investments.
5 Incentives and Rewards for Open Science: National Governments While it is widely acknowledged that Open Science initiatives and practices need to emerge
“bottom-up”, thus aligning with the experiences and needs of researchers, top-down legislation
also serves an important role as a framework within which incentives can be positioned and
motivated. National governments need to take responsibility for fostering Open Science
activities and making it as easy as possible for researchers and research institutions to
implement the required changes. From the discussions held during the course of the MLE, and
the responses provided by delegates (see section 6), it is clear that researchers and research
institutions are widely expecting governments to explicitly endorse Open Science policy at the
national level, and provide at least some resources and funding to help implement and
coordinate its implementation across all relevant stakeholders.
5.1 Improving Transparency and External Accountability
A survey among participants to this MLE (details below) has revealed a strong interest in Open
Science activities and policies from several member states, but a lack of monitoring and
enforcement for Open Science guidelines.
Investments made towards facilitating Open Science activities are likely to yield high returns in
terms of a more transparent research labour market, better and more reliable research outputs,
and more effective translation of fundamental research into innovative solutions to societal
challenges.
The role of government in providing incentives for Open Science is particularly significant in
member states such as Moldova, Italy and Slovenia where universities have a narrow degree of
autonomy, and most decisions concerning evaluation and resources are made at governmental
level. In such cases, the relevant ministries bear a greater responsibility for setting a clear and
rigorous agenda to promote Open Science activities.
The Netherlands, Finland, the UK and Moldova are providing useful models for how Open
Science can be incentivized by the national government. For instance, the Netherlands
implemented a National Plan Open Science which is responsible for gathering and implementing
relevant training. The UK has made Open Access archiving into a mandatory condition for all
26
research output considered for the Research Excellence Framework, its national research
assessment exercise (HEFCE 2014). In Moldova, journals are only recognized by governmental
assessment if they have an Open Access status, a measure widely seen as very effective in
incentivizing OA. Other countries have started to build national infrastructure to support the
move to Open Access publishing. Croatia has established a national platform of almost 400
Diamond Open Access journals, subsidized by the ministry of education, and created a Croatian
scientific bibliography CRSOSBI (https://bib.irb.hr/) that contains more than 450.000
bibliographic records, allowing scientists to easily archive full-text articles in open access. In
Slovenia the COBISS/SciMet (http://scimet.izum.si/) collects research outputs centrally and the
personal bibliographies of researchers are stored and visualized by the same service, thus
enabling researchers to monitor the performance of their publications by using alternative
metrics and more traditional metrics.
5.2 Promoting Social Engagement and Responsible Innovation
Open science practices promote effective translation and engagement between science, policy
and society. They foster better circulation of researchers between academia and industry;
provide the opportunity to address the ‘citation gap’ separating blue-skies research and
applications; and incentivize the involvement of a broad range of stakeholders in designing and
evaluating research. Crucially, Open Science activities promote trust and reciprocity of
engagement among private and public stakeholders, ensuring that more transparency is
attached to data sharing and analysis for the purposes of innovation, and responsible and
engaged practices are employed for the development of new technology (note that for this to
work, governments need to require private funders to follow Open Science guidelines in the
same way as publicly funded bodies).
This also means better and more efficient documentation of impact and value production,
illustrating the returns that investment in research can yield at the national level. An example of
the successful use of this strategy is provided by the European Bioinformatics Institute (EBI),
whose open science activities have been assessed by an independent consultancy with the aim
of estimating the impact that the data and services freely provided by EBI has had on its users.
The report concluded that “Users reported that EMBL-EBI data and services made their
research significantly more efficient. This benefit to users and their funders is estimated, at a
minimum, to be worth £1 billion per annum worldwide - equivalent to more than 20 times the
direct operational cost. In terms of Return on Investment in R&D: during the last year the use of
EMBL-EBI services contributed to the wider realisation of future research impacts
conservatively estimated to be worth some £920 million annually, or £6.9 billion over 30 years
in net present value” (Beagrie et al 2016). The report makes it clear that such a high return on
investment is largely due to the open access policies enforced by the EBI; and that open
research practices also made it possible to track and document the impact of Open Science
policy.
While governmental policy is crucial to the implementation of Open Science, it is imperative that
such implementation is accompanied by public debate and ‘bottom-up’ support, thus engaging
with stakeholders from all sectors in illustrating the advantages of open research practices and
garnering feedback as to its implications for specific users. The Royal Society & British Academy
2017 Report on Data Governance (2017) warns against mismanaging public perceptions and
recommends multiple opportunities for the public to engage in open research and participate in
shaping both the policies and the specific tools implemented as part of the Open Science agenda.
As part of this mandate, governmental agencies, funding bodies and research institutions can
also play a pivotal role in shaping publishers’ attitudes and services relating to Open Science.
5.3 Enhancing International Relations
Open Science serves as a motor for economic growth, public engagement and social prosperity
across Europe, taking advantage of the common market and of the ERA agreements for
international collaboration. Open Science agreements and work towards common standards
also has the potential to foster networking and communication among member states as well as
with global partners, thus promoting diplomatic relations across institutions and countries, and
increasing the global visibility of European achievements in research and innovation. By
working together, European member states have an opportunity to lead the world in the
development of best practice for Open Science, improve the channels through which
researchers can help address global challenges, and provide future generations with ways to
actively shape and engage with scientific and technological innovation.
The role played by Open Science in fostering science diplomacy brings the science policy agenda
at the regional, national and European levels in close alignment with foreign policy goals and
procedures. Commissioner Moedas highlighted the link between the Open Science Agenda
promoted by his Directorate and the overarching diplomatic mission of the European Union as
follows: “for today’s EU, European research is an important resource for exercising its collective
responsibility in a spirit of international solidarity, as part of its efforts to work with
international partners to solve common and complex global challenges. EU science diplomacy is
therefore becoming an increasingly visible part of the union’s foreign policy, one taken into
account more often and with deeper commitment than ever before (Moedas 2016). Operating in
this way requires each government to establish clear points of contact, communication channels
and venues to debate Open Science implementation at the national level, and engage with
international debates in this domain.
6 Summary The development and implementation of incentives and rewards for Open Science practices
encompasses several different measures, whose application depends on the stakeholders
involved as well as the specific field, community and location in question. Given the highly
international nature of research networks, international coordination is crucial to the effective
implementation of comparable measures. At the same time, each member state, research
funder and research performing organization needs to review the extent to which specific
incentives will work in its specific context, and adapt the requirements discussed in this report
accordingly. Table 8 provides a synoptic view of the approaches to incentivizing and rewarding
Open Science activities that have been identified and discussed in this document, the conditions
under which they are most likely to operate effectively, their potential advantages and
disadvantages, and the main stakeholders responsible for their implementation. In the next
challenge paper, specific examples, strategies and roadmaps for the successful implementation
of Open Science practices will be considered and discussed.
Table 8. Synoptic view of the approaches to incentivizing and rewarding Open Science activities
that have been discussed in this document.
OS-CAM Research
OS Training
Shifts in Citation &
Long-Term Sustainability
Open Science Role Models
Responsible Innovation
Transparency &
International Coordination &
28
Evaluation Provision & Education Resources
Authorship & Public Engagement
Accountability Science Diplomacy
Required conditions
Overhaul of evaluation procedures at research institutions & funding bodies
Resources and personnel to provide training locally and nationally
Overhaul of evaluation procedures and publishing formats
Complex coordination among stakeholders and long-term commitment
Establishment of criteria for successful open science within each field; buy-in from learned societies and science academies
Rewards for social interaction and non-traditional outputs; co-design of research with relevant stakeholders
Systems for tracking, visualizing and discussing the organization, outputs and funding of research.
Clear points of contact and communication channels/venues to debate Open Science implementation.
Pros Most important set of incentives and rewards for researchers
Enables researchers to practice Open Science effectively; produces innovative education tools
Recognition of currently invisible efforts to support Open Science
Crucial incentive for researchers; ensures the long-term fruitfulness of current investments
Exemplifying advantages of Open Science, and ways to successfully implement it; enhance international status of research institutions. Relatively inexpensive.
Embedding of research in society, towards devising ethical and responsible solutions to global challenges.
Improved documentation and scrutiny of research processes and resources. Improved reproducibility of results and evaluation of accountabilities for given outcomes.
Enhanced international visibility, networking and diplomatic relations across institutions and nation states.
Cons Time-intensive evaluation procedures
Investment in training provision and related staff; needs inclusion in researchers workload
Requires new policies tailored to each publication venue
Complex coordination among stakeholders and log-term financial support
Mobilize learned societies and science academies to actively promote Open Science.
Risk of less investment in fundamental research. Increased accountability for all research activities (including privately funded).
Increased administration and more investment in data analysis and qualitative assessments.
Increased national research budgets; need for coordination between science and foreign policy.
Who implements this? (Note: researchers are always involved)
Research institutions, funding bodies, researchers
Funding bodies, libraries
Research institutions, funding bodies, editors, publishers
EU, National governments, research institutions, libraries
National governments, funding bodies, learned societies
Funding bodies, research institutions, EU, National governments
Funding bodies, research institutions, EU, National governments
National governments, policy-makers, research managers
7 MLE Topic 3: Incentives and Rewards for Open Science The goal with the MLE is to learn from each other’s experiences, both from good practices and
failures. To gather background information about whether and how Open Science activities are
being incentivized and rewarded by member states two approaches were taken: discussions
about open science during the previous MLE meetings, and a questionnaire sent to participants
and specifically tailored for this topic. The key points from both approaches are summarized
below, followed by a more detailed overview of the responses to the questionnaire.
7.1 Key points from the MLE discussions and questionnaire responses
From the discussions during the MLE meetings it is clear that a great diversity in the approaches
and the degree of adoption of open science exist between the member states. Depending on the
national context, governments, individual funders and research institutions take up different
levels of responsibility and display different levels of commitment towards the Open Science
29
agenda. There are however recurrent concerns that have arisen both in discussion and in
response to the questionnaire:
• Many governmental agencies, research institutions and funders have put incentives in
place for the Open Access archival of research articles, research institutions and
funders, and indeed the vast majority of the “success stories” included as answers to the
questionnaire provided to delegates (see below) concern Open Access initiatives.
• Very few member states have provisions concerning research components other than
publications (such as Open Data or Education).
• The incorporation of Open Science goals in research evaluation and assessment lags
very far behind, with a majority of countries relying heavily on quantitative
assessments of publications including impact factors. This is due to the cost-
effectiveness and the simplicity of quantitative assessment, and yet it runs counter to all
the consensus among experts that multiple indicators, including both qualitative and
quantitative measures, are the most reliable way to evaluate research.
• Even for the member states that have accepted to be part of the MLE, it is not always
clear who holds responsibility for discussing and implementing Open Science policies at
the national level. This makes it difficult to foster decision-making on how to implement
Open Science mandates, particularly given (1) the transformative nature of the cultural,
evaluative and administrative changes required and (2) the amount of investment
needed to support related infrastructures and services. There is an urgent need for
opportunities and venues to deliberate on Open Science implementation and
investment at the national level.
• Given the international nature of Open Science provisions and standards, member
states expect much of the support, coordination and infrastructural facilities required
for Open Science implementation to come from ERA and related European agencies.
• Researchers and research organisations (including both learned societies and research
performing institutions) are viewed as crucial participants in any decision-making
process mapping future Open Science implementation and related training, so as to
ensure successful and effective uptake by the research community.
• However, there is widespread concern around the conservatism characterizing
particularly senior academic circles, which are typically responsible for research
evaluation in the form of peer review of funding proposals and publications. It is
imperative to provide training and incentives for senior academics to value Open
Science activities and support the career and outputs of junior researchers who operate
in this way.
• The transition towards assessment procedures in line with Open Science is likely to
yield temporary set-backs and difficulties for some research initiatives and
communities, particularly given the different types and levels of uptake of Open Science
ethos among research fields. The transition thus need to be closely monitored and
documented, with clear points of contact within each member state to address and
resolve emerging challenges.
7.2 Detailed overview of questionnaire responses The questionnaire included thirteen open-ended questions of relevance to this challenge paper,
which were grouped under three themes: the current situation of Open Science implementation
30
in each country (four questions), issues specifically of relevance to incentives and rewards (four
questions) and perspectives on future developments in this area (five questions). The questions
and an edited version of the responses are provided below. Sixteen MLE participants
contributed responses to the questionnaire, providing representation for nine member states
(Belgium, Switzerland, Armenia, Moldova, Sweden, Austria, Croatia, Lithuania and Slovenia).
Most respondents did not answer all the questions, or chose to provide responses that
addressed more than one question. Whenever detailed examples of national initiatives were
provided, they are reported verbatim. More discursive responses have been collated and
summarized.
Current situation in your country:
• What Open Science requirements or provisions, if any, have been adopted by the main research funders in your country?
SLOVENIA: Resolution on Research and Innovation Strategy of Slovenia 2011-2020 (http://www.mizs.gov.si/fileadmin/mizs.gov.si/pageuploads/Znanost/doc/Strategije/01.06._RISSdz_ENG.pdf), Chapter 4.5, Action 55 (Preparation of action plan for free access to data from publicly funded research); National strategy of open access to scientific publications and research data in Slovenia 2015-2020 (http://www.mizs.gov.si/fileadmin/mizs.gov.si/pageuploads/Znanost/doc/Zakonodaja/Strategije/National_strategy_for_open_access_21._9._2015.pdf ); Slovenian strategy for strengthening the European Research Area 2016-2020 (ERA Roadmap) (http://www.mizs.gov.si/fileadmin/mizs.gov.si/pageuploads/Znanost/doc/Zakonodaja/Strategije/SI_ERA_Roadmap.pdf ); Research Infrastructure Roadmap 2011-2020, Revision 2016 (http://www.mizs.gov.si/fileadmin/mizs.gov.si/pageuploads/Znanost/doc/Strategije/NRRI_2016_ENG.pdf ). Financial support to universities and research institutes in performing related projects and setting up required infrastructure SWITZERLAND: Open Access policy http://www.snf.ch/en/theSNSF/research-policies/open-access/
SWEDEN: Three out of the four largest Swedish funders have mandates on open access www.vr.se www.formas.se www.forte.se but also some foundations: www.rj.se www.ostersjostiftelsen.se www.wallenberg.com.
BELGIUM: Federal government funded researchers are invited to deposit in a repository and to publish in OA. An elaborate OA policy, including 6/12 month embargoes, quality exigencies, caps on APC and the Liège Model has been drafted but has had no success in being accepted at a higher level. For OD no action has been taken yet, as agreed in consultation between the Federal, Flemish and French speaking administrations. Universitites were the first to adopt OA mandates, following the trend launched by the Université de Liège and its IDOA approach (Immediate Deposit Optional Access) : https://www.ulg.ac.be/cms/c_17700/en/open-access FRS-FNRS has since adopted a similar mandate : http://www.fnrs.be/docs/Reglement_OPEN_ACCESS_EN.pdf A decree is currently in preparation to extend this kind of mandate to all public funding within the Wallonia-Brussels Federation : http://marcourt.wallonie.be/le-libre-acces-aux-publications-scientifiques-en-federation/ CROATIA: Croatian Law on Science and Higher Education mandates that all higher education theses should be available in open access in corresponding university library repository. To my knowledge, the Croatian Science Foundation has so far not adopted any Open Science requirements or provisions. LITUANIA: the main outline for the policy concerning the Open Access is set up in the Article 51
of the Law on Higher Education and Research according which „ the results of all research works carried out in state higher education and research institutions must be communicated to the public (in the Internet or in any other way), to the extent this kind of communication is in compliance with the legal acts regulating the protection of intellectual property, commercial or State secrets.“ The Resolution regarding the approval of The Guidelines on Open Access to Scientific Publications and Data adopted by Research Council of Lithuania (The document http://www.lmt.lt/lt/nuorodos/atvirosios_prieigos_dokumentai.html click pdf 'Lietuvos mokslo tarybos nutarimas dėl atvirosios prieigos prie mokslo publikacijų ir duomenų gairių patvirtinimo, 2016 m. vasaris (anglų kalba)' AUSTRIA: FWF’s Open Access policy: https://www.fwf.ac.at/en/research-funding/open-access-policy/ - Adapting Science Europe’s Principles on Open Access to Research Publications - Signatory of the Berlin Declaration (2003): https://openaccess.mpg.de/Berlin-Declaration - Signatory of OA2020 (2016) - Expression of Interest in the Large-scale Implementation of Open Access to Scholarly Journals https://oa2020.org/mission/ - Supporter of the Open Access recommendations by the Open Access Network Austria (OANA): https://zenodo.org/record/51799#.WQRltR5Qh2M
• What Open Science requirements or provisions, if any, have been adopted by research institutions and governmental agencies in your country?
SLOVENIA: Portal OpenAccess Slovenia ( http://www.openaccess.si/ ), built by Slovenian universities and research institutions (supported by the ministry and research agency); Portal Open Science Slovenia ( http://openscience.si ), built by Slovenian universities (supported by the ministry and research agency); Open access repositories are build up at several universities and research institutes (University of Ljubljana, University of Maribor, University of Primorska, University of Nova Gorica, digital repositories of the Slovenian public research institutes: refer to http://openscience.si); Active participation in OpenAire (https://www.openaire.eu/ ). Formal requirements regarding open access publishing or other open science requirements are not yet adopted by the universities or research institutions. BELGIUM: Publications are required to be deposited in a compatible OA repository with only one research instition at the federal level: The Belgian Health Care Knowledge Center (KCE). Universitites were the first to adopt OA mandates, following the trend launched by the Université de Liège and its IDOA approach (Immediate Deposit Optional Access) : https://www.ulg.ac.be/cms/c_17700/en/open-access MOLDOVA: http://idsi.md/en/academica - probably, the academica network is a sample of requirements adopted by the Academy of Sciences of Moldova SWITZERLAND: Switzerland has set in place specific regulations that mandate research funding and performing institutions to ensure public access to research results, in accordance with the legal provisions. The Federal Act on Promotion of Research and Innovation (RIPA) sets the legal basis for the transfer of knowledge and technology from publicly funded institutions and for the open access to research data and results. Article 50 of the Federal Act on Promotion of Research and Innovation requires “that the results of research are available to the public in accordance with the legal provisions”. In 2015, the SERI mandated local universities and the SNSF to develop a national strategy on Open access and to analyse its financial flow (see swissuniversties and SNSF answer for further information). At the organisational level there are specialised repositories where universities and other research organisations have made available their institutional policies on open access, for example, the Registry of Open Access Repositories Mandatory Archiving Policies (ROARMAP) and the OpenAire’s website. In moving forward, a national study has identified the need to improve infrastructure for research data, while an ongoing study is exploring different measures to promote Open Access and Research Data existing at Swiss universities. The latter study will inform a system “to access, process and
storage science-related digital content” by 2020 (see swissuniversities’ answer for further information). At governmental level, thus indirectly related to Open Science, the federal administration is legally required to open government. According to the Freedom of Information Act from 2004, everyone must in principle be given access to any information or document from the federal administration as long as the privacy of other individuals or the national security are not affected. Currently, different universities follow various policies concerning open access. Some universities require their researchers - as long as there is no legal obstacle - to deposit their issued publications on an institutional repository. Other institutions are less strict but recommend it. Some examples of institutional policies: http://www.unibe.ch/universitaet/dienstleistungen/universitaetsbibliothek/service/elektronisch_publizieren/open_access/open_access_policy_der_universitaet_bern/index_ger.html http://www.library.ethz.ch/ms/Open-Access-an-der-ETH-Zuerich/Open-Access-Policy-der-ETH-Zuerich http://www.ub.unibas.ch/ub-hauptbibliothek/dienstleistungen/publizieren/open-access/open-access-policy/. Since 2017, Switzerland has a national open – access strategy. Its vision is that by 2024,” all scholarly publication activity in Switzerland should be OA, all scholarly publications funded by public money must be freely accessible on the internet. The OA landscape will consist of a mix of OA models.”https://www.swissuniversities.ch/fileadmin/swissuniversities/Dokumente/Hochschulpolitik/Open_Access/P06_7.01-01_Open_Access_strategy_EN.pdf CROATIA: Croatian Law on science and higher education mandates that all higher education theses should be available in open access in corresponding university library repository. Similar mandate on national level for other types of publications doesn’t exist, but Rudjer Boskovic Institute has declared the first Croatian institutional self-archiving mandate which mandates open access for all publications according to publisher’s copyright (Decision on the obligation to store scientific, professional and popular papers at the Ruđer Bošković Institute Repository - FULIR). The mandate was followed by the Faculty of Mechanical Engineering and Naval Architecture at the University of Zagreb and the Physics Department of the Faculty of Science of the University of Zagreb. The University Computing Centre (Srce) of the University of Zagreb fully supports the principles and objectives of the Croatian Declaration on Open Access since 2012. Through its annual work Srce incorporates the idea of open access to scientific information and educational materials in the area of construction and maintenance of open and sustainable national infrastructure and promotes open access and systematic care for data and keeping them reliably. Work of Srce is based on two fundamental principles present in Croatia Declaration on Open Access, as well as in other international documents related to open access and open educational content: 1. open access is of public interest, since it abolished barriers and inequality, improving transparency and quality of public action, wider opportunities for access to knowledge and increase opportunities, competitiveness and the capacity of society as a whole 2. results of the activities financed by public funds, especially in the field of education and science, should be made available in open access. LITHUANIA: Almost all biggest Lithuanian state universities in web-pages declare support to EU policy on opens access to research information and some of them adopted internal documents like a guidelines on open access to scientific publications and data. Some links : https://www.mruni.eu/mru_lt_dokumentai/biblioteka/pdf/Atvira_prieiga.pdf http://www.lsmuni.lt/lt/biblioteka/informacija-vartotojams/elektronine-lsmu-publikaciju-ir-e-dokumentu--registravimo-forma/lsmu-atviros-prieigos-mandatas/ http://www.vgtu.lt/mokslas-ir-inovacijos/mokslo-publikacijos/atviroji-prieiga/274301 http://www.vu.lt/site_files/Senatas_Taryba/S-2016-9/vu_mtdv_gaires_Senatui.pdf http://ktu.edu/uploads/files/Bibliotekos/KTU_AP_nuostatai.pdf
• What are the main challenges in your country towards the implementation of Open Science?
Several respondents mentioned lack of knowledge, interest and initiative at the governmental level, and a lack of commitment towards introducing a national Open Science agenda. In cases where the government has made relevant commitments, respondents mentioned the lack of appropriate financing and resources for the required infrastructure and systemic changes in administration and evaluation procedures. For instance, Slovenia mentioned the fact that while a governmental agenda is in place (the Action plan for Open Access to Scientific Publications and Research Data in Slovenia 2016-2020), it is not yet clear who will be responsible for developing and monitoring the concrete measures and resources necessary to implement it.
Two respondents mentioned academic culture, and particularly the reluctance in some research field to publish in open access journals (which typically have a lower reputation than as top, closed journals) and spend time in archiving articles in OA repositories.
Switzerland emphasized the tension between the public support expected towards the implementation of Open Science, and the fact that “about two thirds of the funding for R&D as well as two thirds of the R&D personnel in Switzerland comes from the private sector (Federal Statistical Office). Private sector research is thus very dominant and given its larger interest in intellectual property and industrial secret than public research institutions, this poses a major challenge for Open Science.”
The publication of monographs in the humanities and social sciences was highlighted as a particularly problematic issue. Additionally, one respondent mentioned the lack of agreed standards as a considerable obstacle, and another highlighted publishers’ policies.
• Is the legal and institutional system in your country, including norms and regulations over intellectual property, conducive to the implementation of Open Science? If so, how? If not, how not?
Some respondents signaled that such system did not yet exist in their countries, but would be needed to establish a balance between protected data and open access to information. Other respondents mentioned that their own country aimed to harmonize their policies with EU practice, though this may present problems when no specific provisions are made for research vis-à-vis other types of activities.
Potential confusion between legislations instantiated by different ministries was mentioned as a barrier (for instance, when one ministry establishes intellectual property protection for the results of applied research, while another ministry emphasizes Open Science licensing for all products of publicly funded research).
Additionally, the following countries offered specifics of their own situation:
SLOVENIA: Present legal and institutional system, including norms and regulations over intellectual property, is not preventing Open Science, but it is not very conducive. The basic provisions (as an instruction measures) are already embedded into existing Act on research and innovation activities in Slovenia. Nevertheless, present legal and institutional system in Slovenia is not supportive enough in terms of addressing and regulating Open Science essential attributes and/or promoting its values. This is now the challenge of the Action plan for Open Access to Scientific Publications and Research Data in Slovenia 2016-2020, which is already prepared by the Ministry of Education, Science and Sport and is just to be submitted to the approval of the Government of the Republic of Slovenia.
34
BELGIUM: At the federal level all author's rights will soon be ceded to the State by Royal Arrest. Should the federal level choose to make research funded by itself open, it could. Moreover, the Belgian law for Legal Deposit will soon be extended to on line digital content so no mandate furthering deposit will be needed any longer. This is currently examinated in the context of the draft decree on Open Acces in Wallonia-Brussels Federation. MOLDOVA: http://agepi.gov.md/en/legislatie/nationale The State Agency for Intellectual Property has regulations including recommendations for Implementation of Open Science LITHUANIA: the Lithuanian legal and institutional system is favourable but not mandatory for implementation of OS policy.
Incentives and rewards
• Are there any types of incentive or reward in place to support Open Science activities in your country? If so, which ones, and who is implementing and monitoring them?
Three respondents answered “no” or “very few”. Additionally:
SLOVENIA: Research data, deposited into data archives and catalogued for the national CRIS (SICRIS – Slovenian Current Research Information System), are rewarded with points. Besides, the ministry, responsible for science is financing repositories for open access in the country. BELGIUM: In Université de Liège, only what has been archived on the Green OA repository ('ORBI') counts in the evaluation. This is a strong stick/carrot kind of incentive. Besides, there is a general feeling that engaging in Citizen science although important, is not well rewarded. Researchers are cautious too about publishing in new OA journals, since the cult of Impact Factor is very present and hard to circumvent. SWEDEN: Swedish research funders give grants to researchers for publishing open access. Some institutions do have publishing funds .These funds support researchers to pay APC.https://www.unisg.ch/en/forschung/foerderung/wissenschaftskommunikation/publikationsfonds http://www.oai.uzh.ch/en/at-the-uzh/funding/publishing-fund CROATIA: There are very rare practical examples of incentive or reward in place to support Open Science activities. One of the examples is the Rudjer Boskovic Institute (IRB)- one of the criteria for the awards for the best scientific papers in a given year is given on the basis that these works are stored in our institutional repository FULIR and if available in open access. This is a very good example of how scientists are willing to store their work in a digital repository if this commitment is linked to some other processes (rewards, advancement in scientific professions, recruitment, etc.). LITHUANIA: The Guidelines on Open Access to Scientific Publications and Data adopted by Research Council of Lithuania there is statement 'Research Council of Lithuania ... establish the transitional period for the implementation of the Guidelines by 31 December 2020. In case during the transitional period any infringements of the Guidelines are established apply warnings only'. Research Council of Lithuania monitors the implementation of Guidelines
• Is there any training structure in place to provide information and skills about Open Science practices? Who is providing it?
Only one respondent answered negatively, with most other respondents noting that the main providers of support are library services. Additionally:
SLOVENIA: Currently, there is no training structure in place for information and skills about Open Science. The Action Plan for Open Access to Scientific Publications and Research Data in Slovenia 2016-2020 contains provisions for the establishment and regular activities of supports services for researchers on open peer-reviewed scientific articles and preparation of research data for open access. Training of open access and open science stakeholders is envisaged. BELGIUM: BELSPO has set up an OA Helpdesk and a yearly training session for institutional OA contact persons. There are quite a lot of information activities that are organized at institutional level, by the universities and the academic libraries themselves. This is particularly focused on Early Stage Researchers, and may be part of their PhD training. But there is no central provision/funding/monitoring for this kind of activities. There is a website that coordinates the OA/OS related activities in Belgium: https://openaccess.be/ MOLDOVA: This is the Institute for Development of Informational Society which is organizing different events in order to inform about the Open Science. http://idsi.md/en/home Also, the librarian community is organizing events about Open Access SWEDEN: The research libraries at Swedish HEI:s are generally active in this field, they arrange lectures etc. One of the HEI:s provides a course for PH.D candidates 'Open Science and reproducible research'. htpp://kiwas.ki.se/katalog/katalog/kurs/2521. SWITZERLAND: Yes, Universities – particularly libraries - provide training on Open-Access. See for instance: http://www.oai.uzh.ch/en/at-the-uzh/events/introductory-course-zora Open-Access is also addressed in doctoral schools. CROATIA: A handful of people are working on open access and open science and / or projects from this area and work on educating the scientific community, librarians and promoting OA and OS. For example, the FOSTER project, currently implemented in Croatia, was designed to educate all stakeholders in Croatia for a couple of months, but it was only a project activity. It is hard to provide adequate education and promotion of OA and OS across Croatia. The IRB Centre for scientific information also carries out education (primarily for librarians) through cooperation with DABAR, as well as through their daily activities. However, this still isn’t sufficient. AUSTRIA: The Open Knowledge Network Austria Branch is very active in training for OS. There is a on-going series of events co-organised with the Technology Transfer Centre East (WTZ-Ost) for example: http://www.wtz-ost.at/veranstaltungen/
• Is there any institutional structure or specific venue dedicated to Open Science implementation and monitoring? Please provide links where possible.
Two respondents answered negatively and many others remarked on the absence of a national level system, noting however that some research organisations are investing efforts into this. Additionally:
SLOVENIA: No institutional structure or specific venue dedicated to Open Science exists presently in Slovenia. Open Science implementation and monitoring is implemented mainly through the dedicated web portals as it is OpenAccess Slovenia ( http://www.openaccess.si/ , built by Slovenian universities and research institutions and supported by the ministry and research agency) and Open Science Slovenia ( http://openscience.si , built by Slovenian universities and supported by the ministry and research agency). The national institute of information science (IZUM) has introduced Altmetrics ( http://scimet.izum.si/en/altmetrics )
36
into bibliographic data of Slovenian researchers ( http://splet02.izum.si/cobiss/BibPersonal.jsp?init=t&lang=eng&code=&type=conor ) in 2016. From the practical and technical point of view, Altmetrics can already be practiced in Slovenia for testing and learning purposes, but are not yet part of the official evaluation system. BELGIUM: The BELSPO Open Science project administrator and the Royal Library Open Access contact person. SWEDEN: When it comes to open access to publications, please see the National Library, www.kb.se/openaccess. Open access to data is going to be monitored by the Swedish Research Council, please see www.vr.se Association of Swedish Higher Education, SUHF, has established a coordinating group on open science www.suhf.se (only in Swedish). SWITZERLAND: The 'Scientific Information' program of swiss universities “promotes the concentration of today's distributed efforts of universities to provide and process scientific information. This includes the development and support of services which may be of use for open science.” https://www.swissuniversities.ch/en/organisation/projects-and-programmes/p-5/ https://www.swissuniversities.ch/fileadmin/swissuniversities/Dokumente/Organisation/SUK-P/SUK_P-2/PgB5_Antrag_2017-2020_kurz_EN.pdf LITHUANIA: Research Council of Lithuania has section in the web-page dedicated to OA with information about LT and EU documents, projects etc. http://www.lmt.lt/lt/atviroji_prieiga.html AUSTRIA: The FWF has implemented a monitoring for Open Access compliance: https://zenodo.org/record/55249#.WQr0Wx5QiP4
Future directions:
• What aspects of Open Science are most discussed and valued in your country, and by whom?
All respondents highlighted open access to scientific publications as the key aspect of Open Science currently under debate in their country. Two respondents (Slovenia and Switzerland) also mentioned some initial interest in Open Data and citizen science, though these issues are not yet viewed as a priority.
• Are you considering changes in incentives and rewards for researchers, funders, universities and research organisations? What kind of changes, and why?
Except for one who reacted negatively, respondents provided positive answers to this question, highlighting a number of national initiatives and committees in charge of considering potential changes to open access policies, research evaluation models, Open Science incentives and support for Open Research Data. These include the National Library of Sweden, the Swedish Research Council, the Region of Brussels-Capital, the Wallonia-Brussels Federation, the Croatian Ministry of Science and Education (working on a future Croatian Current Research Information System), Austrian Science Funding,
In the words of one respondent (Slovenia): “changes are important to enable further progress in development of effective and successful publicly funded research system in Slovenia as well as to support more effective international co-operation. We expect that further progress of EU and ERA in that respects will provide EU member states with the basic source of information, best practices and role models to enable and support us in making proper steps in that direction.” The Croatian delegate similarly remarked that the goal is “a comprehensive system that will be used for systematic design of medium-term and long-term research, development and
innovation priorities in the public research organizations in the future.” At the same time, respondents generally view such a change as requiring several years to be fully implemented, thus overshooting the European requirement of full implementation by 2020.
• What kinds of cultural changes are required for wider adoption of open science?
Respondents provided a variety of answers to this question, which are reported below as each response exemplifies a specific interpretation of what Open Science amounts to. Interpretations include
- Open Science as most closely associated to advances in information and communication technologies, which are shifting the ways in which knowledge is developed and shared.
- The idea that rather than a cultural shift, all that Open Science requires is additional funding.
- Open Science as an opportunity to acknowledge how pluralistic and varied the methods, goals and communities of researchers are.
- Open Science as a shift in the expectations and priority of researchers, which would involve more collaboration and sharing, as well as moving away from the “publish and perish” ethos.
This diversity is notable and important to take into account for further discussions at the national, European and global levels.
Open Science is in particular related to the research which is performed within ICT and web based environments. The promotion of the advances in that environment, that are supporting better interdisciplinary and international collaboration as well as “openness” of the research work to the society and research community, is therefore important. Certain centralised services at the level of research institutions could therefore be beneficial to promote and help get track to researchers with the mainstream solutions. As the developments in ICT and web based environments are extremely fast and versatile, it may be good for research disciplines to develop and promote, within their research communities, best practices in implementing Open Science culture and share them with a larger scope research institutions. Not cultural changes aprt from a general conservatism. What is most needed is better funding of research so that institutions don't have to be worried sick about losing income and researchers about losing impact. Open mind and modern attitude towards science, communication and spreading science Incentives and reward systems, costs for publishing. The better acknowledgement of a diversity of profiles within academia, rather than the focus on the super publisher. A proper acknowledgment of the diversity of research outputs, according to the disciplines and the type of research conducted (curiosity driven, strategic and applied). And definitely, not using anymore IF for assessing individual researchers! In some disciplines, the orientation toward “Publish or Perish” and the demand to publish only in “top-journals” should change toward a stronger orientation to the quality of the content. One of the biggest challenges will be to change the way scientist perceive OS and to improve their willingness to share. This is something they are not used to, for multiple reasons. Most of them are very well defined in ERAC Opinion on Open Research Data. In addition, on the national level it will be important to address and improve the overall digital science policy and governance. For that reason we are considering participating in the OECD “Digital Science and Innovation Policy and Governance” project that should start in 2017. Furthermore, on the EU level, it is important to bring more synergies between
38
different policy streams (cohesion policy, Horizon 2020, ERA policies, Erasmus +, etc.). Without this synergetic effect of all the policies, there will always be contradictions and the research community will have hard time following different sets of regulations and implementation rules when it comes to framework projects and all other types of funding. Almost all initiatives regarding OA or OS comes from Brussels and we lack an active researchers or research leaders promoting OS ideas. Optimization of higher education institutional system is currently on the top of political agenda in Lithuania and OS maters goes on the second plan. OA initiative is actively supported by universities librarians, but part of older academic society needs better understanding about advantages of OS. - Best practice examples - Incentives - Mentality of 'Expert knowledge' by policy-makers on which fields of research are needed/funded, etc... To form a habit requires time. Thereafter implementation of Open Science will require time for the different stakeholders to adjust to the new models, requirements and regulations. It will require time for some scientific communities to embrace the advantages of open science and to capitalize on its opportunities. A clear, transparent and open discussion on the opportunities and the potential limitations of Open Science will certainly contribute to a faster adoption of the new habit.
• What skills and training in open science are necessary for different stakeholders?
Respondents mentioned the need for skills and related training programmes specifically for:
- Policy makers, including awareness of the stakes involved in Open Science and the situation in other countries, and options for Open Access that are not limited to deals with major publishers
- All researchers, including Open Education, copyright issues and understanding of different models of open access publishing and self-archiving
- Specialists in Open Science, such as evaluators of OS implementation and providers of support (which could be provided by the EU and ERA structures)
- Early career researchers, including in science communication and in understanding the impact that publishing choices may have on their future career.
- Industry stakeholders, concerning particularly IPR dimensions. - Editors, including the technical and legal challenges of publishing open access - Librarians and information specialists, which need to provide OA support concerning
publications and research papers, as well as instruct researchers on the options available for publishing.
- Funders, particularly tools to monitor publication of (all types of) research results
• What incentives, reward systems and strategies for Open Science implementation would you like to be able to adopt in the future (whether or not this is currently feasible)? Why?
Respondents disagreed on the extent to which Open Science implementation should be top-down (mandated by governments and research funders) or bottom-up (fostered by researchers and supported by research performing institutions). Responses included the following suggestions:
- “Better and attractive metrics (the carrot) make Open Science part of researcher evaluation and adopting the Liège Model (the stick).”
39
- “Generalization of OS modules in 3d cycle education. Working on/with ESR is the only way to change the mindsets for the better!”
- “Articles should remain central in the ecosystem of science production and dissemination, but the other elements of this ecosystem should be better taken into account. In particular, those research output with as societal impact should be better valorized at each step of the career (and not only for those senior researchers who no longer need to prove anything).”
- “The link between Open Science and Open Education should be investigated, in particular for the development of altmetrics able to track the use of research within (open) educational research.”
- “The bottom-up approach, which is already at the core of the current National open-access, strategy should be maintained.”
- “Research funders and universities should take into account an applicant’s previous practice of green open access and data management plans as evaluation criteria in their funding and hiring decisions.”
- “One of the best solutions would be to link elections in science, teaching and other vacancies with the obligation of storing publications in OA repositories and ensuring OA to so-preserved publications.”
8 Preparation for the 3rd Working Meeting Based on the discussions at the kick-off meeting and the responses to the questionnaire some
specific objectives have been placed for the 3rd Working Meeting.
8.1 Objectives
Discuss and provide feedback to challenge paper and presentation on incentives and rewards.
Define scope of the fourth and final reports, and provide additional examples and materials on
which to structure the challenge paper on experiences, models and strategies.
8.2 Agenda of the meeting
Afternoon of Day 1:
• Presentation on the Background/Challenge paper (Sabina)
• Break-out groups. Discussion questions: what incentives would work best in your country, and why? What do you see as the key obstacles to implementing those incentives?
• Reports back and general discussion
Afternoon of Day 2: • Breakout groups on lessons learnt and questions raised by expert presentations.
Discussion question: how would you envisage a roadmap for Open Science implementation in your country?
• Reports back and general discussion • Feedback on next meeting and topic of the next report (Sabina)
9 References Acord, S. K., & Harley, D. (2013). Credit, time, and personality: The human challenges to sharing scholarly work using Web 2.0. new media & society, 15(3), 379-397.
40
Ankeny, R. A., & Leonelli, S. (2015). Valuing Data in Postgenomic Biology: How Data Donation and Curation Practices Challenge the Scientific Publication System. In H. Stevens & S. Richardson (Eds.), Post-Genomics. Durham: Duke University Press. Balmer, Brian. 2013. Secrecy and science: A historical sociology of biological and chemical warfare. Farnham: Ashgate Publishing, Ltd. Bastow, R. and Leonelli, S. (2010) Sustainable digital infrastructure. EMBO Reports, 11(10): 730-735. Beagrie, N. et al (2016) The Value and Impact of the European Bioinformatics Institute. Report. https://beagrie.com/static/resource/EBI-impact-report.pdf Benedictus, R. , Miedema, F. and Ferguson, M.W.J. 2016. Fewer Numbers, Better Science. Nature
538, 453–455. doi:10.1038/538453a.
Bezuidenhout, L., Leonelli, S., Kelly, A. and Rappert, B (2017) Beyond the Digital Divide: Towards a Situated Approach to Open Data. Science and Public Policy. Moedas, Carlos. 2015. Open Innovation, Open Science, Open to the World. doi:10.2777/061652. Borgman, C. L. (2012). The conundrum of sharing research data. Journal of the American Society for Information Science and Technology, 63(6), 1059-1078. Bratislava Declaration of Young Researchers (2016) http://www.eu2016.sk/data/documents/bratislava-declaration-of-young-researchers-final.pdf Corrall, S., & Pinfield, S. (2014). Coherence of “open” initiatives in higher education and research: Framing a policy agenda. iConference 2014 Proceedings, 7, 293-313. Digital Science Report (2016) The State of Open Data. London, Holtzbrinck Publishing Group. https://dx.doi.org/10.6084/m9.figshare.4036398 Directorate-General for Research and Innovation (2016) European Open Science Agenda. Edwards, P. N., Mayernik, M. S., Batcheller, A. L., Bowker, G. C., & Borgman, C. L. (2011). Science friction: Data, metadata, and collaboration. Social Studies of Science, 41(5), 667-690. Eschenfelder, K. and Shankar, K. 2016. Designing Sustainable Data Archives: Comparing Sustainability Frameworks. iConference 2016 Proceedings, pp. 1–7. doi:10.9776/16243. EU Working Group on Education and Skills Under Open Science (2017) Providing researchers with the skills and competencies they need to practise Open Science. Report of the Working Group on Education and Skills under Open Science. URL. EU Working Group on Open Science Rewards (2017) Evaluation of Research Careers Fully Acknowledging Open Science Practices: Rewards, Incentives and/or Recognition for Researchers practicing Open Science. URL European Commission. 2016. Open innovation, open science, open to the world – A vision for the future. Directorate-General for Research and Innovation. See http://bookshop.europa.eu/en/open-innovation-open-science-open-to-the-world-pbKI0416263/.
European Commission. (2014). Consultation on ‘Science 2.0’: Science in Transition, from http://ec.europa.eu/research/consultations/science-2.0/consultation_en.htm European and Global Young Academies (2016) Position Statement on Open Data by the Young Academies of Europe and the Global Young Academy. https://globalyoungacademy.net/wp-content/uploads/2016/04/Position-Statement-on-Open-Data-by-the-Young-Academies-of-Europe-and-the-Global-Young-Academy.pdf European and Global Young Academies (2016) Position Statement on Open Access by the Young Academies of Europe and the Global Young Academy. https://globalyoungacademy.net/wp-content/uploads/2016/04/Position-statement-on-Open-Access-by-the-Young-Academies-of-Europe-and-the-Global-Young-Academy.pdf Evans, J. A. (2010). Industry collaboration, scientific sharing, and the dissemination of knowledge. Social Studies of Science, 40(5), 757-791. Fecher, B. & Friesike, S. (2014). Open Science: One term, five schools of thought. In: Bartling, S. & Friesike, S. (Eds.). Opening Science: The evolving guide on how the Internet is changing research, collaboration and scholarly publishing. Springer, Heidelberg, p. 17-47. Fecher, B., Friesike, S., & Hebing, M. (2015). What Drives Academic Data Sharing? PLOS ONE, 10(2). Friesike, S. & Schildhauer, T. (2015). Open science: many good resolutions, very few incentives, yet. In: Welpe, I.M., Wollersheim, J., Ringelhan, S. & Osterloh, M. (Eds.). Incentives and Performance. Governance of Research Organizations. Springer. FOSTER (2017) The next phase of FOSTER: what you can get out of it, https://www.fosteropenscience.eu/content/next-phase-foster-%E2%80%93-what-can-you-get-out-it-0 Gaule, P., & Maystre, N. (2011). Getting cited: does open access help? Research Policy, 40(10), 1332-1338. Grubb, A. M., & Easterbrook, S. M. (2011). On the lack of consensus over the meaning of openness: An empirical study. PloS one, 6(8), e23420. Haeussler, C. (2011). Information-sharing in academia and the industry: A comparative study. Research Policy, 40(1), 105-122. HEFCE. (2014). Policy Guide: Open Access Research, from http://www.hefce.ac.uk/whatwedo/rsrch/rinfrastruct/oa/ Hey, A. J., Tansley, S., & Tolle, K. M. (2009). The fourth paradigm: data-intensive scientific discovery. Redmond, Washington. Holmes, D. (2016). A new chapter in innovation. Nature, vol. 533, no. 7602, S54+.
Kitcher, P. (2011). Science in a Democratic Society. Prometheus Books. Kitchin, R. 2014. The data revolution: Big data, open data, data infrastructures and their consequences. London: SAGE. Kleinman, D. L. (ed.). 2000. Science, Technology and Democracy. SUNY Press.
Laakso, M., Welling, P., Bukvova, H., Nyman, L., Björk, B.-C., & Hedlund, T. (2011). The development of open access journal publishing from 1993 to 2009. PLOS ONE, 6(6), e20961. Leonelli, S. (2016) Data-Centric Biology: A Philosophical Study. Chicago, IL: Chicago University Press. Leonelli, S., Spichtinger, D. and Prainsack, B. (2015) Sticks AND Carrots: Incentives for a Meaningful Implementation of Open Science Guidelines. Geo, 2: 12–16 doi: 10.1002/geo2.2 LERU (2017). LEARN toolkit of best practice in research data management, http://learn-rdm.eu/en/research-data-management-toolkit-now-available/ LERU Research Data Working Group (2013) LERU Roadmap for Research Data. http://www.leru.org/files/publications/AP14_LERU_Roadmap_for_Research_data_final.pdf Levin, N. and Leonelli, S. (2016) How Does One “Open” Science? Questions of Value in Biological Research. Science, Technology and Human Values. Levin, N., Leonelli, S., Weckowska, D., Castle, D., and Dupré, J. (2016) How Do Scientists Understand Openness? Exploring the Relationship between Open Science Policies and Research Practice. Bulletin for Science and Technology Studies 36(2): 128-141. http://dx.doi.org/10.1177/0270467616668760 Longino, H. 2002. The Fate of Knowledge. Princeton University Press. Mauthner, N. S., & Parry, O. (2013). Open Access Digital Data Sharing: Principles, Policies and Practices. Social Epistemology, 27(1), 47-67. McKiernan, Erin C., Philip E. Bourne, C. Titus Brown, Stuart Buck, Amye Kenall, Jennifer Lin, Damon McDougall, et al. 2016. “How Open Science Helps Researchers Succeed.” eLife 5 (JULY): 1–19. doi:10.7554/eLife.16800. Merton, R. (1973) [1942] The Sociology of Science: Theoretical and Empirical Investigations, Chicago: University of Chicago Press. Miedema, F. 2012. Science 3.0: Real Science, Real Knowledge. Amsterdam University Press. Minssen, T. and Pierce, J., 2017. Big Data and Intellectual Property Rights in the Health and Life Sciences. In 2016 Annual Conference: Big Data, Health Law, and Bioethics. Cambridge University Press. Mittelstadt, B. D. and Floridi, L. 2016. The ethics of biomedical big data: Current and Foreseeable Issues in Biomedical Contexts. Sci Eng Ethics 22 (2), pp. 303–341. doi:10.1007/s11948-015-9652-2. Moed, H.F., De Bruin, R.E. & Van Leeuwen, T.N. (1995). New bibliometric tools for the assessment of national research performance – database description, overview of indicators and first applications. Scientometrics, 33(3), 381-422. Moedas, C. (2016) “Science Diplomacy in the European Union,” Science & Diplomacy, 5: 1. http://www.sciencediplomacy.org/files/science_diplomacy_in_the_european_union_science__diplomacy.pdf
Morey, RD, Chambers, DC, Peter J. Etchells, Christine R. Harris, Rink Hoekstra, Daniël Lakens, Stephan Lewandowsky, et al. 2016 The Peer Reviewers’ Openness Initiative: Incentivizing open research practices through peer review, Royal Society Open Science 3 (1): 150547. doi:10.1098/rsos.150547. Nelson, B. (2009). Data sharing: Empty archives. Nature, 461. Neylon, C. (2012). Science publishing: open access must enable open use. Nature, 492(7429), 348-349. Nordmann, A., Radder, H. and G. Schiemann, eds, 2011. Science transformed? Debating claims of an epochal break. Pittsburgh: University of Pittsburgh Press. Nuffield Council on Bioethics. (2015). The collection, linking and use of data in biomedical research and health care: ethical issues, from http://nuffieldbioethics.org/wp-content/uploads/Biological_and_health_data_web.pdf Open Science Policy Platform (2017) Recommendations on Open Science Publishing. http://ec.europa.eu/research/openscience/pdf/ospp_open_access_publishing_report.pdf#view=fit&pagemode=none
Piwowar, Heather A. 2013. “Value All Research Products.” Nature 493:159.
Popper, K (2012) [1945]. The Open Society and Its Enemies. Routledge. Prainsack, B. and Buyx, A. 2017. Solidarity in Biomedicine and Beyond. Cambridge: Cambridge University Press. Radder, H. 2010. The Commodification of Academic Research. Pittsburgh University Press. Rappert, B. 2007. Biotechnology, Security and the Search for the Limits: An Inquiry into Research and Methods. Palgrave Macmillan. Research Councils UK. (2014). Pathways to Impact. Available at: http://www.rcuk.ac.uk/innovation/impacts/ Rhee, S. Y (2004) Carpe Diem. Retooling the “Publish or Perish” Model into the “Share and Survive” Model. Plant Physiology 134: 543-547. Royal Society and British Academy (2017) Data Management and Use: Governance in the 21st Century. A Joint Report of the Royal Society and the British Academy. https://royalsociety.org/~/media/policy/projects/data-governance/data-management-governance.pdf Sansone, S.-A., Rocca-Serra, P., Field, D., Maguire, E., Taylor, C., Hofmann, O., . . . Amaral-Zettler, L. (2012). Toward interoperable bioscience data. Nature genetics, 44(2), 121-126. Science International (2015) Open Data in a Big Data World. http://www.icsu.org/science-international/accord/open-data-in-a-big-data-world-long SPARC Europe (2016) Mapping Open Science & Open Scholarship in Europe, December 2016, SPARC Europe. http://sparceurope.org/wp-content/uploads/2017/01/SPARCEuropeMappingOpenScienceinEurope.pdf
Stilgoe, Jack, Richard Owen, and Phil Macnaghten. 2013. “Developing a Framework for Responsible Innovation.” Research Policy 42 (9): 1568–80. doi:10.1016/j.respol.2013.05.008.
Tenopir, C. et al (2015). Changes in Data Sharing and Data Reuse Practices and Perceptions among Scientists Worldwide. PLOS One. http://dx.doi.org/10.1371/journal.pone.0134826 Tenopir, Carol, Suzie Allard, Kimberly Douglass, Arsev Umur Aydinoglu, Lei Wu, Eleanor Read, Maribeth Manoff, and Mike Frame. 2011. “Data Sharing by Scientists: Practices and Perceptions.” Edited by Cameron Neylon. PLoS ONE 6 (6): e21101. doi:10.1371/journal.pone.0021101. The Royal Society. (2012). Science as an Open Enterprise, from https://royalsociety.org/~/media/policy/projects/sape/2012-06-20-saoe.pdf
Vayena, Effy, and John Tasioulas. 2016. “The Dynamics of Big Data and Human Rights: The Case of Scientific Research.” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 374 (2083): 20160129. doi:10.1098/rsta.2016.0129.
Wallis, J. C., Rolando, E., & Borgman, C. L. (2013). If we share data, will anyone use them? Data sharing and reuse in the long tail of science and technology. PloS one, 8(7), e67332. doi:10.1371/journal.pone.0067332. Wilsdon, J. et al. (2015) The Metric Tide: Report of the Independent Review of the Role of Metrics
in Research Assessment and Management. DOI: 10.13140/RG.2.1.4929.1363
Wilsdon, J. et al. (2017) Next-generation metrics: Responsible metrics and evaluation for open science. Report of the European Commission Expert Group on Altmetrics. Brussels, European Commission, March 2017. https://ec.europa.eu/research/openscience/pdf/report.pdf#view=fit&pagemode=none Wolf, B., Lindenthal, T., Szerencsits, J. M., Holbrook, J. B. and Heß, J. (2013). Evaluating Research beyond Scientific Impact. How to Include Criteria for Productive Interactions and Impact on Practice and Society. Gaia, 22(2), p. 11. Wouters, P. (2017) Bridging the Evaluation Gap. Engaging Science, Technology and Society. DOI:
