“The National Academies Roundtable on Aligning Incentives for Open Science, established in 2019, has taken on an important role in addressing issues with open science. The roundtable convenes critical stakeholders to discuss the effectiveness of current incentives for adopting open science practices, current barriers of all types, and ways to move forward in order to align reward structures and institutional values. The Roundtable convened a virtual public workshop on fostering open science practices on November 5, 2020. The broad goal of the workshop was to identify paths to growing the nascent coalition of stakeholders committed to reenvisioning credit/reward systems (e.g., academic hiring, tenure and promotion, and grants)to fully incentivize open science practices. The workshop explored the information and resource needs of researchers, research institutions, government agencies, philanthropies, professional societies, and other stakeholders interested in further supporting and implementing open science practices. This publication summarizes the presentations and discussion of the workshop.”
“PLOS has been awarded a grant from the Wellcome Trust’s 2021 Open Research Fund to accelerate development and testing of new solutions that promote and reward open science. PLOS Pathogens will be piloting the latest version of the Dryad data repository, provided free of charge to authors and integrated into the publishing experience, along with prominent visual links on publications designed to incentivise open research practices….”
Abstract: Peer review is an integral component of contemporary science. While peer review focuses attention on promising and interesting science, it also encourages scientists to pursue some questions at the expense of others. Here, we use ideas from forecasting assessment to examine how two modes of peer review — ex ante review of proposals for future work and ex post review of completed science — motivate scientists to favor some questions instead of others. Our main result is that ex ante and ex post peer review push investigators toward distinct sets of scientific questions. This tension arises because ex post review allows an investigator to leverage her own scientific beliefs to generate results that others will find surprising, whereas ex ante review does not. Moreover, ex ante review will favor different research questions depending on whether reviewers rank proposals in anticipation of changes to their own personal beliefs, or to the beliefs of their peers. The tension between ex ante and ex post review puts investigators in a bind, because most researchers need to find projects that will survive both. By unpacking the tension between these two modes of review, we can understand how they shape the landscape of science and how changes to peer review might shift scientific activity in unforeseen directions.
Looking back at this 2017 post brings a mixed bag of thoughts. First, the fortunes being made with collecting, curating, and selling access to consumer data still haven’t spilled across into research data, and that’s likely because a) relatively few research datasets are available, and b) for the most part, the ones that are available have inadequate metadata and incompatible structures, so that combining datasets for meta-analyses is scarcely worthwhile. Until we address the problem of missing research data – which (full disclosure) we’re trying to do with DataSeer – we can’t really make much headway with getting it all into a consistent format. However, while combining datasets for re-use is a core feature for consumer data, it’s only one of the reasons for sharing research data. Open data also allows readers to check the results for the paper itself, and perhaps this is where our attention for the ‘business model for open data’ should turn. In particular, peer review is considerably simpler when the authors submit computationally reproducible manuscripts. Editors and reviewers can then be sure that the datasets support the analyses and hence the results, allowing them to focus solely on the appropriateness of the experimental design and the significance of the conclusions. It’s therefore conceivable that journals could reduce the APC for computationally reproducible articles (or hike it for non-reproducible ones), thereby incentivizing the extra effort required to required to produce them. No matter what route we choose, it’s clear that our current incentive structures around open science (mostly strongly worded policies and the lure of extra citations) are not getting the job done, and we need to consider alternatives. Money can enter the equation at a few places: by only funding open science, as exemplified by Aligning Science Across Parkinson’s, or by offsetting the extra effort required by researchers with additional financial resources, by making things cheaper or non-open science more expensive. Let’s see where we go.
“In early August, it was announced that UK Research and Innovation (UKRI) would provide significant funding for a new open publishing platform. Called Octopus, this initiative is not yet fully launched, but when it is it plans to “provide a new ‘primary research record’ for recording and appraising research “as it happens’”; UKRI calls Octopus “a ground-breaking global service which could positively disrupt research culture for the better.” I reached out to Octopus’s founder, Dr. Alexandra Freeman, to ask some questions about Octopus and its plans for the future….”
The purpose of this study is to investigate if and how government intervention can nudge students to become ebook readers.
A cross-sectional survey research design was adopted for this study. A total of 1,144 students from four middle and high schools in urban and rural areas of Indonesia participated in this study. The results from statistical analyses were further discussed through the lens of the nudge theory.
This paper founds evidence that government intervention in the form of the Buku Sekolah Elektronik (BSE) policy that has been providing free electronic textbooks for more than a decade can help nudge students to become ebook readers. After controlling for student’s demographic information, this paper founds that their awareness of such a policy is significantly associated with a stronger preference toward ebooks while having no significant effect on their preferences toward printed book format. This paper also founds that mobile device adoption plays an important role where early adopters tend to prefer ebook format, whereas laggards are more associated with printed book format.
Many have studied the benefits of using ebooks in learning, but the literature also shows that most students still prefer reading printed books over ebooks. This is true not only in developing countries where problems with infrastructures can hamper the adoption of ebooks in general but also in developed countries where ebooks are much more prevalent, even among the general population. This paper showed how government interventions have the potency to help tip the scales and nudge students to become ebook readers.
Abstract: Researchers from Uzbekistan are leading the global list of publications in predatory journals. The current paper reviews the principles of implementation of the “publish or perish policy” in Uzbekistan with an overarching aim of detecting the factors that are pushing more and more scholars to publish the results of their studies in predatory journals. Scientific publications have historically been a cornerstone in the development of science. For the past five decades, the quantity of publications has become a common indicator for determining academic capacity. Governments and institutions are increasingly employing this indicator as an important criterion for promotion and recruitment; simultaneously, researchers are being awarded Ph.D. and D.Sc. degrees for the number of articles they publish in scholarly journals. Many talented academics have had a pay rise or promotion declined due to a short or nonexistent bibliography, which leads to significant pressure on academics to publish. The “publish or perish” principle has become a trend in academia and the key performance indicator for habilitation in Uzbekistan. The present study makes a case for re-examining the criteria set by the Supreme Attestation Commission of the Republic of Uzbekistan for candidates applying for Ph.D. and D.Sc. as well as faculty promotion requirements in the light of current evidence for the deteriorating academic performance of scholars. View Full-Text
“One of the underlying ideas of Open Science is that when scientists are open about what they are doing, and what they have been up to, double work can be prevented. For example, Prospero (https://www.crd.york.ac.uk/prospero/), a registry in which authors can file their initiative to execute a systematic review and meta-analysis can indeed fulfill that role. However, Chapelle et al show that only 10 of the 20 meta-analyses were indeed preregistered. …
Another way to reduce redundant publications is to share research before it is peer-reviewed by publishing it on a preprint server such as medRxiv.org. Although the coronavirus disease 2019 pandemic has popularized this practice, it is only used for a small fraction of all research output. Would further adoption of this practice be a way to further reduce research waste? The data collected by Chapelle et al suggest that the time window between “received” and “published online” was short. Preprints will only prevent double work when there is a sufficiently large window between these two timepoints during which other researchers have to decide whether or not to start a new project….
As long as the scientific enterprise incentivizes research waste and science for publications, time and resources are wasted. Open science practices cannot counteract this because they do not address the root cause….”
“This response to the White House Office of Science and Technology Policy’s “Request for Information To Improve Federal Scientific Integrity Policies” is submitted on behalf of the Open Research Funders Group….
The Open Research Funders Group is supportive of the White House Office of Science and Technology Policy’s commitment to explore good practices Federal agencies can adopt to improve scientific integrity, promote transparency, prioritize evidence-based decision making, and promote equity. We believe that the promotion of and adherence to open science principles is a catalytic enabling strategy in support of these goals. Specifically, we recommend that the OSTP prioritize making as much of the research lifecycle openly available to access and reuse. This includes, but is not limited to, preregistrations, protocols, preprints, articles, data, code, and software. The rationale is simple. Research cannot be considered reliable unless it can be tested, replicated, and built upon. Making critical components of the research lifecycle unavailable hampers OSTP’s pursuit of scientific integrity at best, and renders it impossible at worst. Limiting access to research outputs has the further effect of rendering science opaque, which negatively impacts public trust in the research endeavor writ large….”
“The Open Research Funders Group (ORFG) is pleased to submit a formal response to the White House Office of Science and Technology Policy’s “Request for Information To Improve Federal Scientific Integrity Policies”. The comments, which may be found in their entirety here, encourage the federal government to prioritize making as much of the research lifecycle openly available to access and reuse. This includes, but is not limited to, preregistrations, protocols, preprints, articles, data, code, and software. The rationale is simple. Research cannot be considered reliable unless it can be tested, replicated, and built upon. Making critical components of the research lifecycle unavailable hampers OSTP’s pursuit of scientific integrity at best, and renders it impossible at worst. Limiting access to research outputs has the further effect of rendering science opaque, which negatively impacts public trust in the research endeavor writ large….”
Many scientists are transitioning to a new way of working, known as open science, which will require new ways of evaluating researchers’ work. At Utrecht University we are adapting the reward system so it will incentivise this shift. The change that has received the most public attention, ditching the publishing metric known as the journal impact factor, is important, but it’s just one step in a much larger transformation. Through open science, researchers and research administrators seek to improve the quality, reproducibility and social impact of research. Open science includes open access publishing, so citizens and peers can access the fruits of publicly-funded research without paying for the privilege, and moving to a system of FAIR data, making information easy for researchers to find, access, and reuse. Open science also includes software sharing.
From Google’s English: “On July 19, ScienceGuide published an open letter from 171 academics who are concerned about the new Recognition and Valuation of scientists. In fact, the signatories warn that the new ‘Recognize and Appreciate’ leads to more arbitrariness and loss of quality. This will jeopardize the international top position of Dutch science, argue the writers, which will adversely affect young academics in particular. …
It is noticeable that these young scientists, whom the letter speaks of, do not seem to be involved in drafting the message. It is also striking that signatories to the open letter themselves are mainly at the top of the academic career ladder; 142 of the 171 signatories are even professors. As Young Science in Transition, PhD candidates Network Netherlands, PostDocNL, a large number of members of De Jonge Akademies and many other young researchers, we do not agree with the message they are proclaiming. In fact, we worry about these kinds of noises when it comes to our current and future careers. Young academics are eagerly waiting for a new system of Recognition and Appreciation. …”
From Google’s English: “A group of 171 scientists, including 142 professors, warns in this open letter that the new Recognition and Valuation will harm Dutch science. The medical, exact and life sciences in particular are in danger of losing their international top position as a result of the new Recognition and Appreciation, because it is no longer clear how scientists are judged.
An article was recently published in Nature about the new policy of Utrecht University whereby the impact factors of scientific journals are no longer included in the evaluation of scientists. Measurable performance figures have been abandoned in favor of an ‘open science’ system and elevating the team above the individual.
Here 171 academics warn that this new ‘Recognition and appreciation’ will lead to more arbitrariness and less quality and that this policy will have major consequences for the international recognition and appreciation of Dutch scientists. This will have negative consequences in particular for young researchers, who will no longer be able to compete internationally. …”
“During the last few weeks, several opinion pieces have appeared questioning the new Recognition and Rewards (R&R) and Open Science in Dutch academia. On July 13, the TU/e Cursor published interviews with professors who question the usefulness of a new vision on R&R (1). A day later, on July 14, the chairman of the board of NWO compared science to top sport, with an emphasis on sacrifice and top performance (2), a line of thinking that fits the traditional way of R&R in academia. On July 19, an opinion piece was published by 171 university (head) teachers and professors (3), this time in ScienceGuide questioning again the new vision of R&R. These articles, all published within a week, show that as the new R&R gains traction within universities, established scholars are questioning its usefulness and effectiveness. Like others before us (4), we would like to respond. …”
“KEY RESULTS: • Open Science principles: over half (59%) of the surveyed institutions rated Open Science’s strategic importance as very high or high. Open Access to research publications was considered to be highly important for 90% of institutions, but only 60% considered its implementation level to be high. However, the gap between importance and implementation is much wider in data-related areas (RDM, FAIR and data sharing): high importance at between 55-70% of the institutions surveyed, with high levels of implementation at 15-25%. • Open Science policies: 54% of institutions have an Open Science policy and 37% are developing one. Only 9% of surveyed institutions lack an Open Science policy or are not planning to draft one. • Monitoring Open Access to research publications: 80% of institutions monitored the number of publications in their repository and 70% monitored articles published by their researchers in Open Access journals. In addition, almost 60% reported monitoring the cost of publications by their researchers in Open Access journals. • Infrastructure for Open Access to research publications: 90% of the institutions surveyed have their own repository, participate in a shared repository or both. For journal hosting or publishing platforms this figure reaches 66%, and levels out at 57% for monograph hosting/publishing. In addition, 66% of those surveyed reported that their institution has participated in or supported non-commercial Open Access publishing. Data-related skills: over 50% of the surveyed institutions reported that research data skills were only partially available. Moreover, all of the institutions that indicated the absence or partial availability of data skills, considered that more of these skills are needed at institutional level. • Emerging areas of Open Science: Approximately 50% of the respondents know of citizen science and open education activities at their institutions. • Open Science in academic assessment: In 34% of institutions, none of the Open Science elements examined by the survey were included in academic assessments. Amongst the institutions that included Open Science activities in their academic assessments, 77% took into consideration article deposition in a repository. …”