“Nature Methods is proud to publish our very first Registered Report in this issue. Here, we reflect on what we have learned since introducing this article type….”
Registered Reports: what we’ve learned so far | Nature Methods
Reply
Category Archives: oa.preregistration
Advantages and challenges to open science practices (in Russian)
In Russian with this English-language abstract: The article examines open science practices in relation with the Open Science framework. The core values of open science, emerged as a response to the long-standing challenges to scientific knowledge production, include: transparency, scrutiny, critique and reproducibility; equality of opportunities; responsibility, respect and accountability; collaboration, participation and inclusion; flexibility; sustainability. These are the guiding principles for open science practices. The spread of open science practices is uneven, in terms of regional, disciplinary, gender, and institutional differences. The overview of international studies shows that open science practices are beginning to affect the whole research cycle, from idea emergence throughout the dissemination and exploitation of research results. We analyzed four most widespread practices – open data, open peer review, preregistration and registered reports, and open access. Our findings suggest that all these practices, while solving particular problems, simultaneously create new ones. To overcome new challenges, shift in the principles themselves, scheme of funding and workload sharing, evaluation and reward processes are necessary. The most challenging is the need to change research culture in accordance with Open Science values. In Russia, open access is a commonly spread practice, whereas the rest three practices yet to be discussed.
Why should researchers publish all their … | Open Research Europe
“Traditionally, research articles comprise the majority of publications across all disciplines, with journals prioritizing these above all else. However, in Open Research Europe, they represent 51% of all articles – so what are the remaining 49%?
In this blog, we highlight why broadening this focus of publishable work is important and the benefits different article types on Open Research Europe can bring to your research….”
Survey of open science practices and attitudes in the social sciences | Nature Communications
Abstract: Open science practices such as posting data or code and pre-registering analyses are increasingly prescribed and debated in the applied sciences, but the actual popularity and lifetime usage of these practices remain unknown. This study provides an assessment of attitudes toward, use of, and perceived norms regarding open science practices from a sample of authors published in top-10 (most-cited) journals and PhD students in top-20 ranked North American departments from four major social science disciplines: economics, political science, psychology, and sociology. We observe largely favorable private attitudes toward widespread lifetime usage (meaning that a researcher has used a particular practice at least once) of open science practices. As of 2020, nearly 90% of scholars had ever used at least one such practice. Support for posting data or code online is higher (88% overall support and nearly at the ceiling in some fields) than support for pre-registration (58% overall). With respect to norms, there is evidence that the scholars in our sample appear to underestimate the use of open science practices in their field. We also document that the reported lifetime prevalence of open science practices increased from 49% in 2010 to 87% a decade later.
The transparency of quantitative empirical legal research published in highly ranked law journals (2018–2020): an observational study
Abstract: Background: Scientists are increasingly concerned with making their work easy to verify and build upon. Associated practices include sharing data, materials, and analytic scripts, and preregistering protocols. This shift towards increased transparency and rigor has been referred to as a “credibility revolution.” The credibility of empirical legal research has been questioned in the past due to its distinctive peer review system and because the legal background of its researchers means that many often are not trained in study design or statistics. Still, there has been no systematic study of transparency and credibility-related characteristics of published empirical legal research.
Methods: To fill this gap and provide an estimate of current practices that can be tracked as the field evolves, we assessed 300 empirical articles from highly ranked law journals including both faculty-edited journals and student-edited journals.
Results: We found high levels of article accessibility, especially among student-edited journals. Few articles stated that a study’s data are available. Preregistration and availability of analytic scripts were very uncommon.
Conclusion: We suggest that empirical legal researchers and the journals that publish their work cultivate norms and practices to encourage research credibility. Our estimates may be revisited to track the field’s progress in the coming years.
Open Scholarship in the Education Research Community
“The Open Scholarship Survey (OSS) was administered to education researchers between Fall 2020-Summer 2022. The figures below summarize these respondents’ attitudes, behaviors, and perceptions about six core open scholarship practices: sharing data, sharing materials, preregistering studies, replication, sharing preprints, and reporting null results. When relevant, the responses from education researchers are shown alongside responses from the broader pool of respondents who have completed the OSS. Results are presented as figures with corresponding tables included in the appendix.
Education researchers are broadly supportive of open scholarship practices, with over 75% of respondents supporting replication and reporting null results; nearly 70% or more supporting data- and materials-sharing; and 55% supporting sharing preprints. The only behavior where fewer than half of respondents express support is preregistration. Education researchers’ support for these practices slightly trails the broader OSS sample but is still largely comparable – again with the exception of preregistration, where they lag behind their peers by a wider
margin.
While education researchers largely keep pace in their attitudes, they trail their peers more substantially in their behaviors. For example, 34% of education respondents have replicated at least once in their careers, compared to 51% among other OSS respondents. Likewise, 22% shared data in their most recent scholarly work compared to 41% in the broader sample. Education researchers also report being less likely to perform the practices going forward: 46% (compared to 63% in the broader sample) are likely to preregister in the future, and 74% (vs 84%) are
likely to report null results.:
A Targeted Review of Open Practices in Special Education Publications – Bryan G. Cook, Wilhelmina van Dijk, Isabel Vargas, Susan M. Aigotti, Jesse I. Fleming, Sean D. McDonald, Cassidi L. Richmond, Lindsay M. Griendling, Alan S. McLucas, Rachelle M. Johnson, 2023
Abstract: Open practices, such as preregistration, registered reports, open materials, open data, open analytic code, replication, open peer review, open access, and conflict-of-interest and funding statements, support the transparency, accessibility, and reproducibility of research and other scholarship. The purpose of this review was to examine the prevalence of these open practices in the special education literature. We reviewed a randomly selected sample of 250 articles published in special education journals in 2020. Results indicated that conflict-of-interest and funding statements were present in most articles; a small but meaningful proportion of articles provided open materials and were open access; and preregistration, registered reports, open data, open analytic code, open peer review, and replication were rarely or never observed. Recommendations for researching and supporting the use of open practices in special education scholarship are provided.
Embracing Open Science: A Pathway to an Inclusive and Collaborative Society – Open and Universal Science (OPUS) Project
“Transitioning to an open science society is a collective effort that requires the involvement of researchers, policymakers, institutions, and the public. By embracing open access publishing, sharing research data and code, promoting open collaboration, adopting open educational resources, encouraging open review and pre-registration, and advocating for open science policies, we can create a more inclusive, transparent, and collaborative scientific landscape. Embracing open science has the potential to revolutionize research, accelerate scientific discoveries, and address global challenges more effectively. Let us work together to pave the way towards an open science society and unlock the true potential of scientific knowledge for the betterment of humanity.”
Transparency in infectious disease research: meta-research survey of specialty journals | The Journal of Infectious Diseases | Oxford Academic
Abstract: Background
Infectious diseases carry large global burdens and have implications for society at large. Therefore, reproducible, transparent research is extremely important.
Methods
We evaluated transparency indicators (code and data sharing, registration, conflict and funding disclosures) in the 5340 PubMed Central Open Access articles published in 2019 or 2021 in the 9 most-cited specialty journals in infectious disease using the text-mining R package, rtransparent.
Results
5340 articles were evaluated (1860 published in 2019 and 3480 in 2021 (of which 1828 on COVID-19)). Text-mining identified code sharing in 98 (2%) articles, data sharing in 498 (9%), registration in 446 (8%), conflict of interest disclosures in 4209 (79%) and funding disclosures in 4866 (91%). There were substantial differences across the 9 journals: 1-9% for code sharing, 5-25% for data sharing, 1-31% for registration, 7-100% for conflicts of interest, and 65-100% for funding disclosures. Validation-corrected imputed estimates were 3%, 11%, 8%, 79% and 92%, respectively. There were no major differences between articles published in 2019 and non-COVID-19 articles in 2021. In 2021, non-COVID-19 articles had more data sharing (12%) than COVID-19 articles (4%).
Conclusions
Data sharing, code sharing, and registration are very uncommon in infectious disease specialty journals. Increased transparency is required.
Call for Volunteers: TOP Guidelines Advisory Board and Preregistration Template Evaluation Working Group
“Are you passionate about promoting transparency and openness in scientific research? The Center for Open Science (COS) is currently seeking volunteers for two opportunities. We seek colleagues to join (1) the Transparency and Openness Promotion (TOP) Guidelines Advisory Board and (2) the Preregistration Template Evaluation Working Group….”
Open science in health psychology and behavioral medicine: A statement from the Behavioral Medicine Research Council.
Abstract: Open Science practices include some combination of registering and publishing study protocols (including hypotheses, primary and secondary outcome variables, and analysis plans) and making available preprints of manuscripts, study materials, de-identified data sets, and analytic codes. This statement from the Behavioral Medicine Research Council (BMRC) provides an overview of these methods, including preregistration; registered reports; preprints; and open research. We focus on rationales for engaging in Open Science and how to address shortcomings and possible objections. Additional resources for researchers are provided. Research on Open Science largely supports positive consequences for the reproducibility and reliability of empirical science. There is no solution that will encompass all Open Science needs in health psychology and behavioral medicine’s diverse research products and outlets, but the BMRC supports increased use of Open Science practices where possible.
Registered report: Survey on attitudes and experiences regarding preregistration in psychological research | PLOS ONE
Abstract: Background
Preregistration, the open science practice of specifying and registering details of a planned study prior to knowing the data, increases the transparency and reproducibility of research. Large-scale replication attempts for psychological results yielded shockingly low success rates and contributed to an increasing demand for open science practices among psychologists. However, preregistering one’s studies is still not the norm in the field. Here, we conducted a study to explore possible reasons for this discrepancy.
Methods
In a mixed-methods approach, we conducted an online survey assessing attitudes, motivations, and perceived obstacles with respect to preregistration. Respondents (N = 289) were psychological researchers that were recruited through their publications on Web of Science, PubMed, PSYNDEX, and PsycInfo, and preregistrations on OSF Registries. Based on the theory of planned behavior, we predicted that positive attitudes (moderated by the perceived importance of preregistration) as well as a favorable subjective norm and higher perceived behavioral control positively influence researchers’ intention to preregister (directional hypothesis 1). Furthermore, we expected an influence of research experience on attitudes and perceived motivations and obstacles regarding preregistration (non-directional hypothesis 2). We analyzed these hypotheses with multiple regression models and included preregistration experience as a control variable.
Results
Researchers’ attitudes, subjective norms, perceived behavioral control, and the perceived importance of preregistration significantly predicted researchers’ intention to use preregistration in the future (see hypothesis 1). Research experience influenced both researchers’ attitudes and their perception of motivations to preregister, but not the perception of obstacles (see hypothesis 2). Descriptive reports on researchers’ attitudes, motivations and obstacles regarding preregistration are provided.
Discussion
Many researchers had already preregistered and had a rather positive attitude toward preregistration. Nevertheless, several obstacles were identified that may be addressed to improve and foster preregistration.
Journal Observatory – Toward integrated information about the openness of scholarly journals | NWO
“Researchers typically publish their work in scientific journals. In practice, however, it is difficult for them to get a good overview of all the features of these journals. Next to their publications being openly accessible, they want to know if the openness of a journal extends to peer review, pre-printing, preregistration, data sharing, metadata availability and related issues.
There are various data sources that provide information on specific aspects of the openness of journals, but there is hardly any integration of these data sources. This means that collecting information on journals often requires the combined use of multiple data sources, such as the Directory of Open Access Journals (DOAJ), Sherpa Romeo, and Quality Open Access Market, Transpose and Platform for Responsible Editorial Policies and TOP Factor.
The Journal Observatory brings together this information from different existing data sources on how journals organize their peer review, their quality assurance processes and all kinds of other open science practices that journals support. This enables researchers to adopt open science practices more easily. Moreover, research funders and other stakeholders are putting a significant effort into promoting open science practices in scholarly communication. The Journal Observatory will give them an opportunity to check compliance with their open science policies.’ ”
Nature welcomes Registered Reports
“This year marks the 50th anniversary of Nature’s decision to mandate peer review for all papers. It’s an appropriate time to introduce readers and authors to Registered Reports, a research-article format that Nature is offering from this week for studies designed to test whether a hypothesis is supported (see go.nature.com/3kivjh1).
The fundamental principle underpinning a Registered Report is that a journal commits to publishing a paper if the research question and the methodology chosen to address it pass peer review, with the result itself taking a back seat. For now, Nature is offering Registered Reports in the field of cognitive neuroscience and in the behavioural and social sciences. In the future, we plan to extend this to other fields, as well as to other types of study, such as more exploratory research.
Why are we introducing this format? In part to try to address publication bias, the tendency of the research system — editors, reviewers and authors — to favour the publication of positive over negative results. Registered Reports help to incentivize research regardless of the result. An elegant and robust study should be appreciated as much for its methodology as for its results….”
Nature welcomes Registered Reports
“This year marks the 50th anniversary of Nature’s decision to mandate peer review for all papers. It’s an appropriate time to introduce readers and authors to Registered Reports, a research-article format that Nature is offering from this week for studies designed to test whether a hypothesis is supported (see go.nature.com/3kivjh1).
The fundamental principle underpinning a Registered Report is that a journal commits to publishing a paper if the research question and the methodology chosen to address it pass peer review, with the result itself taking a back seat. For now, Nature is offering Registered Reports in the field of cognitive neuroscience and in the behavioural and social sciences. In the future, we plan to extend this to other fields, as well as to other types of study, such as more exploratory research.
Why are we introducing this format? In part to try to address publication bias, the tendency of the research system — editors, reviewers and authors — to favour the publication of positive over negative results. Registered Reports help to incentivize research regardless of the result. An elegant and robust study should be appreciated as much for its methodology as for its results….”