Clearinghouse Standards of Evidence on the Transparency, Openness, and Reproducibility of Intervention Evaluations | SpringerLink

Abstract:  Clearinghouses are influential repositories of information on the effectiveness of social interventions. To identify which interventions are “evidence-based,” clearinghouses review intervention evaluations using published standards of evidence that focus primarily on internal validity and causal inferences. Open science practices can improve trust in evidence from evaluations on the effectiveness of social interventions. Including open science practices in clearinghouse standards of evidence is one of many efforts that could increase confidence in designations of interventions as “evidence-based.” In this study, we examined the policies, procedures, and practices of 10 federal evidence clearinghouses that review preventive interventions—an important and influential subset of all evidence clearinghouses. We found that seven consider at least one open science practice when evaluating interventions: replication (6 of 10 clearinghouses), public availability of results (6), investigator conflicts of interest (3), design and analysis transparency (3), study registration (2), and protocol sharing (1). We did not identify any policies, procedures, or practices related to analysis plan registration, data sharing, code sharing, material sharing, and citation standards. We provide a framework with specific recommendations to help federal and other evidence clearinghouses implement the Transparency and Openness Promotion (TOP) Guidelines. Our proposed “TOP Guidelines for Clearinghouses” includes reporting whether evaluations used open science practices, incorporating open science practices in their standards for receiving “evidence-based” designations, and verifying that evaluations used open science practices. Doing so could increase the trustworthiness of evidence used for policy making and support improvements throughout the evidence ecosystem.

 

Clearinghouse Standards of Evidence on the Transparency, Openness, and Reproducibility of Intervention Evaluations | SpringerLink

Abstract:  Clearinghouses are influential repositories of information on the effectiveness of social interventions. To identify which interventions are “evidence-based,” clearinghouses review intervention evaluations using published standards of evidence that focus primarily on internal validity and causal inferences. Open science practices can improve trust in evidence from evaluations on the effectiveness of social interventions. Including open science practices in clearinghouse standards of evidence is one of many efforts that could increase confidence in designations of interventions as “evidence-based.” In this study, we examined the policies, procedures, and practices of 10 federal evidence clearinghouses that review preventive interventions—an important and influential subset of all evidence clearinghouses. We found that seven consider at least one open science practice when evaluating interventions: replication (6 of 10 clearinghouses), public availability of results (6), investigator conflicts of interest (3), design and analysis transparency (3), study registration (2), and protocol sharing (1). We did not identify any policies, procedures, or practices related to analysis plan registration, data sharing, code sharing, material sharing, and citation standards. We provide a framework with specific recommendations to help federal and other evidence clearinghouses implement the Transparency and Openness Promotion (TOP) Guidelines. Our proposed “TOP Guidelines for Clearinghouses” includes reporting whether evaluations used open science practices, incorporating open science practices in their standards for receiving “evidence-based” designations, and verifying that evaluations used open science practices. Doing so could increase the trustworthiness of evidence used for policy making and support improvements throughout the evidence ecosystem.

Principles and Standards for OA Arrangements Between Libraries/Consortia and Smaller Independent Publishers

“The transition to Open Access requires change on the part of all stakeholders, and it is particularly crucial that there is active cross-stakeholder alignment focused on enabling smaller independent publishers to transition successfully. In recognition of this, cOAlition S and ALPSP have asked us to convene groups to work on shared principles, data, licenses, and workflows as outlined in our recent report (see https://www.coalition-s.org/open-access-agreements-with-smaller-publishers-require-active-cross-stakeholder-alignment-report-says/).

We are seeking expressions of interest in engaging with this work. Ideally, we would like a diverse array of people knowledgeable about the topic, who can represent their communities and influence working practices, and backed by organisations willing to communicate, champion, implement and maintain the outputs that will emerge from this work….”

COAR releases resource types vocabulary version 3.0 for repositories with new look and feel – COAR

“We are pleased to announce the release of version 3.0 of the resource types vocabulary. Since 2015, three COAR Controlled Vocabularies have been developed and are maintained by the Controlled Vocabulary Editorial Board: Resource types, access rights and version types.  These vocabularies have a new look and are now being managed using the iQvoc platform, hosted by the University of Vienna Library.

Using controlled vocabularies enables repositories to be consistent in describing their resources, helps with search and discovery of content, and allows machine readability for interoperability. The COAR vocabularies are available in several languages, supporting multilingualism across repositories. They also play a key role in making semantic artifacts and repositories compliant with the FAIR Principles, in particular when it comes to findability and interoperability….”

STM’s Peer Review Taxonomy To Be Formalized As An ANSI/NISO Standard | NISO website

“The National Information Standards Organization (NISO) and STM, the International Association of Scientific, Technical and Medical Publishers today announced the formation of a new NISO Working Group to formalize the Peer Review Taxonomy as an ANSI/NISO standard, following approval of the project by NISO Voting Members late last month. NISO invites volunteers to join the soon-to-be-formed Working Group, which will be merged with the existing STM Working Group. 

In 2019, STM recognized the need to support the industry in ensuring greater transparency and openness in peer review, which is an essential element of Open Science. This support includes harmonizing and better communicating definitions of discrete elements of these processes, so that members of the community—whether they be authors, reviewers, editors or readers—can quickly and easily recognize how to more productively participate in the creation and qualification of scholarly content. An STM Working Group was formed, which developed standard definitions and best practice recommendations for the communication of peer review processes, now available in its version 2.0 form. NISO will now take on this output and further develop it into a version 3.0, which will be made available for public comment and then published as a formal ANSI/NISO standard once it has been reviewed and approved by NISO Voting Members. …”

ROR and GRID: The Way Forward

“Earlier today, GRID announced that it will discontinue its schedule of public releases in Q4 2021. This decision marks an important and exciting milestone in the evolution of both organization registries.

ROR’s core mission is to be a community-led registry of open organization identifiers. While GRID has maintained an open registry of organization identifiers available CC0 to the community since 2015, it did not intend to serve as a community-driven initiative. Therefore, it was a natural arrangement to jump-start ROR with seed data from GRID, and accept ongoing updates from GRID while developing ROR to ultimately function independently as the community registry of record. The plan has always been that ROR would inevitably need to be able to diverge from GRID in order to more fully address the requirements and use cases that come with maintaining a community-based initiative. GRID’s recent decision aligns perfectly with the progress ROR has already made towards this goal….”

GRID passes the torch to ROR – Digital Science

“In 2015 Digital Science first released the Global Research Identifier Database (GRID), an open database of unique research-related organisation identifiers they had developed in-house over several years, for public use by the research community. In 2019 ROR, the Research Organization Registry, was founded as a community-driven initiative, mirroring the GRID database. With ROR coming of age and becoming independent from GRID, Digital Science has decided to pass on the torch to ROR and retire GRID from the public space, with a last public release in Q4 of 2021.

This might come as a surprise, as GRID and ROR have been co-existing and collaborating for quite some time now. GRID was initially created to fill a void, as no open organisation identifier was available for the open research space. As a community-driven initiative has now built upon GRID’s first initiative, two open organisation identifiers could be perceived as competing against each other. Digital Science has therefore decided to formally hand the torch over to ROR as the leading open organisation identifier. Digital Science will continue to use GRID internally- but focused on the Digital Science products and their users and clients….”