No evidence that mandatory open data policies increase error correction | Nature Ecology & Evolution

Berberi, I., Roche, D.G. No evidence that mandatory open data policies increase error correction. Nat Ecol Evol (2022). https://doi.org/10.1038/s41559-022-01879-9

Preprint: https://doi.org/10.31222/osf.io/k8ver

Abstract: Using a database of open data policies for 199 journals in ecology and evolution, we found no detectable link between data sharing requirements and article retractions or corrections. Despite the potential for open data to facilitate error detection, poorly archived datasets, the absence of open code and the stigma associated with correcting or retracting articles probably stymie error correction. Requiring code alongside data and destigmatizing error correction among authors and journal editors could increase the effectiveness of open data policies at helping science self-correct.

 

Neither carrots nor sticks? Challenges surrounding data sharing from the perspective of research funding agencies—A qualitative expert interview study | PLOS ONE

Abstract:  Background

Data Sharing is widely recognised as crucial for accelerating scientific research and improving its quality. However, data sharing is still not a common practice. Funding agencies tend to facilitate the sharing of research data by both providing incentives and requiring data sharing as part of their policies and conditions for awarding grants. The goal of our article is to answer the following question: What challenges do international funding agencies see when it comes to their own efforts to foster and implement data sharing through their policies?

Methods

We conducted a series of sixteen guideline-based expert interviews with representatives of leading international funding agencies. As contact persons for open science at their respective agencies, they offered their perspectives and experiences concerning their organisations’ data sharing policies. We performed a qualitative content analysis of the interviews and categorised the challenges perceived by funding agencies.

Results

We identify and illustrate six challenges surrounding data sharing policies as perceived by leading funding agencies: The design of clear policies, monitoring of compliance, sanctions for non-compliance, incentives, support, and limitations for funders’ own capabilities. However, our interviews also show how funders approach potential solutions to overcome these challenges, for example by coordinating with other agencies or adjusting grant evaluation metrics to incentivise data sharing.

Discussion and conclusion

Our interviews point to existing flaws in funders’ data sharing policies, such as a lack of clarity, a lack of monitoring of funded researchers’ data sharing behaviour, and a lack of incentives. A number of agencies could suggest potential solutions but often struggle with the overall complexity of data sharing and the implementation of these measures. Funders cannot solve each challenge by themselves, but they can play an active role and lead joint efforts towards a culture of data sharing.

Facts and Figures for open research data

“Figures and case studies related to accessing and reusing the data produced in the course of scientific production.”

NOT-OD-22-189: Implementation Details for the NIH Data Management and Sharing Policy

“The purpose of this notice is to inform the extramural research community of implementation details for the NIH Policy for Data Management and Sharing (DMS Policy) affecting grant and cooperative agreement applications submitted for receipt dates on or after January 25, 2023. The specific changes to competing grant and cooperative agreement application instructions clarified below will be implemented with application form packages identified with a Competition ID of “FORMS-H” and incorporated into the forthcoming FORMS-H application guides.

Although the DMS Policy will apply also to Research and Development (R&D) contracts, NIH intramural research projects, and other funding agreements (e.g., Other Transactions), the forms changes and other implementation details provided in this Notice apply only to NIH extramural grant and cooperative agreement activities. Details applicable to R&D contracts will be incorporated into the appropriate Requests for Proposals, and details applicable to Other Transactions will be incorporated into the appropriate Research Opportunity Announcement….”

Ten simple rules for maximizing the recommendations of the NIH data management and sharing plan | PLOS Computational Biology

Abstract:  The National Institutes of Health (NIH) Policy for Data Management and Sharing (DMS Policy) recognizes the NIH’s role as a key steward of United States biomedical research and information and seeks to enhance that stewardship through systematic recommendations for the preservation and sharing of research data generated by funded projects. The policy is effective as of January 2023. The recommendations include a requirement for the submission of a Data Management and Sharing Plan (DMSP) with funding applications, and while no strict template was provided, the NIH has released supplemental draft guidance on elements to consider when developing a plan. This article provides 10 key recommendations for creating a DMSP that is both maximally compliant and effective.

 

Adoption of World Health Organization Best Practices in Clinical Trial Transparency Among European Medical Research Funder Policies | Global Health | JAMA Network Open | JAMA Network

Abstract:  Importance  Research funders can reduce research waste and publication bias by requiring their grantees to register and report clinical trials.

Objective  To determine the extent to which 21 major European research funders’ efforts to reduce research waste and publication bias in clinical trials meet World Health Organization (WHO) best practice benchmarks and to investigate areas for improvement.

Design, Setting, and Participants  This cross-sectional study was based on 2 to 3 independent assessments of each funder’s publicly available documentation and validation of results with funders during 2021. Included funders were the 21 largest nonmultilateral public and philanthropic medical research funders in Europe, with a combined budget of more than US $22 billion.

Exposures  Scoring of funders using an 11-item assessment tool based on WHO best practice benchmarks, grouped into 4 broad categories: trial registries, academic publication, monitoring, and sanctions. Funder references to reporting standards were captured.

Main Outcomes and Measures  The primary outcome was funder adoption or nonadoption of 11 policy and monitoring measures to reduce research waste and publication bias as set out by WHO best practices. The secondary outcomes were whether and how funder policies referred to reporting standards. Outcomes were preregistered after a pilot phase that used the same outcome measures.

Results  Among 21 of the largest nonmultilateral public and philanthropic funders in Europe, some best practices were more widely adopted than others, with 14 funders (66.7%) mandating prospective trial registration and 6 funders (28.6%) requiring that trial results be made public on trial registries within 12 months of trial completion. Less than half of funders actively monitored whether trials were registered (9 funders [42.9%]) or whether results were made public (8 funders [38.1%]). Funders implemented a mean of 4 of 11 best practices in clinical trial transparency (36.4%) set out by WHO. The extent to which funders adopted WHO best practice items varied widely, ranging from 0 practices for the French Centre National de la Recherche Scientifique and the ministries of health of Germany and Italy to 10 practices (90.9%) for the UK National Institute of Health Research. Overall, 9 funders referred to reporting standards in their policies.

Conclusions and Relevance  This study found that many European medical research funder policy and monitoring measures fell short of WHO best practices. These findings suggest that funders worldwide may need to identify and address gaps in policies and processes.

Medical research funders across Europe tighten rules on clinical trial reporting

“Eight of the 21 largest public and philanthropic medical research funders in Europe are stepping up their efforts to improve clinical reporting, following an assessment that found widespread gaps in existing research waste safeguards.

 

 

 

At present, many academic clinical trials in Europe fail to make their results public, wasting taxpayers’ money and leaving large gaps in the medical evidence base.

 

 

 

The public institutions that hand out money to medical researchers can prevent such waste by putting into place eleven safeguards recommended by the World Health Organisation. …”

WorldFAIR: Global cooperation on FAIR data policy and practice – Kick-Off Meeting introduces major new initiative to advance implementation of the FAIR data principles – CODATA, The Committee on Data for Science and Technology

“The WorldFAIR project held a successful kick-off meeting online on 9 June 2022, with representatives from the European Commission and all nineteen participating organisations from Europe and beyond.

The WorldFAIR project is a major new global collaboration between partners from thirteen countries across Africa, Australasia, Europe, and North and South America.  WorldFAIR will advance implementation of the FAIR data principles, in particular those for Interoperability, by developing a cross-domain interoperability framework and recommendations for FAIR assessment in a set of eleven disciplines or cross-disciplinary research areas….”

Many researchers were not compliant with their published data sharing statement: mixed-methods study – Journal of Clinical Epidemiology

Abstract:  Objectives

To analyse researchers’ compliance with their Data Availability Statement (DAS) from manuscripts published in open access journals with the mandatory DAS.

 

Study Design and Setting

We analyzed all articles from 333 open-access journals published during January 2019 by BioMed Central. We categorized types of DAS. We surveyed corresponding authors who wrote in DAS that they would share the data. A consent to participate in the study was sought for all included manuscripts. After accessing raw data sets, we checked whether data were available in a way that enabled re-analysis.

 

Results

Of 3556 analyzed articles, 3416 contained DAS. The most frequent DAS category (42%) indicated that the datasets are available on reasonable request. Among 1792 manuscripts in which DAS indicated that authors are willing to share their data, 1670 (93%) authors either did not respond or declined to share their data with us. Among 254 (14%) of 1792 authors who responded to our query for data sharing, only 122 (6.8%) provided the requested data.

 

Conclusion

Even when authors indicate in their manuscript that they will share data upon request, the compliance rate is the same as for authors who do not provide DAS, suggesting that DAS may not be sufficient to ensure data sharing.

New Guidance to Ensure Federally Funded Research Data Equitably Benefits All of America | The White House

“To help ensure that access is shared equitably by all Americans, the White House Office of Science and Technology Policy (OSTP) has been working for nearly a decade to ensure that Federal agencies with research and development budgets of at least $100 million develop plans to deposit Federally funded data into online digital repositories.

To continue this effort, today OSTP is releasing the report Guidance on Desirable Characteristics of Data Repositories for Federally Funded Research. This guidance contains clearly defined desirable characteristics for two classes of online research data repositories: a general class appropriate for all types of Federally funded data—including free and easy access—and a specific class that has special considerations for the sharing of human data, including additional data security and privacy considerations. Federal agencies can use this guidance to provide more consistent information to their research communities about sharing Federally funded data with the public. 

Agencies can also use this guidance to ensure uniformity as they invest in their own digital repository infrastructure and to make their research data resources more findable, accessible, interoperable, and reusable. It is expected that this guidance will not be static, but rather, will be updated as needed, as new modes of data storage and management emerge and agency needs evolve.  Ultimately, this guidance—along with the agency efforts detailed in OSTP’s recent report to Congress—will help make Federally funded research data more accessible to the American public. The release of this guidance is one of many steps that OSTP is taking to advance equitable delivery of research and strengthen Federal public access policies….”

Desirable Characteristics of Data Repositories for Federally Funded Research

“A key element of the required data management plans is specification of the digital, online, public access data repository or repositories researchers will use for preserving, maintaining, and providing access to Federally supported research data. While some agencies designate specific repositories to be used for particular types of data (e.g., genomic data, topographical data) or a particular type of research (e.g., Arctic research, social sciences research), for much Federally funded research, the selection of a suitable repository is delegated to the researcher or their institutions. Some agencies provide information to assist researchers in the selection of data repositories. However, this information is inconsistent across agencies, including among those that support research in similar or related disciplines. Until now, agencies had not identified the desirable characteristics of data repositories on which to base their assistance to researchers and their institutions. To improve the management and sharing of data from Federally funded research, agencies agreed to leverage the SOS to identify a consistent set of desirable characteristics for data repositories that all agencies could incorporate into the instructions they provide to the research community for selecting data repositories. By establishing common expectations, agencies intend to reduce the complexity for the research community–including investigators, program officers, data managers, librarians, and others–in complying with Federal data sharing policies. Federal agencies can also use this set of characteristics to develop or identify suitable repositories for particular types of data. To carry out this work, agencies within the SOS drew upon existing expertise and experience with data management and sharing. They also reviewed existing criteria promulgated by non-governmental organizations involved in the certification of data repositories (e.g., International Standards Organization, International Science Council). Agencies also took into account input received on a draft set of characteristics issued for public comment in January 2020 (Box 1)….

This guidance document presents the set of desirable characteristics for repositories agreed to by Federal agencies, reflecting the input that OSTP and SOS received and evaluated. It addresses a nearterm need to provide greater consistency across agencies, recognizing that future steps will be needed to better coordinate data storage and management to make data from Federally funded research more findable, accessible, interoperable, and reusable (FAIR), 4 as well as more equitable, inclusive, secure, and trustworthy. The endeavor to improve public access to Federally-supported research makes for a more open government, facilitates evidence-based decision making, and yields greater returns on Americans’ investments in R&D. This guidance document constitutes one set of tools that agencies can use to advance those goals….”

 

Gearing Up for 2023 Part II: Implementing the NIH Data Management and Sharing Policy – NIH Extramural Nexus

“NIH has a long history of developing consent language and, as such, our team worked across the agency – and with you! – to develop a new resource that shares best practices for developing informed consents to facilitate data/biospecimen storage and sharing for future use.  It also provides modifiable sample language that investigators and IRBs can use to assist in the clear communication of potential risks and benefits associated with data/biospecimen storage and sharing.  In developing this resource, we engaged with key federal partners, as well as scientific societies and associations.  Importantly, we also considered the 102 comments from stakeholders in response to a RFI that we issued in 2021.

As for our second resource, we are requesting public comment on protecting the privacy of research participants when data is shared. I think I need to be upfront and acknowledge that we have issued many of these types of requests over the last several months and NIH understands the effort that folks take to thoughtfully respond.  With that said, we think the research community will greatly benefit from this resource and we want to hear your thoughts on whether it hits the mark or needs adjustment….”

Using the State of Open Data survey to put the NIH Policy on Data Management and Sharing into practice

“Join us for a webinar on how the State of Open Data survey — the annual survey on researchers’ attitudes toward open data and data sharing — can help your institution put the NIH Policy on Data Management and Sharing into practice. …”