Supporting open and reproducible science | Stanford News

“Open science is a broad goal that includes making data, data analysis, scientific processes and published results easier to access, understand and reproduce. It’s an appealing concept but, in practice, open science is difficult and, often, the costs seem to exceed the benefits. Recognizing both the shortfalls and the promise of open science, Stanford University’s Center for Open and REproducible Science (CORES) – which is part of Stanford Data Science – hopes to make the practice of open science easier, more accessible and more rewarding.

 

Since its launch in September 2020, CORES has been hard at work on the center’s first major efforts. These include developing a guide for open science practices at Stanford – called the “Open by Design” handbook – and producing workshops and a lecture series to help people learn about and contribute to open science across the university….”

FAIR Principles for Research Software (FAIR4RS Principles) | RDA

“Research software is a fundamental and vital part of research worldwide, yet there remain significant challenges to software productivity, quality, reproducibility, and sustainability. Improving the practice of scholarship is a common goal of the open science, open source software and FAIR (Findable, Accessible, Interoperable and Reusable) communities, but improving the sharing of research software has not yet been a strong focus of the latter.

To improve the FAIRness of research software, the FAIR for Research Software (FAIR4RS) Working Group has sought to understand how to apply the FAIR Guiding Principles for scientific data management and stewardship to research software, bringing together existing and new community efforts. Many of the FAIR Guiding Principles can be directly applied to research software by treating software and data as similar digital research objects. However, specific characteristics of software — such as its executability, composite nature, and continuous evolution and versioning — make it necessary to revise and extend the principles.

This document presents the first version of the FAIR Principles for Research Software (FAIR4RS Principles). It is an outcome of the FAIR for Research Software Working Group (FAIR4RS WG).

The FAIR for Research Software Working Group is jointly convened as an RDA Working Group, FORCE11 Working Group, and Research Software Alliance (ReSA) Task Force.”

Glossary Organizing document – instructions for contributors (original doc) – Google Docs

“We invite all interested to: write definitions, comment on existing definitions, add alternative definitions where applicable, and suggest relevant references. If you feel that key terms are missing, please add it – you can let us know, or ask contact us with suggestions in the FORRT slack or email sam.parsons@psy.ox.ac.uk (please CC flavio.azevedo@uni-jena.de during the period Feb 12 to March 1st). The full list of terms will form part of a larger glossary to be hosted on https://FORRT.org, once all terms have been added, the lead writing team (Parsons, Azevedo, & Elsherif) will develop an abridged version to submit as a manuscript. We outline the kinds of contributions and their correspondence to authorship in more detail in the next section. Don’t forget to add your name and details to the contributions spreadsheet….”

Association for Interdisciplinary Meta-Research and Open Science (AIMOS)

“AIMOS (the Association for Interdisciplinary Meta-Research and Open Science) seeks to advance the interdisciplinary field of meta-research by bringing together and supporting researchers in that field.

Science aims to produce robust knowledge and the concept of reproducible experiments is central to this. However the past decade has seen a ‘reproducibility crisis’ in science.

Across a number of scientific fields, such as psychology and preclinical medicine, large-scale replication projects have failed to produce evidence supporting the findings of many original studies. Meta-research will address this challenge head on….”

Enquêter sur les pratiques numériques au sein des communautés scientifiques: regards sur les outils et les données au prisme de la science ouverte – Sciencesconf.org

From Google’s English:  “THE CALL FOR PARTICIPATION AWAITS PROPOSALS ON THE FOLLOWING SUBJECTS:

The epistemological and ethical issues of observation, analysis, representation and communication on the practices of scientific communities
Methodological issues depending on whether the approaches are quantitative, qualitative or mixed
The challenges of visualizing results
The issues of reproducibility and accessibility of results (data, source code) according to legal constraints
The challenges of recounting collective practices in the context of individual observation
The challenges of receiving these surveys in their evaluative, prescriptive or even performative nature
Prescription issues in the development of applications, or any other classification solution for software or other digital services intended for scientific communities…”

Day One Project: Re-envisioning Reporting of Scientific Methods

“The information contained in the methods section of the overwhelming majority of research publications is insufficient to definitively evaluate research practices, let alone reproduce the work. Publication—and subsequent reuse—of detailed scientific methodologies can save researchers time and money, and can accelerate the pace of research overall. However, there is no existing mechanism for collective action to improve reporting of scientific methods. The Biden-Harris Administration should direct research-funding agencies to support development of new standards for reporting scientific methods. These standards would (1) address ongoing challenges in scientific reproducibility, and (2) benefit our nation’s scientific enterprise by improving research quality, reliability, and efficiency. …

Common standards are already proving invaluable for the recognition and reuse of open data. The same principles could be applied to open methods….

Compliance could be achieved through a combination of “push” incentives from publishers and “pull” incentives from funders. As is already happening for open-data standards, federal agencies can require researchers to adhere to open-methods standards in order to receive federal funding, and scientific journals can require researchers to adhere to open-methods standards in order to be eligible for publication….”  

Easyreporting simplifies the implementation of Reproducible Research layers in R software

Abstract:  During last years “irreproducibility” became a general problem in omics data analysis due to the use of sophisticated and poorly described computational procedures. For avoiding misleading results, it is necessary to inspect and reproduce the entire data analysis as a unified product. Reproducible Research (RR) provides general guidelines for public access to the analytic data and related analysis code combined with natural language documentation, allowing third-parties to reproduce the findings. We developed easyreporting, a novel R/Bioconductor package, to facilitate the implementation of an RR layer inside reports/tools. We describe the main functionalities and illustrate the organization of an analysis report using a typical case study concerning the analysis of RNA-seq data. Then, we show how to use easyreporting in other projects to trace R functions automatically. This latter feature helps developers to implement procedures that automatically keep track of the analysis steps. Easyreporting can be useful in supporting the reproducibility of any data analysis project and shows great advantages for the implementation of R packages and GUIs. It turns out to be very helpful in bioinformatics, where the complexity of the analyses makes it extremely difficult to trace all the steps and parameters used in the study.

 

Utrecht University Recognition and Rewards Vision

“By embracing Open Science as one of its five core principles1, Utrecht University aims to accelerate and improve science and scholarship and its societal impact. Open science calls for a full commitment to openness, based on a comprehensive vision regarding the relationship with society. This ongoing transition to Open Science requires us to reconsider the way in which we recognize and reward members of the academic community. It should value teamwork over individualism and calls for an open academic culture that promotes accountability, reproducibility, integrity and transparency, and where sharing (open access, FAIR data and software) and public engagement are normal daily practice. In this transition we closely align ourselves with the national VSNU program as well as developments on the international level….”

Dockstore: enhancing a community platform for sharing reproducible and accessible computational protocols | Nucleic Acids Research | Oxford Academic

Abstract:  Dockstore (https://dockstore.org/) is an open source platform for publishing, sharing, and finding bioinformatics tools and workflows. The platform has facilitated large-scale biomedical research collaborations by using cloud technologies to increase the Findability, Accessibility, Interoperability and Reusability (FAIR) of computational resources, thereby promoting the reproducibility of complex bioinformatics analyses. Dockstore supports a variety of source repositories, analysis frameworks, and language technologies to provide a seamless publishing platform for authors to create a centralized catalogue of scientific software. The ready-to-use packaging of hundreds of tools and workflows, combined with the implementation of interoperability standards, enables users to launch analyses across multiple environments. Dockstore is widely used, more than twenty-five high-profile organizations share analysis collections through the platform in a variety of workflow languages, including the Broad Institute’s GATK best practice and COVID-19 workflows (WDL), nf-core workflows (Nextflow), the Intergalactic Workflow Commission tools (Galaxy), and workflows from Seven Bridges (CWL) to highlight just a few. Here we describe the improvements made over the last four years, including the expansion of system integrations supporting authors, the addition of collaboration features and analysis platform integrations supporting users, and other enhancements that improve the overall scientific reproducibility of Dockstore content.

 

 

REPEAT (Reproducible Evidence: Practices to Enhance and Achieve Transparency)

“Replication is a cornerstone of the scientific method. Historically, public confidence in the validity of healthcare database research has been low. Drug regulators, patients, clinicians, and payers have been hesitant to trust evidence from databases due to high profile controversies with overturned and conflicting results. This has resulted in underuse of a potentially valuable source of real-world evidence.?…

Division of Phamacoepidemiology & Pharmacoeconomics [DoPE]

Brigham & Women’s Hospital and Harvard Medical School.”