Why I am building Arcadia.

“I walked away with the backing to establish a new startup, Trove….

At Trove, we are led by curiosity and remain committed to learning and sharing the knowledge we’ve gained. There is no need to lock up the lessons we’ve learned from others in the tick community. In fact, we have sought their feedback, and we will publish most of our protocols, tools, and datasets without paywalls or delays. It’s the most rigorous any of us have ever had to be, and all of this is in the absence of journals. Our work may ultimately translate into products that could be useful to many more people….

For all these reasons, I have decided to take the best parts of my experiences to build a new research organization called Arcadia Science. I am co-founding Arcadia with yet another fierce woman scientist Prachee Avasthi, who is a leader among leaders in the fight for open science. …”

Clearinghouse Standards of Evidence on the Transparency, Openness, and Reproducibility of Intervention Evaluations | SpringerLink

Abstract:  Clearinghouses are influential repositories of information on the effectiveness of social interventions. To identify which interventions are “evidence-based,” clearinghouses review intervention evaluations using published standards of evidence that focus primarily on internal validity and causal inferences. Open science practices can improve trust in evidence from evaluations on the effectiveness of social interventions. Including open science practices in clearinghouse standards of evidence is one of many efforts that could increase confidence in designations of interventions as “evidence-based.” In this study, we examined the policies, procedures, and practices of 10 federal evidence clearinghouses that review preventive interventions—an important and influential subset of all evidence clearinghouses. We found that seven consider at least one open science practice when evaluating interventions: replication (6 of 10 clearinghouses), public availability of results (6), investigator conflicts of interest (3), design and analysis transparency (3), study registration (2), and protocol sharing (1). We did not identify any policies, procedures, or practices related to analysis plan registration, data sharing, code sharing, material sharing, and citation standards. We provide a framework with specific recommendations to help federal and other evidence clearinghouses implement the Transparency and Openness Promotion (TOP) Guidelines. Our proposed “TOP Guidelines for Clearinghouses” includes reporting whether evaluations used open science practices, incorporating open science practices in their standards for receiving “evidence-based” designations, and verifying that evaluations used open science practices. Doing so could increase the trustworthiness of evidence used for policy making and support improvements throughout the evidence ecosystem.

 

Methods as a scientific asset – The Official PLOS Blog

“Clear, complete, and open methods increase credibility and support lasting impact. Documenting and sharing methodologies has interrelated scientific and reputational benefits for individuals and the community. 

Making methods public creates a positive impression. Having the option to review detailed methods increases readers’ trust, whether or not they consult the documentation. 
Researchers can more easily reproduce results with detailed open methods. Authors who want to apply the method in their own research can do so more efficiently if the approach is described in detail and easy to find online.
Strong, easy-to-follow methods are more likely to be used in future research, and by extension more likely to be cited, bringing fresh eyes to the original and helping it to remain relevant over time….”

Public access to protocols of contemporary cancer randomized clinical trials | Trials | Full Text

Abstract:  Access to randomized clinical trial (RCT) protocols is necessary for the interpretation and reproducibility of the study results, but protocol availability has been lacking. We determined the prevalence of protocol availability for all published cancer RCTs in January 2020. We found that only 36.1% (48/133) of RCTs had an accessible protocol and only 11.3% of RCTs (15/133) had a publicly accessible protocol that was not behind a paywall. Only 18.0% (24/133) of RCTs were published in conjunction with the protocol on the journal website. In conclusion, few cancer RCTs have an accessible research protocol. Journals should require publication of RCT protocols along with manuscripts to improve research transparency.

 

Communicating reusable research with peer-reviewed protocols from PLOS ONE and protocols.io

“Register below for our up-coming webinar where PLOS, protocols.io and the research community will introduce an innovative new publishing option that gives researchers recognition for their contributions to developing and optimising research methods, and advances open science. Developed with researchers and in partnership with the protocols.io team, Lab Protocol articles in PLOS ONE consist of two interlinked components that together describe peer-reviewed, reusable methods.

 

This webinar will cover:

The importance of sharing peer-reviewed protocols and methods to advance open science and meet researchers’ needs

A new, innovative partnership between protocols.io and PLOS ONE

An overview of the options for publishing protocols and methods at PLOS ONE

How peer-reviewed protocols at PLOS ONE complement other types of publication

The researcher perspective and experience with sharing and reusing verified methodologies

Questions from the audience and discussion with panelists…”

Day One Project: Re-envisioning Reporting of Scientific Methods

“The information contained in the methods section of the overwhelming majority of research publications is insufficient to definitively evaluate research practices, let alone reproduce the work. Publication—and subsequent reuse—of detailed scientific methodologies can save researchers time and money, and can accelerate the pace of research overall. However, there is no existing mechanism for collective action to improve reporting of scientific methods. The Biden-Harris Administration should direct research-funding agencies to support development of new standards for reporting scientific methods. These standards would (1) address ongoing challenges in scientific reproducibility, and (2) benefit our nation’s scientific enterprise by improving research quality, reliability, and efficiency. …

Common standards are already proving invaluable for the recognition and reuse of open data. The same principles could be applied to open methods….

Compliance could be achieved through a combination of “push” incentives from publishers and “pull” incentives from funders. As is already happening for open-data standards, federal agencies can require researchers to adhere to open-methods standards in order to receive federal funding, and scientific journals can require researchers to adhere to open-methods standards in order to be eligible for publication….”  

Dockstore: enhancing a community platform for sharing reproducible and accessible computational protocols | Nucleic Acids Research | Oxford Academic

Abstract:  Dockstore (https://dockstore.org/) is an open source platform for publishing, sharing, and finding bioinformatics tools and workflows. The platform has facilitated large-scale biomedical research collaborations by using cloud technologies to increase the Findability, Accessibility, Interoperability and Reusability (FAIR) of computational resources, thereby promoting the reproducibility of complex bioinformatics analyses. Dockstore supports a variety of source repositories, analysis frameworks, and language technologies to provide a seamless publishing platform for authors to create a centralized catalogue of scientific software. The ready-to-use packaging of hundreds of tools and workflows, combined with the implementation of interoperability standards, enables users to launch analyses across multiple environments. Dockstore is widely used, more than twenty-five high-profile organizations share analysis collections through the platform in a variety of workflow languages, including the Broad Institute’s GATK best practice and COVID-19 workflows (WDL), nf-core workflows (Nextflow), the Intergalactic Workflow Commission tools (Galaxy), and workflows from Seven Bridges (CWL) to highlight just a few. Here we describe the improvements made over the last four years, including the expansion of system integrations supporting authors, the addition of collaboration features and analysis platform integrations supporting users, and other enhancements that improve the overall scientific reproducibility of Dockstore content.

 

 

An interview with protocols.io: CEO Lenny Teytelman on partnering with PLOS – The Official PLOS Blog

“Our ongoing partnership with protocols.io led to a new and exciting PLOS ONE article type, Lab Protocols, which offers a new avenue to share research in-line with the principles of Open Science. This two-part article type gives authors the best of both platforms: protocols.io hosts the step-by-step methodology details while PLOS ONE publishes a companion article that contextualizes the study and orchestrates peer review of the material. The end result is transparent and reproducible research that can help accelerate scientific discovery.

Read our interview with protocols.io CEO and co-founder Dr. Lenny Teytelman where he discusses what triggered the concept for the company, how researchers can benefit from this collaboration and how his team settled on such an adorable racoon logo….”

An interview with protocols.io: CEO Lenny Teytelman on partnering with PLOS – The Official PLOS Blog

“Our ongoing partnership with protocols.io led to a new and exciting PLOS ONE article type, Lab Protocols, which offers a new avenue to share research in-line with the principles of Open Science. This two-part article type gives authors the best of both platforms: protocols.io hosts the step-by-step methodology details while PLOS ONE publishes a companion article that contextualizes the study and orchestrates peer review of the material. The end result is transparent and reproducible research that can help accelerate scientific discovery.

Read our interview with protocols.io CEO and co-founder Dr. Lenny Teytelman where he discusses what triggered the concept for the company, how researchers can benefit from this collaboration and how his team settled on such an adorable racoon logo….”

Assessment of transparency indicators across the biomedical literature: How open is open?

Abstract:  Recent concerns about the reproducibility of science have led to several calls for more open and transparent research practices and for the monitoring of potential improvements over time. However, with tens of thousands of new biomedical articles published per week, manually mapping and monitoring changes in transparency is unrealistic. We present an open-source, automated approach to identify 5 indicators of transparency (data sharing, code sharing, conflicts of interest disclosures, funding disclosures, and protocol registration) and apply it across the entire open access biomedical literature of 2.75 million articles on PubMed Central (PMC). Our results indicate remarkable improvements in some (e.g., conflict of interest [COI] disclosures and funding disclosures), but not other (e.g., protocol registration and code sharing) areas of transparency over time, and map transparency across fields of science, countries, journals, and publishers. This work has enabled the creation of a large, integrated, and openly available database to expedite further efforts to monitor, understand, and promote transparency and reproducibility in science.

 

 

What’s Next for Open Science – Making the Case for Open Methods – The Scholarly Kitchen

“But data alone is not enough, and an enormous hole in the open science movement has been the lagging attention paid to the reporting of research methodologies. Being able to review the data behind a study does indeed allow one to see if a researcher’s analysis and the conclusions drawn are accurate for that dataset. But it does little to validate the quality and accuracy of the dataset itself. If I don’t know how you got that data, I have no idea if it’s any good, and I certainly don’t stand any chance of replicating it.

A big problem here is that the scant information offered by most journals’ Materials and Methods sections is insufficient to have any chance of repeating what the original authors did. Often when describing a technique, an author will merely cite a previous paper where they used that technique…which also cites a previous paper, which also cites a previous paper and the wild goose chase is on. This lack of detailed methodology reporting is something of an anachronism, driven by decades of a print-dominant publication model aimed at reducing the number of pages in journal issues, along with a lack of incentives to improve methods reporting.

As open data requires the public availability of the data behind any published research conclusions, so open methods would require the public availability of detailed documentation of the procedures used to gather and analyze those data. Like open data, this can happen through a variety of routes — publication of the method as a standalone paper cited by the research paper, detailed documentation of the methods used in the paper itself (or its supplementary materials), or citation of a deposited documentation of the method in a repository such as protocols.io….”

Submit your Lab and Study Protocols to PLOS ONE! – The Official PLOS Blog

“PLOS ONE’s array of publication options that push the boundaries of Open Science continues to expand. We’re happy to announce two new article types that improve reproducibility and transparency, and allow researchers to receive credit for their contributions to study design: Lab Protocols and Study Protocols. 

These new article types complement other Open Science developments at PLOS ONE, such as Registered Reports, and support PLOS’ mission to accelerate progress in science and medicine. Adding reliable and accessible methods that can be built upon to the scientific record, Lab and Study Protocols support robust, reproducible science.  As reproducible science helps accelerate discoveries, such contributions deserve notice. …”