Attitudes, behaviours and experiences of authors of COVID-19 preprints

Abstract:  The COVID-19 pandemic caused a rise in preprinting, apparently triggered by the need for open and rapid dissemination of research outputs. We surveyed authors of COVID-19 preprints to learn about their experience of preprinting as well as publishing in a peer-reviewed journal. A key aim was to consider preprints in terms of their effectiveness for authors to receive feedback on their work. We also aimed to compare the impact of feedback on preprints with the impact of comments of editors and reviewers on papers submitted to journals. We observed a high rate of new adopters of preprinting who reported positive intentions regarding preprinting their future work. This allows us to posit that the boost in preprinting may have a structural effect that will last after the pandemic. We also saw a high rate of feedback on preprints but mainly through “closed” channels – directly to the authors. This means that preprinting was a useful way to receive feedback on research, but the value of feedback could be increased further by facilitating and promoting “open” channels for preprint feedback. At the same time, almost a quarter of the preprints that received feedback received comments resembling journal peer review. This shows the potential of preprint feedback to provide valuable detailed comments on research. However, journal peer review resulted in a higher rate of major changes in the papers surveyed, suggesting that the journal peer review process has significant added value compared to preprint feedback.

 

Open Science

“For a growing number of scientists, though, the process looks like this:

The data that the scientist collects is stored in an open access repository like figshare or Zenodo, possibly as soon as it’s collected, and given its own Digital Object Identifier (DOI). Or the data was already published and is stored in Dryad.
The scientist creates a new repository on GitHub to hold her work.
As she does her analysis, she pushes changes to her scripts (and possibly some output files) to that repository. She also uses the repository for her paper; that repository is then the hub for collaboration with her colleagues.
When she’s happy with the state of her paper, she posts a version to arXiv or some other preprint server to invite feedback from peers.
Based on that feedback, she may post several revisions before finally submitting her paper to a journal.
The published paper includes links to her preprint and to her code and data repositories, which makes it much easier for other scientists to use her work as starting point for their own research.

This open model accelerates discovery: the more open work is, the more widely it is cited and re-used. However, people who want to work this way need to make some decisions about what exactly “open” means and how to do it. You can find more on the different aspects of Open Science in this book.

This is one of the (many) reasons we teach version control. …”

PLOS partners with DataSeer to develop Open Science Indicators – The Official PLOS Blog

“To provide richer and more transparent information on how PLOS journals support best practice in Open Science, we’re going to begin publishing data on ‘Open Science Indicators’ observed in PLOS articles. These Open Science Indicators will initially include (i) sharing of research data in repositories, (ii) public sharing of code and, (iii) preprint posting, for all PLOS articles from 2019 to present. These indicators – conceptualized by PLOS and developed with DataSeer, using an artificial intelligence-driven approach – are increasingly important to PLOS achieving its mission. We plan to share the results openly to support Open Science initiatives by the wider community.”

Individual Award 2021 – Paul Ginsparg – Einstein Foundation Berlin

“Preprints have been shared in the physics community since the early 1950s but mostly among well established professors. Physicist Paul Ginsparg, who receives the Einstein Foundation’s Individual Award, set out to democratize access to scientific results. Today, his preprint server arXiv has spread to many other fields—and made science progress more efficient and fairer….”

Preprints in Chemistry: a Research Team’s Journey** – Ciriminna – ChemistryOpen – Wiley Online Library

Abstract:  The benefits of publishing research papers first in preprint form are substantial and long-lasting also in chemistry. Recounting the outcomes of our team’s nearly six-year journey through preprint publishing, we show evidence that preprinting research substantially benefits both early career and senior researchers in today’s highly interdisciplinary chemical research. These findings are of general value, as shown by analyzing the case of four more research teams based in economically developed and developing countries.

 

Easing Into Open Science: A Guide for Graduate Students and Their Advisors | Collabra: Psychology | University of California Press

Abstract:  This article provides a roadmap to assist graduate students and their advisors to engage in open science practices. We suggest eight open science practices that novice graduate students could begin adopting today. The topics we cover include journal clubs, project workflow, preprints, reproducible code, data sharing, transparent writing, preregistration, and registered reports. To address concerns about not knowing how to engage in open science practices, we provide a difficulty rating of each behavior (easy, medium, difficult), present them in order of suggested adoption, and follow the format of what, why, how, and worries. We give graduate students ideas on how to approach conversations with their advisors/collaborators, ideas on how to integrate open science practices within the graduate school framework, and specific resources on how to engage with each behavior. We emphasize that engaging in open science behaviors need not be an all or nothing approach, but rather graduate students can engage with any number of the behaviors outlined.

 

Five Years of ChemRxiv: Where We Are and Where We Go From Here | Chemical Education | ChemRxiv | Cambridge Open Engage

“ChemRxiv was launched on August 15, 2017 to provide researchers in chemistry and related fields a home for the immediate sharing of their latest research. In the past five years, ChemRxiv has grown into the premier preprint server for the chemical sciences, with a global audience and a wide array of scholarly content that helps advance science more rapidly. On the service’s fifth anniversary, we would like to reflect on the past five years and take a look at what is next for ChemRxiv.”

August ASAPbio Community Call – Promoting equity in visibility, curation and evaluation of preprints

“Preprints are freely accessible, but there are persistent disparities in the visibility and attention paid to preprints according to the authors’ institutions, geographical area, language and other backgrounds. In this interactive session, we will discuss how to promote equity in making preprints visible.”

ARC bans preprints, again

“The information pack for Excellence in Research for Australia 2023 advises that the Australian Research Council consulted on including preprints and feedback, “was overwhelmingly supportive of not including preprints as an eligible research output type.”

And lest anyone miss the point, “as a consequence, preprints will not be eligible for ERA submission.”

This follows last year’s fiasco when the ARC enforced a rule that many Research Offices had missed, banning pre-prints from grant applications, generating outrage on behalf of excluded applicants who did not know about it (umpteen CMM stories last year but September 23 covers it)….”

Reliability of citations of medRxiv preprints in articles published on COVID-19 in the world leading medical journals | PLOS ONE

Abstract:  Introduction

Preprints have been widely cited during the COVID-19 pandemics, even in the major medical journals. However, since subsequent publication of preprint is not always mentioned in preprint repositories, some may be inappropriately cited or quoted. Our objectives were to assess the reliability of preprint citations in articles on COVID-19, to the rate of publication of preprints cited in these articles and to compare, if relevant, the content of the preprints to their published version.

Methods

Articles published on COVID in 2020 in the BMJ, The Lancet, the JAMA and the NEJM were manually screened to identify all articles citing at least one preprint from medRxiv. We searched PubMed, Google and Google Scholar to assess if the preprint had been published in a peer-reviewed journal, and when. Published articles were screened to assess if the title, data or conclusions were identical to the preprint version.

Results

Among the 205 research articles on COVID published by the four major medical journals in 2020, 60 (29.3%) cited at least one medRxiv preprint. Among the 182 preprints cited, 124 were published in a peer-reviewed journal, with 51 (41.1%) before the citing article was published online and 73 (58.9%) later. There were differences in the title, the data or the conclusion between the preprint cited and the published version for nearly half of them. MedRxiv did not mentioned the publication for 53 (42.7%) of preprints.

Conclusions

More than a quarter of preprints citations were inappropriate since preprints were in fact already published at the time of publication of the citing article, often with a different content. Authors and editors should check the accuracy of the citations and of the quotations of preprints before publishing manuscripts that cite them.

Nature’s Take: what’s next for the preprint revolution

“In this first episode of Nature’s Take, we get four of Nature’s staff around microphones to get their expert take on preprints. These pre-peer-review open access articles have spiked in number over recent years and have cemented themselves as an integral part of scientific publishing. But this has not been without its issues.

In this discussion we cover a lot of ground. Amongst other things, we ask whether preprints could help democratise science or contribute to a loss of trust in scientists. We pick apart the relationship between preprints and peer-reviewed journals and tackle some common misconceptions. We ask how preprints have been used by different fields and how the pandemic has changed the game. And as we look to the future, we ask how preprints fit into the discussion around open access and even if they could do away with journals all together….”

Guest Post – Has Peer Review Created a Toxic Culture in Academia? Moving from ‘Battering’ to ‘Bettering’ in the Review of Academic Research – The Scholarly Kitchen

” It seems that many reviewers see their primary role as deflating the arguments and methodologies of the manuscripts they receive, often without any concern for the way the author will receive the comments or whether the critique can be addressed and revised….

A format that might be appealing, not only to authors and reviewers but to publishers as well, takes a page out of the work of HSS book publishers and how they review manuscripts.

 

One of the main differences between the STEM journal and HSS book submission process is that book acquisitions editors get involved in the process before the manuscript is complete (and sometimes before there is a manuscript at all). This process starts at a relatively early stage in the writing process, creating a situation whereby editors are incentivized to help authors and sign them up before other publishers can swoop in and publish it themselves. Consider the potential parallels with the increasing use of journal preprints, as a place where journal editors could hop in and start the process of working with authors at an early stage in the process. (It may also help that you can submit book proposals to multiple publishers simultaneously)….”

 

Frontiers | neuPrint: An open access tool for EM connectomics

Abstract:  Due to advances in electron microscopy and deep learning, it is now practical to reconstruct a connectome, a description of neurons and the chemical synapses between them, for significant volumes of neural tissue. Smaller past reconstructions were primarily used by domain experts, could be handled by downloading data, and performance was not a serious problem. But new and much larger reconstructions upend these assumptions. These networks now contain tens of thousands of neurons and tens of millions of connections, with yet larger reconstructions pending, and are of interest to a large community of non-specialists. Allowing other scientists to make use of this data needs more than publication—it requires new tools that are publicly available, easy to use, and efficiently handle large data. We introduce neuPrint to address these data analysis challenges. Neuprint contains two major components—a web interface and programmer APIs. The web interface is designed to allow any scientist worldwide, using only a browser, to quickly ask and answer typical biological queries about a connectome. The neuPrint APIs allow more computer-savvy scientists to make more complex or higher volume queries. NeuPrint also provides features for assessing reconstruction quality. Internally, neuPrint organizes connectome data as a graph stored in a neo4j database. This gives high performance for typical queries, provides access though a public and well documented query language Cypher, and will extend well to future larger connectomics databases. Our experience is also an experiment in open science. We find a significant fraction of the readers of the article proceed to examine the data directly. In our case preprints worked exactly as intended, with data inquiries and PDF downloads starting immediately after pre-print publication, and little affected by formal publication later. From this we deduce that many readers are more interested in our data than in our analysis of our data, suggesting that data-only papers can be well appreciated and that public data release can speed up the propagation of scientific results by many months. We also find that providing, and keeping, the data available for online access imposes substantial additional costs to connectomics research.

 

FAST principles to foster a positive preprint feedback culture – ASAPbio

“In order to foster broad and inclusive participation in this conversation on preprints, we need a positive culture around preprint feedback, but what does this look like in practice? This is the question that the ASAPbio preprint review cultural norms Working Group is tackling. The group is discussing what behaviors reflect the preprint feedback culture we would like to see, with the goal of developing a set of norms for all participants in preprint feedback. 

The Working Group is pleased to share an initial draft of principles for creating, responding to, and interpreting preprint feedback, clustered around four broad themes: Focused, Appropriate, Specific, Transparent – FAST. The table below provides elaboration for each of the FAST themes, and we invite feedback from the community on these principles….”

Supporting public preprint review through collaborative reviews – an update on ASAPbio’s crowd preprint review – ASAPbio

“Through our crowd preprint review activities we seek to draw on the collective input of a group of commenters who each can comment on the preprint according to their level of expertise and interest. We are midway through our activities for 2022 and we wanted to share an update on our progress.

What have we accomplished so far?

We had a great response from the community with over 120 crowd reviewers signed up so far, with strong representation of early career researchers. We have three groups which complete reviews of preprints in each of the disciplines below:

Cell biology – a crowd of 70 members reviews preprints posted on bioRxiv 
Biochemistry – a crowd of 35 researchers reviews preprints from bioRxiv 
Infectious diseases preprints in Portuguese – a crowd of 30 researchers provide reviews in Portuguese for preprints posted in SciELO Preprints

For each of the groups, a group of ASAPbio Fellows and partners from SciELO Preprints are involved in selecting preprints to review and summarizing the comments received. They also provide regular feedback on aspects of the process that can be adjusted or improved. 

We circulate a new preprint to each group every week and invite comments via a Google document. We have seen a great level of engagement from reviewers, and are particularly pleased to see the interactions among reviewers in the collaborative documents, where they provide comments and feedback to each other, not only about the preprints but also about queries that may arise during their review….”