Peer Review Models, Publication Types, Open Access, and the Future

Abstract:  First off, we acknowledge that digital publishing can take many forms, from standard article or book formats that are enhanced by digital visualizations or interactivity to a variety of less text-centered formats. Likewise, digital projects may enter the world by means of self-publishing (Daniel’s main area of expertise) or through more established academic and commercial publishers (Martha’s main area of expertise). Peer review will shake out differently across these contexts, as will other factors around digital publishing, with each presenting its own challenges and opportunities.

 

Peer Review (beta)

“Peer Review is an experiment in scholarly publishing currently in Beta. It is a platform that enables crowdsourced peer review and public dissemination of scientific and academic papers. For now, the platform can only handle pre-prints. It is and will remain open source and diamond open access. It is currently being maintained by a single developer as a side project.

Peer Review uses a reputation system to ensure that review and refereeing is done by qualified peers. Reputation is primarily gained from publishing, but can also be gained from giving constructive reviews. Review is separated into pre-publish “review” and post-publish “refereeing”. Review is entirely focused on giving authors constructive, supportive feedback. Refereeing is intended to help maintain the integrity of the overall literature by identifying spam, malpractice, and misinformation. To learn more, please read how it works.”

Model(s) of the future? Overlay journals as an overlooked and emerging trend in scholarly communication | The Canadian Journal of Information and Library Science

Abstract: Overlay journals, a potentially overlooked model of scholarly communication, have seen a resurgence due to the increasing number of preprint repositories and preprints on coronavirus disease 2019 (COVID-19) related topics. Overlay journals at various stages of maturity were examined for unique characteristics, including whether the authors submitted their article to the journal, whether the peer reviews of the article were published by the overlay journal, and whether the overlay journals took advantage of opportunities for increased discovery. As librarians and researchers seek new, futuristic models for publishing, overlay journals are emerging as an important contribution to scholarly communication.

 

Towards peer review as a group engagement | JLIS.it

Abstract:  I discuss from an economic perspective two of the most recent suggestions to reform the peer review system: (a) payment to referees; (b) ex post peer review. I show that strong economic arguments militate against these ideas.

With respect to payment to referees I use results from the economic analysis of prosocial behavior and the private production of public goods, which show that the supply of monetary incentives has the paradoxical effect of reducing the willingness of agents to collaborate, insofar as they substitute intrincic motivation with extrinsic motivation.

With respect to ex post peer review, I show that it fails to offer sufficient incentives to researchers, since it is anonymous, depersonalized, and weak in its marginal impact on publishing decisions. I take this argument to criticize the lack of theorizing, in the side of radical proponents of Open access, about the conditions for transition from the subscription model to the Open model. It is this lack of critical attention to economic arguments that has led to the unintended but dramatic outcome of a net increase in the cost of scientific publishing, as documented in very recent papers.

Finally, I advance a proposal for admitting payments to referees, but not as individuals but as groups of researchers. I offer this idea to open discussion.

Preprint review should form part of PhD programmes and postdoc training

“We need a recognized, equitable way for PhD graduates to demonstrate the transferable skills they have gained.

For me, that way is to train them in preprint review….

Peer reviewing preprints would guarantee young researchers some concrete outputs that illustrate their ability to critique work, write about science and discuss subjects outside their immediate focus of research. By building such training into our scientific institutions, rather than relying on outside opportunities to which many do not have access, we can create a fairer system in which not just the well-connected can demonstrate their abilities….”

Let peer review be transparent | Communications Earth & Environment

“For all peer reviewed articles submitted from 23rd January 2023, we will publish the editor decision letters, reviewer reports and author responses, together with the published paper. Reviewers can choose to remain anonymous or reveal their identity….

At Communications Earth & Environment, we are convinced that opening up the scholarly discussions that precede publication of our articles will deepen understanding of the scientific process and help spark trust in science. We are enormously grateful for the time and effort our reviewers put into elaborating on the merits and shortcomings of papers with the aim to improve them. We are impressed by the detailed and positive letters our authors send back along with their revisions in response to the points raised by the reviewers. And we are proud to put care and thought into our editorial decisions and give constructive guidance to our authors by explaining our take on the reviewer comments….”

Can artificial intelligence assess the quality of academic journal articles in the next REF? | Impact of Social Sciences

“For journal article prediction, there is no knowledge base related to quality that could be leveraged to predict REF scores across disciplines, so only the machine learning AI approach is possible. All previous attempts to produce related predictions have used machine learning (or statistical regression, which is also a form of pattern matching). Thus, we decided to build machine learning systems to predict journal article scores. As inputs, based on an extensive literature review of related prior work, we chose: field and year normalised citation rate; authorship team size, diversity, productivity, and field and year normalised average citation impact; journal names and citation rates (similar to the Journal Impact Factor); article length and abstract readability; and words and phrases in the title, keywords and abstract. We used provisional REF2021 scores for journal articles with these inputs and asked the AI to spot patterns that would allow it to accurately predict REF scores….”

Tired of the profiteering in academic publishing? Vote with your feet. – Spatial Ecology and Evolution Lab

“First, let’s say one of the Olympian Editors asks you to review a manuscript for one of the profit-making esteem engines. You record on your CV that you have been asked to review for this journal (esteem points!), but you politely decline the invitation, explaining that you would rather your professional service go towards open science initiatives.

The editor at the esteem factory finds that her job has just become a lot harder than it used to be. It is hard to find reviewers, and the reviews aren’t as thorough or as good anymore. She keeps the line on her CV stating that she has been an editor at X (esteem points!), and then steps down at the next opportunity. She has better things to do than spend her days cajoling reluctant reviewers. And so it goes.

Being a discerning reviewer has nothing but benefits. There are no esteem points lost for the individual, and there is a higher turnover of editorial staff at high-esteem journals. This turnover means more opportunity and less competition for these positions, and it means the esteem hierarchy is flattened somewhat because, well, who hasn’t been an editor for Nature, and, besides, the stuff published there isn’t as good as it used to be. Overburdened reviewers have an important reason to do less reviewing; they are, through individual decision, changing the face of academic publishing and making science accessible to all….”

Enhancing community participation & understanding of preprint review – YouTube

“In this video, Jane Alfred (Catalyst Editorial Ltd) and Iratxe Puebla (ASAPbio) provide an overview of preprint review, its benefits to researchers and the research community and different platforms available for preprint review. The video also discusses good practices in preprint review and ways in which individual researchers can participate in preprint review.”

The Preprint Club – A cross-institutional, community-based approach to peer reviewing | bioRxiv

Abstract:  The academic community has been increasingly using preprints to disseminate their latest research findings quickly and openly. This early and open access of non-peer reviewed research warrants new means from the scientific community to efficiently assess and provide feedback to preprints. Yet, most peer review of scientific studies performed today are still managed by journals, each having their own peer review policy and transparency. However, approaches to uncouple the peer review process from journal publication are emerging. Additionally, formal education of early career researchers (ECRs) in peer reviewing is rarely available, hampering the quality of peer review feedback. Here, we introduce the Preprint Club, a cross-institutional, community-based approach to peer reviewing, founded by ECRs from the University of Oxford, Karolinska Institutet and Icahn School of Medicine at Mount Sinai. Over the past two years and using the collaborative setting of the Preprint Club, we have been discussing, assessing, and providing feedback on recent preprints in the field of immunology. In this article, we provide a blueprint of the Preprint Club basic structure, demonstrate its effectiveness, and detail the lessons we learned on its impact on peer review training and preprint author’s perception.

 

The Preprint Club – A cross-institutional, community-based approach to peer reviewing | bioRxiv

Abstract:  The academic community has been increasingly using preprints to disseminate their latest research findings quickly and openly. This early and open access of non-peer reviewed research warrants new means from the scientific community to efficiently assess and provide feedback to preprints. Yet, most peer review of scientific studies performed today are still managed by journals, each having their own peer review policy and transparency. However, approaches to uncouple the peer review process from journal publication are emerging. Additionally, formal education of early career researchers (ECRs) in peer reviewing is rarely available, hampering the quality of peer review feedback. Here, we introduce the Preprint Club, a cross-institutional, community-based approach to peer reviewing, founded by ECRs from the University of Oxford, Karolinska Institutet and Icahn School of Medicine at Mount Sinai. Over the past two years and using the collaborative setting of the Preprint Club, we have been discussing, assessing, and providing feedback on recent preprints in the field of immunology. In this article, we provide a blueprint of the Preprint Club basic structure, demonstrate its effectiveness, and detail the lessons we learned on its impact on peer review training and preprint author’s perception.

 

The Preprint Club – A cross-institutional, community-based approach to peer reviewing | bioRxiv

The academic community has been increasingly using preprints to disseminate their latest research findings quickly and openly. This early and open access of non-peer reviewed research warrants new means from the scientific community to efficiently assess and provide feedback to preprints. Yet, most peer review of scientific studies performed today are still managed by journals, each having their own peer review policy and transparency. However, approaches to uncouple the peer review process from journal publication are emerging. Additionally, formal education of early career researchers (ECRs) in peer reviewing is rarely available, hampering the quality of peer review feedback. Here, we introduce the Preprint Club, a cross-institutional, community-based approach to peer reviewing, founded by ECRs from the University of Oxford, Karolinska Institutet and Icahn School of Medicine at Mount Sinai. Over the past two years and using the collaborative setting of the Preprint Club, we have been discussing, assessing, and providing feedback on recent preprints in the field of immunology. In this article, we provide a blueprint of the Preprint Club basic structure, demonstrate its effectiveness, and detail the lessons we learned on its impact on peer review training and preprint author’s perception.

Recognizing Preprint Peer Review (Day 1, Part 1) – YouTube

“On December 1 and 2, 2022, HHMI, ASAPbio, and EMBO, co-sponsored a meeting, held at Janelia Research Campus, to promote community consensus and support for preprint peer review and to create funder, institutional, and journal policies that recognize both preprints with reviews, and reviews of preprints….”

To better reproducibility, peer-reviewing musts move forward! – ScienceDirect

“Free access to raw data and scripts is an avenue to be considered: it would encourage authors to transmit the clearest and most exhaustive methodology and would give readers a guarantee of confidence that would go beyond the very random anonymous peer-reviewing. Transparent reviewing, requiring time and skills, could then no longer be voluntary. Could we imagine that publishers themselves have a department dedicated to “guaranteeing reproducibility”? The reviewer could then concentrate on his core competence, which is to interpret results that he considers reliable….”