Applying Librarian Created Evaluation Tools to Determine Quality and Credibility of Open Access Library Science Journals

Abstract:  This article explores the application of journal quality and credibility evaluation tools to library science publications. The researchers investigate quality and credibility attributes of forty eight peer-reviewed library science journals with open access components using two evaluative tools developed and published by librarians. The results identify common positive and negative attributes of library science journals, compare the results of the two evaluation tools, and discuss their ease of use and limitations. Overall, the results show that while library science journals do not fall prey to the same concerning characteristics that librarians use to caution other researchers, there are several areas in which publishers can improve the quality and credibility of their journals.

 

Charting the Future of Science: Reforming scientific publishing for a new era of open knowledge – International Science Council

“Given the vital importance of these processes, the International Science Council has undertaken a rigorous review1 of current practices so as to identify contemporary needs for science publishing and to assess the extent to which the current system serves those needs. The Council’s studies led to development of eight essential principles for modern scientific publication, which were endorsed by over 90% of the membership present at its 2021 General Assembly. The principles are listed in Paper One, The Key Principles for Scientific Publishing accompanied by an analysis of the extent to which they are observed operationally….

The conclusion of this exercise has been that the current operation of the science publishing system falls far short in the principles that are essential to its effective operation, and that significant reform is needed. This conclusion is all the more important in light of the current global commitment to a new era of open science, in which new forms of openness are believed to be vital in enhancing the trustworthiness and utility of science as an essential human enterprise. The achievement of this open science vision depends fundamentally upon an effective, globally pervasive knowledge network based on the Council’s eight principles….

The ideal outcomes of reform would be systems that support four basic functions: 

funding of publishing that is internationally coordinated in ways that calibrate national financial contributions through indices of capacity to pay, with no charges for authors or readers; 
standards for publication that incorporate the ISC’s eight principles, and standard setting that is accountable to the international scientific community; 
agreement amongst universities to only use articles published in ways that adhere to these standards when evaluating scientific contributions; 
creating a functional record of the complete output of scientific articles as characterized above, making the record of science freely available to all. …”

Open Science Charter makes urgent appeal for open access | Research Information

“The Frontiers Research Foundation has launched an Open Science Charter, calling upon governments, research institutions and funders, the scientific community, and citizens everywhere to support mandatory open access to all publicly funded scientific knowledge by 2030.? 

The foundation stated: “The climate emergency poses an existential threat, demanding immediate and far-reaching actions. Our planet is edging closer to several irreversible tipping points, with dire consequences for all life. We need a wide spectrum of science solutions urgently and the greatest accelerator is simply to mandate open access to publicly funded articles and data.” …

The Charter will be presented at COP28 to participants looking for solutions during the panel ‘Open Science for Inclusive and Transformative Climate and Sustainability Innovation’ on 2 December in Dubai. Read and sign The Charter here.?”

The Open Science Charter

“We call on all academic publishers to prioritize the public good, reinforcing trust in science by committing to the following fundamental principles and actions:

Universal Access by 2030: Commit to transitioning all academic journals to fully open-access models by 2030.

Uphold Peer-Review Quality: Preserve and champion the core values of scientific publishing, including registration, validation, certification, and conservation of scientific findings. Publishers bear the responsibility of protecting the integrity of peer review, ensuring it adheres to universally recognized standards of research ethics.

Transparent Pricing Linked to Quality: Adopt transparent financial models that directly correlate the price of publication with the quality of services offered. This foundational principle, prevalent in other industries, is glaringly absent in academic publishing.

Strengthen Trust in Science: Make all scientific findings openly accessible and promote transparent publishing practices. The opacity and economic flaws of traditional subscription models enable and embolden malpractice in academic publishing. A competitive environment, paired with transparent pricing tied to service quality, can effectively root out these side effects….”

Prestigious journals struggle to reach even average reliability – JuSER

Abstract: The journal in which a scientist publishes is considered one of the most crucial factors determining their career. The underlying common assumption is that only the best scientists manage to publish in a highly selective tier of the most prestigious journals. However, data from several lines of evidence suggest that the methodological quality of scientific experiments does not increase with increasing rank of the journal. On the contrary, an accumulating body of evidence suggests the inverse: methodological quality and, consequently, reliability of published research works in several fields may be decreasing with increasing journal rank.The data supporting these conclusions circumvent confounding factors such as increased readership and scrutiny for these journals, focusing instead on quantifiable indicators of methodological soundness in the published literature, relying on, in part, semi-automated data extraction from often thousands of publications at a time. With the accumulating evidence over the last decade grew the realization that the very existence of scholarly journals, due to their inherent hierarchy, constitutes one of the major threats to publicly funded science: hiring, promoting and funding scientists who publish unreliable science eventually erodes public trust in science.

The Winners of the 2023 Einstein Foundation Award for Promoting Quality in Research – Einstein Foundation Award

“Initiative for Transparency in the Social Sciences (BITSS) advocates for ethical, transparent, and reproducible research in the social sciences in order to address the credibility crisis in the sciences and ensure that political decisions are based on robust data. BITSS develops the infrastructure required to support transparent social science research practice, such as the Preprint Service MetaArXiv and the Social Science Reproduction Platform (SSRP), which crowdsources attempts to assess and improve the reproducibility of social science research. The Initiative also conducts its own meta-research to validate scientific findings. BITSS has reached tens of thousands of social scientists across the world through its training and learning materials on Open Science practices. The Initiative was founded at the Center for Effective Global Action at the University of California in Berkeley in 2012 and has developed into one of the world’s most active organizations in the field of Open Science in the social sciences. The winner of the Institutional Award receives €200,000.”

Addressing the challenges in scholarly communication: the role of critical open access literacy | PUBMET

Abstract:  After twenty years of the open access movement, there has been growth in the uptake of open access, and some progress has been made in achieving its original goals; however, results have been far from satisfactory, with more than half of the research literature still closed (COKI, 2022; Curry, 2018). Moreover, new concerns have arisen, such as questionable quality and the reliability of peer review, in particular predatory publishing. There are also threats to equity, including stratifications of publishing as a consequence of the exclusionary character of the author-pays model of open access, and new risks of bias and exclusion in the means of transparent evaluation (Ross-Hellauer et al., 2022). It is argued that these challenges are the result of the uncritical narratives of openness and their narrow focus on access alone which fail to address inequitable power dynamics, systemic problems and the structural barriers in scholarly publishing and knowledge production (Perry, 2020; ibid.).

These cha(lle)nges in the system of scholarly communication, coupled with recent advances in technology and the tectonic transformations in the information environment, require new (pedagogical) approaches and foci that would enable researchers and students to understand and navigate such a complex environment; for this, a holistic and integrative approach to scholarly communication and information literacy is needed (ACRL, 2013; Špiranec, 2015).

Scholarly communication, including open access, is impossible without information literacy (Hebrang Grgi?, 2016). As they are both concerned with (access to) information, open access and in particular critical information literacy largely share the same goals, ethical dimensions and values of (social and epistemic) justice, equity and democratisation. Indeed, they have been considered instrumental to achieving these, and even proclaimed a panacea and deus ex machina for the current scientific, social and political challenges and crises (Hebrang Grgi?, 2016; Kapitzke, 2003). However, literature on the intersections and interplay between open access and information literacy has been limited.

This paper seeks to intertwine these two concepts more strongly, for mutual exchange and benefit, by analysing their convergent aspects as well as the role information literacy has in the context of the complexities of the scholarly communication system and in achieving open access. The paper builds on the few empirical studies of the intersections between the two concepts (e.g. Hebrang Grgi? (2016) which confirms the importance of ‘open access literacy’), but goes beyond their functional approaches, advocating a more holistic and critical approach to open access to help reinvent it and make a more substantial progress in open access. It argues that the transformations in the information environment and the scholarly communication system require not only basic skills and competences at the core of information literacy but also specific skills of, for instance, managing scientific data and publishing in open access sources. More importantly, what is also required is critical consciousness of all the aspects of the research process including the context, power relationships, and the privileged positions in knowledge production, publishing and dissemination, and an ability to evaluate the quality and reliability of information.

The paper proposes critical open access literacy as a pedagogical methodology and strategy to confront the challenges and enable a critical understanding of the contemporary information environment and scholarly communication. In line with the tenets of critical information literacy, critical open access literacy is directed at an analysis and critique of the economic, social, political, legal and technological conditions, aspects and implications of open access and the overall scholarly communication system, as well as the power dynamics, tensions and flaws within them. This will empower students and researchers to navigate this environment successfully and potentially become champions of its transformation to make it more meaningful, reliable, equitable and democratic.

Generative AI, Synthetic Contents, Open Educational Resources (OER), and Open Educational Practices (OEP): A New Front in the Openness Landscape – Open Praxis

Abstract:  This paper critically examines the transformation of the educational landscape through the integration of generative AI with Open Educational Resources (OER) and Open Educational Practices (OEP). The emergence of AI in content creation has ignited debate regarding its potential to comprehend and generate human language, creating content that is often indistinguishable from that produced by humans. This shift from organic (human-created) to synthetic (AI-created) content presents a new frontier in the educational sphere, particularly in the context of OER and OEP. The paper explores the generative AI’s capabilities in OER and OEP, such as automatic content generation, resource curation, updating existing resources, co-creation and facilitating collaborative learning. Nevertheless, it underscores the importance of addressing challenges like the quality and reliability of AI-generated content, data privacy, and equitable access to AI technologies. The critical discussion extends to a contentious issue, ownership in OER/OEP. While AI-generated works lack human authorship and copyright protection, the question of legal liability and recognition of authorship remains a significant concern. In response, the concept of prompt engineering and co-creation with AI is presented as a potential solution, viewing AI not as authors, but powerful tools augmenting authors’ abilities. By examining generative AI’s integration with OER and OEP, this paper encourages further research and discussion to harness AI’s power while addressing potential concerns, thereby contributing to the dialogue on responsible and effective use of generative AI in education.

 

New criteria for special issues – DOAJ News Service

“There has been a proliferation of special issues recently. DOAJ is responding to concerns from the community by adding new criteria to its Basic criteria. After today, all new applications submitted must comply with them….

Journals that publish special issues or other content curated by guest editors must adhere to these additional criteria:

The Editor-in-Chief must be responsible for the content of the entire journal, including all special issues, which must fall within the scope of the journal
Special issue articles must have the same editorial oversight as regular papers, including external peer review, and be clearly labelled
Journals must ensure that guest editors’ credentials are checked and approved
The Editor-in-Chief or dedicated board members must oversee the guest editors
Papers submitted to a special issue by the guest editor(s) must be handled under an independent review process and make up no more than 25% of the issue’s total

DOAJ will not accept a journal if all content in the last year/volume is published as special issues….”

Public funds being swallowed up by scientific journals with dubious articles | Science | EL PAÍS English

“A strange phenomenon has transformed the world scientific system. Suddenly, academic journals that were previously weekly or biweekly have started publishing several special issues each day. There are unusual cases, such as the hyperprolific environmental and health research journal International Journal of Environmental Research and Public Health (IJERPH), which last year published 17,000 scientific studies, 13 times more than in 2016, according to engineer Pablo Gómez Barreiro’s calculations. The theoretically biweekly journal has reached an output speed of six special issues per day and in recent years has been the preferred journal of Spanish scientists seeking to publish their work. The publishing house that owns it, MDPI, was founded in Switzerland by the Chinese chemist Shu-Kun Lin and has multiplied its income, thanks to a transfer of millions of euros of public money budgeted for science. It is a bubble that is about to burst, as a study by Gómez Barreiro and three other colleagues suggests….

Gómez Barreiro gives an extreme example: Professor Elsayed Tag Eldin, dean of the Faculty of Engineering at the University of the Future, a private institution in Cairo (Egypt) had barely published studies before, but this year he has already published 418, more than one every day, on all kinds of topics: Covid-19, solar panels, nanofluids, agriculture, even cyberattacks. He publishes more studies than anyone other scientist in the world….”

 

 

Conclusions and Way Forward

“Diamond OA is ultimately a means to an end: equitable access to scholarly publishing and reading, with a focus on the quality of the content rather than on the publishing venue. It recognises and rewards all contributions to the publication process. Furthermore, Diamond OA journals, repositories, and platforms represent community-driven, academic-led and -owned publishing initiatives. It is a publication model at the service of a diverse and more just society in which quality scholarly knowledge is a public good.

Bringing together the international community of Diamond OA stakeholders for the first time and sharing the common goal to support the quality, sustainability, usability, and equity of access to scholarly communication, we acknowledge that Diamond OA journals, repositories, and platforms can be as diverse as required by different research disciplines and epistemic traditions. We also recognise that different actors in various world regions and countries possess varying capacities to adopt and implement Diamond OA practices.

From 23 to 27 October 2023, we met in the city of Toluca, México, at the Global Summit on Diamond Open Access to reflect on and advance Diamond OA initiatives and practices. Extensive, insightful discussions led us to propose to strengthen collaboration within the Diamond OA sector at global level, adhering to the concepts and values stated above….”

Improving Wikipedia verifiability with AI | Nature Machine Intelligence

Abstract:  Verifiability is a core content policy of Wikipedia: claims need to be backed by citations. Maintaining and improving the quality of Wikipedia references is an important challenge and there is a pressing need for better tools to assist humans in this effort. We show that the process of improving references can be tackled with the help of artificial intelligence (AI) powered by an information retrieval system and a language model. This neural-network-based system, which we call SIDE, can identify Wikipedia citations that are unlikely to support their claims, and subsequently recommend better ones from the web. We train this model on existing Wikipedia references, therefore learning from the contributions and combined wisdom of thousands of Wikipedia editors. Using crowdsourcing, we observe that for the top 10% most likely citations to be tagged as unverifiable by our system, humans prefer our system’s suggested alternatives compared with the originally cited reference 70% of the time. To validate the applicability of our system, we built a demo to engage with the English-speaking Wikipedia community and find that SIDE’s first citation recommendation is preferred twice as often as the existing Wikipedia citation for the same top 10% most likely unverifiable claims according to SIDE. Our results indicate that an AI-based system could be used, in tandem with humans, to improve the verifiability of Wikipedia.

 

When We Publish: Accuracy and Quality Control in the Time of Open Access | Annual Review of Cell and Developmental Biology

“Many aspects of publishing are being fiercely discussed. If open-access policies make sense to remove barriers to information, especially publicly funded science, then straying from a subscription-based model raises other questions. Who pays for the editorial work that a journal puts into publishing a manuscript? This includes the salaries of editors and editorial staff who handle each submission up through text and figure checks, manuscript layout, and final production. Neither authors nor reviewers are paid for their services and contributions. Indeed, authors often have to pay hefty publication fees to have their research published, shifting the subsistence burden to the scientists in the form of author charges. Leading journals produced by some for-profit companies charge many thousands of dollars. For example, the Springer publication house charges $11,690 for immediate open-access publication in its premier journal Nature and Elsevier charges $9,900 to publish in Cell. This shifts the barrier from access-to-read to access-to-publish. The demand by funders and the public to provide open-access dissemination of research results becomes a special financial burden for journals that are published by nonprofit scientific societies or scientific publishers. Some of these already rely heavily on expert volunteers from the scientific community to act as editors to keep the editorial cost low, but publishing fees must still be recovered from either subscriptions or authors, making it difficult for truly grassroots journals to be competitive. One way to deal with this pressure is to have wealthy backers. In the case of the open-access journal eLife, for example, substantial funding is provided by the Wellcome Trust, the Max Planck Society, and the Howard Hughes Medical Institute. This allows the journal to charge a reasonable $2,000 for publication. Without this type of generous backing, nonprofit organizations are struggling to keep the cost of immediate open access low for their authors. For example, the American Society for Cell Biology’s open access charge for publication in Molecular Biology of the Cell is $3,396 (members) or $3,995 (nonmembers), the Rockefeller Press Journal of Cell Biology costs $6,000, and the Cold Spring Harbor Press Genes & Development is priced at $5,200. The Annual Reviews journals are pioneering a different approach, called Subscribe to Open (S2O). Instead of relying on authors to carry the burden of publishing costs, S2O places them on institutions. This is an evolution of the traditional subscription model in that libraries play a central role. It might be considered an honors system which trusts that libraries that have purchased subscriptions to the journal in the past, will continue to do so, raising the funds that allow the entire journal to be open for everyone. This year, the company anticipates that all 51 Annual Reviews volumes, including this volume of the Annual Review of Cell and Developmental Biology, will participate in the S2O model. Clearly, while access to much research is now freely accessible for all, the financial challenges have not yet been entirely solved….”

SocArXiv Papers | Preprint review services: Disrupting the scholarly communication landscape?

Abstract:  Preprinting has gained considerable momentum, and in some fields it has turned into a well-established way to share new scientific findings. The possibility to organise quality control and peer review for preprints is also increasingly highlighted, leading to the development of preprint review services. We report a descriptive study of preprint review services with the aim of developing a systematic understanding of the main characteristics of these services, evaluating how they manage preprint review, and positioning them in the broader scholarly communication landscape. Our study shows that preprint review services have the potential to turn peer review into a more transparent and rewarding experience and to improve publishing and peer review workflows. We are witnessing the growth of a mixed system in which preprint servers, preprint review services and journals operate mostly in complementary ways. In the longer term, however, preprint review services may disrupt the scholarly communication landscape in a more radical way.