Does grant funding foster research impact? Evidence from France*

Abstract:  Over the last fifteen years, European countries have increasingly relied on competitive grants to allocate research funding, replacing the more traditional block funding model. Policymakers are interested in assessing the effectiveness of the grant funding model in producing impactful research. However, the literature aiming to quantify the effect of grants on the resulting research’s impact is scant. In the French context, we compare the impact of scientific articles resulting from the support of competitive grants from the main national funding agency with the impact of articles not supported by grants. We rely on publication acknowledgments to retrieve funding information and on citation data to assess the articles’ impact. We find that articles supported by competitive grants receive more citations than articles not supported by grants in the long run, while the difference is not statistically significant in the short run. We find heterogeneity in the effect of grant funding on citations across fields.

Open Access Publishing: A Study of UC Berkeley Faculty Views and Practices

Abstract:  This project focused on open access (OA) publishing, which enhances researcher productivity and impact by increasing dissemination of, and access to, research. The study looked at the relationship between faculty’s attitudes toward OA and their OA publishing practices, including the roles of funding availability and discipline. The project team compared University of California Berkeley (Berkeley) faculty’s answers to questions related to OA from the 2018 Ithaka Faculty Survey with the faculty’s scholarly output in the Scopus database. Faculty Survey data showed that 71% of Berkeley faculty, compared to 64% of faculty nationwide, support a transition to OA publishing. However, when selecting a journal to publish in, faculty indicated that a journal having no cost to publish in was more important than having no cost to read. After joining faculty’s survey responses and their publication output, the data sample included 4,413 articles published by 479 Berkeley faculty from 2016 to 2019. With considerable disciplinary differences, the OA publication output for this sample, using data from Unpaywall, represented 72% of the total publication output. The study focused on Gold OA articles, which usually require authors to pay Article Processing Charges (APCs) and which accounted for 18% of the publications. Overall, the study found a positive correlation between publishing Gold OA and the faculty’s support for OA (no cost to read). In contrast, the correlation between publishing Gold OA and the faculty’s concern about publishing cost was weak. Publishing costs concerned faculty in all subject areas, whether or not their articles reported research funding. Thus, Berkeley Library’s efforts to pursue transformative publishing agreements and prioritize funding for a program subsidizing publishing fees seem like effective strategies to increase OA. 

Library Impact Research Report: Open Access Publishing: A Study of UC Berkeley Faculty Views and Practices – Association of Research Libraries

Overall, the UC Berkeley study found a positive correlation between publishing gold OA and the faculty’s support for OA (no cost to read). In contrast, the correlation between publishing gold OA and the faculty’s concern about publishing cost was weak. Publishing costs concerned faculty in all subject areas, whether or not their articles reported research funding. Therefore, UC Berkeley Library’s efforts to pursue transformative publishing agreements and prioritize funding for a program subsidizing publishing fees seem like effective strategies to increase OA.

Correlating article citedness and journal impact: an empirical investigation by field on a large-scale dataset | SpringerLink

Abstract:  In spite of previous research demonstrating the risks involved, and counsel against the practice as early as 1997, some research evaluations continue to use journal impact alone as a surrogate of the number of citations of hosted articles to assess the latter’s impact. Such usage is also taken up by research administrators and policy-makers, with very serious implications. The aim of this work is to investigate the correlation between the citedness of a publication and the impact of the host journal. We extend the analyses of previous literature to all STEM fields. Then we also aim to assess whether this correlation varies across fields and is stronger for highly cited authors than for lowly cited ones. Our dataset consists of a total of almost one million authorships of 2010–2019 publications authored by about 28,000 professors in 230 research fields. Results show a low correlation between the two indicators, more so for lowly cited authors as compared to highly cited ones, although differences occur across fields.

 

Open access in scholarly publishing: Where are we now? | Research Information

“Notably, 2023 marks a decade since two important events. Not only David Bowie’s return to releasing records, but Research Councils UK’s (the predecessor to UKRI) launch of its open access policy. This was a watershed moment for UK research, a clear statement of intent to make open access a full-scale reality. But 10 years on, it is pertinent to ask, where are we now?…

In fact, 2022 certainly witnessed a continuing paradigm shift, particularly UKRI’s open access policy coming into effect for articles and conference proceedings. This represents a step-change to full and immediate open access for publicly funded research, and essentially incorporates Plan S into the UK research landscape. Similar policies have been launched by other funders, including the National Institute for Health & Care Research and Cancer Research UK. 

 

Moreover, 2022 saw the release of the Research Excellence Framework (REF) 2021 results, marking another milestone for open access. REF 2021’s open access mandate for journal articles and conference proceedings has arguably had the greatest impact in driving open access engagement by researchers. What was once a niche pursuit that was opposed by many researchers is now overwhelmingly regarded as an everyday part of the research lifecycle. There is a growing sense of positive engagement too, with researchers increasingly publishing open access because they want to and not just because they have to….”

SocArXiv Papers | The potential of inclusive and collaborative Open Research processes at the science-policy interface

Abstract:  Proponents of Open Research often assert that it can support evidence-based policy-making by making scientific outputs more readily available to policy-makers and other policy actors, yet there is little empirical work to support or deny this claim. This paper fills this void by reporting the results of a qualitative study with researchers who regularly work at the science-policy interface. We found that there is little evidence that Open Research products, namely Open Access and open data, which aim at increasing access and transparency, are useful in integrating science into policy-making. Instead, we found that the cognitive accessibility of research outputs is more important than their physical accessibility, and that inclusive and collaborative Open Research processes, like upstream engagement, co-creation and Citizen Science, are most effective at doing so.

 

‘The attitude of publishers is a barrier to open access’ | UKSG

“Transitioning to open research is incredibly important for the University of Liverpool for two reasons: the external environment we are now operating in, and our own philosophy and approach to research.

But there are barriers, particularly the research culture and the attitude of publishers….

In my experience, the biggest barrier is culture: researchers are used to operating in a particular way. Changing practice and mindset takes time and must be conducted sensitively.

Open research benefits all researchers, so having their support on this journey is vitally important.

Some researchers are concerned that publishing their work open access has implications for their intellectual property (IP) rights. In fact, this is a perceived problem, since the same IP protections apply to all work, whether published behind a paywall or published open access.

Despite the recognition that citation metrics are not a suitable proxy for research assessment, some researchers continue to seek the kudos of publishing in a so-called prestige journal with a high-impact factor, such as ‘Nature’.  They see this as a key career goal and worry their progression will falter without this achievement….

So, while I acknowledge there has been significant progress towards open access globally, and in particular compliance with UKRI’s open access policy, the attitude of publishers which are driven by profit margins continues to be an unacceptable barrier….”

The Effect of Open Access on Scholarly and Societal Metrics of Impact in the ASHA Journals | Journal of Speech, Language, and Hearing Research

Abstract:  Purpose:

 This study examined the effect of open access (OA) status on scholarly and societal metrics of impact (citation counts and altmetric scores, respectively) across manuscripts published in the American Speech-Language-Hearing Association (ASHA) Journals.

Method:

 

Three thousand four hundred nineteen manuscripts published in four active ASHA Journals were grouped across three access statuses based on their availability to the public: Gold OA, Green OA, and Closed Access. Two linear mixed-effects models tested the effects of OA status on citation counts and altmetric scores of the manuscripts.

Results: 

Both Green OA and Gold OA significantly predicted a 2.70 and 5.21 respective increase in citation counts compared with Closed Access manuscripts (p < .001). Gold OA was estimated to predict a 25.7-point significant increase in altmetric scores (p < .001), but Green OA was only marginally significant (p = .68) in predicting a 1.44 increase in altmetric scores relative to Closed Access manuscripts.

Discussion:

 

Communication sciences and disorders (CSD) research that is fully open receives more online attention and, overall, more scientific attention than research that is paywalled or available through Green OA methods. Additional research is needed to understand secondary variables affecting these and other scholarly and societal metrics of impact across studies in CSD. Ongoing support and incentives to reduce the inequities of OA publishing are critical for continued scientific advancement.

The Effect of Open Access on Scholarly and Societal Metrics of Impact in the ASHA Journals | Journal of Speech, Language, and Hearing Research

Abstract:  Purpose:

 This study examined the effect of open access (OA) status on scholarly and societal metrics of impact (citation counts and altmetric scores, respectively) across manuscripts published in the American Speech-Language-Hearing Association (ASHA) Journals.

Method:

 

Three thousand four hundred nineteen manuscripts published in four active ASHA Journals were grouped across three access statuses based on their availability to the public: Gold OA, Green OA, and Closed Access. Two linear mixed-effects models tested the effects of OA status on citation counts and altmetric scores of the manuscripts.

Results: 

Both Green OA and Gold OA significantly predicted a 2.70 and 5.21 respective increase in citation counts compared with Closed Access manuscripts (p < .001). Gold OA was estimated to predict a 25.7-point significant increase in altmetric scores (p < .001), but Green OA was only marginally significant (p = .68) in predicting a 1.44 increase in altmetric scores relative to Closed Access manuscripts.

Discussion:

 

Communication sciences and disorders (CSD) research that is fully open receives more online attention and, overall, more scientific attention than research that is paywalled or available through Green OA methods. Additional research is needed to understand secondary variables affecting these and other scholarly and societal metrics of impact across studies in CSD. Ongoing support and incentives to reduce the inequities of OA publishing are critical for continued scientific advancement.

Research on the relationships between discourse leading indicators and citations: perspectives from altmetrics indicators of international multidisciplinary academic journals | Emerald Insight

Abstract:  Purpose

This paper aims to analyze the relationships between discourse leading indicators and citations from perspectives of integrating altmetrics indicators and tries to provide references for comprehending the quantitative indicators of scientific communication in the era of open science, constructing the evaluation indicator system of the discourse leading for academic journals and then improving the discourse leading of academic journals.

Design/methodology/approach

Based on the theory of communication and the new pattern of scientific communication, this paper explores the formation process of academic journals’ discourse leading. This paper obtains 874,119 citations and 6,378,843 altmetrics indicators data from 65 international multidisciplinary academic journals. The relationships between indicators of discourse leading (altmetrics) and citations are studied by using descriptive statistical analysis, correlation analysis, principal component analysis, negative binomial regression analysis and marginal effects analysis. Meanwhile, the connotation and essential characteristics of the indicators, the strength and influence of the relationships are further analyzed and explored. It is proposed that academic journals’ discourse leading is composed of news discourse leading, social media discourse leading, peer review discourse leading, encyclopedic discourse leading, video discourse leading and policy discourse leading.

Findings

It is discovered that the 15 altmetrics indicators data have a low degree of centralization to the center and a high degree of polarization dispersion overall; their distribution patterns do not follow the normal distributions, and their distributions have the characteristics of long-tailed right-peaked curves. Overall, 15 indicators show positive correlations and wide gaps exist in the number of mentions and coverage. The academic journals’ discourse leading significantly affects total cites. When altmetrics indicators of international mainstream academic and social media platforms are used to explore the connotation and characteristics of academic journals’ discourse leading, the influence or contribution of social media discourse, news discourse, video discourse, policy discourse, peer review discourse and encyclopedia discourse on the citations decreases in turn.

Originality/value

This study is innovative from the academic journal level to analyze the deep relationships between altmetrics indicators and citations from the perspective of correlation. First, this paper explores the formation process of academic journals’ discourse leading. Second, this paper integrates altmetrics indicators to study the correlation between discourse leading indicators and citations. This study will help to enrich and improve basic theoretical issues and indicators’ composition, provide theoretical support for the construction of the discourse leading evaluation system for academic journals and provide ideas for the evaluation practice activities.

H index, journal citation indicator, and other impact factors in neurosurgical publications – Is there a ‘cost factor’ that determines the quality? – ScienceDirect

Abstract:  Objective

There has been an increase in number of Neurosurgical publications including open access approach over the recent years. We aim to compare the Journal’s performance and its relationship to the submission fee incurred in publication. We have performed an in-depth analysis of various Neurosurgical journals’ performance in terms of the bibliometrics and have attempted to determine if there is any impact of the cost incurred to the quality of Journal’s output.

Methods

We identified 53 journals issuing neurosurgical-related work. Quantitative analysis from various search engines involved obtaining H indices, journal citations indicators, and other journal’s metrics such as immediacy index and 5-year impact factor utilising Journal Citation Reports from Clarivate software. Open access fees, coloured print costs, and individual subscription fees were collected. Correlations were produced using Spearmen Rho (?), p<0.05.

Results

Median H indices for 53 journals is 54 (range: 0-292), with journal citation indicators median reported at 0.785 (range: 0-2.45). Median immediacy indices are 0.797 (range: 0-4.076) and the median for 5-year impact factor is 2.76 (range: 0-12.704). There is a very strong positive correlation between JCI and immediacy indices, JCI and 5-year impact factor and 5-year impact factor and immediacy indices (? >0.7, p < .05). There is a moderate positive correlation between the H index and JCI (?= 0.399, p = 0.004). It is unclear whether there is any correlation between the indices and the OA costs and subscription costs for personal usage respectively (p > 0.05).

Conclusions

Our analysis indicates that larger costs incurred for open access fees and subscription costs for personal use are not clearly reflected upon the journals’ performance and this is quantified by utilising various indices. There appears to be a strong association within performance across the journals’ metrics. It would be beneficial to include learning about the bibliometric indices’ impact for research publications in the medical education training to maximise the quality of the scientific work produced and increase the visibility of the information produced. The potential full movement to OA exclusive journals would form a significant barrier for junior researchers, small institutions, or full time-trainee doctors with limited funding available. This study suggests the need for a robust measurement of the journals’ output and the quality of the work produced.

Do open-access dermatology articles have higher citation counts than those with subscription-based access? | PLOS ONE

Abstract:  Background

Open-access (OA) publishing is increasingly prevalent in dermatology, and many journals now offer hybrid options, including conventional (subscription-based access [SA]) publishing or OA (with an author publishing charge) in a subscription journal. OA publishing has been noted in many disciplines, but this has been rarely studied in dermatology.

Methods

Using the Clarivate Journal Citation Report, we compiled a list of English-language dermatology hybrid OA journals containing more than 5% OA articles. We sampled any OA review or original research article in 4 issues from 2018 to 2019 and matched an equal number of SA articles. Citation count, citation count excluding self-citations and view counts found using Scopus and Altmetrics score were recorded for each article. Statistical analyses were performed using logistic and negative binomial models using R software.

Results

Twenty-seven hybrid dermatology journals were found, and 538 articles were sampled (269 OA, 269 SA). For both original research and review articles, OA articles had significantly higher mean citation counts (mean 13.2, standard deviation [SD] 17.0) compared to SA articles (mean 7.9, SD 8.8) (odds ratio [OR] 1.04; 95% CI 1.02–1.05; P < .001) including when adjusted for time from publication. Original research OA articles had significantly higher citation counts than original research SA articles (excluding self-citations; OR, 1.03; 95% CI, 1.01–1.05; P = .003), and review articles also had OA citation advantage than review SA articles (OR, 1.06; 95% CI, 1.02–1.11; P = .008). There was, however, no significant difference in citation counts between review articles and original research articles (OR, 1.00; 95% CI, 0.19–5.31; P = 1.000).

There was no significant difference seen in view counts (OA: mean±SD 17.7±10.8; SA: mean±SD 17.1±12.4) and Altmetric score (OA: mean±SD 13.2±47.8; SA: mean±SD 6.3±25.0) between OA and SA articles. Potential confounders included the fact that more OA articles were published in Europe than in Asia, and pharmaceutical-funded articles were more likely to be published OA.

Conclusions

We noted a higher citation count for OA articles than SA articles in dermatology hybrid journals. However, dermatology researchers should take into account confounding factors when deciding whether to increase the impact of their work by selecting OA over SA publishing.

Does it pay to pay? A comparison of the benefits of open-access publishing across various sub-fields in Biology | bioRxiv

Abstract:  Authors are often faced with the decision of whether to maximize impact or minimize costs when publishing the results of their research. For example, to potentially improve impact via increased accessibility, many subscription-based journals now offer the option of paying a fee to publish open access (i.e., hybrid journals), but this solution excludes authors who lack the capacity to pay to make their research accessible. Here, we tested if paying to publish open access in a subscriptionbased journal benefited authors by conferring more citations relative to closed access articles. We identified 146,415 articles published in 152 hybrid journals in the field of biology from 2013-2018 to compare the number of citations between various types of open access and closed access articles. In a simple generalized linear model analysis of our full dataset, we found that publishing open access in hybrid journals that offer the option confers an average citation advantage to authors of 17.8 citations compared to closed access articles in similar journals. After taking into account the number of authors, journal impact, year of publication, and subject area, we still found that open access generated significantly more citations than closed access (p < 0.0001). However, results were complex, with exact differences in citation rates among access types impacted by these other variables. This citation advantage based on access type was even similar when comparing open and closed access articles published in the same issue of a journal (p < 0.0001). However, by examining articles where the authors paid an article processing charge, we found that cost itself was not predictive of citation rates (p = 0.14). Based on our findings of access type and other model parameters, we suggest that, in most cases, paying for access does confer a citation advantage. For authors with limited budgets, we recommend pursuing open access alternatives that do not require paying a fee as they still yielded more citations than closed access. For authors who are considering where to submit their next article, we offer additional suggestions on how to balance exposure via citations with publishing costs.

 

[2212.05416] In which fields are citations indicators of research quality?

Abstract:  Citation counts are widely used as indicators of research quality to support or replace human peer review and for lists of top cited papers, researchers, and institutions. Nevertheless, the extent to which citation counts reflect research quality is not well understood. We report the largest-scale evaluation of the relationship between research quality and citation counts, correlating them for 87,739 journal articles in 34 field-based Units of Assessment (UoAs) from the UK. We show that the two correlate positively in all academic fields examined, from very weak (0.1) to strong (0.5). The highest correlations are in health, life sciences and physical sciences and the lowest are in the arts and humanities. The patterns are similar for the field classification schemes of Scopus and this http URL. We also show that there is no citation threshold in any field beyond which all articles are excellent quality, so lists of top cited articles are not definitive collections of excellence. Moreover, log transformed citation counts have a close to linear relationship with UK research quality ranked scores that is shallow in some fields but steep in others. In conclusion, whilst appropriately field normalised citations associate positively with research quality in all fields, they never perfectly reflect it, even at very high values.