Raising research quality will require collective action

” Institutions are committing to working together to determine how their cultural practices, such as emphasizing the importance of novelty, discovery and priority, undermine the value of replication, verification and transparency. That is the goal of the UK Reproducibility Network, which I co-founded earlier this year. It started as informal groups of researchers at individual institutions that met with representatives from funders and publishers (including Nature) who were open to discussions about how best to align open-science initiatives — reproducibility sections in grant applications and reporting checklists in article submissions, for example. Now institutions themselves are cooperating to consider larger changes, from training to hiring and promotion practices….

Our ten university members span the United Kingdom from Aberdeen to Surrey, and we expect that list to grow. Each will appoint a senior academic to focus on research quality and improvement. Figuring out which system-level changes are needed and how to make them happen will now be someone’s primary responsibility, not a volunteer activity. What changes might ensue? Earlier this year, the University of Bristol, where I work, made the use of data sharing and other open-research practices an explicit criterion for promotion….

But these cultural changes might falter. Culture eats strategy for breakfast — grand plans founder on the rocks of implicit values, beliefs and ways of working. Top-down initiatives from funders and publishers will fizzle out if they are not implemented by researchers, who review papers and grant proposals. Grass-roots efforts will flourish only if institutions recognize and reward researchers’ efforts.

Funders, publishers and bottom-up networks of researchers have all made strides. Institutions are, in many ways, the final piece of the jigsaw. Universities are already investing in cutting-edge technology and embarking on ambitious infrastructure programmes. Cultural change is just as essential to long-term success.”

Full article: Open Access and Promotion and Tenure Evaluation Plans at the University of Wisconsin–Eau Claire

Abstract:  Department and program evaluation plans at the University of Wisconsin–Eau Claire were examined to see if these documents provide evidence that could be used to justify supporting the publication of peer-reviewed open access articles toward tenure and promotion. In an earlier study, the authors reveal that faculty members at the University of Wisconsin–Eau Claire are more unaware of open access publishing than their counterparts at larger universities. These findings dovetail with other studies that show that faculty members are reluctant to publish in open access journals because of concerns about the quality of those journals. The existing body of scholarship suggests that tenure-line faculty fear publishing in open access journals because it could adversely impact their chances of promotion and tenure. The authors of this current study sought to determine if department and program evaluation plans could influence negative perceptions faculty have of open access journals. The implications of this study for librarians, scholarly communication professionals, tenure-line faculty, departments, and programs are addressed.


CAUT signs the San Francisco Declaration on Research Assessment | CAUT

“The Canadian Association of University Teachers (CAUT) has signed the San Francisco Declaration on Research Assessment (DORA), an international initiative to address the overreliance on journal-based metrics in hiring, promotion, and funding decisions and to promote and support equity in the academy….”

Introduction to DORA: a short presentation at the Global Research Council’s virtual Responsible Research Assessment Conference | DORA

“DORA chair, Prof. Stephen Curry made a short introduction to DORA for the Global Research Council conference on Responsible Research Assessment, which was held online over the week of 23-27 November 2020. He briefly explains the origins of DORA, the meaning of the declaration, and how DORA developed into an active initiative campaigning for the world-wide reform of research assessment. In advance of the conference, Curry, and Program Director, Dr. Anna Hatch, contributed to a working paper outlining the state of play regarding responsible research assessment, exploring what it means and describing existing initiatives in the space….”

Rethinking Research Assessment for the Greater Good | DORA

“The ScholCommLab in Canada conducted a multi-year project examining more than 850 review, promotion, and tenure (RPT) guidelines in the United States and Canada to better understand academic career advancement. The lab examined how the public dimensions of faculty work, use of the Journal Impact Factor, and non-traditional scholarly outputs were recognized and rewarded in review, promotion, and tenure. Key findings have been represented in a series of infographics for the scholarly community….”

The transformative power of values-enacted scholarship | Humanities and Social Sciences Communications

Abstract:  The current mechanisms by which scholars and their work are evaluated across higher education are unsustainable and, we argue, increasingly corrosive. Relying on a limited set of proxy measures, current systems of evaluation fail to recognize and reward the many dependencies upon which a healthy scholarly ecosystem relies. Drawing on the work of the HuMetricsHSS Initiative, this essay argues that by aligning values with practices, recognizing the vital processes that enrich the work produced, and grounding our indicators of quality in the degree to which we in the academy live up to the values for which we advocate, a values-enacted approach to research production and evaluation has the capacity to reshape the culture of higher education.


ACRL Framework for Impactful Scholarship and Metrics

“ACRL recommended “as standard practice that academic librarians publish in open access venues.” ….In June 2019, ACRL outlined priorities and plans to reshape the current system of scholarly communications to increase equity and inclusivity.  While by no means an exhaustive list of the values that institutions should discuss and balance, both of these priorities place value on a scholarly infrastructure that is new, emerging, different, and may not completely align with current evaluative practices. We urge institutions to discuss their core institutional values and priorities, and how support for open access, equity, and inclusion, and impact will be represented by the codified institutional guidelines, expectations, and rank/tenure/promotion/evaluation processes….”

The document is undated. But the announcement is dated December 11, 2020.


Open Access Legislation and Regulation in the United States: Implications for Higher Education | Journal of Copyright in Education & Librarianship

Accessing quality research when not part of an academic institution can be challenging. Dating back to the 1980s, open access (OA) was a response to journal publishers who restricted access to publications by requiring a subscription and limited access to knowledge. Although the OA movement seeks to remove costly barriers to accessing research, especially when funded by state and federal governments, it remains the subject of continuous debates. After providing a brief overview of OA, this article summarizes OA statutory and regulatory developments at the federal and state levels regarding free and open access to research. It compares similarities and differences among enacted and proposed legislation and describes the advantages and disadvantages of these laws. It analyzes the effects of these laws in higher education, especially on university faculty regarding tenure and promotion decisions as well as intellectual property rights to provide recommendations and best practices regarding the future of legislation and regulation in the United States.

Publishing, P&T, and Equity, an Open Access Week Miniseries, Part 3: How Librarians Became Experts on Publishing and Equity

Happy Open Access Week! This is the final installment in our 3-part mini-series of blog posts on Publishing, P&T, and Equity. The overarching issue: how to reform our research evaluation processes to eliminate bias and promote structural equity. On Monday I argued for ending P&T standards that reward journal ‘prestige.’ On Wednesday I wrote about why institutions who want to build structural equity should reward open publishing practices in their research evaluation processes. Today I will conclude with a little meta-piece on the Library’s place in all this.

Publishing, P&T, and Equity, an Open Access Week Miniseries, Part 3: How Librarians Became Experts on Publishing and Equity

Happy Open Access Week! This is the final installment in our 3-part mini-series of blog posts on Publishing, P&T, and Equity. The overarching issue: how to reform our research evaluation processes to eliminate bias and promote structural equity. On Monday I argued for ending P&T standards that reward journal ‘prestige.’ On Wednesday I wrote about why institutions who want to build structural equity should reward open publishing practices in their research evaluation processes. Today I will conclude with a little meta-piece on the Library’s place in all this.

Publishing, P&T, and Equity, an Open Access Week Miniseries, Part 1: Stop Rewarding Journal “Prestige”

“-Part 1 (this post!) will discuss why updating P&T standards to eschew journal level metrics and journal prestige is an important strategy for advancing equity and inclusion, as well as open access.

-Part 2 (Wednesday-ish) will suggest that rewarding open practice and open publishing in P&T standards is an important step toward affirmatively promoting equity and inclusion in the academy (and in the communities we serve).

-Part 3 (Friday-ish) is a kind of postscript that explains a bit about why libraries are especially interested in these issues and how we see the intersection of Open Access, Equity/Inclusion, and Promotion and Tenure with perhaps a unique clarity relative to other parts of the scholarly ecosystem….”

Journal- or article-based citation measure? A study… | F1000Research

Abstract:  In academia, decisions on promotions are influenced by the citation impact of the works published by the candidates. The Medical Faculty of the University of Bern used a measure based on the journal impact factor (JIF) for this purpose: the JIF of the papers submitted for promotion should rank in the upper third of journals in the relevant discipline (JIF rank >0.66). The San Francisco Declaration on Research Assessment (DORA) aims to eliminate the use of journal-based metrics in academic promotion. We examined whether the JIF rank could be replaced with the relative citation ratio (RCR), an article-level measure of citation impact developed by the National Institutes of Health (NIH). An RCR percentile >0.66 corresponds to the upper third of citation impact of articles from NIH-sponsored research. We examined 1525 publications submitted by 64 candidates for academic promotion at University of Bern. There was only a moderate correlation between the JIF rank and RCR percentile (Pearson correlation coefficient 0.34, 95% CI 0.29-0.38). Among the 1,199 articles (78.6%) published in journals ranking >0.66 for the JIF, less than half (509, 42.5%) were in the upper third of the RCR percentile. Conversely, among the 326 articles published in journals ranking <0.66 regarding the JIF, 72 (22.1%) ranked in the upper third of the RCR percentile. Our study demonstrates that the rank of the JIF is a bad proxy measure for the actual citation impact of individual articles. The Medical Faculty of University of Bern has signed DORA and replaced the JIF rank with the RCR percentile to assess the citation impact of papers submitted for academic promotion.  

Statement on the Scholarly Merit and Evaluation of Open Scholarship in Linguistics | Linguistic Society of America

“The Linguistic Society of America values the open sharing of scholarship, and encourages the fair review of open scholarship in hiring, tenure, and promotion. The LSA encourages scholars, departments, and personnel committees to actively place value on open scholarship in their evaluation process with the aim of encouraging greater accessibility, distribution, and use of linguistic research….”

Best Practices in Research Metrics: A Conversation with Dr. Diana Hicks Registration, Tue, Oct 13, 2020 at 4:00 PM | Eventbrite

“Join Professor Diana Hicks of the Georgia Tech School of Public Policy for a conversation about the Leiden Manifesto for Research Metrics. There will be a high level overview of the 10 principles to guide research evaluation followed by a participant driven Q&A with Professor Hicks.”

Gaps in Academic Communication – The Scholarly Kitchen

“Recently I saw a LinkedIn advertisement looking for Chinese speaking Associate Editors for the Directory of Open Access Journals (DOAJ). I approached Dominic Mitchell who posted the message, and suggested he post a message on Chinese social media. The next day I noticed a post in Chinese by the DOAJ WeChat account, and shared it with my network of STM publishers. There was immediate interest, and issues. The application form was a Google Doc which cannot be opened by Chinese users since Google does not have a license to operate in China. Realizing that this could be a problem, the DOAJ post offered a Word file, available for download from a Google Drive account which was still inaccessible to Chinese users….

A few weeks ago, an old document from a double-first-class university in south China triggered a lot of discussion among researchers and publishers. This internal document states that, “starting from January 1, 2019, papers published in open access journals will not be recognized by the university” in performance evaluation. It was rumored that at least one other top-100 university has the same regulation. As incredible as it sounds, this reflects the perception of open access by many Chinese researchers and publishers. Twenty years ago, the same skepticism on open access publications was not uncommon in the West, but it remains surprisingly prevalent in China.

I also often encounter a lack of interest in open access from Chinese STM publishers, which is perplexing considering how hot this topic has been in the West. I attribute this to the uncommercialized status of STM publishing in China. The majority of journals are fully funded and published by universities or research institutes. Reputation and quality have always been placed above all other metrics, and the business side is much less of a concern. This attitude towards open access is changing slowly. From 2016 to 2019, the total number of open access papers published by Chinese authors in gold or hybrid journals has more than doubled, and the percentage of these papers among all papers by Chinese authors increased by 1-2% every year, to 24% in 2019 (source of data: Dimensions). Meanwhile there is more and more vocal support of open access, so it could be just a matter of time until China fully embraces it….”