DORA Community Engagement Grants: Supporting Academic Assessment Reform | DORA

“DORA sought to fund ideas to advance assessment reform at academic institutions at any stage of readiness. Projects could be targeted to any level within an academic institution, including (but not limited to) reform efforts at the graduate program, department, library, or institution level, and should address one or more key aspects of education, planning, implementing, training, iteratively improving, and scaling policies and practices. More than 55 ideas were submitted from individuals and teams in 29 countries! After careful review, members of the Steering Committee selected 10 proposals to support….”

Wiley Signs Declaration on Research Assessment, Deepens Commitment to Responsible Research Assessment | John Wiley & Sons, Inc.

“Global research and education leader Wiley today announced it has signed the Declaration on Research Assessment (DORA), which is a world-wide initiative designed to improve the ways in which the outputs of scholarly research are evaluated. 

As the publisher of nearly 2,000 academic journals, Wiley will deliver more ways to assess and recognize research outputs, which in turn supports healthy scholarship and allows more researchers to thrive in their careers. To this end, Wiley will roll out a broad range of journal and article metrics across its journal portfolio with the aim of providing a holistic, well-rounded view of the value and impact of any author’s research. This includes metrics that measure levels of impact beyond citation value, including usage, re-use, reproducibility, peer review assessment, geographic reach, and public recognition via references in media outlets….”

Open Science in the recently adopted Resolution on the Slovenian Scientific Research and Innovation Strategy 2030 – OpenAIRE Blog

“At the end of March 2022, the National Assembly of the Republic of Slovenia adopted the Resolution on the Slovenian Scientific Research and Innovation Strategy 2030 which was published in the Official Gazette of the Republic of Slovenia in April 2022. This is a key Slovenian strategic document for research and innovation until 2030, which will be the basis for formulating policies related to social and economic development as well as to societal challenges. The Scientific Research and Innovation Strategy is inextricably intertwined with the Resolution on National Programme of Higher Education 2030, and both are harmonized with the Slovenian Development Strategy 2030. At the implementation level, the Scientific Research and Innovation Strategy will be supplemented by action plans and sectoral strategic documents (e.g., Research Infrastructure Development Plan, Open Science Action Plan, Action Plan for Technology Transfer Offices, Equal Opportunities Action Plan), which will define the set goals and upgrade them with measurable monitoring indicators.

The Scientific Research and Innovation Strategy, which is based on the Article 10 of the new Scientific Research and Innovation Activities Act, introduces Open Science as an important integral part of the scientific research. In the context of the horizontal objectives under Item 6.2., the Strategy defines six key measures in the field of Open Science, as follows:

1. Effective management and financing of the development of the national Open Science ecosystem and related national structures and infrastructures, ensuring their international alignment as well as integration into international associations and infrastructures.

2. Introduction of modern approaches to the evaluation of scientific research activity in accordance with Open Science principles to increase the quality and impact of research (e.g., DORA – San Francisco Declaration on Research Assessment, Leiden Manifesto for Research Metrics, ERAC – European Research Area and Innovation Committee Guidelines).

3. Ensuring that the results of scientific research comply with the FAIR principles (Findable, Accessible, Interoperable, and Reusable), and that full and immediate open access is provided (subject to legitimate exceptions).

4. Establishment of a National Open Science Community for the introduction and monitoring of Open Science in Slovenia, as well as its integration into ERA and beyond.

5. Promoting the development of citizen science and public involvement in scientific research.

6. Promoting the development of a national scientific publishing system to operate according to the principles of Open Science.

These measures, which are recognised as essential for the development of Open Science in Slovenia, are the basis for the Action Plan on Open Science in preparation. Its adoption by the Government of the Republic of Slovenia is expected later this year.”

 

Towards a new reward system for open science

The transition to an open science system affects the entire research process. The reward systems also need to be adjusted in order to support and mirror the open research landscape, but what will this work look like, and what will change? We met Gustav Nilsonne, chair of the European working group dealing with the issue and a participant in the SUHF working group on merit reviews.

Rethinking Research Assessment for the Greater Good: Findings from the RPT Project – Scholarly Communications Lab | ScholCommLab

“The review, promotion, and tenure (RPT) process is central to academic life and workplace advancement. It influences where faculty direct their attention, research, and publications. By unveiling the RPT process, we can inform actions that lead towards a greater opening of research. 

Between 2017 and 2022, we conducted a multi-year research project involving the collection and analysis of more than 850 RPT guidelines and 338 surveys with scholars from 129 research institutions across Canada and the US. Starting with a literature review of academic promotion and tenure processes, we launched six studies applying mixed methods approaches such as surveys and matrix coding.

So how do today’s universities and colleges incentivize open access research? Read on for 6 key takeaways from our studies….”

Societal impact of university research in the written press: media attention in the context of SIUR and the open science agenda among social scientists in Flanders, Belgium | SpringerLink

Abstract:  Transferring scientific knowledge to non-academic audiences is an essential aspect of the open science agenda, which calls for scholars to pursue a popularization of their research. Accordingly, purposefully introducing scientific insights to the public at large is almost univocally deemed commendable. Indeed, in today’s models of research evaluation, the objects and activities considered are being extended beyond peer-reviewed journal articles to include non-scholarly popular communication. Although altmetrics offer one instrumental way to count some interactions with lay audiences, their reliance on social media makes them susceptible to manipulation, and mostly reflect circulation among niche audiences. In comparison, attention from non-scholarly media like newspapers and magazines seems a more relevant pathway to effectuate societal impact, due to its recognition in qualitative assessment tools and its broad, societal reach. Based on a case study of social scientists’ attention by newspapers and magazines in Flanders (northern Dutch-speaking region of Belgium) in 2019, this paper highlights that frequent participation in the public debate is reserved for high-status researchers only. Results show highly skewed media appearance patterns in both career position and gender, as eight male professors accounted for almost half of all 2019 media attention for social scientists. Because media attention is highly subject-dependent moreover, certain disciplines and fields offer easier pathways to popularization in media than others. Both the open science agenda and research assessment models value presence of researchers in popular media, adding written press attention to existing evaluation assessments however would disproportionately disadvantage early career researchers and exacerbate existing inequalities in academia.

 

 

At what point do academics forego citations for journal status? | Impact of Social Sciences

“The limitations of journal based citation metrics for assessing individual researchers are well known. However, the way in which these assessment systems differentially shape research practices within disciplines is less well understood. Presenting evidence from a new analysis of business and management academics, Rossella Salandra and Ammon Salter and James Walker¸ explore how journal status is valued by these academics and the point at which journal status becomes more prized than academic influence….”

At what point do academics forego citations for journal status? | Impact of Social Sciences

“The limitations of journal based citation metrics for assessing individual researchers are well known. However, the way in which these assessment systems differentially shape research practices within disciplines is less well understood. Presenting evidence from a new analysis of business and management academics, Rossella Salandra and Ammon Salter and James Walker¸ explore how journal status is valued by these academics and the point at which journal status becomes more prized than academic influence….”

CoNOSC Member needs report published

SPARC Europe took on the facilitation of the Council for National Open Science Coordination (CoNOSC) late last year. To kick-start its efforts and plan the work ahead, it investigated the needs of CoNOSC members. We are pleased to publish this report today. 

The report investigates the needs of national policymakers, presenting key takeaways from interviews with CoNOSC members’ representatives – national OS coordinators, ministry officials, research funders and other policymaking decision-makers.

SPARC Europe conducted interviews with 30 representatives from 18 different European nations during January and February 2022, as well as with the Deputy Head of the Open Science Unit from the European Commission, which attends CoNOSC as an observer. The interviews were designed to identify what activities and areas would provide the most significant value through collaboration at CoNOSC without duplicating any current efforts. 

Based on responses, the report identifies the priority areas for CoNOSC as follows:

Data management
Policy monitoring
Research assessment
Copyright and licensing
Open Access funding
Bibliodiversity

 

Implementation of promotion standards to discourage publishing in questionable journals: the role of the library – ScienceDirect

Abstract:  To discourage faculty members from publishing in questionable journals, tenure and promotion standards in which the librarians play an active role can been developed. These standards have been effective in terms of identifying publications in questionable outlets. However, we need to explore how these systems are perceived by the main actors in research, which are the researchers. This study explores the perception of the researchers at a university in Ghana who have been evaluated by a system implemented to discourage publishing in questionable publication outlets. We collected data using an online, largely qualitative questionnaire distributed to all faculty members that had applied for promotion since the implementation of the verification process. The results show that the majority of the faculty members are satisfied or very satisfied with the new tenure and promotion standards. There are differences across faculties, and this seems to be tied to concerns about the choice of publication outlets. Furthermore, the dissatisfied faculty members are concerned with the role of the library in the verification process whereas the satisfied trust the judgement of the librarians. We discuss implications of the results as well as future development of the standards.

 

Elena Giglia, Open Science is here to stay — Filologia Classica e Italianistica – Ficlit

“What lessons did we learn from the COVID pandemic? How has the crisis impacted the current scholarly communication system? And, all in all, does the current scholarly communication work? Are you happy with the way your research is evaluated? During this workshop we shall see the reasons why we need Open Science, how it works, and what you can do starting tomorrow to open up every step of your research – without harming your career. We shall also try to overcome common misunderstandings on Open Access, Open Science and FAIR data and we shall discuss the future of research in the EOSC – European Open Science Cloud era.”

Handbook on Research Assessment in the Social Sciences (2022) – Emanuel Kulczycki

This Handbook provides a comprehensive overview of current developments, issues and good practices regarding assessment in social science research. It pays particular attention to the challenges in evaluation policies in the social sciences, as well as to the specificities of publishing in the area. The Handbook discusses the current societal challenges facing researchers, from digital societies, to climate change and sustainability, to trust in democratic societies. Chapters provide ways to strengthen research assessment in the social sciences for the better, by offering a diverse range of experiences and views of experts from all continents. The Handbook also outlines major data sources that can be used to assess social sciences research, as well as looking at key dimensions of research quality in the social sciences including journal peer review, the issue of identifying research quality, and gender disparities in social science research. This book will be an essential read for scholars interested in research assessment in the social sciences. It will also be useful to policy makers looking to understand the key position of the social sciences in science and society and provide appropriate frameworks for key societal challenges.

Do German university medical centres promote robust and transparent research? A cross-sectional study of institutional policies | Health Research Policy and Systems | Full Text

Abstract:  Background

In light of replication and translational failures, biomedical research practices have recently come under scrutiny. Experts have pointed out that the current incentive structures at research institutions do not sufficiently incentivise researchers to invest in robustness and transparency and instead incentivise them to optimize their fitness in the struggle for publications and grants. This cross-sectional study aimed to describe whether and how relevant policies of university medical centres in Germany support the robust and transparent conduct of research and how prevalent traditional metrics are.

Methods

For 38 German university medical centres, we searched for institutional policies for academic degrees and academic appointments as well as websites for their core facilities and research in general between December 2020 and February 2021. We screened the documents for mentions of indicators of robust and transparent research (study registration; reporting of results; sharing of research data, code and protocols; open access; and measures to increase robustness) and for mentions of more traditional metrics of career progression (number of publications; number and value of awarded grants; impact factors; and authorship order).

Results

While open access was mentioned in 16% of PhD regulations, other indicators of robust and transparent research were mentioned in less than 10% of institutional policies for academic degrees and academic appointments. These indicators were more frequently mentioned on the core facility and general research websites. Institutional policies for academic degrees and academic appointments had frequent mentions of traditional metrics.

Conclusions

References to robust and transparent research practices are, with a few exceptions, generally uncommon in institutional policies at German university medical centres, while traditional criteria for academic promotion and tenure still prevail.

Time to recognize authorship of open data

“The open data revolution won’t happen unless the research system values the sharing of data as much as authorship on papers….

Such a practice is neither new nor confined to a specific field. But the result tends to be the same: that authors of openly shared data sets are at risk of not being given credit in a way that counts towards promotion or tenure, whereas those who are named as authors on the publication are more likely to reap benefits that advance their careers.

Such a situation is understandable as long as authorship on a publication is the main way of getting credit for a scientific contribution. But if open data were formally recognized in the same way as research articles in evaluation, hiring and promotion processes, research groups would lose at least one incentive for keeping their data sets closed….”