Category Archives: oa.rra

The SCOPE framework – implementing the ideals of responsible research assessment

Posted on by
Reply

Abstract:  Background: Research and researchers are heavily evaluated, and over the past decade it has become apparent that the consequences of evaluating the research enterprise and particularly individual researchers are considerable. This has resulted in the publishing of several guidelines and principles to support moving towards more responsible research assessment (RRA). To ensure that research evaluation is meaningful, responsible, and effective the International Network of Research Management Societies (INORMS) Research Evaluation Group created the SCOPE framework enabling evaluators to deliver on existing principles of RRA. SCOPE bridges the gap between principles and their implementation by providing a structured five-stage framework by which evaluations can be designed and implemented, as well as evaluated.

Methods: SCOPE is a step-by-step process designed to help plan, design, and conduct research evaluations as well as check effectiveness of existing evaluations. In this article, four case studies are presented to show how SCOPE has been used in practice to provide value-based research evaluation.

Results: This article situates SCOPE within the international work towards more meaningful and robust research evaluation practices and shows through the four case studies how it can be used by different organisations to develop evaluations at different levels of granularity and in different settings.

Conclusions: The article demonstrates that the SCOPE framework is rooted firmly in the existing literature. In addition, it is argued that it does not simply translate existing principles of RRA into practice, but provides additional considerations not always addressed in existing RRA principles and practices thus playing a specific role in the delivery of RRA. Furthermore, the use cases show the value of SCOPE across a range of settings, including different institutional types, sizes, and missions.

CoARA Working Group Towards Open Infrastructures for Responsible Research Assessment announced – GraspOS

Posted on by
Reply

“A few months have passed since the Coalition for Advancing Research Assessment (CoARA) launched its call for members to propose Working Groups and National Chapters on 28 March 2023. Being the first since the beginning of CoARA in December 2022, the call was an important step in the Coalition’s action to tackle the need to reform research assessment and highlighted how the working groups will support the implementation of the commitments contained in the Agreement on Reforming Research Assessment. Members of the Coalition had until 6 July 2023 to send their proposals which were then reviewed by the CoARA Steering Board.

We are very pleased to announce that the proposal for the CoARA Working Group Towards Open Infrastructures for Responsible Research Assessment (OI4RRA) has been accepted. A group of 23 Coalition members* from around the globe joined forces to propose a Working Group focused on making research assessment more transparent and responsible, consequently enabling the research community to be in full control of the data and indicators it relies on in assessments.

Research assessment generally relies on proprietary databases which often use proprietary analytic tools. Still, open research information is crucial for responsible research assessment which in turn needs data, tools and infrastructures that ensure transparency, reproducibility and geographic-discipline-output coverage in data and indicators. The existing Open Infrastructure community has created and operates open solutions (e.g., OpenAIRE, OpenCitations, COKI, OpenAlex, Semantic Scholar), most of which have yet to find their way in official research assessment operations….”

‘Responsible use of what?’ Navigating US university governance to approve an institutional statement on the responsible use of metrics

Posted on by
Reply

A slide presentation by Rachel Miles, Research Impact Coordinator at Virginia Tech University Libraries. 

 

Navigating Responsible Research Assessment Guidelines – Leiden Madtrics

Posted on by
Reply

“Responsible Research Assessment is discussed and used in many contexts. However, Responsible Research Assessment does not have a unifying definition, and likewise its guidelines indicate that the implementation of Responsible Research Assessment can have many different scopes.

Research assessment has a long history continuously introducing new methods, tools, and agendas, for example, peer review of publications dating back to 17th century and catalogues from the 19th century that facilitated publication counting. This blog post discusses Responsible Research Assessment (RRA), an agenda gaining attention today. The blog post gives an introduction to RRA and discusses how to navigate RRA guidelines, which can be a complex task….”

The changing role of funders in responsible research assessment: progress, obstacles and the way ahead

Posted on by

“This working paper explores what RRA [Responsible Research Assessment] is, and where it comes from, by outlining fifteen initiatives that have influenced the shape and direction of current RRA debates. It goes on to describe the responses that these have elicited, with a particular focus on the role and contribution of research funders, who have more freedom and agency to experiment and initiate change than other actors in research systems.

The paper also presents the findings of a survey of RRA policies and practices in the participant organisations of the Global Research Council (GRC)—mainly national public funding agencies—with responses from 55 organisations worldwide….”

The changing role of funders in responsible research assessment: progress, obstacles and the way ahead

Posted on by

“This working paper explores what RRA [Responsible Research Assessment] is, and where it comes from, by outlining fifteen initiatives that have influenced the shape and direction of current RRA debates. It goes on to describe the responses that these have elicited, with a particular focus on the role and contribution of research funders, who have more freedom and agency to experiment and initiate change than other actors in research systems.

The paper also presents the findings of a survey of RRA policies and practices in the participant organisations of the Global Research Council (GRC)—mainly national public funding agencies—with responses from 55 organisations worldwide….”

The changing role of funders in responsible research assessment: progress, obstacles and the way ahead

Posted on by

“Encouraging interim results of different vaccine trials reflect the speed, innovation and dedication that the research community has shown in its response to Covid-19. But the pandemic has also shone a spotlight on the inner workings of research, and in lots of ways—good and bad—has intensified scrutiny of how research is funded, practiced, disseminated and evaluated, and how research cultures can be made more open, inclusive and impactful.

 

The uncertain possibilities that flow from this moment follow a period in which concern has intensified over several long-standing problems, all linked to research assessment. As attention shifts from describing these problems, towards designing and implementing solutions, efforts are coalescing around the idea of responsible research assessment (RRA). This is an umbrella term for approaches to assessment which incentivise, reflect and reward the plural characteristics of high-quality research, in support of diverse and inclusive research cultures.

This working paper explores what RRA is, and where it comes from, by outlining fifteen initiatives that have influenced the content, shape and direction of current RRA debates. It goes on to describe some of the responses that these have elicited, with a particular focus on the role and contribution of research funders, who have more freedom and agency to experiment and drive change than many of the other actors in research systems.

The paper also presents the findings of a new survey of RRA policies and practices in the participant organisations of the Global Research Council (GRC)—most of which are national public funding agencies—with responses from 55 organisations worldwide….”

Home – Responsible Research Assessment – a virtual conference from the Global Research Council

Posted on by

“Around the world, the COVID-19 pandemic has reaffirmed the importance of international collaboration in research and innovation. The impact of research has become ever more apparent during the pandemic, and so there is a renewed urgency for funders to come together and reconsider how research is assessed and evaluated. 

At the GRC Responsible Research Assessment Conference 2020, participants will be invited to consider the existing sector-wide frameworks on responsible research assessment and have a global discussion on how funders can drive a positive research culture through research assessment criteria and processes. The discussions will reflect on how to support a diverse, inclusive and thriving research sector….”

A pivotal moment for responsible research assessment – Research Professional News

Posted on by

“We’ve been involved in diagnosing, assembling evidence and banging drums about these problems, through initiatives such as the Declaration on Research Assessment (Dora), the Metric Tide report and the UK Forum for Responsible Research Metrics.

So we welcome signs that attention is shifting towards implementing solutions, and coalescing around a more expansive agenda for responsible research assessment (RRA). Early debates on metrics and measurement have expanded to encompass questions about how to create a healthy work culture for researchers, how to promote research integrity, how to move from closed to open scholarship, and how to embed the principles of equality, diversity and inclusion across the research community.

This more holistic approach can be seen, for example, in UK Research and Innovation’s commitment to a healthy research culture, and in the recent guidelines on good research practice from the German Research Foundation (DFG).

Next week’s Global Research Council virtual conference on RRA—hosted by UKRI in ?collaboration with the UK Forum for Responsible Research Metrics and South Africa’s National Research Foundation—comes at a pivotal time….

Declarations and statements of principle have been an important part of this story. But even though we have co-authored some of these, we feel the time for grand declarations has passed. They risk becoming substitutes for action.

RRA now needs to focus on action and implementation—testing and identifying what works in building a healthy and productive research culture. Institutional commitments must be followed by the hard graft of reforming cultures, practices and processes….”