“Since being released in July 2022, an Agreement on Reforming Research Assessment has been signed by more than 400 European research performing and funding organizations. It is intended to guide a reform and mutual learning process within a coalition of its signatories, CoARA. This policy brief analyses the agreement and provides recommendations for the next steps….”
“The European Commission is making a big push to reform research assessment, but Germany’s university leaders are not convinced the call for change from above is the right way to deliver it.
While the German Rector’s Conference sees value in rethinking how research is assessed in Europe, it believes the debate is not independent enough. “The agreement itself was designed and conceptualised along the policy lines of the EU Commission and, at its core, is basically the result of a top-down process,” says Walter Rosenthal, the Rectors’ Conference vice president for research, academic career paths and transfer.
The reform process was kicked off by the European Commission in the beginning of 2022, as part of moves to create a single European Research Area (ERA).
With research becoming more impact-driven, multidisciplinary and open, the aim is to align research assessment with these changes, moving away from metrics such as the number of papers published and the number of times these are cited, to a broader set of achievements reflecting the wider impact of research….”
“Research Performing Organisations (RPOs) are encompassing all Universities and Institutions that enable researchers to conduct and perform their research and duties by ensuring the presence of infrastructures and (human) resources to support and produce valuable research products (publications, data, software, patents, etc.).??
Thousands of organisations have expressed an interest in shifting the new research culture vision, enforcing the need for policy changes in research assessment. The embrace of this vision has been translated into signing several declarations such as the DORA declaration, the?Leiden Manifesto, the?Metric Tide, and the?Hong Kong Principles for Assessing Researchers just to name a few. RPOs are key stakeholders of the more recent Agreement on Research Assessment, being part of the Coalition for Advancing Research Assessment (CoARA).?…”
“Launched in January 2022 as a co-creation exercise, the process of drafting an agreement for reforming research assessment has reached an important milestone. On 8 July, the final version of the agreement was presented at a Stakeholder Assembly bringing together the 350+ organisations from 40+ countries having expressed interest in being involved in the process. Today, the final Agreement is made public with this news.
Organisations involved include public and private research funders, universities, research centres, institutes and infrastructures, associations (and alliances thereof), national and regional authorities, accreditation and evaluation agencies, learned societies and associations of researchers, and other relevant organisations. They represent a broad diversity of views and perspectives….”
“In the early summer of 2022, all European Union Ministers agreed that in order to accelerate the implementation and the impact of Open Science policies and practices across Europe, action must be taken in three areas, one being reforming research assessment. This new policy was published in European Council Conclusions in June 2022. Reforming the assessment system for research, researchers and institutions is also a priority for the European Research Area in ERA Action 3 for the period 2022-2024.
In line with this, we welcome the establishment of the Coalition for Advancing Research Assessment (CoARA) since it provides a common direction to reform research assessment practices across nations. We envisage that this is of great interest to governments, research funders, research performing organisations and researchers.
The transition to Open Science as the new normal will be accelerated through such a more balanced research assessment system.”
“Launched in January 2022 as a co-creation exercise, the process of drafting an agreement for reforming research assessment has reached an important milestone. On 8 July, the final version of the agreement was presented at a Stakeholder Assembly bringing together the 350+ organisations from 40+ countries having expressed interest in being involved in the process. Today, the final Agreement is made public with this news….
Signatories will commit to a common vision, which is that the assessment of research, researchers and research organisations recognises the diverse outputs, practices and activities that maximise the quality and impact of research. This requires basing assessment primarily on qualitative judgement, for which peer-review is central, supported by responsible use of quantitative indicators….”
“Following the approval of the General Assembly, UAS4EUROPE [Universities of Applied Sciences for Europe] has signed the ‘Agreement on Reforming Research Assessment’ and joined the ‘Coalition for Advancing Research Assessment’ (CoARA).
After its kick-off in early 2022, many actors in the R&I community discussed how the assessment of research should develop in a way that values different types of research and research outputs as well as researchers at different career stages.
The European University Association (EUA), Science Europe, and Dr. Karen Stroobants have been leading this process as the drafting team of the agreement, facilitated by the European Commission. The UAS4EUROPE Secretariat followed the discussions throughout the year in several assembly meetings that included individual researchers, research institutions, umbrella organisations and the European Commission….”
“Today, following the decisions of its Executive Board and the approval of its member institutions, YERUN becomes an official signatory of the Agreement on Reforming Research Assessment and officially expresses its willingness to join the Coalition for Advancing Research Assessment (CoARA).
YERUN has been strongly involved in the process leading to the reform of Research Assessment since the very beginning of the initiative. It has been part of the Core Group supporting the development of the Agreement, and since September, is member of the Implementation Group that has advised the development of the CoARA initiative. In light of its strong commitment, YERUN values its future involvement in CoARA and will contribute at the best of its possibilities for the initiative to be useful, relevant and a real enabler of change.
Contextually with the signature, the network published a position paper with 4 suggestions to be considered within the implementation of the initiative and more specifically towards the 1st Constitutive Assembly of CoARA:
Ensuring ownership of the initiative by those that are making change happen.
Promoting an open and flexible governance model.
Encouraging a new approach to change.
Reaching critical mass is the key to becoming a global actor….”
“The European University Association is pleased to announce that the Association will sign the Agreement on Reforming Research Assessment and join the Coalition for Advancing Research Assessment (CoARA).
Both decisions were unanimously approved by the EUA Council, at its meeting in Brussels on 28 October. The decision to formally endorse the agreement by the Association, which represents more than 850 universities and national rectors’ conferences across Europe, comes at an opportune time for this initiative, as CoARA’s Constitutive Assembly will take place on 1 December….”
“Today, the Commission has signed the Agreement on Reforming Research Assessment. The Agreement sets a common direction for changes in assessment practices for research, researchers and research organisations, with the goal to maximise the quality and impact of research. It covers the principles, commitments and timeframe for reforms and lays out the principles for the Coalition for Advancing Research Assessment (CoARA). The Coalition is a group of organisations willing to work together to implement the reform. The Coalition’s establishment is one of the main expected outcomes of the European Research Area (ERASearch for available translations of the preceding linkEN•••) Policy Agenda for 2022-2024Search for available translations of the preceding linkEN•••, which includes an action to advance the reform of the assessment system for research, researchers and institutions.
Together with the Agreement’s signature, today also marked the Commission’s endorsement of the San Francisco Declaration on Research Assessment(DORA), which sets recommendations to improve the evaluation of researchers and the outputs of scholarly research.The Commission signed the Agreement and endorsed DORA in its capacity of a research funding organisation….”
OPERAS has signed the Agreement on Reforming Research Assessment, an initiative that gathers more than 180 signatories, including public and private research funders, universities, research centres, institutes and infrastructures, associations (and alliances thereof), national and regional authorities, accreditation and evaluation agencies, learned societies and associations of researchers, and other relevant organisations, representing a broad diversity of views and perspectives. Many OPERAS members have already signed the document.
The agreement aims to recognise the diverse outputs, practices and activities that maximise the quality and impact of research for the research, researchers and research organisations assessment. The document sets a shared direction for changes in the research assessment practices, including principles, commitments and timeframe for reforms. The new vision for the research assessment requires that it will be based primarily on a qualitative judgement, for which peer review is central, and supported by responsible use of quantitative indicators.
As the Research Infrastructure dedicated to supporting the whole research lifecycle of Social Sciences and Humanities (SSH) in Europe, OPERAS is taking part in CoARA and invites all its members to sign the agreement. OPERAS supports CoARA’s vision of innovative and open research assessment practices, which can be adapted to the diversity of scientific disciplines.
OPERAS commitment in signing includes the provision of services that can enable the full range of practices in the SSH field, where there is still a lack of tools and indicators. Some examples would be
The issue of reforming research assessment will be also included among the topics of the relevant OPERAS Special Interest Groups (Advocacy and Standards).
The agreement’s initiative followed a call of interest by the European Commission, in January 2022, to be drafted in a co-creation exercise. Since July 2022, the document was publicly presented (read it via this link). The Coalition for Advancing Research Assessment (CoARA) is now being created and has an interim secretariat provided by Science Europe, the European University Association (EUA) and the European Commission.
Commitment of existing and new members
By advocating for a change in the research assessment and supporting CoARA, OPERAS is including the commitment to recognise all research outputs and to experiment new indicators and tools in its new members’ application form. The infrastructure encourages existing members to sign the Agreement and to be active parts of the Coalition as research performing organisations.
The Agreement on Reforming Research Assessment is based on 10 Commitments, which include topics such as the recognition of the diversity of contributions of research according to its needs and nature; the basing of research assessment primarily on qualitative evaluation; the purpose to abandon inappropriate uses of journal and publication-based metrics; the avoidance of ranking usage for research organisations; the commitment of resources in organisations to the reform of the research assessment; the reviewing and development of criteria, tools, and processes; the raising of awareness on the reform; the communication on progress made on the implementation of the Commitments; and the evaluation of practices, criteria and tools based on state-of-the-art research. The list of Commitments, their scopes and purposes are available in this link.
As stated by CoARA’s announcement on the Agreement, the reform movement underpinned by both the document and the coalition intends to be a space to advance together towards a higher quality, more impactful and more efficient, inclusive system.
| More information |
Read the Agreement full-text here.
“Our vision is that the assessment of research, researchers and research organisations recognises the diverse outputs, practices and activities that maximise the quality and impact of research. This requires basing assessment primarily on qualitative judgement, for which peer review is central, supported by responsible use of quantitative indicators.”
“Launched in January 2022 as a co-creation exercise, the process of drafting an agreement for reforming research assessment has reached an important milestone. On 8 July, the final version of the agreement was presented at a Stakeholder Assembly bringing together the 350+ organisations from 40+ countries having expressed interest in being involved in the process. Today, the final Agreement is made public with this news.
Organisations involved have provided feedback to the evolving drafts of the agreement, as prepared by a team composed of representatives from the European University Association (EUA), Science Europe, and the European Commission, alongside Dr Karen Stroobants in her individual capacity as researcher with expertise in research on research.
A core group of 20 research organisations, representing the diversity of the research community across Europe, also contributed to the drafting process, while EU Member States and Associated Countries have been consulted on the agreement in the framework of the ERA Forum and the European Research Area Committee (ERAC).
The Agreement on Reforming Research Assessment sets a shared direction for changes in assessment practices for research, researchers and research performing organisations, with the overarching goal to maximise the quality and impact of research. The Agreement includes the principles, commitments and timeframe for reforms and lays out the principles for a Coalition of organisations willing to work together in implementing the changes.
Signatories will commit to a common vision, which is that the assessment of research, researchers and research organisations recognises the diverse outputs, practices and activities that maximise the quality and impact of research. This requires basing assessment primarily on qualitative judgement supported by responsible use of quantitative indicators.”