It’s REF, but not as you know it

by | Jun 15, 2023 | Blog, News

The initial decisions on the next Research Excellence Framework have been published. They are informed by the work of the Future Research Assessment Programme, a long term collaboration between the UK funding bodies begun in 2021 that has thought long and hard about what the assessment should focus on, how it should be conducted, and what it should reward. 

There are significant differences to the last exercise – indeed institutions have been guided to “not assume that the approach to assessment in REF2028 will be the same as that utilised in REF2021”. 

In assessing performance there is a full shift away from individuals to “the contribution institutions and disciplines make to healthy, dynamic and inclusive research environments”. This is reflected in the new approach to the ‘Volume measure’ or the number of full time equivalent staff within each disciplinary area at an institution.

No longer will institutions submit lists to the exercise, this data will be sourced from HESA over multiple years and an average taken to determine the FTE. The submissions can then be drawn from all activity from within those disciplinary areas, by any individual or team with a ‘demonstrable link to the institution’.

There is a significant recalibration of the three domains of the assessment. The triad of Environment, Outputs and Impact have been replaced by new terms with more nuance that reflect the turn to research culture and research process.

People, Culture and Environment: This will be assessed at both Institutional Level – intended to focus on the institution’s policies, strategies and priorities with evidence of their impact on the whole institution – and Disciplinary Level – intended to focus on areas of specific disciplinary relevance, providing evidence of the impact of mechanisms to support a healthy research culture and environment in the specific disciplinary context – using structured questionnaires. This element has an overall weighting of 25%.

Contribution to Knowledge and Understanding: This will be assessed through appraisal of outputs and via a structured explanatory statement – “an evidence-based description of the institution’s wider contribution to knowledge and understanding in the disciplinary area” which also explains how the submission is representative of the research and researchers in this disciplinary area. This element has an overall weighting of 50%.

Engagement and Impact: This will be assessed through impact case studies, supplemented by quantitative and qualitative evidence, focused on outcomes and supported by data where appropriate, which sets out the wider contribution of the research activities to society and the economy. Impact case studies no longer require underpinning research of 2* quality or higher. This element has an overall weighting of 25%.

Perhaps most significant is the move away from published research outputs as the main way in which excellence is demonstrated, to a larger emphasis and weight given to the environment in which the research happens. This is intended to “encourage broader contributions to research and research processes” and for those elements to receive better recognition for the crucial part they play in creating and sustaining health research cultures, and vibrant disciplines. 

It should be said that REF has always welcomed a variety of output types and different ways of doing research; the material submitted just hasn’t always reflected that openness. The changes made for REF2028 can be seen as a determined nudge to see more plurality, more inclusivity, and more variety.

What is in there for smaller and specialist institutions?

GuildHE has worked closely with colleagues at Research England and across the sector throughout the last REF cycle and the FRAP activity to highlight particular areas of the assessment where being small or being specialist has led to disproportionate burden. We are pleased to see these have been listened to. 

Importantly the minimum number of impact case studies has been lowered and put on a sliding scale. This will mean that very small submissions of less than 9.99 FTE will no longer need to find a second case study from an almost impossibly small pool of activity. 

Being able to submit impacts from any research, not just that qualifying as 2* is also helpful; this will broaden out what could be considered for submission and hopefully highlight a variety of impact and engagement activity that would otherwise be invisible. 

It is good to see some elements stay unchanged, particularly those parts of REF 2021 that were burdensome to implement for smaller institutions, like identifying staff with Significant Responsibility for Research. These kinds of categorisations have long tendrils and have effects far beyond the submission to the REF, with many institutions now embedding them in strategies and development plans; regular tinkering with such fundamental elements can cause significant disruption.

And allowing the inclusion of research from individuals across the staff body will also be welcome. In many emerging research environments experienced and productive researchers necessarily take on roles of leadership that would have previously left them out of scope.

It is good to see these initial decisions published, and GuildHE Research will study them closely over the next few months. GHER and its constituent universities and colleges will contribute fully to the UKRI consultations as we move towards the next Research Excellence Framework.  Research England’s stated intention that REF 2028 will be a more inclusive research assessment exercise is an admirable one, and we look forward to helping fashion a REF which will capture the contributions of our researchers and research-enabling staff.
Professor John Strachan

Pro-Vice-Chancellor Research and Enterprise, Bath Spa University , Director of GuildHE Research

The devil will be in the detail

As to be expected with an exercise as extensive and detailed as REF, there are areas where we have misgivings.

Some aspects of institutional level assessment remain challenging for institutions submitting to single or very limited numbers of units of assessment. There can be duplication of effort, and a sense of not being on a level playing field with multi-faculty, research intensive institutions which can bring far more to the table for an institutional review of research.

On issues of equality, diversity, and inclusion care will need to be taken that the more light touch approach – essentially removing the need to account for each individuals’ circumstances and ability to research productively – is accompanied by detailed and clear guidance about what an equitable, healthy, and inclusive research culture looks like and how inequalities of any kind are addressed.

The trouble REF has always had is that for all the good intentions, and sound decision making, interpretation of the guidance is not under such close control.

Your starters for ten

There will now be a period of consultation, conducted in stages, to establish the details behind the main headlines. The final guidance will be with us in late 2024, and there is plenty left to decide. 

For now, the funders are consulting on:

  • the proposed approach to the volume measure;
  • the impacts and potential unintended consequences, and some practical details concerning output submissions;
  • the impacts and potential unintended consequences of changes to the number and weighting of impact case studies;
  • the UOA structure;
  • potential measures to take into account the impact of the Covid-19 pandemic.
Respond to the consultation by 6 October 2023