Science Europe presents a new position statement with recommendations on research assessment processes for research funding and performing organisations.
Research organisations must have processes in place to ensure that assessments of researchers and research quality are effective, efficient, fair and transparent. The shortage of funding and research positions as well as the current transformation of the research system towards Open Science make such assessments ever more important and force organisations to adapt. Science Europe, the European association representing the interests of major public research performing and research funding organisations, has thus published a new position statement with recommendations on the topic on 9 July. The paper was developed based on a study and a consultation on research assessment in 2019. It contains a set of guidelines for the evaluation of research assessment processes, which are directed towards Science Europe Member Organisations and other research organisations alike, and provide the ground for mutual learning and knowledge exchange.
The position statement builds on the identification of common challenges and a set of seven detailed recommendations. The 2019 study shows that research organisations struggle to address discrimination and biases in assessment processes, have to consider costs and efficiency of assessments in view of funding limitations and applicant investment of time and efforts, and do not know how to deal with ‘reviewer fatigue’ and lower the burden placed on (peer) reviewers. It concludes that research stakeholders need to act in a concerted way in order for the research system to evolve effectively.
First, research assessment processes must be clear and transparent at all stages and for all the involved parties, the paper claims. This means that organisations should provide user-friendly guidelines on all their processes and consider the inclusion of ‘right-to-reply’ mechanisms as well as publish the general results of assessments. Second, research organisations should continually monitor and evaluate the robustness of assessment processes and share best practices. Will say, whenever reforms to assessments are implemented or problems are identified; the processes will need a re-evaluation as well.
Third, to address discrimination and bias, Science Europe recommends re-appraising anti-bias processes against the results of assessment programmes and striving to include diverse profiles in reviewer pools, panels and boards. Fourth, research organisations should streamline assessment processes to reduce costs and the burden on reviewers and applicants. It may help considering a standardisation of processes within and even between organisations. To counteract a ‘reviewer fatigue’, organisations should consider broadening the criteria for selecting reviewers and emphasising the importance of internationality. In addition, review activities must be recognised appropriately and should figure in researchers CVs and add to their career profiles and development.
Research assessments should focus on the substance and content of applications rather than on metrics, and the processes behind assessments must aid reviewers in conducting qualitative assessments taking a broad spectrum of outputs into account. This is especially important in the light of the current transition to Open Science, where the value of impact factors and the like are changing. Last, research organisations should consider the use of novel assessment techniques, document them well and share their experience widely. Such novel processes may also include the involvement of other stakeholder, for instance industry partners, in their design and development.
Science Europe will continue its work on research assessment under its priority area ‘Quality of Science’ and will look into ways to drive and facilitate changes in the assessment system. It encourages its members and research organisations beyond to use the current position statement and recommendations to reflect on and improve their own processes.