Across the research funding community, there is a growing appetite for more sophisticated approaches to the review and allocation of research funding. There is also a desire to minimise burden and bureaucracy across the research system. Charities, as forward-looking funders, often look to optimise and innovate in their expert review processes.  

We recognise there are many different ‘innovative’ expert review methods. If chosen, charities must incorporate these methods without compromising AMRC’s six principles of expert review. Below is a list of innovations that diverge from the standard expert review process that charities may wish to experiment with and/or implement. With all these methods, charities are advised to consider the limitations, for example, any implications for equity, diversity, and inclusion (EDI).   

Expression of interest/pre-proposal step

This step asks applicants to submit an outline of the proposed research before they complete a full application. From these shorter applications, only those which meet a minimum threshold are invited to complete a full application. This saves more time for reviewing applications and potentially increases the relevance and/or quality of the pool of full applications.  

Triage

Also known as eligibility or scope check, triage aims to reduce the number of applications that are sent for in-depth review. It can take the form of a ‘pre-proposal step/expression of interest’ but can also be performed on a full application.  

Where triage is based on eligibility, scope, strategic fit, or completeness of an application, it may be undertaken by charity staff. Where it is based on research quality, it should be undertaken by independent experts, for example via a specific triage panel or by members of the research review committee. In both triage and ‘expressions of interest’ staff should only decide that an application is unsuitable for funding when it clearly falls outside the charity’s remit. If using either step, the process must be fair and made transparent to researchers. Where staff are unsure about the appropriateness of an application, they should refer to the chair of the research review committee.  

Parkinson’s UK uses a triage system for project grants and non-drug approaches grants. Applicants are required to submit a pre-proposal application, which is reviewed by three scientific members of their College of Experts. For more about how Parkinson’s UK uses their college of experts click here. For non-drug approaches, grants are additionally reviewed by lay reviewers. Unsuccessful pre-proposal applicants are given feedback based on the comments from the reviewers. Successful applicants are invited to submit a full application by the specified deadline.  

Applicant anonymisation

Reviewers evaluate proposals without details of the applicants’ personal and professional information. This intervention is widely used by funders and aims to reduce bias. There is some evidence to show that this can also attract more ‘unconventional’ research ideas.

Panel-only review

This method is based on virtual or in‐person discussions among reviewers who convene to discuss their reviews and provide a consensus view as a group. Only by exception can a supplementary in-depth review (usually written) be requested by the panel to provide additional technical expertise. Using only panels in the review process tends to reduce proposal processing time and the time-to-decision for applicants. It also places less burden on the reviewer community.

Cancer Research UK have updated the way they operate their expert review process by discontinuing written review as a default across their regular response-mode funding schemes, with the aim of reducing the burden  on their research community. Instead, where relevant, their expert review panels thoroughly assess the applications, providing written comments to applicant(s) prior to interview.

Pool/College of experts

A large group of experts identified as relevant to the research field and funding priorities of the charity, are approached to be part of a pool of experts on which the charity can call. From the group, individuals can be selected as a bespoke research review committees or panels for an individual grant round or funding call.

Sandpits

In these intensive residential workshops, lasting up to five days, research groups are formed and carry out review in real time to allocate research funding. Sandpits can be useful for developing interdisciplinary and innovative research ideas. It is important to note that a residential workshop hosted over several days might reduce opportunities of participation for those with caring responsibilities or disabilities and/or favour applicants with good presentation skills. It is important to put mitigating measures in place to counter this. For example, offering to pay for carers to enable the applicant to attend and adapting the facilitation style of the event to make it more accessible. 

Pancreatic Cancer UK collaborated with Cancer Research UK and the Engineering and Physical Sciences Research Council (EPSRC) for an Early Detection of Pancreatic Cancer Innovation Workshop, focusing on novel technological approaches for the early detection of pancreatic cancer. The three-day event was dedicated to developing new multidisciplinary and revolutionary research ideas, with up to five of the best proposals being awarded £100,000 seed-funding to support the subsequent pilot and feasibility studies, including covering associated running costs and named research staff. Applications were welcomed from across a range of academic, industry, and community sectors. Participants were expected to engage constructively with each other, the event facilitators, and the sandpit academic Director to develop brand new collaborative research ideas during the sandpit. Teams who successfully pitched proposals at the workshop received seed-funding for 18 months to cover the costs of pilot/feasibility studies. 

Partial randomisation

Once expert review has been conducted to confirm the quality of applications, a lottery approach is used to determine which best quality applications are funded. Partial randomisation aims to remove bias and to reduce administrative burden in the selection process, while maintaining the core mechanism for the necessary quality control: expert reviewers’ judgement.

Wellcome's Institutional Fund for Research Culture was an invite-only grant call in 2023, providing institutions with up to £1 million of grant funding to take on ambitious projects that advance research cultures and environments. Recognising the broad range of topics and ideas for advancing positive research cultures, this is the first Wellcome funding scheme to use partial randomisation to allocate funding. Applications were grouped by a funding committee into three categories (Gold, Silver and Bronze), with the applications selected for Gold being directly recommended for funding and applications grouped into Bronze not funded. A randomisation (lottery) process using a Python script decided which applications in the Silver group to fund. For transparency in this process, the code used for the randomisation has been shared by Wellcome here.