Grant applications: Common reviewer criticisms (and how to avoid them)

Some of the most common points of criticism that I see raised time and again by grants evaluators in the life sciences

Reviewers and panel members’ criticisms fall broadly into two categories: those that are linked to highly-specific aspects of the science, approach and methodology; and those that highlight a more general flaw in the proposal. By definition, the former tend to be one-offs, more or less unique to a particular grant application. But the latter are, in my experience, frequently identifiable as recurring themes that come up with frustrating regularity.

I can’t offer you any words of wisdom on avoiding specific scientific and methodological criticisms, beyond urging you to do your best to design your project so as to avoid criticisms like the following: “It is apparent that no data will be collected if an asteroid strike doesn’t occur during the period of the study”. (Yes, I made that one up.) But, given how often I see the same sorts of general criticism crop up, I think it should be reasonably easy to prepare a checklist of common bear traps, and offer some thoughts on how to avoid them. So, in no particular order:

1. “The study seems (over-)ambitious”

When the word ‘ambitious’ appears anywhere in a proposal review or panel feedback it’s a huge red flag. Although the word can be laudatory in other contexts, it’s usually a killer blow in a review – along similar lines to remarks like “that dress is a really brave colour!” Ouch. Sometimes it’s used somewhat euphemistically – “the proposed study appears highly ambitious”. Other times, the gloves are well and truly off – “the project is over-ambitious”.

Accusations of over-ambition might relate to the timeframe, the (insufficient) amount of funding requested, the size and experience of the team, and the availability of resources – often several of these things. Early-career researchers in particular seem prone to proposing projects that get shot down for being over-ambitious by seasoned reviewers who know how much – or indeed how little – can actually be achieved in the course of a three-year project.

Avoiding this criticism

No one wants to submit a project proposal that’s obviously under-ambitious. So the answer here definitely isn’t to propose doing one year’s worth of research over a three-year period.

Part of knowing how to avoid this criticism comes with experience. An experienced academic who’s previously delivered a number of substantial research projects is reasonably unlikely to suddenly put together a proposal that’s wildly unrealistic. If you don’t yet have that experience yourself then try to borrow it. Ask a seasoned colleague (such as your mentor, supervisor, group lead or perhaps a senior-level collaborator) to look at your proposal specifically through the lens of whether it’s realistic and feasible. For your own part, try to break down the project with a degree of granularity into tasks and sub-tasks. Think carefully about who will do each task, what they’ll need in order to do it, how long it should reasonably take, and how much contingency might be sensible to build in. Once you’ve done this it should be easy to prepare a Gantt chart to illustrate your project timelines – a visual representation of your key steps, their scheduling and the time needed for each. Ask yourself what could derail your plans, how likely this is to happen, and how that risk could be minimized. As a general rule, long lists of objectives will terrify reviewers. Three main scientific objectives is often plenty for a three-year research project.

When it comes to budget, the overambition-related killer blow often looks like the following: “The proposed study cannot be completed with the requested resources/funding”. This should be quite simple to avoid. When you do your granular-breakdown exercise, work out exactly what will be needed for each task in terms of human resources, equipment (including access charges) and consumables; get reasonably-accurate costs for everything; and be sure to justify everything in your budget narrative (justification of resources – more on this below). Leave no one in any doubt that you very obviously need everything that you’ve included in the budget – no less, but definitely no more.

2. “It is unclear who will be doing what in the project”

This criticism comes in various different guises – “certain team members’ inclusion is not justified”, “the roles of individual team members are not clearly explained” and so on. The reviewer is basically thinking “why on earth is this person named on the grant?” They may also be suspecting that someone has been named because they’re a high-profile ‘big beast’, or perhaps because you owed them a favour.

Avoiding this criticism

This should be really easy to avoid. Firstly, only include people on the grant if you genuinely need to do so. Perhaps you need them to carry out specific tasks, or to lead a particular work package, because they have specific expertise and access to infrastructure or resources. Maybe you need them to bring their expertise to the table in an advisory capacity.

Secondly, explain clearly what they will be doing in the project. Your explanation doesn’t have to be lengthy, but it should be specific. Do make sure that their time commitment on the project is sufficient for them to get everything that they’ve signed up to done.

Part of justifying an individual’s inclusion on a grant is to make clear that they have the specific skills and experience needed. CVs help in this respect, and narrative-style CVs are perfect. But it’s often wise to cover this briefly in your proposal too – for example in a work package description (“Dr X will lead this work package, where she will use her expertise in LC-MS for the detection of novel psychoactive substances in blood samples”).

3. “PI/Co-I time commitment is not sufficiently justified” / “PI/Co-I time commitment seems insufficient”

This is closely related to both of the above criticisms. An evaluator is worrying either that it’s not clear why someone needs to be costed into the grant for however many hours per week; or that it seems very (too) ambitious for the person or people to complete all of their project tasks in the time allowed.

Avoiding this criticism

Again, this should be easy to avoid. In fact, if you’ve taken steps to avoid the above two criticisms (scale of ambition and division of tasks) then it should never even crop up. Work out carefully who will be doing what, and how long it should take them; ensure that your time allocations and budget properly account for this; and describe and explain it all clearly in your proposal.

4. “Costs and resources are insufficiently justified”

Here we go again. You’re asking for something in your budget, but it’s not at all clear why. Why do you need all those people on the team, and how was each person’s time commitment worked out? Why do you need that bit of kit? How did you arrive at that travel figure – and why do all those people have to go? Indeed, why go at all?

Avoiding this criticism

Explain and justify everything – don’t just list it! Give sufficient breakdown where necessary rather than just including high-level totals, and explain and justify your choices (e.g. choice of conference) and numbers (e.g. number of attendees). You’ll often be given a whole section of the proposal in which to explain your requests for resources, and the clue here is very much in the name: Justification of resources. So do be sure to justify them.

I’ve previously written a whole blog post on this subject, so rather than repeat myself at length here I’ll simply provide a link. 

5. “Sample size is not justified” / “The sample size seems insufficient” / “The study appears inadequately powered”

You said that you were going to include X number of subjects/participants/samples in your study – but how did you arrive at that number? Why shouldn’t it be 10 more, and why couldn’t it be 10 less?

I see this criticism a lot. Reviewers and panels seem very fixated on sample size and power calculations, and not without reason. Every extra participant or sample that you include is likely to come at a cost to the project budget; but if you fail to include enough of them then your data, results and conclusions are likely to be incomplete, unreliable, misleading or otherwise meaningless. In short, sample-size calculation can make or break a research project.

Avoiding this criticism

To the uninitiated, sample-size calculation can be something of a minefield. It’s not uncommon to see concerns over sample size raised by reviewers, only for the panel subsequently to complain that the applicant has failed to fully address these concerns in their rebuttal. You could fit everything I know about sample-size and power calculations into one word - nothing (I’m a bid writer, not a statistician). So the best advice I can give is to find a good statistician working in your field of research (e.g. a biostatistician or trial statistician) and seek their sage advice. Depending on the nature of your project it may be appropriate to include them on the grant as a collaborator or co-investigator. If you’re not going to include them then remember the old adage – ‘what’s in it for me?’ At the very least you’re going to owe them a favour.

If you want to find out more about sample-size calculation, so as to reduce your reliance on others, then plenty has been written on the subject. See for example this paper on sample-size calculation in medical studies.

6. “PPI activity is insufficiently developed/described” / “There is little evidence of genuine co-creation”

PPI – patient and public involvement – is increasingly being expected by funders in the life sciences, even for basic biomedical research. They want it to be meaningful not tokenistic, and they’re (rightly) emphatic that involvement is not synonymous with engagement (and it’s certainly not the same as being a participant). Telling people about your research is important, but wherever possible and appropriate you should look to involve them. While no one is expecting you to get members of the public to do your lab experiments, you should always remember that, for research that’s designed to affect people’s lives, it’s better to do things with people rather than just to them (or for them).

Ideally, if PPI is appropriate to your research then you’ll have started to do it well in advance of any grant application. This will help you set your research agenda and define key research questions, and it will inform and shape the development of your proposal. If you can indicate in your proposal that your project demonstrably reflects the needs and priorities of the groups of people whom it has been designed to benefit, then you’ll be well on your way to convincing reviewers that it warrants funding. Your project itself should if appropriate incorporate on-going PPI, through which your patient/public team members can help to steer the project’s direction and keep it relevant; contribute to key decisions; and interpret and disseminate findings. A genuinely co-produced project will include PPI partners as equals when it comes to things like agenda setting and decision making. You can generally request budget for PPI activities where they’re appropriate.

Conversely, if you fail to include meaningful PPI when you should then your reviewers won’t be impressed. At the very least they’ll want to see PPI activities within the project, and they’ll often expect you to demonstrate how PPI helped to shape the project. They’ll be looking for a clear and specific description of PPI activities that you have undertaken to date and/or planned for the project.

Avoiding this criticism

The answer here is straightforward to the point of flippancy; include PPI partners in the co-development of your project wherever possible; organise and plan some specific and meaningful PPI activities whenever appropriate; and describe all of this very clearly in your proposal.

PPI can sometimes seem challenging, particularly in disciplines like lab-based pre-clinical research where opportunities for meaningful involvement are not always obvious. And for anyone seeking to do pre-project PPI activities, it can sometimes be a challenge to find the financial resources needed to cover the costs involved. There are some good resources and sources of support for PPI on the NIHR and Health and Care Research Wales websites.

7. “The PI has no prior experience of managing a project on this scale” / “The lead applicant has not previously led a large grant”

There’s clear potential for a chicken-and-egg situation here. If previously having led a large grant was a prerequisite for being awarded a large grant, then no one would ever get one. The reality is a bit more nuanced. Funders tend to be risk averse when it comes to handing out their money, and giving a million or two to someone whose previous largest grant was £50k represents a clear and unacceptable risk to most. If you apply for very substantial funding that’s an order of magnitude or so greater than anything you’ve been awarded before, the answer’s quite likely to be no.

Avoiding this criticism

The key here is to show stepwise progression, where the award of this large grant would be a logical next step in terms of both the applicant’s career progression and their research programme. If you have previously delivered two or three £300k – £500k grants, for example, and now you’re applying for £1.2 million to host a research centre in the same field of research, then your position as lead applicant should be pretty credible. Be sure to explain in the proposal narrative why your relevant track record equips you to lead on this latest major proposal.

If you just don’t have the track record, then realistically the odds are probably stacked against you. There’s no quick way around this – it obviously takes time to build that track record. Perhaps the only quick fix would be to find someone else with more experience to lead the grant, with you as a substantive co-applicant.

Seven textbook criticisms – is that the lot?

To write this blog post I drew up a list, and it ended up being rather a long one. It turns out that there are quite a few textbook errors that people commonly make in grant applications, that result in textbook criticisms from reviewers. Excessive risk with insufficient mitigation; unsubstantiated impact claims; even bad writing, sloppiness and typos. The list goes on – long enough, in fact, to fill another blog post. So watch this space.

What to do if your proposal receives these criticisms

In some cases you’ll get a second chance to put things right. Where reviewers have raised a criticism and you have a chance to respond to their reviews, be sure to grasp the opportunity in full. Give the panel the extra information they’ll need, and/or clarify whatever it was that you failed to explain clearly the first time around. If necessary, act on the reviewers’ advice to make key changes to your proposal – for example to reduce the level of (over-)ambition. I’ve previously blogged about writing a PI response to reviews – here’s the link.

Failing that, there’s little more you can do than just take it on the chin, and learn from the experience – and resolve to make sure that you never receive that particular criticism again.

The views and opinions expressed in this blog are mine alone and are in no way endorsed by my employer. Factual information and guidance are provided on a 'best-endeavour' basis and may become out of date over time. Web-links were correct at time of writing but commonly go out of date. No responsibility can be taken for any action or inaction taken or not in respect of the content of this blog. If you think you recognise your own panel rejection letter in the above list, please be assured that it is a composite of the main recurring themes taken from a large number of different proposal reviews