Ten Fatal Flaws of NIH Ten Fatal Flaws of NIH Grant Submissions Grant Submissions

(and how to avoid them)(and how to avoid them)Steffanie A. Strathdee, PhDSteffanie A. Strathdee, PhDThomas L. Patterson, PhDThomas L. Patterson, PhD

DisclaimersDisclaimers

• These points reflect the opinion of a few seasoned These points reflect the opinion of a few seasoned NIH-funded researchers who are also chartered NIH-funded researchers who are also chartered reviewers; but are admittedly somewhat subjectivereviewers; but are admittedly somewhat subjective

• Most refer to applications aimed at social, Most refer to applications aimed at social, behavioral and epidemiologic topicsbehavioral and epidemiologic topics

• Points do not include scientific misconduct Points do not include scientific misconduct • Points in italics reflect actual quotes that we have Points in italics reflect actual quotes that we have

received in our own summary statements , and received in our own summary statements , and those we have written in reviews of others’ those we have written in reviews of others’ applicationsapplications

10) Waiting Until the Last Minute10) Waiting Until the Last Minute

• GoalGoal: Drafts should be circulated : Drafts should be circulated to coauthors at least a few weeks to coauthors at least a few weeks before the deadlinebefore the deadline

• ConsequencesConsequences: : −No time for feedbackNo time for feedback−Typos, details can lack consistencyTypos, details can lack consistency−Grant lacks polish, fabric, cross-Grant lacks polish, fabric, cross-referencing, and appearance of a ‘single referencing, and appearance of a ‘single voice’voice’

9) Wrong Funding Mechanism9) Wrong Funding Mechanism• GoalGoal: : – Communicate with funders to determine agency Communicate with funders to determine agency

interest and appropriate funding mechanisminterest and appropriate funding mechanism– For an R01, present preliminary data to demonstrate For an R01, present preliminary data to demonstrate

feasibility (especially for a trial)feasibility (especially for a trial)– If lacking, consider other mechanisms, such as R21, If lacking, consider other mechanisms, such as R21,

R03 or R34. R03 or R34. • ConsequencesConsequences: : – Feasibility questionedFeasibility questioned– Study appears prematureStudy appears premature– Often considered to be a fatal flawOften considered to be a fatal flaw

8) Human Subjects Concern8) Human Subjects Concern

• GoalGoal: To ensure safety of subjects and staff, : To ensure safety of subjects and staff, addressing ‘4 points’, upholding equipoise in the addressing ‘4 points’, upholding equipoise in the case of RCTscase of RCTs

• ConsequencesConsequences::– Actual or perceived human subjects violationActual or perceived human subjects violation– Infers inexperience and/or disregard for ethical Infers inexperience and/or disregard for ethical

scientific conduct scientific conduct – Can be a fatal flawCan be a fatal flaw– If proposal receives a fundable score, NGA is not If proposal receives a fundable score, NGA is not

awarded until HRPP removes Human Subjects awarded until HRPP removes Human Subjects ConcernConcern

7) Weak Statistical Plan or Study Power7) Weak Statistical Plan or Study Power• GoalGoal:: – Study design factors in sufficient power in real-world Study design factors in sufficient power in real-world

situations (e.g. attrition, missing data, control for situations (e.g. attrition, missing data, control for confounders)confounders)• Exception: pilot studyException: pilot study

– Power and analysis sections shown for each aim and Power and analysis sections shown for each aim and hypothesishypothesis

– should link back to conceptual framework and should link back to conceptual framework and measuresmeasures

– present alternative strategies present alternative strategies – Should include up to date statistical techniques and Should include up to date statistical techniques and

softwaresoftware

7) Weak Statistical Plan or Study Power7) Weak Statistical Plan or Study Power

• ConsequencesConsequences:: –Reviewers will question feasibility for Reviewers will question feasibility for

meeting aims, (‘believability factor’) meeting aims, (‘believability factor’) –PI appears inexperiencedPI appears inexperienced–Often a fatal flawOften a fatal flaw–Statistical plan appears to have been Statistical plan appears to have been

‘written by someone else’, or ‘cookie-‘written by someone else’, or ‘cookie-cutter’, inferring inexperience or lazinesscutter’, inferring inexperience or laziness–Methods can appear passéMethods can appear passé

6) Lack of a Back-up Plan6) Lack of a Back-up Plan

• GoalGoal: Present a logical, feasible plan for : Present a logical, feasible plan for alternate strategies in case experiment or alternate strategies in case experiment or hypothesis is not borne out as hopedhypothesis is not borne out as hoped

• Success of one aim should not depend on the Success of one aim should not depend on the success of another success of another

• ConsequencesConsequences: : – Reviewers will consider this a fatal flawReviewers will consider this a fatal flaw– Aims appear as a ‘house of cards’Aims appear as a ‘house of cards’

5) Gaps in Expertise5) Gaps in Expertise• GoalGoal: : – Every content area and method matched to at Every content area and method matched to at

least one investigator with relevant expertiseleast one investigator with relevant expertise– Co-investigator % effort matches what is required Co-investigator % effort matches what is required

to meet the aimsto meet the aims– Consultants included (with letters of support) to Consultants included (with letters of support) to

fill any gaps in expertisefill any gaps in expertise

• ConsequencesConsequences: : – Proposal appears overly ambitiousProposal appears overly ambitious– Fatal flaw for a new PIFatal flaw for a new PI

4) Proposal Poorly Organized 4) Proposal Poorly Organized • GoalGoal: : – Background/significance should be concise, present Background/significance should be concise, present

both sides of controversiesboth sides of controversies– Write for the layman, not the expertWrite for the layman, not the expert– Half the proposal should be dedicated to methodsHalf the proposal should be dedicated to methods

• ConsequencesConsequences: : – Background is too long, no room for methods Background is too long, no room for methods – Lit review appears one-sided, biasedLit review appears one-sided, biased– Background too technical, reviewer is lost in jargonBackground too technical, reviewer is lost in jargon– Methods lacking sufficient detail or appear overly Methods lacking sufficient detail or appear overly

dense and hard to digestdense and hard to digest

3) Missing /Problematic Hypotheses or 3) Missing /Problematic Hypotheses or Weak/Absent Conceptual FrameworkWeak/Absent Conceptual Framework• GoalGoal:: – Aims should be linked to clear, testable Aims should be linked to clear, testable

hypotheses for which the outcome is not already hypotheses for which the outcome is not already well established well established

– Aims and hypotheses should map onto conceptual Aims and hypotheses should map onto conceptual framework, measures, power and analysisframework, measures, power and analysis

3) Missing /Problematic Hypotheses or 3) Missing /Problematic Hypotheses or Weak/Absent Conceptual FrameworkWeak/Absent Conceptual FrameworkConsequencesConsequences: : – Application appears merely ‘descriptive’Application appears merely ‘descriptive’– Hypotheses appear ‘pedestrian in nature’ Hypotheses appear ‘pedestrian in nature’ – Research questions and design appear murkyResearch questions and design appear murky– Study design ‘lacks focus’Study design ‘lacks focus’– Power and statistical analysis section appears to Power and statistical analysis section appears to

be ‘cookie cutter’ since they do not tie back to be ‘cookie cutter’ since they do not tie back to hypotheses/framework.hypotheses/framework.

2) Lack of Significance/Innovation2) Lack of Significance/Innovation• GoalGoal: Proposal deals with an important, : Proposal deals with an important,

exciting topic re: public health and/or clinical exciting topic re: public health and/or clinical decision-making, or moves the field forward.decision-making, or moves the field forward.

• ConsequencesConsequences: : - Reviewers will be bored, significance rating will significantly Reviewers will be bored, significance rating will significantly

hamper overall scorehamper overall score- Proposal has a hard time competing with othersProposal has a hard time competing with others- A beautifully designed study that has no real significance or A beautifully designed study that has no real significance or

innovation will not be fundedinnovation will not be funded

““And now, for the #1 fatal flaw ofAnd now, for the #1 fatal flaw ofNIH grant submissions…”NIH grant submissions…”

1) Overly Ambitious1) Overly Ambitious• Goal: Goal: – Project is designed to be feasible within the time frameProject is designed to be feasible within the time frame– Aims support one coherent project, not 2 or moreAims support one coherent project, not 2 or more– Provide enough detail for reviewers to understand novel Provide enough detail for reviewers to understand novel

methods and measuresmethods and measures– Project generates preliminary data to guide future studiesProject generates preliminary data to guide future studies

1) Overly Ambitious1) Overly AmbitiousConsequencesConsequences: : – Threatens the ‘believability factor’Threatens the ‘believability factor’– Budget may not realistically support the aimsBudget may not realistically support the aims– Makes PI appear inexperienced; possible fatal flawMakes PI appear inexperienced; possible fatal flaw– Reviewers may propose cutting an entire aim or 2, Reviewers may propose cutting an entire aim or 2,

or may unscore the proposal after deciding they or may unscore the proposal after deciding they ‘cannot re-write it for the PI’‘cannot re-write it for the PI’

– If you are funded, you stand to risk not being able to If you are funded, you stand to risk not being able to meet aims, which can risk your reputationmeet aims, which can risk your reputation

GOOD LUCK!GOOD LUCK!