The Center for Symptom Management

The NIH review process

Kathryn Lee, RN, PhDApril 3, 2009

MDP

The Center for Symptom Management

The Center for Symptom Management

Grant Mechanisms

R series (research projects) R01 -individual research R03- small, short-term R21- exploratory/developmental

R15 – certain institutes with little NIH funding

The Center for Symptom Management

1946

One Review Platform for 62 years

The First NIH Study Section The Last NIH Study Section

2008

The Center for Symptom Management

The Letter of Intent (LOI)

Used by federal agencies/foundations Filter applications to their

interest area Appoint appropriate reviewers

Specific to the agency, typically ask for Abstract NIH Biosketch Nomination letter (some, but not all)

The Center for Symptom Management

Writing an LOI- Start Early

Limited to 1-2 pages

Title of proposal Background of applicant (or Biosketch)

Objectives Design and Methods Statistical analysis plan

The Center for Symptom Management

LOI: Can really help your process

finalize key aims/questions get prepared to submit a well designed application in short time frame

Start a relationship with your future program officer

The Center for Symptom Management

Letters of Intent

They’re Not Easy ! “If I had more time, I would have written you a shorter letter.”

Mark Twain

The Center for Symptom Management

National Institutes of Health

NIH Mission: Promote biomedical

and behavioral research to help improve the health of all Americans

Carried out through 27 Institutes and Centers

The Center for Symptom Management

FY 2007 NIH Budget is $28.6 Billion

SpendingSpendingOutside NIHOutside NIH

$24.1 B$24.1 B

SpendingSpending

at NIHat NIH

$4.5 B$4.5 B

The Center for Symptom Management

Goal of Peer Review

Independent, fair, thorough, and competent review of each application

Identify and rank appropriately those applications that show the greatest promise of advancing biomedical science and/or improving disease prevention, diagnosis and treatment

The Center for Symptom Management

Role of Reviewer

Provide judgment of the scientific merit of each application; NOT TO DETERMINE FUNDING

Criteria for selecting reviewers Record of scientific excellence Able to see “big picture” Fair and balanced Willing to follow guidelines and stay w/in required time frame

Articulate opinions in a clear, concise manner

Open minded to the views of other reviewers

The Center for Symptom Management

Conflict of Interest

Worked with key personnel in past 3 years or currently

Financial gain Close relative of key personnel PI is from your institution Recognized scientific disagreement PI was your student/major dissertation advisor

You are applying for job at PI’s institution

PI is applying for job at your institution

The Center for Symptom Management

Review Details

Each application 3-4 assigned reviewers

Primary , Secondary – full written critique

Discussant(s)- summary paragraph

The Center for Symptom Management

Tips for Reviewers

Focus on science, not “grantsmanship”

Keep in mind “big picture” Distinguish between major problems and minor concerns or differences in approach

“lack of detail” – what is missing and why it matters

“Trust me” proposal

The Center for Symptom Management

Written Critiques

Address each of 5 review criteria Address human subjects/inclusion Overall evaluation/summary paragraph Be specific, constructive, and concise

Not a time for mentoring Do not identify yourself

The Center for Symptom Management

Review Criteria

Significance 1= exceptional 9 = poor

Approach (1 - 9) Innovation (1 - 9) Investigators (1 - 9) Environment (1 - 9)

Overall IMPACT

The Center for Symptom Management

New Scoring

Strengths

Weaknesses

Poor9

Marginal8

Fair7

Low Impact

Satisfactory6

Good5

Very Good4

Moderate Impact

Excellent3

Outstanding2

Exceptional1

HighImpact

Guidance on weighing strengths

and weaknesses

Overall Impact Score

The Center for Symptom Management

Chair’s Role

Ensure that all appropriate viewpoints are expressed

Ensure that discussion is fair, balanced, and appropriate

Promote consistent scoring Summarize panel’s views Time management

The Center for Symptom Management

Percentiles

Scores are translated into percentiles which are used to make funding decisions

Currently most institutes funding at 8-15th percentile

Score clustering results makes it easier to fund out of order

The Center for Symptom Management

Dual Review System for Applications

2nd Level of Review NIH Institute/Center Council

1st Level of Review

Scientific Review Group (SRG)

The Center for Symptom Management

Streamlining

Purpose: make more efficient use of time at meetings

Goal: streamline 50-60%, so proposals that are not competitive are not discussed

Streamlining does NOT equal BAD

The Center for Symptom Management

CSR Peer Review: 2008

77,000 applications received

56,000 applications reviewed

16,000 reviewers

240 Scientific Review Officers

1,600 review meetings

The Center for Symptom Management

Major Complaints About NIH Peer Review

Process is too slow

Not enough senior/experienced reviewers

Process favors predictable research instead of significant, innovative, or transformative research

Time and effort required to write and review are a heavy burden on applicants and reviewers

The Center for Symptom Management

Assign Applications Accurately & Efficiently

Retooled for electronic submission

Applications are now submitted electronically

Assign applications using text fingerprinting, and text mining programs

Full Implementation by early 2009

The Center for Symptom Management

Fund best research earlier & reduce burden on applicants, reviewers, & NIH

More flexible deadlines Abolish A2 applications

The Center for Symptom Management

Improve Quality & Transparency of Peer ReviewMay-July meetings 2009 Shorten summary statements, follow template

for each criteria Change the rating system

Use 1-9 integers Score each criteria Provide score for all applications (even those not discussed)

Spring 2010 Shorten applications, aligning with review

criteria Impact, investigator, innovation (if applicable), research strategy, facilities

The Center for Symptom Management

0

20000

40000

60000

80000

Number of Applications Submitted

Historical Growth

The Center for Symptom Management

0

2

4

6

8

10

12

Reviewer’s Load

Applications Per Reviewer

October Council Rounds

The Center for Symptom Management

RO1 Resubmission Within 4 Months of Original Application

0

100

200

300

400

500

600

2006 2007 2008

New PI

Estab.PI

The Center for Symptom Management

Confidentiality

“What happens in study section stays in study section”

Materials are proprietary Don’t discuss outside of the meeting

Don’t show application to anyone else

Avoid web sites associated with grant

The Center for Symptom Management

NIH resources

Proposal writing guides Avoiding common mistakes in an application

SON website link to NIH website at: 

http://nurseweb.ucsf.edu/www/ix-rs.shtml.