Date post: | 29-Aug-2018 |
Category: |
Documents |
Upload: | truongthien |
View: | 216 times |
Download: | 0 times |
Rules of the Road: Successful Navigation through the NCI/NIH Peer Review Process
Jeannette F. Korczak, Ph.D.Scientific Review Officer (SRO)
Resources and Training Review BranchDivision of Extramural Activities
National Cancer Institute
NIH’s “Enhancing Peer Review”
• Initiative began in early 2009 with a phased-in implementation
• Goals include:– Improving reliability of scoring– Providing quantitative feedback on
ALL applications– Easing administrative burden on
applicants, reviewers, and NIH staff
NIH’s “Enhancing Peer Review”
• Peer review emphasis is on IMPACT, rather than methodological details
• 9-point scoring system used to assess Overall Impact and each of 5 Core Review Criteria:– Only applications discussed at the
meeting receive an Overall Impact score• Other applications are “ND” (Not Discussed)
– All applications receive scores for the 5 Core Review Criteria
9-Point Scoring System
SCORE DESCRIPTOR IMPACT1 EXCEPTIONAL HIGH2 OUTSTANDING HIGH3 EXCELLENT HIGH4 VERY GOOD MODERATE5 GOOD MODERATE6 SATISFACTORY MODERATE7 FAIR LOW8 MARGINAL LOW9 POOR LOW
Scoring Process• Prior to review meeting, reviewers:
– Prepare critiques and assign scores from 1 to 9 to Preliminary Overall Impact and 5 Core Review Criteria
• At meeting, reviewers assign Final Impact Score to Discussed applications– Scores are averaged and multiplied by 10– Final Impact Scores: 10 (best) to 90 (worst)– Not discussed applications are scored ND
• After meeting, reviewers update criterion scores and critiques
Impact and 5 Core Review Criteria
• Definition of “Impact” varies across Funding Mechanisms:– For R01/R21/R03, etc., “Impact” is
likelihood for project to exert a sustained, powerful impact on the field
• in consideration of “pertinent” review criteria– For K01/K07/K99, etc., “Impact” is
likelihood for candidate to achieve scientific independence
• in consideration of “pertinent” review criteria
Impact and 5 Core Review Criteria
• 5 Core Review Criteria vary across Funding Mechanisms
• For R01/R21/R03, etc., they are:– Significance; Investigator(s); Innovation;
Approach; and Environment• For K01/K07/K99, etc., they are:
– Candidate; Career Development Plan; Research Plan; Mentor(s), Co-mentor(s), Consultant(s), Collaborator(s); and Environment and Institutional Commitment to Candidate
•
Scoring Based on Balance of Strengths and Weaknesses
• Reviewers list strengths/weaknesses for each Criterion and for Overall Impact, and assign scores based on:– Number of strengths/weaknesses – Whether strengths/weaknesses are
major, moderate, or minor – Examples:
• Score = 2: Extremely strong with negligible weaknesses
• Score = 5: Strong but > 1 moderate weakness
Maximizing Your Scores
• ALWAYS consider the guidelines for REVIEWERS when preparing your application
• See SECTION V: APPLICATION REVIEW INFORMATION of the Funding Opportunity Announcement for the review criteria guidelines
Section V for “K” Applications• Candidate:
– Potential to become independent, productive researcher?
– High quality academic, clinical, and research record?
– Commitment to becoming independent investigator in research?
– Letters of reference from well-established scientists who address review criteria and attest to candidate’s potential for independent research?
Section V for “K” Applications
• Career Development Plan:– Will contribute significantly to
candidate becoming scientifically independent?
– Appropriate content, scope, phasing, and duration, given candidate’s background and objectives?
– Adequate plans to monitor candidate’s research and career development progress?
Section V for “K” Applications
• Research Plan:– Significant scientific and technical merit
of research question, design, methods?– Relevant to candidate’s research career
objectives?– Appropriate to candidate’s stage of
research development and future goals?– Adequate plans to include both genders,
minorities and children, as appropriate, for human subjects research?
Section V for “K” Applications
• Mentors, Co-mentors, Consultants, Collaborators:– Mentor has appropriate research
qualifications?– Adequate description of quality and
extent of mentor’s role in advising candidate?
– Previous experience of team in mentoring independent investigators?
– Productive research team with peer-reviewed support?
Section V for “K” Applications
• Environment and Institutional Commitment to Candidate:– Assurance of > 75% of candidate’s effort
for “K” project?– Evidence that candidate will be integral
part of institution’s research enterprise?– Appropriate facilities, resources, and
training opportunities, including faculty?– High quality environment for candidate’s
scientific and professional development?
Other Components, as Applicable
• Additional Review Criteria - can affect the Impact Score:– Protections for Human Subjects; Inclusion
of Women, Minorities, and Children; Vertebrate Animals; Biohazards
• Additional Review Considerations -do not affect Impact Score:– Training in the Responsible Conduct of
Research; Budget and Period of Support; and Resource Sharing Plan
Protections for Human Subjects
• If research is exempt under 45 CFR Part 46, you MUST describe:– Justification for the exemption and
sources of materials– Involvement of human subjects and their
characteristics
Protections for Human Subjects• If research is NOT exempt under 45
CFR Part 46, you MUST describe:– Risk to subjects– Adequacy of protection against the risks– Potential benefits to subjects and others– Importance of the knowledge to be gained– Data and safety monitoring for clinical
trials– Plans for inclusion of minorities, members
of both genders, and children, as appropriate, for clinical research
Vertebrate Animals/Biohazards• For live vertebrate animals, describe:
– Proposed use of animals, and species, strains, ages, sex, and numbers
– Justification for the use of animals– Adequacy of veterinary care– Limitation of discomfort, distress, pain, and
injury– Methods of euthanasia
• For biohazards, describe:– Materials/procedures that are hazardous to
research personnel and/or environment and protections to be used
Training in the Responsible Conduct of Research (RCR)
• See NOT-OD-10-019 and NOT-OD-11-059• “K” applicants have 1 page to describe
RCR training in regard to:– Format: must include face-to-face training– Subject Matter: e.g., conflict of interest,
collaborative research, research misconduct– Faculty Participation: should include
mentors and training faculty– Duration of Instruction: > 8 contact hours– Frequency of Instruction: at least once every
career stage/every 4 years
Budget / Resource Sharing Plan
• Budget and Period of Support:– Requested funds must be adequately
explained in Budget Justification– Make sure Period of Support and Indirect
Costs are correct• Resource Sharing Plan:
– Data Sharing Plan– Model Organism Sharing Plan– Genome-Wide Association Studies
(GWAS)
Restructured Applications
• See NOT-OD-10-002 and NOT-OD-11-027• Key components of “K” applications
total < 12 pages and include:– Candidate’s Background– Career Goals & Objectives– Career Development/Training Activities
During Award Period– Research Strategy
Restructured Applications• Other sections with page limits:
– Project Description (Abstract) – 1 page– Specific Aims – 1 page– Training in Responsible Conduct of
Research – 1 page– Statements by Mentor, Co-Mentors,
Contributors – 6 pages– Description of Institutional Environment – 1
page– Commitment to Candidate’s Research
Career Development – 1 page– Biographical Sketches – 4 pages/person
Restructured Applications• Sections without page limits:
– Budget Justification– Protection of Human Subjects– Inclusion of Women and Minorities– Inclusion of Children– Vertebrate Animals– Appendixes
• Sage advice: Avoid the temptation to “Overstuff” your application!– e.g., by including Research Strategy in
Budget Justification or Vertebrate Animals sections
Biographical Sketches
• Biographical Sketches (“Biosketches”) – 4 pages/person:– Include a biosketch for you, your
mentor(s), and all collaborators– Biosketch contains a Personal Statement
which should be relevant to each person’s role on the project
– Recommended (not required!) that it includes full citation for up to 15 of most relevant publications
• Also indicate total number of publications after “Selected Peer-Reviewed Publications”
Appendixes• See NOT-OD-07-018• Allowable appendixes include:
– Up to 3 of the following:• Accepted but not yet published
manuscripts and/or abstracts• Published manuscripts/abstracts if free,
online, publicly available journal link not available
• Patent materials relevant to project– Surveys, questionnaires, data collection
instruments, clinical protocols, informed consent documents
More Recent NIH Policies on Grants• Resubmission applications (A1) must
be received within 37 months of the initial application receipt date– See NOT-OD-10-140
• No more 2-day “error correction window” – all corrections must be complete by due date– See NOT-OD-10-123
• No more 5-day grace period for Letters of Reference – must be in by due date – See NOT-OD-11-036
More Recent NIH Policies on Grants• Additional restrictions on submission
of late grant application materials– See NOT-OD-10-115
• Policy on late submission of grant applications: e.g., up to a 2-week extension for submitting an application under certain conditions– See NOT-OD-11-035