Bridges to the Future Technical Assistance

Workshop

Division of Minority Opportunities in ResearchNational Institute of General Medical Sciences

National Institutes of Health, USDHHS

June 19, 2009

NIGMS

Bridges To the Future Bridges To the Future ProgramProgram

Clifton Poodry, Ph.D.

Director, MORE DivisionNational Institute of General Medical Sciences

National Institutes of Health, USDHHS

June, 19 2009

Outline of PresentationOutline of Presentation

•Overview of Bridges to Baccalaureate (PAR-07-411) & Bridges to Doctorate (PAR-07-410) programs

•Guidance for Bridges Program Development

Bridges Program GoalsBridges Program Goals

An institutional program with a focus on increasing the number of community college/master’s degree students from underrepresented groups and/or health disparities populations (URMs) who transfer and complete the baccalaureate/PhD degree, respectively, in biomedical and behavioral sciences.

The Bridges Program Emphasizes:

Institutional Focus: impact on Bridges and non-Bridges students alike so more URMs transfer and complete the baccalaureate/PhD degree

Partnerships: 2-year community college(s) with four-year institution(s), & master’s degree-granting institution(s) with PhD degree-granting institution(s)

Developmental Activities: well-integrated activities that will provide students with the necessary academic preparation and skills to enable their transition and successful completion of the baccalaureate/PhD degree in biomedical/behavioral sciences

Clear Expectations

• Purpose: To facilitate a seamless transition of targetedstudents from associate to baccalaureate degree-granting institution, and from master’s to the PhD degree-granting institution in biomedical/behavioral sciences

• Size: Up to 4 institutions for BTB, and 3 for BTD, including the applicant institution, unless strongly justified otherwise

• An institution may participate in more than one Bridges partnership if strongly justified by the potential to magnify the programs’ and institution’s outcomes

Partnership/Consortium

• Students from groups underrepresented in the biomedical and behavioral research enterprise of the nation and/or populations disproportionately affected by health disparities (targeted groups). Nationally, these groups include, but are not limited to, African Americans, Hispanic Americans, Native Americans (including Alaska Natives), Natives of the U.S. Pacific Islands, and/or rural Appalachians.

• Must be U.S. citizens or non-citizen nationals or permanent residents.

• Must be matriculated full-time in associate/master’s degree program in biomedically relevant science fields at the partner community college/master’s degree institution.

Bridges Students((Students who receive support in the form of salaries/wages)

• Student Selection

• Institution’s responsibility to establish student qualifications

• Describe the criteria for selection and retention of Bridges students into the program

• Critical Mass: Number of students in the program each

year

• BTB: 15-20, with 4 from each CC

• BTD: 6-8, with 3 from each master’s degree institution

Student Selection & Critical Mass

Baseline Data & Goals to Improve

Effective Partnerships

Strong Pool of Targeted Students Well-integrated Student Development

Activities year-round

Strong Institutional Commitment

Sound Evaluation Plan

Key Program Expectations

Baseline• The baseline is a starting point; it serves as a guide to capacity,

and is necessary to gauge the impact of the program. For example, an increase of 100% on a base of 1 is not nearly as impressive as if it were on a base of 10.

• Usually it’s an average data over a 3-5 year period

Measurable Objectives• These are brief (and focused) statements of end results –

connected to the long-term goal. They can be qualitative and quantitative, but must be measurable.

• They are not a means to an end, or a checklist of “to do” list.

• The activities proposed are the means to achieve your goals and objectives.

What Are the Baseline Data & Measurable Objectives?

Examples of Goals, Baseline Data & Measurable Objectives

Goal: To increase the number of CC/master’s degree students who transfer to four-year/PhD degree-granting institutions in biomedically relevant sciences.

Baseline: How many and what percent of the students currently transfer to baccalaureate/PhD degree programs?

Measurable objective: What is the proposed number and percentage of students who will transfer to baccalaureate/PhD degree programs during the grant period?

Examples of Baseline Data & Objectives, cont.

Institutional Baseline: Out of a total of 40 students in biomedically relevant fields (40% URM & 60% non-URM), 4 URMs (25%) & 9 non-URMs (37.5%) actually transfer to programs in biomedically relevant disciplines per year (average data from 2005-2008).

Institutional Objective:The URM transfer in biomedically relevant disciplines will increase from the current rate of 25% to 37.5% (from an average of 4 to 6 students per year) by the fourth year of the grant award.

Examples of Baseline Data & Objectives, cont.

Bridges Baseline on Transfer - An average of 4 out of 8 (50%) Bridges students transferred to programs in biomedically relevant disciplines per year (average data from 2005-2008).

Objective - The transfer of Bridges students to programs in biomedically relevant disciplines will increase from the current rate of 50% to 75% (from an average of 4 to 6 students per year) by the fourth year of the grant award.

Bridges Baseline on Degree Completion - An average of 2 out of 4 (50%) transferring Bridges students completed the baccalaureate/PhD degree in biomedically relevant disciplines per year (average data from 2005-2008).

Objective - Bridges students’ degree completion in biomedically relevant disciplines will increase from the current rate of 50% to 75% (to an average of 4.5 graduates per year) by the fourth year of the grant award.

• Clearly stated goals, and measurable objectives relative to the baseline and in context of NIH expectations

• Must improve on the previous record and make substantial progress towards meeting the NIH expectations during the next 5 years

Goals to Improve the Track Record

• Institutional• Increase in the institutional transfer of targeted students

(Bridges & non-Bridges) to baccalaureate/PhD degree programs in biomedically relevant sciences by 50% in five years

• Bridges• Increase in academic preparation and skills development

• Increase in the transfer of students to baccalaureate/PhD degree programs (70-75% in five years)

• Increase in the baccalaureate/PhD degree completion of transferring students (75-80% in five years)

NIH ExpectationsNIH Expectations

BASELINE DATA AND OBJECTIVES

Applicant Institution/

Unified Institutional

Program

Partner Institutions

Institutional Baseline for Transfer of URMs and non-URMs (# of

students who transfer/ # of full-time students in

biomedically relevant disciplines)

Institutional Objectives for Transfer

of URMs

Bridges Baseline for Transfer (# of Bridges students who transfer/ total

# of Bridges students in the program) & Degree Completion (# of

Bridges students who complete the baccalaureate/PhD degree/ # of transferring Bridges students)

Bridges Objectives for Transfer

& Degree Completion

Great University

Champion College

       

         

Valley State College

       

       

Unified Institutional Program

         

To Move the Institution Forward from Baseline to NIH Expectations

Bridges to Baccalaureate:Bridges to Baccalaureate:

• Developing community college courses and curricula that are fully Developing community college courses and curricula that are fully transferable to the baccalaureate institutiontransferable to the baccalaureate institution

• Faculty from the four-year institution serving as visiting lecturers at the two-year institution, or developing joint team-taught courses

• Introduction of research concepts into the community college Introduction of research concepts into the community college curriculum and/or offering a research skills course to stimulate curriculum and/or offering a research skills course to stimulate students’ interest in sciencestudents’ interest in science

• Supplementary instruction in “gate-keeping” courses at the Supplementary instruction in “gate-keeping” courses at the community college community college

• Mentored research experiences for the Bridges students, Mentored research experiences for the Bridges students, including summer research internshipsincluding summer research internships

Examples of Developmental Activities

Bridges to Baccalaureate, cont.Bridges to Baccalaureate, cont.

• Skills development (e.g., critical thinking, communications skills, study and time management skills) workshops, and research careers seminars

• Preparing community college students, through college orientation classes, etc., for transfer to the 4-year institution

• Peer mentoring and tutoring, and research career seminars

• Research education conferences for Bridges faculty (CC), and advanced or special courses at the partner 4-year institution

• Research conferences for 4-year faculty mentors, if accompanying students making presentations

• Others

Examples of Developmental Activities, Examples of Developmental Activities, Cont. Cont.

Bridges to Doctorate:Bridges to Doctorate:

• Faculty from the two types of institutions jointly developing courses Faculty from the two types of institutions jointly developing courses and curricula, including updating existing or developing and curricula, including updating existing or developing new/advanced courses at the master’s degree institution new/advanced courses at the master’s degree institution

• Faculty from the doctorate institution serving as visiting lecturers, Faculty from the doctorate institution serving as visiting lecturers, offering lectures and/or laboratory courses at the master’s degree offering lectures and/or laboratory courses at the master’s degree institution institution

• Faculty from the doctorate institution providing mentored research Faculty from the doctorate institution providing mentored research experiences to Bridges students, including the summer research experiences to Bridges students, including the summer research internships, and serving on their thesis advisory committees internships, and serving on their thesis advisory committees

• Fostering research capacity of the master’s degree institution via Fostering research capacity of the master’s degree institution via research collaborationsresearch collaborations

Examples of Developmental ActivitiesExamples of Developmental Activities

Bridges to Doctorate, cont.Bridges to Doctorate, cont.

• Allowing Bridges students to take some courses, and complete part Allowing Bridges students to take some courses, and complete part of their master’s thesis research at the doctoral institutionof their master’s thesis research at the doctoral institution

• Providing Bridges students access to computer and library facilities, Providing Bridges students access to computer and library facilities, seminars, and workshops,seminars, and workshops, etc., at the doctoral institution etc., at the doctoral institution

• Establishing a mentoring and academic counseling program for Establishing a mentoring and academic counseling program for master’s students with faculty at the doctoral institutionmaster’s students with faculty at the doctoral institution

• Advanced or special courses and scientific research conferences for Advanced or special courses and scientific research conferences for Bridges faculty from the master’s degree institutionBridges faculty from the master’s degree institution

Examples of Developmental

Activities

Strong commitment to the goals of the proposed program from all participating institutions, including commitment to:

• Provide the institutional data on the transfer of URMs and non-URMs in biomedically relevant sciences

• Provide the data on transfer and baccalaureate/PhD degree completion by Bridges students in biomedically relevant sciences

• Track Bridges students over a ten-year period as they progress through the pipeline

• Institutionalize Institutionalize the most effective activities supported by the Bridges program

Institutional Commitment

• Purpose: Provide information useful to the applicant and the partner institutions for improving the program, and for institutionalizing the most effective activities supported by the Bridges program.

• Provide a sound evaluation plan, with timeline, that’s in-line with measurable goals and objectives

• Identify a qualified evaluator and include his/her biosketch

EvaluationEvaluation

• Consortium agreement between the associate and baccalaureate, and between master’s and PhD degree-granting institutions define the

participating institutions’ respective roles in administering the Bridges program.

• The application must include a letter from each collaborating

institution signed by the appropriate institutional officials and

program director/program coordinator, acknowledging participation in the program. These letters must also include the following:

“THE APPROPRIATE PROGRAMMATIC AND ADMINISTRATIVE PERSONNEL OF EACH INSTITUTION INVOLVED IN THIS GRANT APPLICATION ARE AWARE OF THE NIH CONSORTIUM GRANT POLICY AND ARE PREPARED TO ESTABLISH THE NECESSARY INTER-INSTITUTIONAL AGREEMENT(S) CONSISTENT WITH THAT POLICY.”

Consortium Agreement

• The program provides support for student, faculty, and institutional development activities

• Average awards range from $150,000 to $300,000 (DC) per year.

• Budget must be reasonable, well documented, and fully justified and commensurate with the scope of the proposed program

• Awards for up to five years

Budget and Years of Support

Guidance for Bridges Program Development

Develop a Plan

• Read the FOA and F&Qs

• Conduct an institutional assessment and gather baseline data: how many transferring and graduating; why not more transferring and graduating? what can change so more will transfer and graduate?

• What are the institutional needs?

• What is your long range goal?

• What are your measurable objectives?

• What activities will help your institution achieve these objectives

• How will you evaluate your program outcomes?

• Follow all of the instructions in writing your application!!

• Project Summary, & Performance site Project Summary, & Performance site (s)(s)

• Facilities & other ResourcesFacilities & other Resources

• Key Persons and their Biographical Key Persons and their Biographical SketchesSketches

• BudgetBudget

• Research Plan:Research Plan:

Application Components/Organization

Research Plan:

• (Introduction)

• Specific Aims and Measurable Objectives

• Background and Significance: Institutional & student data; vision and anticipated value

• Preliminary Studies or Progress Report: Specific Outcome Data & Impact on the Institution

• Research Design & Methods:

Application Components/Org., Cont. Application Components/Org., Cont.

Research Design & Methods:Research Design & Methods:

• PD, PCs, and Program FacultyPD, PCs, and Program Faculty

• Developmental Activities:Developmental Activities:

The rationale and detailed description

Who will implement and the timeline

Possible pitfalls and alternative approaches

• Responsible Conduct of ResearchResponsible Conduct of Research

• Evaluation PlanEvaluation Plan

• Consortium AgreementsConsortium Agreements

Application Components/Org., Cont. Application Components/Org., Cont.

Presentation of Data

• Present data in figures, graphs, tables or text

• Place figures, tables, and graphs close to where they are referred to in the text

• Make all figures, tables, and graphs clearly legible

• Avoid irrelevant information

• Do not use the Appendix to circumvent the page limitations of the Research Plan

Common Reasons for Failure

• Missing or inadequate baseline data

• Lack of clear and well-defined measurable objectives

• Lack of adequate progress

• Activities poorly related to the objectives

• Poorly developed or missing evaluation plan

• Lack of institutional commitment or support

• Lack of coordination with other institutional programs

aimed at accomplishing similar goals

• “Program-centric” application vs. institutional

application

• Institutional and Bridges baseline data

• Clear statement of program goals, specific aims, and measurable objectives

• Detailed progress report, if applicable• PD/PI, Coordinators, and program faculty with appropriate training

and experience

• Detailed description of developmental activities

• Sound evaluation plan (see for example, http://oerl.sri.com/ and http://www.nsf.gov/pubs/2002/nsf02057/start.htm)

• A plan for training in responsible conduct of research

• Evidence of strong institutional commitment

• Consortium Agreement with “specific language”

• A well justified budget that is commensurate with the scope of the proposed program

Summary: Bridges “Essentials”

Program Evaluation• BRIDGES TO THE DOCTORATE (PAR-07-410)

• BRIDGES TO THE BACCALAUREATE (PAR-07-411)

Clifton Poodry, Ph.D.Division of Minority Opportunities in ResearchNational Institutes of General Medical Sciences

National Institutes of Health, USDHHS

June 19, 2009

NIGMS

Evaluation: What is it? Program evaluations are

individual, systematic studies that use objective measurement and analysis to answer specific questions about how well a program is working.

- #GAO/GGD-00-204 Program Evaluation

Program evaluation and the tracking of students are not the same

thing.

NIGMS

Program Evaluation Answers Questions Like….

• Does it work?• How well does it work?• Does it do what we want it to? • Does it work for the reasons we think

it does? • Is it cost effective? • Are the benefits worth it? • What are the unintended

consequences?

NIGMS

Why bother? Why bother? Supports continuous program

improvement

Increases understanding of the program – how are activities and strategies linked to results?

Leads to improved planning and management

Provides shared understanding of program

NIGMS

Guidelines for Conducting Successful Evaluations

• Invest heavily in planning early on

• Use knowledgeable, experienced evaluators (usually social scientists)

• Integrate evaluation into ongoing activities of the program

NIGMS

Typical Evaluations

Needs Assessment What is nature & extent of the issues

program should address? Planning phase

Process Evaluation Is program being conducted & producing

output as planned? How can process can be improved?

Outcome Evaluation Extent to which a program’s goals have

been met?

NIGMS

Needs Assessment

What problem is the program attempting to address?

Whom does this program serve; to what extent are their needs met?

What should be the documented goals of the program?

Process Evaluation

Is the program being implemented as planned? If not, why?

How could the program’s processes be improved?

Has the program achieved recognized standards of performance?

Sample Study QuestionsNIGMS

Outcome Evaluation

To what extent has the program achieved its goals?

Is the current performance different from the past?

Has the program been more successful than a comparable program?

Which characteristics/ activities are most related to success?

What are the intended/ unintended effects of the program?

Sample Study QuestionsNIGMS

Why should you care?

If you don’t know where you’re going, any road will take you there.

- Lewis Carroll, Alice in Wonderland-Illustration by Sir John Tenniel, eBooks@ Adelaide, 2004

NIGMS

Key Steps in Evaluation

1. Engage stakeholders2. Describe the program3. Focus the evaluation design4. Gather credible evidence5. Justify conclusions (present

data, analysis used, and findings)

6. Ensure use and share lessons

NIGMS

1. Engage 1. Engage StakeholdersStakeholders

Who are the stakeholders?

Those involved in program operations, those affected by the program operations, and primary users of evaluation

results

NIGMS

2. Describe the program What are the goals and specific aims of

the program?

What problem or need is it designed to address?

What are the measurable objectives?

What are the strategies to achieve the objectives?

What are the expected effects?

What are the resources and activities?

How is the program supposed to work?

NIGMS

Activity ≠ Program

You cannot evaluate a program by assessing only

an activity

Remember: NIGMS

Model of a Training Program Resources Activities Impact

Research base

Workshops & Seminars

Mentoring by faculty member

Training in scientific methods

Short term KnowledgeSkills Attitudes

IntermediateBehaviorsPractices

Long term

Enter PhD Program

Faculty & Staff

Money

Equipment & Technology

What is invested?

(Inputs) (Outputs) (Outcomes)

What is invested? What is done? What are the changes or benefits?

NIGMS

3. Focus the evaluation design

What do you want to know? (key questions)

Who will be involved in or affected by the evaluation or use the findings? (stakeholders)

To focus an evaluation, consider its purpose, uses, questions, methods, roles, budgets, deliverables etc.

An evaluation cannot answer all questions for all stakeholders

NIGMS

4. Gather credible evidence

Evidence must be believable, trustworthy, and

relevant

• Select methodological approach & data collection instruments

• Determine who is studied and when

NIGMS

5. “Justify” conclusionsConsider data:

• Analysis and synthesis - determine findings

• Interpretation - what do findings mean?

• Judgments - what is the value of findings based on

accepted standards?

• Recommendations – - what claims can be made?- what are the limitations of your design?

NIGMS

An evaluation plan should include:

• Program description with baseline data

• Purpose & rationale for evaluation

• Evaluation Design • Data Collection & Analyses • Products of evaluation & their

use• Project Management • Budget estimate

NIGMS

If you remember nothing else…

Evaluation is a tool to help you make decisions about program

management

NIGMS

June 2009

Application Submission and Review Considerations

Mona R. Trempe, Ph.D.Scientific Review Officer

NIGMS Office of Scientific Review

NIGMS

June 2009

Application Process

First step - Grants.gov

Electronic application via this site now required for all R and most K mechanisms

Tentative transition dates for other mechanisms: F transition scheduled for August 2009

T and K12 scheduled for January 2010 P and U transition indefinite at this point

NIGMS

June 2009

Application Process

Second Step - eRA Commons

Retrieves the application from Grants.gov and checks it against NIH-specific requirements

Allows applicants to electronically track the status of submissions and to receive/transmit application and award information

Provides contact information for assigned NIH staff: MORE Division, Review Office, and Grants Management

NIGMS

June 2009

Online Resources

Overview of Electronic Submission http://era.nih.gov/ElectronicReceipt/

Frequently Asked Questions http://era.nih.gov/ElectronicReceipt/faq.htm

Avoiding Common Errors http://era.nih.gov/ElectronicReceipt/avoiding_errors.htm

Training Resources, Videos, Quick Reference Materials http://era.nih.gov/ElectronicReceipt/training.htm

NIGMS

June 2009

Finding Help

Grants.gov Contact Center Toll-free: 1-800-518-4726Email : support@grants.gov

Hours : Mon-Fri, 7 a.m. to 9 p.m. ESTSupport for: Grants.gov registration, Mac issues,

Adobe forms

eRA Commons Help Desk Phone: 301-402-7469

Hours : Mon-Fri, 7a.m. to 8 p.m. EST Online Help Ticket : http://ithelpdesk.nih.gov/eRA/ Support for: Commons registration, application status,

post-submission questions

NIGMS

June 2009

Review Process: Usual Timeline

Timeframe Activity

1 - 2 months Referral

2 - 6 months Review Panel

6 - 7 months Summary Statement Available

7 - 8 months Advisory Council

8 - 9 months Funding Decisions

9 - 10 months Award Start Date

NIGMS

June 2009

Review Process: NIH Contacts

Who and when? Scientific Review Officer - Prior to summary statement Program Officer - After summary statement Grants Manager - During award activities

How to find them? Information always available in eRA Commons If need help, ask your SRO or PO

NIGMS

June 2009

Review Process: Reviewers

Review Panel Organized by a Scientific Review Officer in the NIGMS

Office of Scientific Review Special Emphasis Panel or Standing Committee Temporary committee members included on standing

committees for particular expertise

Reviewer Characteristics Experience with multiple education levels Involvement with research training programs Educators, researchers, and institutional administrators

NIGMS

June 2009

Enhanced Peer Review: Criteria

NIGMS

June 2009

Enhanced Peer Review: ScoringScale Nine-point whole number scale (1 = exceptional; 9 = poor)

Overall Impact and Criterion Scores All discussed applications receive a two digit

impact/priority score; this is the average of the score given by each eligible review committee member

Scores are given for each of five core review criteria by each of the assigned reviewers

“ND” = not discussed (previously unscored) applications will receive criterion scores but no impact/priority score

NIGMS

June 2009

Enhanced Peer Review: Amendment Limits

For original new applications (never submitted) and competing renewal applications only one resubmission (previously amended or revised application) will be accepted

If funding is not received after two submissions, the program must be substantially re-designed rather than slightly altered in response to previous reviews in order to be submitted as a new application.

There are no more “A2” applications For details, see NIH Guide Notice NOT-OD-09-003

NIGMS

June 2009

Details of Enhanced Peer Review

NIH Guide NOT-OD-09-024, NOT-OD-09-025, NOT-OD-09-003

http://enhancing-peer-review.nih.gov

NIGMS

June 2009

Tips about Format

Page LimitsSupply all requested materials within page limitsDo not use appendices to get around the limits

AppendicesFollow general guidelines (NIH Guide NOT-OD-07-018) and ones specific to the program announcementRelevant material only, such as large tables, survey instruments, publications that are NOT available onlineNO catalogs, lengthy reports, or material that should be in the body of the application

NIGMS

June 2009

Application Preparation Tips

Content Read the program announcement and ensure that your

application contains the necessary elements Successful submission through Grants.gov and eRA

Commons does not mean appropriate responsiveness to the program announcement

Context Present the institutional framework and environment of

your program Be realistic in your program’s goals

NIGMS

June 2009

Application Preparation Tips

Comprehensive Address all of the requirements of the program

announcement• For example:

If you don’t have institutional baseline data, explain how you plan to obtain it

If you haven’t fully formed your evaluation plan, at least acknowledge that you are working on it

Describe how your program “works”• For example:

How are students recruited and selected? By whom? What does the advisory committee do? How often do they meet? How have you used evaluation information in designing/improving

your program?

NIGMS

June 2009

Application Preparation Tips

Clear Don’t bury important information in appendices or

expansive prose Don’t expect reviewers to “read between the lines” to

figure out what you are proposing Present outcomes data in a straightforward manner:

• Don’t exaggerate• Don’t hide data (reviewers will “do the math”)• It is far better to present results as they are and address how the

program aims to improve

NIGMS

June 2009

Application Preparation Tips

Current Make sure faculty biosketches are up-to-date Provide data on current and prior students Use the most recent institutional data

Consistent Data in tables and text should match Data should be consistent across tables Match justification to budget items Refer to the correct program in text and tables

ESSENTIAL “MUSTS” FOR THE BRIDGES PROGRAM

APPLICATIONS

BRIDGES TO THE DOCTORATE (PAR-07-410)

BRIDGES TO THE BACCALAUREATE (PAR-07-411)

Jermelina Tupas, Ph.D.Division of Minority Opportunities in ResearchNational Institutes of General Medical Sciences

National Institutes of Health, USDHHS

June 19, 2009

NIGMS

ESSENTIAL “MUSTS” FOR THE BRIDGES PROGRAM

APPLICATIONS

BRIDGES TO THE DOCTORATE (PAR-07-410)

BRIDGES TO THE BACCALAUREATE (PAR-07-411)

Jermelina Tupas, Ph.D.Division of Minority Opportunities in ResearchNational Institutes of General Medical Sciences

National Institutes of Health, USDHHS

June 19, 2009

NIGMS

9 Essential Items for A Bridges Program

Application

1. Institutional program2. Partnership3. Applicant pool4. Student Development5. Specific Aims6. Budget7. Responsible Conduct of Research8. Evaluation Plan9. Progress Report

NIGMS

1. Institutional Program

baseline data

goals and measurable objectives

activities or interventions

evaluation plan

NIGMS

Create a partnership program that will focus attention and adequate resources on the institution(s) granting associates or master’s degrees and enhance competitiveness of its (their) science graduates and science programs

NIGMS

Challenge to the consortium:

• Significance Are you proposing a program that is

important to the mission of NIGMS and NICHD and the science they support?

What will be the impact of the proposed program on each participating institutions, i.e. the number of students transferring and graduating (with baccalaureate or PhDs) in the biomedical/behavioral sciences?

NIGMS

Institutional Program: Review Qs

• Innovation Is this a duplication of other programs

currently supported at the applicant institution or available within the consortium?

• Environment What are the unique features of the

institutions that will contribute to achieving the proposed goals?

Are the plans to coordinate the planned activities in multiple sites adequate?

Is the institutional commitment to the proposed program appropriate?

NIGMSInstitutional Program:

Review Qs

Tracking must be carried out.

Official letter of agreement to:

a)provide the institutional data on transfer and subsequent graduation of its students in biomedical and behavioral sciences;

b)track Bridges students over a ten-year period as they progress through the pipeline; and

c)provide the data on transfer, and degree completion (bachelor’s or Ph.D.) for Bridges and non-Bridges students at the partner institutions.

NIGMS

Institutional Commitment

2. Partnership• Consortium-no more than 4 baccalaureate

or 3 doctorate institutions• Partners-one must be a 2-year or master’s

degree institution, another must be a bachelor’s or Ph.D. degree institution

• Applicant institution (only one)- must name the PD

• Role of each partner institution- must be well-defined; include partnership/consortium agreement in Appendix

NIGMS

Requirement for the partners(Bridges to the Baccalaureate)

• 2-year institution: offers the associate degree as the only undergraduate degree in the biomedical and behavioral sciences within the participating departments

• Baccalaureate institution: must be a college or university granting the baccalaureate degree in biomedical or behavioral sciences.

NIGMS

NIGMS

Requirement for the partners(Bridges to the Doctorate)

• Master’s institution: MS- terminal degree in biomedical or behavioral sciences as the highest degree; must have a high enrollment of targeted MS students

• Ph.D. institution: must be a research institution; have a significant number of mentors with extramural research support; awards doctoral degrees in biomedical or behavioral sciences.

• MS institutions with RISE programs that include master’s students must justify the need for the Bridges program and provide evidence that there is a large pool of targeted students

• A list of faculty research mentors at the doctoral institution must be provided; include in their biosketches past student training record and extramural support

NIGMS

Requirement for the partners(Bridges to the Doctorate)

Partnership: Review Qs

• ApproachAre the roles of the participating

institutions well developed, well integrated, and appropriate to the aims of the program?

Are the partnership arrangements reasonable and are likely to facilitate the seamless transition of the students?

NIGMS

Partnership: Review Qs

• Investigators Is there an adequate pool of

research mentors at the PhD institution who are extramurally funded?

Do the key personnel selected for program implementation have experience in mentoring students, particularly URMs?

NIGMS

Partnership: Review Qs• Environment Is there appropriate collaboration among

participating departments and institutions?

Do the consortium agreements and letters from each participating institution provide adequate documentation and assurance that each will contribute to the success of the proposed Bridges to the (Baccalaureate or Doctorate) program?

Does the proposed consortium involve appropriate number of institutions

NIGMS

3. Applicant Pool

Targeted students• groups underrepresented in the biomedical &

behavioral sciences; those disproportionately affected by health disparities

• U.S. citizens, nationals, or permanent residents

• matriculated full-time in (Associate or MS) in targeted areas at the (2-yr or MS)

• pool at each (2-yr or MS) institution must be clearly described; include total number of targeted students at 2-yr or MS institutions & total number of targeted Bridges participants

NIGMS

Applicant Pool: Review Qs

• Approach Are the recruitment, retention, and follow-

up activities adequate to ensure a large pool of eligible participants?

• Environment Is there an adequate pool of students from

targeted groups/populations in the participating science department(s) at the (2-year or master’s degree-granting) institutions who are interested in research careers in biomedical/behavioral fields?

NIGMS

• integrated set of student development activities

• rationale for each developmental activity

• targeted students’ needs and requirements

• contribution of each activity to realization of the objectives

• interventions to increase the number of student transfer and completion of B.S. or Ph.D. degree in biomedical/behavioral sciences

• overall impact on the capabilities of the master’s degree institution(s) to provide competitive training to their students, must be measured (i.e., measurement of outcomes)

NIGMS

4. Student Development

• Significance Are the proposed developmental activities likely

to improve the academic preparation of (associates or master’s degree) students and allow them to be admitted to BS or Ph.D. degree programs in biomedical/behavioral sciences?

What will be the impact of the proposed program on the number of students from the targeted groups who transfer (to 4-yr or doctoral institutions) and complete (B.S. or Ph.D.) degrees?

NIGMS

Student Development: Review Qs

• Approach: Does the applicant acknowledge potential

problem areas and consider alternative approaches?

Is there evidence that the program is based on sound research concepts and educational principles?

Does the program provide details and rationale for the activities proposed to enhance the academic preparation of targeted students?

NIGMS

Student Development: Review Qs

• Innovation Does the project challenge existing paradigms

and address the critical barriers that prevent targeted students from pursuing bachelor or doctorate degree programs?

Do the proposed academic development activities employ novel concepts, approaches, or methods to attract, retain, and prepare (associate or master’s degree) students for more challenging academic programs at the (B.S. or Ph.D.) degree-granting institution?

NIGMS

Student Development: Review Qs

• Environment Does the scientific/educational environment in

which the program will be conducted contribute to the probability of success?

Does the proposed research education program benefit from unique features of the scientific environment, subject populations, or employ useful collaborative arrangements?

NIGMS

Student Development:

Review Qs

5. Specific Aims

• must address the overall goals and specific measurable objectives (including anticipated milestones) that the consortium institutions expect to accomplish by the end of the project period

• objectives must be presented as percent improvement over the current baseline and the baseline must be clearly defined

NIGMS

Specific Aims: Review Qs

• Significance How will implementation of the proposed

program advance the objectives of this funding opportunity announcement as well as the mission of the NIGMS and NCMHD?

• Approach: Are the conceptual framework, design,

methods, and analyses adequately developed, well integrated, well reasoned, and appropriate to the aims of the project?

NIGMS

6. Budget(clearly justified single consolidated budget )

• fully itemized; well justified participant cost• requested equipment must be critical to

improving laboratory instruction at the 2-yr or MS institution ($25k per project period)

• cost for faculty teaching/developing new courses: reasonable and commensurate with the scope of the proposed program; no salary for mentoring activities

• cost for evaluator who is an employee of an institution in the consortium must be included in key personnel and listed as person months

• expenses for items generally available for educational programs at the institutions should not be duplicated

NIGMS

7. Progress ReportNIGMS

Information needed:1. summary of the overall progress-

• overall transfer data on Bridges and non-Bridges students from targeted groups

• number of Bridges students who transferred to ( 4-yr or Ph.D.) institutions

• number of Bridges students who completed (B.S. or Ph.D.) degrees in biomedical sciences)

7. Progress ReportInformation needed:2. student development activities implemented;

number of students served; faculty member(s) conducting the activity; progress made in relation to the original goals and objectives

3. list of Bridges students who were supported by the Program during the previous grant period

4. lessons learned from the program evaluation; changes made in the program as a result of the evaluation

5. Bridges activities that are now continuing (or will continue) on institutional funds

NIGMS

Progress Report: Review Qs

Has the research education program successfully achieved its stated objectives, especially in the context of Program’s expectations during the prior project period?

What is the track record of the participating institutions on the transfer and graduation rates of students,

How does this record compare to the Bridges Program goals (50% increase in the overall institutional transfer rate; and 75% transfer and 80% degree completion rates for Bridges students)?

NIGMS

Progress Report: Review Qs

Has the program been innovative in the past and does it continue to demonstrate innovation?

Has the program been adequately evaluated, and is the proposed approach for the next project period responsive to the results of this evaluation?

NIGMS

8. Responsible Conduct of Research

Include:1. the subject matter of the instruction,

the format of the instruction, the degree of program faculty participation, participant attendance, and the frequency of instruction

2. the rationale for the proposed plan of instruction.

NIGMS

9. Evaluation Plan

Remember:

• evaluation must be used as advisory to the PD and the participating institutions

• specific plans, benchmarks, and procedures must be described to capture, analyze, and report outcome measures that would determine the success of the research education program in achieving its objectives

NIGMS

Evaluation Plan: Review Qs

Is the evaluation plan and timeline adequate for assessing the effectiveness (process and outcome) of the program in achieving its goals and objectives?

Does the application identify an individual with appropriate credentials to conduct the proposed evaluation?

NIGMS

BRIDGES TECHNICAL WORKSHOP

GRANTS MANAGEMENT PAR-07-411: http://grants1.nih.gov/grants/guide/pa-files/PAR-07-

411.html PAR-07-410: http://grants1.nih.gov/grants/guide/pa-files/PAR-07-

410.html

Lori Burge Grants Management Officer

National Institute of General Medical Sciences

June 19, 2009

New BRIDGES to the Baccalaureate Program and Doctorate Program

Current/New PAR

Submission style Electronic-Grants.Gov SF424

Project Period Up to 5 years

Receipt Date 2009 January 22 and September 18

Number of Trainees

BAC Min 4 from each partner institution DOC Min 3 from each partner institution

Length of TrainingUp to 2 years

Features (cont’d)Just-in-Time Concepts

Used (Other Support, IRB, IACUC submitted to eRA Commons)

Budget Format Categorical

Number of Institutions

BAC Maximum 4 (includes the applicant institution)DOC Maximum 3

Salary for Students

BAC level comparable to that of other students employed in similar

activities DOC $12 an hour max/level comparable to other students

Total Direct Costs

No limit

Features – BRIDGES to the Doctorate

Support for students:- salary support for up to 20 hours a week during the

academic year while they are fulfilling their course requirements and 40 hours a week during the summer if no courses are being taken. Bridges students in the master’s degree program are also allowed tuition remission as part of a compensation package.

Limited tuition costs of participating faculty (from the master’s institution) to take one advanced course per year at the partner doctoral institution is allowed

Features – BRIDGES to the Doctorate

Students must be matriculated full-time. Salary support:

- for the PD to administer the program is limited to a max of 1.8 person months

- for a program coordinator at the partnering institutions should not exceed 1.8 person months

Equipment costs are limited to a maximum of $25,000 per project period

Research supplies for Bridges students (not to exceed $2,000/student/year)

Features Bridges to the Baccalaureate

Students must be Matriculated full-time. Salary support:

- for the PD to administer the program is limited to a max of 2.4 person months

- for a program coordinator at the partnering institutions should not exceed 1.8 person months

Equipment costs are limited to a maximum of $25,000 per project period

Research supplies for Bridges students (not to exceed $1,000/student/year)

Allowable Costs for all Bridge Grants

Consultant costs- If evaluator is an employee of an

institution within the consortium the cost must be included in the category of key personnel salary and listed in person months

Supplies and equipment Travel for key persons

Allowable Costs for all Bridge Grants

Personnel

- Limited administrative and clerical salary costs associated distinctly with the Bridges program that are not normally provided by the institution when justified

Per new guide notice each institution may participate in more than one Bridges partnership if justified http://grants.nih.gov/grants/guide/notice-files/NOT-GM-07-

116.html

Participants Allowable Costs

Must be itemized in section B (other Personnel) of Research & Related Budget

Students may be supported with salary/wages (not stipends)

STIPENDS

All requested costs– All must be justified as specifically required by

the proposed research education program and must not duplicate items generally available for same purpose at the applicant institution

– Items must be itemized in appropriate sections on 424 forms, section C (Equipment), D (Travel) and F (Other Direct Costs) of Research & Related Budget

– PLEASE JUSTIFY ALL COSTS CLEARLY OR REVIEW MAY NOT ALLOW THEM

Unallowable Costs for Bridges to the Baccalaureate

Costs for students not matriculated full-time at the partner associate degree-granting institution

Salary support for students who have completed the associate degree

Faculty: if mentoring and other activities with students are considered a regular part of an individuals academic duties these would not be an allowable costs from Bridge grant funds

Unallowable Costs for Bridges to the Doctorate

Costs for students not matriculated full-time at the partner master’s degree-granting institution

Salary support for students who have completed their master’s degree and enter a Ph.D program

Faculty: if mentoring and other activities with students are considered a regular part of an individuals academic duties these would not be an allowable costs from Bridge grant funds

Participant Terms & Conditions on Notice of Award

For compensation the following conditions must be met:

The student must belong to the targeted groups/populations, must be a U.S. citizen or non-citizen national or permanent resident and must be matriculated full-time in degree programs in biomedical or behavioral science fields at the partner institution

The student must be performing necessary work relevant to the proposed program

There is an employer-employee relationship between the student and the institution (NO STIPENDS)

The total compensation is reasonable for the work performed

The institution provides compensation for all students under similar circumstances, regardless of the source of support for the activity

Unallowable Costs

Foreign Travel Alterations and renovations Housing, food, books, recruitment

costs incentives to encourage or motivate students (such as laptops, internet subscriptions)

Consortium Agreements

Are a required part of the Bridges Program

Agreements between the eligible parent grantee and the bridging institution define their respective roles in administering the program

Application must include a signed letter from each collaborating institution

Consortium Agreements Language

Each letter should include the following statement:

“THE APPROPRIATE PROGRAMMATIC AND ADMINISTRATIVE PERSONNEL OF EACH INSTITUTION INVOLVED IN THIS GRANT APPLICATION ARE AWARE OF THE NIH CONSORTIUM GRANT POLICY AND ARE PREPARED TO ESTABLISH THE NECESSARY INTER-INSTITUTIONAL AGREEMENT(S) CONSISTENT WITH THAT POLICY.”

EXPANDED AUTHORITIES

Beginning this fiscal year all Bridge grants are under Expanded Authorities

http://grants1.nih.gov/grants/policy/nihgps_2003/NIHGPS_Part7.htm#_Toc54600128

Expanded AuthoritiesWhat does it mean?

Carryover of unobligated balances from one budget period to the next is allowed

No cost extension of final budget period can be submitted automatically

Financial Status Reports (FSR) are no longer due after every budget year. FSRs are due 90 days after the final budget year of the project period

No Cost Extensions

1st No Cost Extension (NCE) may be requested through the eRA Commons.

Only students already participating in the current Bridges program may continue in a NCE

No new students may be added to the Bridges program during a NCE

Submit Annual Progress Reports by eSNAP

Bridge progress reports are now eligible for Streamlined Non-Competing Award Procedures (SNAP) and may file their annual Progress Reports electronically using the eSNAP feature of the eRA Commons

SNAP Questions

# 3 SNAP Question

If you answer YES to question # 3 “unobligated balance will be greater

than 25% of the current year’s total budget”

- Provide a detailed explanation of why there is a large unobligated balance and your specific plans to spend these funds.

Restrictions to Expanded Authorities for Bridge Grants

The Bridge grantees may not appoint more than the approved number of Bridges Participants unless approval is first obtained from NIGMS.

Reminder

Grantees are reminded that they must continue to exercise proper stewardship over Federal funds and that costs charged to awards are allowable, allocable, reasonable, necessary, and consistently applied regardless of the source of funds.

Personnel on 424 Forms For Bridge grants all

students must be listed under personnel in block B

DO NOT add students in block E under Participant/Trainee Support Costs

Add Students to B. Other Personnel

Inclusion Enrollment Report If you mark YES to Human Subjects on the

BRIDGES application then the 424 format will expect the applicant to complete and provide the Target Inclusion Enrollment Report.

For the Study Title add Bridges to the Baccalaureate or Doctorate and add zeros in all the total fields.

If the application is marked NO to Human Subject involvement then no action is needed on the Target Inclusion Enrollment Report

For More Information Contact Your Grants Management Specialist

Robert Altieri altierir@nigms.nih.gov

Irina Alva-Weinstein alvair@nigms.nih.gov

Michael Mace macem@nigms.nih.gov

Justin Rosenzweig rosenzwj@nigms.nih.gov

Lori Burge burgel@nigms.nih.gov

6/12/09

WORKSHOP SUMMARY& THE MORE Website

Jermelina L.G. Tupas, Ph.D.

Division of Minority Opportunities in ResearchNational Institute of General Medical Sciences

National Institutes of Health, USDHHS

June 19, 2009

NIGMS

6/12/09

NIGMSWORKSHOP SUMMARY

Key features of the new Program Announcements

Bridges to the Doctorate:(PAR-07-410) http://grants1.nih.gov/grants/guide/pa-files/PAR-07-410.html

Bridges to the Baccalaureate: (PAR-07-411)http://grants1.nih.gov/grants/guide/pa-files/PAR-07-411.html

- Read the correct Program Announcement - Be responsive to the Program Announcement

Case Studies: Strengths & Weaknesses

6/12/09

NIGMSWORKSHOP SUMMARY

Program Evaluation: How much did we do?How well did we do it?Are the community colleges; baccalaureate

degree-granting institution; and students better off ?

Are the Master’s granting institution; Ph.D. granting institution; and students better

off ?

6/12/09July 25, 2008

NIGMS

WORKSHOP SUMMARY

Review Issues Essential Items for Bridges Applications http://www.nigms.nih.gov/Minority/Bridges/EssentialsBridgesApps.htm

also see FAQs: http://www.nigms.nih.gov/Minority/Bridges/FAQs.htm

Grants Management: - -allowable/unallowable costs-No caps- “Justify, justify, justify”

6/12/09

SUMMARY

real partnership

measurable program objectives

measurable program outcomes, impact, & implementation for improvement & advice

transfer and graduation focusing on institutional impact

6/12/09July 25, 2008

NIGMS

The MORE website:http://www.nigms.nih.gov/Minority/

Other useful website:

1. Frequently asked questions:http://www.nigms.nih.gov/Minority/Bridges/FAQs.htm2. Participating institutions:http://www.nigms.nih.gov/Minority/Bridges/PartInstBacc.htmhttp://www.nigms.nih.gov/Minority/Bridges/PartInstDoct.htm3. Program announcements:http://grants1.nih.gov/grants/guide/pa-files/PAR-07-411.html (Bac)http://grants1.nih.gov/grants/guide/pa-files/PAR-07-410.html (Doc)4. MORE Staff Contactshttp://www.nigms.nih.gov/Minority/more_staffcontacts.htm