Grant Writing Workshop for Exploration Projects

Allen RubyNational Center for Education Research

Getting Started

Request for Applicationshttp://ies.ed.gov/funding/11rfas.aspFuture RFAs: sign up for newsflash

http://ies.ed.gov/newsflash/

IES Grants.gov Application Submission Guide http://ies.ed.gov/funding/11rfas.asp

Application Packagehttp://www.grants.gov/

Education Research Grant Program (84.305A)

Special Education Research Grant Program (84.324A)

Education Research Grant Program (84.305A)• Reading and Writing• Mathematics and Science Education• Cognition and Student Learning• Teacher Quality (Reading & Writing; Math & Science)• Social and Behavioral Context for Academic Learning• Education Leadership• Education Policy, Finance, and Systems• Postsecondary Education• English Learners• Early Learning Programs and Policies• Education Technology• Adult Education• Organization and Management of Schools and Districts• Analysis of Longitudinal Data to Support State & Local

Education Reform

Special Education Research Program (84.324A)

• Early Intervention and Early Childhood Special Education• Reading, Writing, and Language Development• Mathematics and Science Education• Social and Behavioral Outcomes to Support Learning• Transition Outcomes for Special Education Secondary

Students• Cognition and Student Learning in Special Education• Professional Development for Teachers and Related

Service Providers• Special Education Policy, Finance, and Systems• Autism Spectrum Disorders

Solution Driven Research

Research intended to contribute to the solution of practical education problems

– Exploration – Development – Efficacy Evaluation– Scale-up Evaluation– Measurement

Before Development Work Begins...

Need to understand the problem:• Examine the underlying processes that

affect education outcomes• Look for malleable factors that might be

targets for interventions• Identify what distinguishes between

effective and less effective practices

Before Evaluation Begins...

Identify education interventions or components of them that appear promising due to their association with improved student outcomes before committing resources to evaluation

In a Nut Shell...Development work and some evaluations begin with Exploration Projects that examine underlying processes for the purpose of informing the development of new interventions, modifying existing interventions, or identifying promising interventions for evaluation

Exploration Goal

• Explore the association of education outcomes with malleable factors

• Examine factors or conditions that mediate or moderate the relationship between malleable factors and education outcomes

• Small primary data studies, secondary analyses, and meta-analyses

Malleable Factors• Malleable factors must be under the control

of the education system – something that can be changed by the system

• Examples– Student characteristics: behavior, skills– Teacher characteristics: practices, credentials– School characteristics: size, climate, organization– Education interventions: practices, curricula,

instructional approaches, programs, and policies

Exploration Projects Should• Identify underlying processes that enhance or

inhibit learning. These may contribute to the development or modification of interventions

• Identify education interventions that may deserve rigorous evaluations

• Identify factors that mediate or moderate the underlying processes or the interventions

• Generate hypothesis concerning causal relations between factors and outcomes

• Contribute to theories of action

Exploration Projects Should NOT

• Test the efficacy of education interventions

• Examine non-malleable factors

• Examine malleable factors outside the control of the school system

Research Narrative includes 4 sections

• Significance• Methodological Requirements (Research

Plan)• Personnel• Resources

Significance Section• Describes the overall project

– Research question to be answered– Variables to be examined

• Provides a compelling rationale for the project– Theoretical justification

• Logic Models, Change Models

– Empirical justification• Related work supporting the project

– Practical justification • importance of the variables (malleable factors,

mediators, moderators, outcomes) examined

Significance Section

• How work will lead to useful next step– Development or modification of

interventions to address the identified malleable factors or underlying process to improve student outcomes

– Identification of interventions for more rigorous evaluation

• Overall importance

Significance – 2 Problem Areas

• Unclear theory of change– Why should the malleable factor be related

to improved outcomes– A well laid out theory of change makes clear

what is expected to happen and in what order

– Easy for reviewers to understand research plan – why measure certain outcomes

– Graphic can be helpful – e.g. a logic or change model

Significance – 2 Problem Areas• Unclear Description of Intervention

– What the intervention is• Many components and may be applied at

different times – how fit together – Graphic may help

– Intervention not shown to be strong enough to expect an impact• Informational• Ensure fidelity

– Overly focused on actions not content• Ex.: 20 hours of PD held over 10 weeks but no

detail on what is to be covered in the sessions

Research Plan• Describe the work you intend to do

– How you will answer your research question

• Make certain Research Plan is aligned to Significance section– All research questions should have

justification in Significance.

• Step-by-step process– Timeline to show when everything will be

done

Methods will vary by research question and design

• Secondary data analysis of existing datasets

• Original data collection with appropriate statistical analyses

• Secondary data analysis complemented by primary data collection

• Meta-analyses designed to determine moderators or moderators of effects

All Research Plans Should Include• Clear, concise hypotheses or research

questions• Well-specified relations between

hypotheses, measures, and independent & dependent variables

• Clear description of data sources• Detailed descriptions of data analysis

procedures

Secondary Data• Database(s)

– Sample characteristics– Variables to be used– Missing data – how it will be handled– Access to and permission to use the data– Links if multiple datasets used

• Measure of malleable factor• Primary outcome measures to be used,

including their reliability and validity• Mediators & moderators

Primary Data Collection

• Describe sample characteristics• Describe measures to be collected,

including reliability & validity• Procedures proposed for data collection• Describe links to secondary data if

relevant

If observational data are collected

• How will inter-observer reliability be maintained

• How will data be coded• How will data be quantified to support

prediction of relation between what was observed and outcomes of interest

Meta-analysis

• Criteria for including or excluding studies, and rationale

• Search procedures used • Coding scheme & procedures for

extracting data from studies• Procedures for ensuring reliability of the

coding

Meta-analysis• Demonstrate that sufficient numbers of studies

are available to support meta-analysis• Demonstrate that relevant information is

reported frequently enough that a database can be constructed

• Clearly describe effect size statistics, associated weighting functions, procedures for handling outliers, and any other adjustments

Data Analysis

• Detailed description of data analysis– Describe method to analyze data– Show model – Link method to all research questions– Include analyses for mediators and

moderators– Sensitivity tests to address assumptions

Personnel

• Show expertise in every aspect of project:– Relevant content domain– Appropriate methodological expertise for

work to be done in project– Working with schools, or other education

agencies (if appropriate)• Analysis of Longitudinal Data to Support State

and Local Education Reform requires SEA/LEA personnel

For Each Key Personnel Describe• Link each person and their expertise to

their role in the project• Qualifications• Roles• Responsibilities• Percent of time devoted to the projectShow every aspect of project has person

with expertise and time to do it

Personnel Strategies for PI• Senior Researcher

– Show adequate time to be PI– Make credentials clear: not all reviewers may know

• Junior Researcher as PI/PD– Show you have adequate expertise not only to do

work but to manage project• Continuation of graduate research• Management skills as graduate student

– Reviewers more comfortable if you have senior person(s) on project to turn to for advise• Co-PI, Co-I, contractors, advisory board• Have them on for enough time to be taken seriously

Resources• Show the institutions involved have the

capacity to support the work– Don’t use university boilerplate

• Show that all organizations involved understand and agree to their roles– What will each institution, including schools,

contribute to the project– Show strong commitment of schools and districts– Have alternatives in case of attrition

• Appendix A should back this up with – Detailed Letters of Support from research

institutions, States, districts, schools

Resources• Data issues

– Document permission to use and access to confidential data

– Show familiarity with data – show that it can be used to do the proposed work

– If merging datasets, show that it can be done

Award Information• Secondary data analysis or meta-analysis:

– $100,000 to $350,000 per year total cost (direct + indirect)

– Up to 2 years

• Primary data collection analysis (with or without secondary analysis):– $100,000 to $400,000 per year total cost (direct

+ indirect)– Up to 4 years

Write from Reviewers’ Perspective

• Write clearly and concisely• Address the points described in RFA• Organize information in logical

sequence• Label sections and number pages• Make it easy for reviewers to find and

understand the information

Submitting a Proposal

• All proposals must be submitted electronically to:– http://www.grants.gov

• By 4:30:00 pm Washington DC time on the date listed in the RFA for the competition to which you are submitting

Key Dates

Application Deadline

Letter of Intent

iesreview.ed.gov

Application Package

www.grants.gov

Start Dates

9/16/104:30:00 PM

7/19/10 7/19/10 7/1/11to

9/1/11

Final Application Submission• On-line forms are complete• PDFs of proposal contents have been

uploaded• Use “Check Package for Errors” button on

grant application package• Authorized representative has completed the

final step of the electronic process.• You have received email acknowledging

receipt of your application.

Peer Review

• Proposal is reviewed for compliance.• Proposal reviewed for responsiveness• Compliant and responsive proposals are

assigned to a review panel.• Two or three panel members conduct

primary review of each application.• At panel meeting, the most competitive

applications are reviewed by full panel.

Peer Review Process Information

http://ies.ed.gov/director/sro/peer_review/index.asp

Notification

• All applicants will receive email notification of the status of their application.

• All applicants receive copies of reviewer comments.

• If you are not granted an award the first time, plan on resubmitting, and talk to your program officer.

Final Reminders

• Start early• Read the Request for Applications• Talk with the program officer• Start the online submission process

early

ies.ed.gov

Allen Rubyallen.ruby@ed.gov