+ All Categories
Home > Documents > IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Date post: 05-Jan-2016
Category:
Upload: scott-leonard-wheeler
View: 216 times
Download: 2 times
Share this document with a friend
Popular Tags:
102
IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011
Transcript
Page 1: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

IES Grant Writing Workshop

Institute of Education Sciences

U.S. Department of Education

April 2011

Page 2: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Agenda• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and

Special Education Research Grants (84.324A)– Grant Topics– Grant Research Goals

Page 3: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Agenda (continued)• Four Sections of the Research Narrative

– Significance– Research Plan– Personnel– Resources

• Other IES Grant Programs• Application Submission and Review

Page 4: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

A bit about IES

Page 5: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

IES Structure

Office of the Director

National Board for Education

Sciences

National Center for Education Research

National Center for Education Evaluation

National Center for Education Statistics

National Center for Special Ed Research

Office of Standards &

Review

Page 6: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Overall Research Objectives

• Develop or identify education interventions (practices, programs, policies, and approaches) that enhance academic achievement and that can be widely deployed

• Identify what does not work and thereby encourage innovation and further research

• Understand the processes that underlie the effectiveness of education interventions and the variation in their effectiveness

Page 7: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Final Outcomes of Interest are for StudentsPreschool• School readiness• Developmental outcomes for infants and toddlers with

disabilities

Kindergarten through Grade 12• Academic outcomes in reading, writing, math, and science• Behaviors, interactions, and social skills that support learning in

school and successful transitions to post-school opportunities• High school graduation• Functional outcomes that improve educational results,

transitions to employment, independent living, and postsecondary education for students with disabilities

Page 8: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Final Outcomes of Interest (continued)

Postsecondary• Access, persistence, completion• Achievement in gateway math and science courses• Achievement in introductory composition courses

Adult Education• Reading, writing, and math for basic and secondary

education and English Language Learners

Page 9: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Agenda• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and

Special Education Research Grants (84.324A)

• Four Sections of the Research Narrative• Other IES Grant Programs• Application Submission and Review

Page 10: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Research and Research Training Grant Programs• Education Research Grant Programs (84.305A)• Special Education Research Grant Programs (84.324A)

• Postdoctoral Research Training Grant Programs (84.305B and 84.324B)

• National Research and Development Centers (84.305C and 84.324C)

• Statistical and Research Methodology in Education (84.305D)• Evaluation of State and Local Education Programs and

Policies (84.305E)

Page 11: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Key Dates

Application Deadline

Letter of Intentiesreview.ed.gov

Application Package

www.grants.gov

Start Dates

6/23/11 4/21/11 4/21/11 3/1/12 to

9/1/12

9/22/11 7/21/11 7/21/11 7/1/12to

9/1/12

Page 12: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Information for Applying

Requests for Applications

Letter of Intent

IES Grants.gov Application Submission Guide

Application Package

Page 13: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Requests for Applications (RFA)

• A separate RFA for each grant program

• Describes the requirements for an application

• Requests for Applications are available on: http://ies.ed.gov/funding

• To be informed about the release of future RFAs, sign up for the IES Newsflash: http://ies.ed.gov/newsflash/

Page 14: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Letter of Intent (LOI)• A short description of your intended application

– PI, institution, collaborators– Budget – rough estimate– Up to 1 page abstract describing the work

• Not used in the review process – superseded by your application

• Submitted on http://iesreview.ed.gov

Page 15: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

IES Grants.gov Application Submission Guide

• Instructions for completing and submitting the application package

• Available on http://ies.ed.gov/funding about May 1, 2011

Page 16: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Application Packages for FY 2012

• Available at www.grants.gov– Help: [email protected] or 1-800-518-4726

• For the June 23, 2011 deadline, packages will be available starting April 21, 2011

• For the September 22, 2011 deadline, packages will be available starting July 21, 2011

• Packages are specific for grant program and deadline

Page 17: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Eligibility to Apply

• Applicants that have the ability and capacity to conduct scientifically valid research

• Include, but are not limited to, nonprofit and for-profit organizations and public and private agencies and institutions, such as colleges, universities, and school districts

Page 18: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Identify the Appropriate Grant Program

• Read the appropriate Request for Applications– http://ies.ed.gov/funding

• Look at the abstracts of projects funded– http://ies.ed.gov/ncer/projects/ – http://ies.ed.gov/ncser/projects/

• Talk to the appropriate Program Officer

Page 19: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Agenda• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and

Special Education Research Grants (84.324A)

• Four Sections of the Research Narrative• Other IES Grant Programs• Application Submission and Review

Page 20: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Grant Topics

• All applications to 84.305A and 84.324A must be directed to a specific topic– Note: on SF 424 Form, Item 4b (Agency Identifier

Number)– Note: at top of Abstract and Research Narrative

Page 21: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Education Research Topics (84.305A)

• Reading and Writing• Mathematics and Science Education• Cognition and Student Learning• Social and Behavioral Context for Academic Learning• Education Technology• Effective Teachers and Effective Teaching

• Improving Education Systems: Policies, Organization, Management, and Leadership

• Postsecondary and Adult Education• Early Learning Programs and Policies• English Learners

Page 22: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Identify Education Research Topic

• Purpose: This study will examine the association between aspects of preschool quality and child health, behavioral and cognitive outcomes in community-based and school-based early care and education programs.

• Purpose: The purpose of this research is to test several possible ways to influence participation in college savings plans and subsequent savings behavior.

• Purpose: This study will provide a detailed examination of factors that predict gender differences in elementary school mathematics performance.

• Purpose: This project is designed around findings from a local needs assessment of teachers, which found: a) a need for more support for laboratory work; b) a need for greater access to subject matter experts; and c) a strong desire to plan together.

Page 23: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Special Education Research Topics (84.324A)• Early Intervention and Early Learning in Special Education• Reading, Writing, and Language Development• Mathematics and Science Education• Social and Behavioral Outcomes to Support Learning• Transition Outcomes for Special Education Secondary Students• Cognition and Student Learning in Special Education• Professional Development for Teachers and Related Service

Providers• Special Education Policy, Finance, and Systems• Autism Spectrum Disorders• Technology for Special Education• Families with Children with Disabilities

Page 24: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Identify Special Education Research Topic• Purpose: This research group will develop and preliminarily

evaluate SELF: Social-Emotional Learning Foundations, to promote emotional and behavioral self-regulation for children in Kindergarten and first grade who are at risk for emotional and behavioral disorders.

• Purpose: This research will providing guidance for speech-language pathologists by examining how dosage, techniques, and context are associated with language outcomes.

• Purpose: Federal regulations issued in April 2007 allow states to develop "modified academic achievement standards" that are challenging for eligible students but are less difficult than grade-level achievement standards. This project will develop and validate an assessment based on modified academic achievement standards.

Page 25: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Issues Specific to Topics• All require student outcomes (Effective Teachers, Systems)• Grade range varies by topic

– Most topics are for K-12 students only – Early Learning: for pre-K (ages 3-5) and their teachers

• Exception if project is to follow pre-K students into later grades applicant can choose most appropriate topic

– Education Technology: pre-K to adult except science (grade 12)– Cognition: pre-K to adult (voc ed, adult ed, remedial post-sec)– Postsecondary and Adult: for older students

• Postsecondary includes high school programs to get students into postsecondary

• Postsecondary limited to sub-baccalaureate and baccalaureate

• Adult: adult basic education, adult secondary education, and English Learner

Page 26: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Topics (continued)

• Improving Education Systems: Policies, Organization, Management, and Leadership

– Anything designed to improve the overall functioning of a school, district, state, or national education system

• Programs

• Finance

• Leadership

• Organization and Management

– Combined into 1 topic because interventions may include all of these approaches

Page 27: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Topics (continued)• Topics can Overlap

– Effective Teachers & Effective Teaching• Read/Write & Math/Science: determine if focus on Prof

Dev or on curriculum/instructional approach• Cognition: applying cognitive science to teacher practice• Early Learning topic is for pre-K teachers• English Learners: if for EL teachers can be either• Improving Ed Systems: teacher certification, recruitment,

and retention can go to either topic

– Ed Technology with all programs• Is focus/team on tech development or on substance

– English Learners with Read/Write & Math/Science• Is EL the primary focus or a secondary focus

– Improving Ed Systems with all programs • Except Early Learning Programs and Policies and

Postsecondary and Adult Education

Page 28: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Choosing among Overlapping Topics

• What literature are you citing?• To which topic is your area of expertise best aligned?• If your focus is on a specific population of

students/teachers, go to that program/topic:– Is your focus on a specific type of student/teacher

(e.g., special education or English Learners), or are you studying them as subgroup of your sample?

Page 29: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Topics (continued)• Pre-service programs

– Only exploratory research can be done on teacher pre-service programs – no development of pre-service programs, evaluation of them, or measures-development for them

– Can develop or evaluate pre-service components with in-service teachers

– Support for leadership pre-service programs if the programs last 24 months or less

Page 30: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Issues Specific to Special Education Topics• Children with disabilities or at risk for developing

disabilities: – At Risk based on individual assessment not population

characteristics (e.g. low SES)– Be specific about which disabilities you are addressing

• Specify the inclusion/screening criteria

• Applicants to the following topics must address students with a disability only and not students at risk for a disability: – Transition Outcomes – Autism Spectrum Disorders – Families of Children with Disabilities

Page 31: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Special Education Topics (continued)• Grade Coverage

– Early Intervention: infants to age 5– Cognition: infants to grade 12– Technology: infants to grade 12– Autism: pre-K – 12– Transition: secondary students only (middle/high school)– Others: K -12

• Overlaps– Autism and Other Topics

• Comprehensive interventions with multiple outcomes to Autism• One outcome goes to that topic (e.g. Math/Science)

– Early Intervention and Other topics if follow pre-K students to later grades

Page 32: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Decide Which Topic Your Research Idea Would Fall Under

• Think about your research question(s)• Decide which topic it best fits under• If not sure

– Check RFA– Discuss with program officer

Page 33: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Grant Research Goals

• All applications to 84.305A and 84.324A must be directed to a specific research goal (1 of 5)– Note: on SF 424 Form, Item 4b– Note: at top of Abstract and Research Narrative

• The goal describes the type of research to be done

• So every application is directed to a specific topic/goal combination

Page 34: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

The 5 Research Goals

• Exploration• Development and Innovation• Efficacy and Replication • Scale-up Evaluation • Measurement

Page 35: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Exploration

• Exploration of the association between education outcomes and malleable factors (non-causal) – A factor that can be changed by the education system

be it a student, teacher, or school characteristic, or an education program or policy

– Underlying processes that enhance or inhibit learning– Aspects of a school, district, or community associated

with beneficial education outcomes– Education interventions associated with beneficial

education outcomes

Page 36: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Exploration• Exploration of the factors that mediate or

moderate the relationship between malleable factors and student outcomes

• Small primary data studies, secondary analyses, and meta-analyses– Secondary data:

• Typical $100,000 to $300,000 per year (direct and indirect)

• Maximum 2 years and $700,000

– Include primary data: • Typical: $100,000 to $400,000 per year

• Maximum 4 years and $1,600,000

Page 37: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Would these Research Questions fit under the Exploration Goal?

• Do middle school girls score higher on English achievement tests than boys?

• Is hands-on science teaching associated with better grades for boys?

• Is increasing foster care payments linked to better academic outcomes of foster children?

• Does Bluebird Reading Curriculum cause higher student achievement on reading tests?

• Do students with certain types of disabilities have shorter attention spans?

Page 38: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Development and Innovation• Develop new interventions

– E.g., instructional practices, curricula, teacher professional development, principal practices, policies

– Iterative development process– Define “operating as intended” and how to measure

• Feasibility of implementation – Implement the intervention in an authentic education delivery

setting– Small sample of users– Demonstrate “operating as intended”

Page 39: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Development and Innovation

• Collect pilot data on promise of intervention to achieve intended outcomes– Does not need to be causal study– Stronger with comparison group– Can be no more than 30% of grant budget– Obtain evidence to support grant for evaluation

• Typical award: $150,000 to $400,000 per year– Maximum of 3 years and $1,500,000

Page 40: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Would these fit under Development and Innovation?• Develop 9th grade biotechnology course over

summer, implement from September to December, and measure student gains in knowledge.

• Give half the student iPads, monitor how they’re used, and compare test scores at end of year.

• New writing program – Develop with 10 teachers over 1 year – try components

out in class and revise accordingly– Feasibility test with the 10 teachers in Year 2– Compare writing scores of students of the 10 teachers to

scores of students from 10 other teachers in Year 3

Page 41: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Efficacy and Replication• Causal test of whether or not a fully developed

intervention has a beneficial impact on student outcomes relative to a counterfactual in an authentic educational setting– Interventions already in wide use– Interventions not in wide use

• Takes place under “ideal” conditions– Homogenous sample of students/schools– Extra assistance to support high implementation

Page 42: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Efficacy and Replication• Detailed description of intervention

– Theory of Action– Empirical evidence– Practical importance

• Random assignment to intervention and comparison conditions preferred– Strong quasi-experiment designs can be proposed when

experiment not feasible– Single-subject methods / single-case designs– Check What Works Clearinghouse evidence standards

(RCTs, RDD, Single Case, Attrition)

Page 43: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Efficacy and Replication• Address power of design to identify impacts• Address fidelity of implementation of treatment

and comparison groups• Address important moderators• Detail analysis plan• Avoid apparent conflicts of interest for evaluation

team• Typical award: $250,000 to $650,000 per year

– Maximum of 4 years and $3,500,000

Page 44: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Efficacy and Replication Follow-up Study

• Follow students who received intervention into later grades, or

• Follow teachers (principals or schools) who received intervention after the project ends to see if sustained effect on practice and on student outcomes

• Typical award: 150,000 to $300,000 per year– Maximum of 3 years and $1,200,000

Page 45: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Would these fit under the Efficacy and Replication Goal?

• Randomly assign iPads to treatment and control groups

• Intervention will provide 3 weeks of teacher training, ongoing coaching, plus classroom materials

• Match 30 schools who adopted an anti-bullying program to 30 schools who did not based on % minorities and FSL and average test scores

• 4 districts agree to take part in a study that will randomly assign a math curriculum to 2 of them

Page 46: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Scale-up Evaluation• Independent causal test of whether or not a

fully developed efficacious intervention has a beneficial impact on student outcomes relative to a counterfactual in an authentic educational setting under routine implementation– Independent: evaluation team has no financial

interest in intervention– Efficacious: evidence of intervention’s efficacy– Routine implementation: as implemented by

practitioners with expected level of support if adopted by a school or district

Page 47: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Scale-up Evaluations• Limit of 25% of budget for implementation of the

intervention• Other requirements similar to Efficacy and

Replication• Replications of Scale-up Evaluations allowed

with different populations (students, schools)• Typical award: $350,000 to $900,000 per year

– Maximum of 5 years and $5,000,000

Page 48: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Scale-up Evaluation Follow-up Study

• Follow students who received intervention into later grades

• Typical award: $250,000 to $400,000 per year– Maximum of 3 years and $1,500,000

Page 49: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Would these fit under Scale-up Evaluation?• Test new in-service math teacher training program

developed under a Development and Innovation grant in 60 randomly assigned classrooms

• A district wants to compare 2 Algebra 1 curricula, and the companies agree to provide them at cost along with teacher coaching

• A charter management company that has 2 small efficacy studies receives funds from a millionaire to take over 40 schools. 80 schools apply, and the company will randomly select half if it receives IES

funds to do an evaluation

Page 50: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Measurement• Develop and validate assessments or other

measurement tools– Typically to be used by practitioners: screening,

progress monitoring, and outcome assessment– Some cases for use by researchers– Validation of non-student measures involves

student outcomes (e.g., Effective Teachers)– Program specific, e.g., cost-accounting under

Improving Education Systems

Page 51: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Measurement

• Not for evaluating an assessment used as an intervention

• The measure is the primary product – Not creating a measure as part of a larger study

• Typical awards $150,000 to $300,000 per year– Maximum of 4 years and $1,600,000

Page 52: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Would these fit under Measurement?• Developing a formative chemistry

assessment to help students learn how to balance formulas

• Developing a measure of teacher instruction in fractions and validating it against teacher logs and principal observations

• Develop a measure of student attention and validate it against student grades as part of a project to evaluate an intervention to increase student time on task

Page 53: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

The Goals Build on One Another• Exploration should lead to:

– Development or modification of an intervention– Efficacy evaluation of an intervention

• Development and Innovation should lead to an Efficacy evaluation if found feasible and pilot data is supportive

• Efficacy and Replication should lead to a Scale-up evaluation if impact found

• Measurement should feed into the other goals

Page 54: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Decide Which Goal Your Research Idea Would Fall Under

• Think about your research question(s)• Decide which goal it best fits under• If not sure:

– Check RFA

– Discuss with program officer

– If your idea straddles several goals, consider breaking it into smaller pieces

– Choose goal with best fit, not the one that offers greatest funding

Page 55: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Expected Products Under the Goals• Exploration

– Identify a malleable factor associated (or not) with student outcomes to support a future Development project

– Identify a mediators and/or moderators of the relationship between a malleable factor and student outcomes to support a future Development project

– Identify initial evidence of the association of a program or policy with student outcomes to support a future Efficacy Project

• Development and Innovation: an intervention ready to implement and evaluate

Page 56: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Expected Products• Efficacy & Replication: a methodological sound

evaluation of an intervention• Scale-up Evaluation: a methodologically sound

independent evaluation of an intervention carried out under routine conditions

• Measurement: a validated instrument ready for use

Page 57: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Dissemination Expected for all Goals• Publications in peer-reviewed journals

• Quick release of findings: working papers, presentations and posters, seminars

• Products others can use: software, manuals, instruments, monographs

• Teaching others to use findings/products: short courses, long courses, on-line tutorials

• Long-term collaborations with practitioners

Page 58: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Agenda• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and

Special Education Research Grants (84.324A)

• Four Sections of the Research Narrative• Other IES Grant Programs• Application Submission and Review

Page 59: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

The Application’s Research Narrative

• Key part of your application• 4 Sections

– Significance

– Research Plan

– Personnel

– Resources

• Each section scored and an overall score given• Requirements vary by program & goal• 25 pages, single spaced

Page 60: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Significance• Describes the overall project

– Your research question to be answered; intervention to be developed or evaluated, or measure to be developed and/or validated

• Provides a compelling rationale for the project– Theoretical justification

• Logic Models, Change Models

– Empirical justification– Practical justification

Page 61: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Significance• Do not assume reviewers know significance of

your work

• Do not quote back RFA on general importance of a topic, – e.g., RFA paragraph on lack of reading proficiency

of 8th and 12th graders based on NAEP data

• Do quote back RFA if a specific topic is highlighted and your work will address that topic

Page 62: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Significance: Exploration Goal• Describe the malleable factors, moderators, and

mediators to be examined• Justify their importance

– Theoretical rationale– Empirical rationale– Practical importance

• How work will lead to useful next step– Development or modification of interventions to address the

identified malleable factors or underlying process to improve student outcomes

– Identification of interventions for more rigorous evaluation

• Overall importance

Page 63: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Significance: Development Goal• Context for proposed intervention

– Why needed: what problem exits– What exists now (may be many alternatives already)

• Detailed description of intervention to be developed– Clearly identify components already developed, partially

developed, and to be developed (no jargon)– Don’t overextend (# grades, full vs. part year)

• Theory of change (theoretical support)• Empirical support• Practical importance:

– Meaningful impact, feasibility, affordability

• Answer the question: Why will this intervention produce better student outcomes than current practice?

• Overall importance

Page 64: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Significance: Efficacy & Replication• Detailed description of intervention

– Show fully developed, implementation process, and ready to be evaluated

• Justification for evaluating the intervention– Importance of practical problem it is to address– If in wide use, show it has not been rigorously evaluated– If not in wide use, show evidence of feasibility and promise

to address the practical problem• Theory of change: why lead to expected outcomes

– Theoretically and empirical rationale– Direct impact on student outcomes or through mediators

• Justify that it could lead to better outcomes than current practice

• Overall importance

Page 65: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Significance: Scale-Up Evaluation• Detailed description of intervention• Justification for evaluating the intervention

– Evidence of meaningful impacts (Efficacy study)

• Theory of change• Justify that it could lead to better outcomes than

current practice• Implementation under normal conditions• Independent evaluation• Evidence that implementation can reach high enough

fidelity to have meaningful impacts• Overall importance

Page 66: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Significance: Measurement

• Description of assessment and how it will be used

• Theoretical basis for constructs to be measured

• Empirical evidence for constructs• Practical need for the assessment• Feasibility of use• Overall importance

Page 67: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Significance – 2 Key Problem Areas• Unclear Description of Malleable Factor or

Intervention– May have many components and these may be applied

at different times– graphic may help– Unclear how to be implemented to ensure fidelity– Unclear why strong enough to expect an impact– Overly focused on actions not content

• Example: provide 10 professional development sessions of 4 hours apiece - no detail on what occurs in sessions

Page 68: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Significance – 2 Key Problem Areas• Lack of a Theory of Change

– Why a malleable factor should be related to a student outcome

– Why an intervention should improve outcomes versus current practice

– Why an assessment/instrument should measure a specific construct

– A well laid out theory of change makes clear what is expected to happen and in what order

– Easy for reviewers to understand research plan – why measure certain outcomes

– Graphic can be helpful – e.g. a logic model

Page 69: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Four Sections of the Research Narrative

• Significance• Research Plan• Personnel• Resources

Page 70: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Research Plan• Describe the work you intend to do

– How you will answer your research question, develop your intervention, evaluate the intervention, or develop and/or validate your assessment

• Make certain Research Plan is aligned to Significance section– All research questions should have justification in

Significance.

• Step-by-step process– Timeline to show when everything will be done

Page 71: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Research Plans Differ by Research Goal• However all should describe:

– Setting

– Population and sample• Sampling plan: inclusion and exclusion criteria• Size (power issue) and attrition• External validity

– Measures• Outcomes: proximal and distal; answer research questions• Other measures: fidelity, feasibility, operating as intended, feedback• Quantitative and qualitative• Reliability and validity• Relevance: sensitivity vs. broad interest• Multiple comparisons issue

Page 72: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

All Research Plans Should Include (cont.)

– Research Design (more detail on following slides)– Analysis

• Describe how it answers research questions

• Show your model: show different types of models used

• Address clustering

• Describe how missing data will be handled

• Check for equivalency at start of study and attrition bias throughout

• Describe sensitivity tests of assumptions

• Describe analysis of qualitative data and links to quantitative analysis

Page 73: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Research Design• Start off with you research questions

• The research design should answer your questions– Do not have the design section written independently by

a methodologist– If sections are written by different people have everyone

read through the whole application

• Issues common to designs across goals– Attrition and missing data– Obtaining access to and permission to collect/use data

Page 74: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Research Design Varies by Goal• Exploration

– Primary data• Sampling strategy

• Data collection and coding processes

– Secondary data• Descriptive analysis

• Statistical correlational analysis

• Analyses attempting to address selection issues

• Mediation analysis

• Development– Focus should be on iterative development process

– Feasibility study: use in authentic education setting

– Pilot study: comparison to a similar group

Page 75: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Research Design Varies by Goal

• Efficacy and Replication– Randomized Control Trail (RCT) favored

• Unit of randomization and justification• Procedures for assignment

– Strong quasi-experiment - justify why RCT not possible• How it reduces or models selection bias• Discuss threats to internal validity – conclusions to be drawn

– Describe the control/comparison group– Power analysis/MDES – show calculation and assumptions– Fidelity of implementation study in both T and C– Mediator and moderator analyses– Contamination issues: schools vs. classrooms

Page 76: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Research Design Varies by Goal• Scale-up Evaluation

– Same as Efficacy & Replication except requires implementation under routine conditions, independent evaluator, and a cost study

• Measurement– The plan to develop or refine the assessment

• Evidence of constructs• Interpretation of assessment results• Item development and selection• Procedures for administering and scoring

– Reliability and validity studies

Page 77: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Four Sections of the Research Narrative

• Significance• Research Plan• Personnel• Resources

Page 78: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Personnel Section• Describe key personnel

– Link each person and their expertise to their role in project - show that every aspect of project has person with expertise to do it

• Methodologists: show expertise in particular method to be used

• Substantive person for all issues addressed

• Do not propose to hire a key person with X expertise

• Project management skills

– Give time contribution for each - show that every aspect has enough time from expert

• Orient CVs same way – specific to project– 4 pages plus 1 page for other sources of support

Page 79: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Personnel Requirements• Publication record and projected publications from

this grant are considered• Developers should discuss past success getting

developed interventions evaluated• If previous IES grant, discuss results• Evaluations require attention to objectivity should a

developer or persons with financial interest be involved– Efficacy projects: address how objectivity maintained– Scale-Up: Independent evaluation: developer can provide

routine implementation support

Page 80: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Personnel Strategies for PI

• Senior Researcher– Show adequate time to be PI– Make credentials clear: not all reviewers may know

• Junior Researcher as PI/PD– Show adequate expertise not only to do work but to

manage project– Reviewers may be more comfortable if you have

senior person(s) on project to turn to for advice

Page 81: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Resources

• Show the institutions involved have the capacity to support the work– Do not use university boilerplate

• Show that all organizations involved understand and agree to their roles– What will each institution, including schools,

contribute to the project– Show strong commitment of schools and districts– Have alternatives in case of attrition

Page 82: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Resources (continued)• Appendix C should back this up with

– Detailed Letters of Support from research institutions, States, districts, schools

• Data issues– Document permission to use and access to

confidential data (letters in Appendix C)– Show familiarity with data – show that it can be

used to do the proposed work– If merging datasets, show that it can be done

Page 83: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Appendices• Appendix A (15 page limit)

– Figures, charts, and tables– Examples of measures– 3 pages to address past reviewer comments or to

argue that a proposal is a new submission

• Appendix B (10 page limit)– Examples of materials used in an intervention or assessment

• Appendix C (no page limit)– Letters of agreement (districts, schools, data providers, other

partners, consultants)– Clearly state responsibilities of the writer

Page 84: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Budget and Budget Narrative• Provide a clear budget and budget narrative

for overall project and each sub-award• IES Grants.gov Application Submission Guide

describes budget categories• Check RFA for specific budget requirements

for Research Goals and Grant Programs• Ensure agreement among Research

Narrative, Budget, and Budget Narrative

Page 85: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Agenda• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and

Special Education Research Grants (84.324A)

• Four Sections of the Research Narrative• Other IES Grant Programs• Application Submission and Review

Page 86: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Other Grant Programs

• Do not use topic/goal structure• Use a similar Research Narrative

– Postdoctoral Training uses a Fellowship Plan in place of Research Plan and the Research Narrative has a 15 page limit

– Centers address Management and Institutional Resources, include a 5th component: Plans for Other Center Activities, and the Research Narrative has a 35 page limit

– Stats/Methods and State/Local use same Research Narrative

• Have only 1 application deadline

Page 87: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Postdoctoral Training Program (84.305B & 84.324B)

• Grants to institutions to establish postdoctoral training programs to train researchers in the skills necessary to conduct the type of research that IES funds

• Institution must grant doctoral degrees in fields relevant to education

• Up to 5 years with up to 8 fellow/years for a maximum of $687,000– Funding for fellow recruitment, stipend, benefits, travel, other costs– No funding for faculty research, faculty salaries, or facilities

• 6/23/11 deadline for Special Education Training Program• 9/22/11 deadline for Education Training Program

Page 88: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Statistical and Research Methodology in Education(84.305D)

• Expand and improve the methodological and statistical tools available for mainstream education researchers

• Funding– Typical Range: $45,000 to $300,000 per year– Maximum: 3 years, $1,000,000

• Applications accepted for 9/22/11 deadline only

Page 89: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Evaluation of State and Local Education Programs and Policies (84.305E)

• Rigorous evaluations of education programs or policies implemented by state or local education agencies.

• Requires collaboration with the state or district agency.

• Funding– Typical Range: $500,000 to $1,000,000 per year – Maximum: 5 years, $5,000,000

• Applications accepted for 9/22/11 deadline only

Page 90: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

National Research and Development Centers• R&D Centers address key complex education issues,

create solutions, and contribute to knowledge and theory

• A focused program of research– Tightly linked set of studies on key issue– 50-75% of funds used to address focused

program• Supplemental studies and leadership activities• Cooperative agreement with $1 - $2 million a year

(direct and indirect) for up to 5 years• Applications accepted for 9/22/11 deadline only

Page 91: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Centers – Key Problem Areas in Applications

• Focused program appears to be one person’s research agenda versus a national issue

• Focused program is a set of poorly linked studies, i.e., several peoples’ individual research agendas

• A set of linked studies are proposed but not one is given enough detail to allow reviewers to evaluate its quality

Page 92: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Education Research and Development Centers

• Center on Cognition and Adult Literacy

• Center on State and Local Policy

Page 93: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Special Education Research and Development Centers

• Center on Interventions for Families of Students with Autism Spectrum Disorders

• Center on Interventions for Families of Students with Emotional and Behavioral Disorders

• Center on School-Based Interventions for Secondary Students with Autism Spectrum Disorders

• Center on Reading Instruction for Deaf and Hard of Hearing Students

Page 94: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Grant Program Application Deadline

Letter of Intent

Due Date

Application Posted

Start Dates

305A: Ed Research324A: Sp Ed Research324B: Sp Ed Postdoc

6/23/11 4/21/11 4/21/11 3/1/12 to

9/1/12

305A: Ed Research324A: Sp Ed Research305B: Ed Postdoc305C: Ed R&D Centers324C: Sp Ed R&D Centers305D: Stats/Methods305E: State/Local

9/22/11 7/21/11 7/21/11 7/1/12 to

9/1/12

Page 95: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Agenda• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and

Special Education Research Grants (84.324A)

• Four Sections of the Research Narrative• Other IES Grant Programs• Application Submission and Review

Page 96: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Grant Submission• Make sure your institution is registered on grants.gov• Complete your online forms and upload PDFs• Authorized representative completes the process• Submit by 4:30:00 EST on deadline – earlier is safer• If problems uploading

– Contact Help Line 1-800-518-4726 and get a case number

• You should receive four emails – Grants.gov: assigns you a number that starts with GRANT– Grants.gov: your application is validated or rejected due to

errors. If the latter, correct and resubmit until validated.– Dept. of Ed: retrieved your application from Grants.gov– Dept. of ED: assigns you a number that starts with R305 or R324

Page 97: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Application Review (Office of Standards & Review)• Compliance screening for format requirements• Responsiveness screening to program/goal requirements• Assigned to review panel

– 2-3 reviewers (substantive and methodology)– If scored high enough, application is reviewed by full panel

• Many panelists will be generalists to your topic• There will an expert in every procedure you use

– Overall score plus scores on Significance, Research Plan, Personnel, and Resources

– So far, all applications with overall score of Outstanding and Excellent have been funded

• Resubmissions encouraged: address comments• See

Page 98: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Application Review (Office of Standards & Review)• Compliance screening for format requirements• Responsiveness screening to program/goal requirements• Assigned to review panel

– 2-3 reviewers (substantive and methodology)– If scored high enough, application is reviewed by full panel

• Many panelists will be generalists to your topic

• There will an expert in every procedure you use– Overall score plus scores on Significance, Research Plan,

Personnel, and Resources– So far, all applications with overall score of Outstanding and

Excellent have been funded

• Resubmissions encouraged: address comments

Page 99: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Peer Review Process Information

http://ies.ed.gov/director/sro/peer_review/index.asp

Page 100: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Notification

• All applicants will receive e-mail notification of the status of their application

• All applicants receive copies of reviewer comments

• If you are not granted an award the first time, plan on resubmitting and talk to your program officer

Page 101: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

Upcoming Application Process Webinars http://ies.ed.gov/funding/webinars/index.asp

Monday, May 2, 2011 1:00 p.m. – 2:30 p.m. EST

Full, Registration Closed

Monday, May 23, 20111:00 p.m. – 2:30 p.m. EST

Open, Registration still available

Page 102: IES Grant Writing Workshop Institute of Education Sciences U.S. Department of Education April 2011.

http://ies.ed.gov/

Allen Ruby

[email protected]


Recommended