ies.ed.gov
Connecting Research,Policy and Practice
Meredith Larson, Ph.D.National Center for Education Research
andAmy Sussman, Ph.D.
National Center for Special Education Research
First Impressions:How to Win Grants and Influence
Your Research Career
ies.ed.gov
1. Identify two things you should have with you at all times.2. Describe a theory of change and discuss why it’s essential
grant writing.3. Have the tools necessary for refining your research
statement of purpose.4. Have a sense of what program officers and reviewers want
from you and your applications.
All of this is to help prepare you to write winning proposals and articulate your research agenda to the broader (research)
community.
Goals
ies.ed.gov
Our Assumptions of You and What You Want
• We assume that you want– to help improve education – to carve out your own research agenda– to get funding to conduct your research
• We assume that you have– expertise in a wide range of topic areas and
research methods/analyses– varying levels of familiarity with grant writing
ies.ed.gov
What You’re Facing
Whenever you ask someone to support your research, remember that it all comes down to convincing them that they should care about your topic and that you have the necessary competencies to reach your goals.
…so how do you do it?
ies.ed.gov
What You’re Facing
Audience awareness1. Seek common ground. • Highlight core values or concerns that your research
addresses.• Look for ways to build off of things that are familiar.
2. Be helpful and thorough.• Make things easy to understand, remember, and find.• Be engaging and helpful.
ies.ed.gov
What You Need to Do
Sell your research idea, promote yourself as the best person to do the research, and build
goodwill and trust.
How?
By demonstrating that you know what the problem is and have a way to address it.
ies.ed.gov
Explaining Your Research
You should have two things ready at all times:1. Statement of Purpose (preferably an “elevator
speech” or “research pitch”)2. Theory of Change
ies.ed.gov
Statement of Purpose
We will come back to this later, but in general a ready-made Statement of Purpose should– Be short and attention-getting and– Contain the problem statement and your contribution to
solving it.
Your fellow researchers, friends, and family members should be able to understand it and see its relevance.
NOTE: It’s not as easy to do as you may think. You have to know a lot about what it is you want to do before you can describe it succinctly.
ies.ed.gov
Theory of Change (ToC)
• This is the model underlying your research. • It is a roadmap to your grant narrative.• It is a source for generating research
questions.• It is constantly evolving.
NOTE: Some fields and scholars use terms like Logic Model or Logical Framework to mean things similar to what we are calling a Theory of Change here.
ies.ed.gov
ToC: What is It?
Theories of Change help define how and why an intervention (e.g., curriculum, policy, practice) should lead to particular outcomes.– Makes assumptions explicit– Suggests the causal relations
ies.ed.gov
ToC: What is it?
• Some associate them with program evaluation studies, but they are also useful for exploration work, work on cognitive or behavioral processes, etc.
• They are frequently represented visually, but they can be simply text.
ies.ed.gov
ToC: Why You Should Care
Reviewers will evaluate your research on how clear your theory and model are. A strong ToC will help make your case for you.
Four Criteria Used to Judge ToCs (Connell & Klem, 2000):1. Plausible2. Doable3. Testable4. Meaningful
ies.ed.gov
ToC: How to Make One
• Start with the long-term outcome and work backwards.– What preconditions need to exist to lead to the outcome?– What is occurring in the context that could hinder or
support the outcome? – What assumptions are you making?
• Draw it out.• Write it up.• Get feedback. Revise. Repeat.
ies.ed.gov
Create the General Framework
Come up with the broad framework.
Generally, you should start with the outcomes. You can then jump all the way back to thinking about the initial state or hop around.
Long-term Outcomes
Intermediate Outcomes
Strategies/Activities
Initial State Context
ies.ed.gov
The Basic Components
INITIAL STATE: What’s there before the intervention including things such as resources or student characteristics, etc. (sometimes called input)STRATEGIES/ACTIVITIES: The things that happen (e.g., activities, events, curriculum) to the people who participate or who are the targets of the intervention (sometimes called output)OUTCOMES: The proximal and distal changes for people involved in or that are the targets of the interventionASSUMPTIONS: What you believe to be true of the resources, the people involved, the people targeted, and the intervention in generalCONTEXT: The environment (internal or external) in which the intervention takes place (e.g., other policies or practices occurring simultaneously)
Initial State(resources)
Strategies/Activities
Outcomes(proximal &
distal)
Assumptions
Context
ies.ed.gov
Example
4-year-oldpre-K children
DemographicsEligibility/need
Exposed to intervention1-hour lessonWrap-around
servicesPositive supports
Focus on early literacy
PD for providers
Positive attitudes to
school
Improved pre-literacy
skills
Learn appropriate
school behavior
Increased school
readiness
Greater gains in K literacy
Setting/contextPersonal and family characteristics
Prior experienceTeacher/provider experience
ies.ed.gov
Detailed ComponentsNow that you have the general aspects, you should specify the details. These will lead into your research design, measures, etc. in your narrative.
Indicators: Operationalizing the outcomes (e.g., what indicates success?)Populations: Identifying who should show which change (could be multiple groups)Thresholds: Setting your expectations for change for each outcome (e.g., how much is good enough?)Timeline: Determining when we should meet the threshold for each outcome
(Connel & Klem, 2000)
ies.ed.gov
Example
OUTCOME = Improved pre-literacy skillsIndicator: Correctly identifies letters and soundsPopulation: 4-year-olds, especially those with little print exposureThreshold: 1.5 effect greater than comparison Timeline: After full intervention, 16-weeks
ies.ed.gov
Another Example (Slightly different layout )
Instructional ContextText and task characteristicsNature of instructional support
Reader CharacteristicsDecoding and fluency proficiencyVerbal knowledgeAttention and behavior
Text Comprehension
Motivation to ReadDecision to engage in reading and task persistence
Expectancies for successSelf-efficacyPerceived control
ValueAchievement goalsIntrinsic motivationUsefulnessSocial motivation
Reading Engagement PhysicalCognitive: Building and maintaining coherence
Adapted from a model presented by Dr. Sharon Vaughn (R305F100013).
ies.ed.gov
Don't Do This
1. Be overly simplistic2. Overwhelm the reader3. Use color as a key (audience awareness!)
ies.ed.gov
Example: Too Simple
Increased Academic
Achievement
The Increasing
School Success Program
Students with
Disabilities
ies.ed.gov 22
PLTProfile AnalysisSet instructional goals & WL focus
[4 weeks at end of prior year or beg of current
year]
PLTBegins weekly
meetings PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
PLT
WLDebrief
[Processes 4-11 repeat to mid-year]
[PLT appoints SLT1 to address PD Topic 1]
SLT 1Research
SLT 1Implement Prof Devt
PLT = Primary Leadership team
SLT = Secondary Leadership team
= Begin Process
= Feedback for next process
= Delegation of PD
10. reviews/revises
model based on findings
14a. SLT and Coaches
create PD unit
1a. PI Recruits
and Trains Coaches
6a. Coaches facilitate PLT
identification of annual goal
1b. develops
data collection
tools
11. Coaches share/
implement revisions with PLTs
3a. Coaches collect 3 yrs. stud. ach. & demo. data per school2a. PI
matches Coaches
to schools (n=5)
2b. trains coaches to
use all data tools
5. guide Coaches and PLTs in
analyzing data collected
4. Coaches work with
PLT 3b.
develops electronic
tool
7a. Coaches teach PLT to conduct
WL
approx. 3 wks after school begins
8a2. Coaches assist PLTs in
weekly WLs and Debriefing
through mid-year
6-8b. observe PLTs, documents implementation
9. interviews coaches and
PLTs about WL process
approx. 9 wks after school begins
13a. researchers train SLT to
research best-practices in PD
area
12. Coaches & PLTs choose 2-4 teachers (based on WLs) to become Dynamic
Leadership Team 1]
13-15b. Observe SLTs,
document implementation
15a.Coaches assist SLT in
implementing PD with faculty
16. interview coaches, SLTs,
and PLTs about PD and WL processes
17. reviews/revise model based on findings
8a1. Coaches assist PLTs in using all data
[Processes 3-17 repeat twice in Year 2]
18a. submit Annual Report
to schools
ies.ed.gov
Draw your Theory of change
Take a few minutes and write a couple words about the following:
1. What are the initial states (inputs)?2. What are the strategies/activities (outputs)?3. What are the outcomes? Proximal, Intermediate,
Distal…4. What is in the context?
ies.ed.gov
So Now What?
Your Theory of Change acts like a roadmap that can help lead you to research questions, a
proposal, and so much more.
ies.ed.gov
ToC & Research Questions
Questions about initial states or inputs– Were resources provided and used?
Questions about strategies/activities or outputs– Did the activities occur as planned?
Questions about outcomes– Were there changes in the proximal/distal outcome relative
to the control/comparison?
Questions about context– Did the context suppress or reinforce effects?
(Milanowski & Kimball, 2009)
ies.ed.gov
ToC & Your Application(s)
In IES applications, there are four major parts of the application: Significance, Research Plan, Personnel, and Resources. The first two of them can draw heavily from the ToC.
Significance – Draws heavily from the outcomes, context, and assumptionsResearch Plan – Draws heavily from the strategies/activities and the outcomes
ies.ed.gov
ToC & Your Research Plan
In your research plan, you need to specify exactly what it is you’re exploring, creating, validating, or testing. You also need
to specify how you will do these things.
Strategies/Activities What are the pieces that you’ll be exploring, creating, testing, etc.?Outcomes Indicators: What will you measure, and how you will measure it?
Populations: Who and where (both in treatment and control/comparison)?Thresholds: What effect (size) should you expect?Timeline: When should you be collecting what data?
ies.ed.gov
ToC & Your Application(s)
Other components of your narrative and your application can also be informed by the ToC:
Personnel – For example, you want someone with experience in each of the outcome areas and with knowledge of how to assess the strategies/activities.
Budget – For example, how many observations will you need to take (# of researchers and % effort) and when (during what grant year) and what assessments you will need (how many of which one should you buy).
ies.ed.gov
ToC & Your Statement of Purpose
Your Theory of Change also contains the core elements of a good statement of purpose, mission statement, elevator speech, etc.
You have the pieces.Put them in a short, digestible format that conveys
the problem’s significance.And now you have your statement.
(It actually requires a bit more than that.)
ies.ed.gov
An Example
Now we’re going to walk through an example of how you might take a ToC and generate an “elevator speech”, a statement of purpose, even an opening paragraph.
Note: Sometimes people have their statement and then create their ToC. You need both, and they inform one another. You do not need to have a ToC to write a statement.
ies.ed.gov
From ToC to Statement of PurposeDistal Outcome: Increase student achievement in scienceProximal Outcome: Young children (preK – K) with greater ability to think scientificallyStrategies/Activities:
Professional development curricula Four 1-hour long workshops for teachers Training on science content and early childhood pedagogy Current, practicing educators PreK students
Initial State: Early childhood educators who lack science content and pedagogical knowledge, lack of curricula to help teachersAssumption: Teachers are essential to student outcomes. Teachers need both content and pedagogical knowledge and expertise. Teaching teachers will improve student outcomes.
ies.ed.gov
…continuedSpeech:Being able to think scientifically at a young age predicts long-term academic success in science. However, many children coming into elementary school lack this ability. Even if they are exposed to science in PreK programs, they are still not acquiring the necessary skills. One reason PreK students may be struggling is that early childhood educators often do not have the background knowledge of science content or instruction to help their students. Because teachers are so important to the learning process, we need to help better prepare them for their role. So I am developing an intervention that will increase early childhood educators’ content and pedagogical knowledge so that they can better prepare their students.
ies.ed.gov
Components of a Good Statement
• Story! Characters, goals, drama, resolution. • Facts – but not too many. This will vary based
on audience and use.• Short – shoot for a paragraph or < 90 seconds.
Make them care.
ies.ed.gov
Now you try
• What is the problem? The tension? The drama?
• What evidence do you have that it’s important?
• How can you help solve it?
ies.ed.gov
Uses of Your Statement
• Opening paragraph(s) of your applications or your purpose paragraph– Note: This is true for all documents, such as dissertation proposals, not
just IES applications.
• Job talks• Communicating with the general public (e.g.,
when you give your NPR interviews)
ies.ed.gov
Importance of Opening Paragraph
• Opening paragraph sets the scene for readers.– Identifies the significance of the work to be
done and what actually will be done– Readers use it to organize information in rest of
the application– You can lose your readers right off with an
unclear opening
ies.ed.gov
NCER/NCSER Abstracts
As an example of an opening/purpose paragraph, we’d like to use examples from some of the abstracts we have posted online:
http://ies.ed.gov/funding/grantsearch
ies.ed.gov
Example of Opening/PurposePurpose: Teaching children how to think scientifically in the preschool years has the potential to address an existing achievement gap in early science and provide children with the skills necessary to continue learning and thinking critically throughout the school years.
Unfortunately, early childhood teachers typically lack content and pedagogical knowledge in science and are not prepared to provide developmentally appropriate experiences that support children's early science learning and readiness. In order to address these challenges, the researchers will develop the Cultivating Young Scientists (CYS) intervention, which includes a professional development program for early childhood educators, science curricular content, and a set of formative assessment tools. The intervention is intended to lead to an increase in teachers' use of science instructional content and practices in preschool settings and improvements in young children science content knowledge and scientific thinking skills.
Problem
Goal:How will you address this
problem?
Background
ies.ed.gov
Example of Opening/PurposeCore diagnostic features of autism include deficits in social-communicative functioning. Two pivotal skills for young children with autism include joint attention and pretend play, which constitute early foundations upon which later social-communicative skills are built. Joint attention (characterized by behaviors such as pointing, showing, and coordinated looking to share attention toward objects or events with another person) and symbolic play (characterized by the ability to pretend), play important roles in language development and social engagement with peers.
Children with autism show deficits in these skills, and as a consequence may lag behind their peers academically and socially. Advancing Social-Communication and Play (ASAP) is an intervention that has recently been developed to help preschool children with autism learn and practice these important skills. However, we do not know how effective it is.
The purpose of this research is to conduct a cluster randomized trial to evaluate the efficacy of ASAP. The major goals of the project include investigating whether children experiencing the intervention demonstrate greater gains in the proximal child outcomes of social-communication and play skills as well as the more distal outcomes of language development and engagement with classroom objects and peers. The study will also examine whether child-level and teacher-level (i.e., teacher burnout, general classroom quality) characteristics moderate the impact of the intervention.
Problem
Goal:How will you address this
problem?
Background
ies.ed.gov
Final Comments: Helpfulness of the Program Officer
• Share your framework and statement of purpose with the Program Officer– Ensure you are submitting to the correct
competition/topic – Springboard for further discussion
ies.ed.gov
Final Comments: Importance of Clarity of Writing
• Readers (e.g., application reviewers) often complain about lack of clarity. – Significance too general – Lack of detail regarding intervention,
development cycle, or data analysis– Use of jargon and assumptions of knowledge– Poor writing (e.g., grammar), awkward
constructions, etc.
ies.ed.gov
Final Comments: Importance of Practicing
All forms of communication require practice.
Things only other people can tell you:– Is your description of your theory too long or complex?– Is your statement of purpose too long or complex?– Is your idea engaging, and does it invite questions that
build off of your ideas (rather than those that try to figure out what your idea really is)?
ies.ed.gov
Pulling It All Together
Recall Why You’re Here:You want to write winning grants and build a solid research career.
But How? By getting people interested in your work and the questions you feel are important and by building trust that you have a plan for addressing an important issue.
ies.ed.gov
Pulling It All Together
• Have a theory of change– Should be as fully explicated as possible– Is always being refined– Should be made with and informed by others
• Have a clear statement of purpose at all times– Your “elevator speech”– Simple, elegant (hard to do, actually)
ies.ed.gov
More infoRemember: Some scholars distinguish between a Theory of Change and a Logic Model. Others do not. We use the term “Theory of Change” for the purposes of this presentation as a way to discuss how people can present the causal relations among elements. Please use the term most appropriate for your field. (Note that the IES RFAs use “Theory of Change.”)
This presentation was informed in part by the following resources:Connell, J., & Klem, A. (2000). You can get there from here: Using a theory of change approach to plan urban education reform. Journal of Educational and Psychological Consultation, 11(1), 93-120.
Center for Theory of Change: http://www.theoryofchange.org/. Retrieved May 1, 2013.
Harris, E. (2005). An Introduction to Theory of Change (Issue Topic: Evaluation Methodology, vol. 11). Retrieved May 1, 2013, from http://www.hfrp.org/evaluation/the-evaluation-exchange/issue-archive/evaluation-methodology/an-introduction-to-theory-of-change.
Kimball , S. & Tony Milanowski, T. (2009). Establishing a Theory of Action and Logic Model for Your Project Session 1 [PowerPoint slides]. Retrieved on May 1, 2013 from Center for Educator Compensation and Reform website: http://www.cecr.ed.gov/pdfs/september2009meeting/TheoryofAction.pdf.
Reisman, J., Gienapp, A., Langley, K., & Stachowiak, S. (2004). Theory of change: A practical tool for action, results and learning. Organizational Research Services for Annie E. Casey Foundation.
