ies.ed.gov
Connecting Research,Policy and Practice
Rebecca Kang McGill-Wilkinson, Ph.D.Institute of Education Sciences
Overview of Education Research & Development Centers Program
(84.305C)
ies.ed.gov 2
Outline
• Overview of IES• Research & Development Centers– Developmental Education Assessment & Instruction– Knowledge Utilization
• General requirements• Deadlines and notification
ies.ed.gov
Legislative Mission of IES
• Describe the condition and progress of education in the United States
• Identify education practices that improve academic achievement and access to education opportunities
• Evaluate the effectiveness of Federal and other education programs
3
ies.ed.gov 4
Organizational Structure of IES
4
National Board for Education
SciencesStandards &
Review Office
Office of the Director
National Center for Education Evaluation
National Center for Education Statistics
National Center for Education Research
National Center for
Special Education Research
ies.ed.gov
Missions of the Research Centers
• NCER– supports rigorous research that addresses the nation’s
most pressing education needs, from early childhood to adult education
• NCSER– sponsors a rigorous and comprehensive program of special
education research designed to expand the knowledge and understanding of infants, toddlers, and students with or at risk for disabilities from birth through high school
5
ies.ed.gov
IES Grant Programs: Research Objectives
• Develop or identify education interventions (practices, programs, policies, and approaches) – that enhance academic achievement– that can be widely deployed
• Identify what does not work and thereby encourage innovation and further research
• Understand the processes that underlie the effectiveness of education interventions and the variation in their effectiveness
6
ies.ed.gov
FY 2014 Research & Research Training Grant Programs
• Education Research Grants Program (84.305A)
• Education Research & Development Centers (84.305C)
• Statistical & Research Methodology in Education (84.305D)– Statistical & Research Methodology Grants – Early Career Statistical & Research Methodology Grants
• Partnerships & Collaborations Focused on Problems of Practice & Policy (84.305H)– Researcher-Practitioner Partnerships in Education Research– Continuous Improvement Research in Education– Evaluation of State & Local Education Programs & Policies
• Research Training Grant Programs in the Education Sciences (84.305B)– Predoctoral Interdisciplinary Research Training – Methods Training for Education Research– Training in Education Research Use & Practice
7
ies.ed.gov
NCER Investments by Competition
Program Number of Awards
2002-2012
Investment (in millions)2002-2012
Education Research 548 $897.2Research & Development Centers 20 $204.7Postdoctoral Research Training 35 $22Predoctoral Research Training 26 $121.8Evaluation of State & Local Programs & Policies 13 $53.4Statistical & Research Methodology in Education 33 $23.9Reading for Understanding 6 $113.4Preschool Curriculum Evaluation Research 12 $20.2Social & Character Development Research 7 $13.6Unsolicited 31 $44.9Small Business Innovation Research 58 $51
8
ies.ed.gov 9
Research & Development Centers: Purpose
• Contribute to solving U.S. education problems, improve the education system, and increase student achievement through– Research – Development – Evaluation – National leadership
ies.ed.gov 10
Research & Development Centers: Activities
• Focused research program– Contribute to solution of specific education problem within
the Center’s topic– Generate new knowledge and theories
• Supplemental studies– Research on supplemental questions within the Center’s
topic• Leadership and outreach– To researchers, practitioners, and policymakers– Advance the field and practice through evidence-based
research, practice, and policy
ies.ed.gov 11
Center Partnerships
• Institute encourages partnerships with stakeholder groups to increase– Relevance of work– Accessibility of publications– Usability of findings
ies.ed.gov 12
National Research & Development Center on Developmental Education Assessment
& Instruction
ies.ed.gov 13
Purpose
• Research– Document current developmental education practices– Identify promising programs– Support innovation– Evaluate programs that serve large numbers of students
and/or that can be expanded• Assess scalability of programs found to benefit students
• Leadership and outreach– Bring together policymakers, practitioners, and researchers – Assist states and postsecondary institutions to bring
effective models of developmental education to scale
ies.ed.gov 14
Background: Research
• Large numbers of students beginning postsecondary education – especially in community colleges and open-access 4-year institutions – are placed into developmental (remedial) reading, writing, or mathematics courses before they can accumulate credits– Many students are required to take multiple
developmental education courses
ies.ed.gov 15
Background: Research
• Research suggests tests and other practices used to place students in these courses may fail to predict who will benefit from developmental education
• Studies that track progress through coursework show mixed results– Students may not complete coursework– Students assigned to developmental education may be less
likely to graduate
ies.ed.gov 16
Background: Policy Concerns
• Decline in support for developmental education– In some corners, seen as more of a hindrance to students
than a support– Expenditures on developmental education
• Target for cost savings
• More students may be assigned to developmental education than need it
• Placement and instruction need to be revised or replaced
ies.ed.gov 17
Background: Ongoing Work to Improve Developmental Education
• Change placement procedures– Replacing standardized test with other measures
• E.g., high school transcript, college-readiness indicators– Change cut scores for tests
• Identify high school students’ need for developmental education and provide remediation – During high school rather than college– During a “bridge program” before college
• Providing additional supports during college– E.g., learning communities
ies.ed.gov 18
Background: Ongoing Work to Fundamentally Change Developmental Education
• Acceleration– Compress 1-year’s worth of material into a shorter period
• Modularization– Students complete a module of study at their own pace– Students who start with more knowledge can finish sooner
• Mainstreaming – Development education students take college-level courses
with additional support• Replacement– Replace conventional developmental math education
sequence with one on quantitative reasoning and statistics
ies.ed.gov 19
The Center’s Focused Program of Research
• One descriptive study– To document current developmental education practices
(assessment and instruction) in use by community colleges and open-access institutions
• Two evaluation studies– To determine if specific assessments or instructional
practices lead to improved student outcomes in reading, writing, math, English language proficiency, or any combination of these
ies.ed.gov 20
The Descriptive Study: Research Questions
• What are the assessment tools and practices used to assess reading, writing, and math skills of incoming students and place them in developmental education (versus college-level) courses?– Why do they use these tools and practices?
• What are the major strategies used to teach developmental education?
• Are different assessment and instructional approaches used for nontraditional students?
• Are the assessment and instructional approaches aligned with the Common Core State Standards?
• Applicants may expand or add additional research questions (and should provide justification for doing so)
ies.ed.gov 21
The Descriptive Study: Research Approach
• Institutional survey– Stronger if nationally representative of community colleges
and open-access institutions– If not national, stronger if focus on State or postsecondary
institution in which evaluation studies will take place• Qualitative interviews• Secondary data analysis• Combination of the above
ies.ed.gov 22
The Descriptive Study: Methodological Requirements
• Research questions• Population from which study sample will be selected• Measures• Data analysis plan• Timeline
ies.ed.gov 23
Evaluation Study 1: Assessment & Placement
• Possible Research Questions– How can institutions improve on current assessments and
placement practices?– What are the effects of alternative assessment and
placement strategies on students’ overall academic performance, persistence, and progress?
• You may propose expanded or modified research questions as long as they meet the purpose and requirements of the evaluation
ies.ed.gov 24
Evaluation Study 2: Instruction
• Possible Research Questions– How can institutions prepare students for college-level
courses in the shortest amount of time?– What are the impacts of different instructional approaches
on student learning, overall academic performance, persistence, and progress?
• You may propose expanded or modified research questions as long as they meet the purpose and requirements of the evaluation
ies.ed.gov 25
Evaluation Studies: Methodological Requirements
• Research questions• Sample and setting• Appropriate research design (meets WWC standards)• Power analysis• Measures– Proximal and distal outcomes
• Including student outcomes in reading, writing, math, English language proficiency, or some combination of these
– Fidelity of implementation– Comparison group practices
ies.ed.gov 26
Evaluation Studies: Methodological Requirements
• Moderators or mediators• Plan to analyze implementation fidelity and
comparison group practices• Data analysis plan– Overall impacts– Subgroup impacts (e.g., nontraditional students and
students taking more vs. fewer developmental ed courses)• Issues of scalability– Including cost-benefits of the programs evaluation
• Timeline
ies.ed.gov 27
Supplemental Studies
• May conduct smaller, supplemental research projects– Usually can be completed within 9 to 12 months– May include exploratory research, piloting innovations, or
further development of innovations• Center will work cooperatively with the Institute to
select and design supplemental studies• Must provide two examples of supplemental studies
in application– Short rationale and brief description of approach (2 to 3
paragraphs)
ies.ed.gov 28
Leadership & Outreach
• The Center will work with the Institute on its leadership and outreach activities. These may include:– Convening policymakers, practitioners, and researchers– Assisting efforts by States, colleges, and universities to
bring effective models to scale• The Institute encourages a Center website, webinars,
podcasts, and other innovative uses of technology• You should describe at least two examples of
activities in application
ies.ed.gov 29
Significance of Center
• Significance section at start of Research Narrative• Discuss the significance of your proposed Center– Your vision of the Center – What problems it is intended to address– How your set of research and leadership activities will
work together to help students in developmental ed– How the research will be relevant to community colleges
and open-access 4-year institutions• Describe active partnerships with States or
postsecondary systems and their role in the work
ies.ed.gov 30
Maximum Awards
• 5 years• $10,000,000• Maximum of one award• Focused program of research must comprise at least
75% of the total budget
ies.ed.gov 31
National Research & Development Center on Knowledge Utilization
ies.ed.gov 32
Purpose
• Research– Develop tools for observing and measuring research use in
schools– Identify conditions under which practitioners use research
and factors that promote or inhibit research use in schools– Identify or evaluate strategies that researchers can adopt
to make their work more meaningful to and impactful on education practice
• Leadership and outreach– Effective means of knowledge transfer to improve school
performance and student outcomes
ies.ed.gov 33
Background
• Accumulation of high-quality research and evaluation on impact of education policies and interventions
• Concern that this work is not used by practitioners– To adopt interventions with beneficial effects on students– To discard interventions with null or adverse effects
ies.ed.gov 34
Background: Potential Causes
• Misalignment between research questions (e.g., “Does it work?”) and practitioners’ concerns (e.g., “How do I use it with my students?”)
• Misunderstanding of social context of practitioners– Research use is multi-level phenomenon
• Need new ways of working with practitioners– Partnerships– Bidirectional model of relationship– Intermediary organizations
ies.ed.gov 35
The Center’s Focused Program of Research
• One measurement study– Develop and validate tools for observing and measuring
research use in schools and school districts• Two descriptive studies, or one descriptive study +
one evaluation/pilot study– Identify the conditions under which research is used and
factors that promote research/inhibit research use in schools and districts
– Identify or evaluate the skills and strategies researchers can adopt to make their work more meaningful and have a greater impact on practice
ies.ed.gov 36
The Measurement Study: Research Questions
• What are the most reliable and valid indicators of school staff using education research in their decision making?
• What are the most reliable and valid approaches to capture variation in research use across schools, school districts, and over time?
• Applicants may expand or add additional research questions (and should provide justification for doing so)
ies.ed.gov 37
The Measurement Study: Methodological Requirements
• Conceptual framework to provide basis for assessment, intended population, and validation
• Sample and setting• Characteristics, size, analytic adequacy of sample• Methods for developing and validating• Methods for gathering psychometric evidence• Appropriateness for end users• Need for adaptation for different contexts and purposes• Timeline
ies.ed.gov 38
Descriptive Study 1: Research Questions
• What role do State and local education agencies play in making research available to school administrators, faculty, and staff?
• What role do school leaders play?• How do education agencies, school leaders, and teachers obtain
and process scientific information?• How do social networks within schools and school districts appear
to support or hinder research use?• What are the major incentives or disincentives for practitioners to
better understand and apply education research?• What contextual factors seem to affect the use of research in
schools and school districts?• Applicants may expand or add additional research questions (and
should provide justification for doing so)
ies.ed.gov 39
Descriptive Study 2: Research Questions
• Are partnerships between researchers and practitioners associated with greater use?
• Are intermediaries associated with greater use?• What communication strategies or technological
innovations are associated with greater use?• Applicants may expand or add additional research
questions (and should provide justification for doing so)
ies.ed.gov 40
Optional Evaluation or Pilot Study
In lieu of second descriptive study• Examine efficacy of researcher practices on research
useOR
• Rigorous evaluation of impact of particular practiceOR
• Pilot study to show promise of particular practice
ies.ed.gov 41
Descriptive Studies: Methodological Requirements
• Research questions• Population from which study sample will be selected– Institute encourages you to select two or more research
sites• Measures• Data analysis plan• Timeline
ies.ed.gov 42
Optional Evaluation/Pilot Study: Methodological Requirements
• Research questions• Sample and setting• Appropriate research design– if doing an evaluation, the design should meet WWC
standards• Power analysis• Measures– Proximal and distal outcomes– Fidelity of implementation– Comparison group practices
ies.ed.gov 43
Optional Evaluation/Pilot Study: Methodological Requirements
• Moderators or mediators• Plan to analyze implementation fidelity and
comparison group practices• Data analysis plan• Timeline
ies.ed.gov 44
Leadership & Outreach
• Innovative approaches to fostering dialogue among practitioners and researchers
• May take advantage of annual conferences• Institute encourages websites• Institute is open to learning how other applied social
sciences encourage research use among practitioners• Describe at least two examples of activities in
application
ies.ed.gov 45
Supplemental Studies
• May conduct smaller, supplemental research projects– Usually can be completed within 9 to 12 months
• Center will work cooperatively with the Institute to select and design supplemental studies
• Must provide two examples of supplemental studies in application– Short rationale and brief description of approach (2 to 3
paragraphs)
ies.ed.gov 46
Significance of Center
• Significance section at start of Research Narrative• Discuss the significance of your proposed Center– Your vision of the Center – What problems it is intended to address– How your set of research and leadership activities will
work together to promote the use of scientific education research in schools and districts
• Describe how you will engage researchers and practitioners in the work and communication of findings
ies.ed.gov 47
Maximum Awards
• 5 years• $5,000,000• Maximum of one award• Focused program of research must comprise at least
75% of the total budget
ies.ed.gov 48
General Requirements
ies.ed.gov 49
Additional Sections of the Research Narrative for Both Centers
• In the 35-page Research Narrative, discuss…– Significance– Research Plan for the Focused Program of Research
(including supplemental studies)– Other Center Activities (Leadership and Outreach)– Management and Institutional Resources– Personnel
• We’ve discussed the first three and now let’s move to the last two
ies.ed.gov 50
General Requirements: Management & Institutional Resources
• Describe plans and procedures for overall management of the Center and its activities
• If plan is to work in schools or other education delivery settings in first year, must include letter(s) of agreement from education organization(s)
• If developer/distributer of program or intervention is part of Center, describe role and how objectivity will be maintained
ies.ed.gov 51
General Requirements: Personnel
• Expertise in content area(s)• Methodological expertise• Sufficient experience working with education
delivery settings• Experience relevant to national leadership– Explain why proposed Center staff is qualified to fulfill
leadership role
ies.ed.gov
Application Deadline
Application Package Posted
Start Dates
Sept 4, 20134:30:00 PM
DC Time
June 6, 2013 July 1, 2014to
Sept 1, 2014
Important Dates & Deadlines
52
ies.ed.gov
Notification
• All applicants will receive e-mail notification that the following information is available via the Applicant Notification System (ANS):• Status of award• Reviewer summary statement
• If you are not granted an award the first time, consider resubmitting and talking with your program officer