Quality of the project … · Need for project ... requirements allowing different graduation...

i

Oregon State Personnel Development Grant 84.323A

Table of Contents

Need for project……………………………………………………………………...page 1

Significance…………………………………………………………………………..page 10

Quality of the project design…………………………………………………………page 21

Quality of project personnel………………………………………………………….page 43

Adequacy of resources……………………………………………………………….page 48

Quality of the management plan…………………………………………………….page 61

Quality of the project evaluation……………………………………………………..page 75

Logic Model………………………………………………………………………..Appendix A

Adoption Timeline…………………………………………………………………Appendix B

District and Interagency System Support Plans……………………………………Appendix C

Year five SPDG report…………………………………………………………….Appendix D

EBISS Coaches Manual……………………………………………………………Appendix E

Key Personnel (Resumes)…………………………………………………………Appendix F

Letters of Support…………………………………………………………………Appendix G

Sample Evaluation Tools………………………………………………………….Appendix H

ii

Oregon State Personnel Development Grant 84.323A

Priorities

PAGE # REQUIREMENTS

43 (a)Projects funded under this notice must make positive efforts to employ and advance in

employment qualified individuals with disabilities in project activities. (See

Section 606 of IDEA)

43 (b)Applicants and grant recipients funded under this notice must involve individuals with

disabilities or parents of individuals with disabilities ages birth through 26 in

planning, implementing, and evaluating the projects. (See Section

682(a)(1)(A) of IDEA)

43 (c)Applicant must describe steps to ensure equitable access to, and participation in, its

program for students, teachers, and other program beneficiaries with special

needs. (See Section 427, GEPA)

61 (d)Projects funded under these priorities must budget for a three-day Project’s Directors’

meeting in Washington, D.C. during each year of the project.

Oregon Personnel Development Application 84.323A _2011 Page 1

NEED FOR PROJECT

Specific gaps or weaknesses in services, infrastructure, or opportunities and the nature

and magnitude of those gaps or weaknesses.

Oregon’s Broad Areas of Need: This section will provide a general overview of the factors

that directly contribute to Oregon‘s need for a project that devotes resources and focus to the

professional development of specialized educational personnel.

A logic model of Oregon‘s personnel development plan is included in Appendix A.

Oregon Specific Factor Identified Need

Increases and changes in standards for all students

as well as for high school graduation and

proficiency.

Assurance of common approach to

content standards for all state-endorsed

trainers.

Evidence-based practices focused on

interventions for students with diverse

needs.

Rapid population/demographic changes over the

past decade.

Evidence-based practices focused on

interventions for students with diverse

needs

Challenges to meet the training needs in our diverse

geographical regions

An Implementation Science that is

geographically intentional (zoning)

Budget reductions related to education funding

infrastructure

An Implementation Science that

capitalizes on efficiencies across

activities


Initiative overload and fragmentation of well-

intentioned services, providers, and miscellaneous

experts.

An Implementation Science that

defragments and provide a framework for

activities

Continuing deficits of specialized personnel. A recruitment and incentive plan to

increase areas of deficit

Changes to Higher Standards: Several recent policy decisions have had substantial impact

on Oregon‘s education system. Courageous moves by the state to implement rigorous standards

for the future benefit of our students, and our state, place another immediate strain on Oregon‘s

educational infrastructure that must be borne by the state‘s educators (See Appendix B for

adoption timeline). In 2007 the Board of Education instituted major changes in the form of

upgrades to Oregon‘s high school diploma requirements to more adequately prepare high school

graduates. These changes were made in response to information from the report, Ready for

College and Ready for Work: Same or Different (ACT, 2006). The report made it clear that high

school graduates must leave school adequately prepared with the skills (reading and math) that

will allow them to either take entry level college credit bearing courses or to enter workforce

training programs associated with family-wage jobs and potential for career advancement.

In 2007 the Legislature also passed statewide requirements for modified diplomas and

alternative certificates. Although alternative options for high school completion had been in

existence in Oregon for many years the requirements for modified diplomas and alternative

certificates were not consistent between districts. Rules were established for standardized and

consistently high quality expectations for all students (OAR 581.022.1130, ORS 329.451). In


2009 the Legislature amended the statewide requirements for a modified diploma (OAR

581.022.1134) and alternative certificate (OAR 581.022.1135) and established an extended

diploma (OAR581.022.1133). These changes created a hierarchical approach to diploma

requirements allowing different graduation options for students with diverse learning needs.

Many schools need to adjust course content to provide students with scaffolded instruction and

guidance in working toward the higher standards. Pre-requisite course content required

adjustment in many schools so students were provided with scaffolded instruction and guidance

in working toward the higher standards. Instruction and curriculum changes are still ongoing,

partially as a result of the scale of change necessary to meet the higher standards and partially

due to the gradual rollout of the Essential Skill deadlines through the year 2014.

As a further demonstration of Oregon‘s commitment toward rigorous expectations for

students, in October 2010 the Oregon Board of Education joined with other participating states in

adopting the Common Core State Standards (CCSS). The primary purpose for this adoption is to

ensure that Oregon‘s graduates are adequately prepared to enter the 21st century job market and

compete for jobs with students from other states and countries many of which have more

rigorous standards (Education Northwest, Spotlight on the Common Core State Standards;

March 2011).

The CCSS for English language arts and mathematics are content standards designed to help

ensure all students are college and career ready in literacy and mathematics no later than the end

of high school. Developed as part of the Common Core State Standards Initiative, a voluntary

effort led by multiple states across the nation, this set of standards describes what students in our

nation should know and be able to do in order to compete in a global economy. The standards are


research and evidence-based, internationally benchmarked, aligned with college and work

expectations and include rigorous content and skills. Common standards help ensure that all

students, no matter where they live in the state or the nation, are prepared for success in

postsecondary education and the workforce. Common standards will help Oregon ensure that

students are receiving a high quality education consistently, from school to school and district to

district. Common standards will also provide a greater opportunity to share experiences and best

practices within and across states which will also improve our ability to serve the needs of

students. Oregon is in the process of developing an implementation plan for the statewide use of

these standards. Funds from this grant will assist in that effort. Oregon‘s commitment to equip

students with access to the CCSS as a contribution to their post K-12 success is included

throughout this application in fulfillment of the expectations for Competitive Priority three.

Population Changes: In recent years there has been an increase in the number of students in

Oregon who experience additional challenges. Students who qualified for free and reduced-price

meals in 2003-2004 made up 40.1% of all students. By 2008-2009 44.6% qualified for free and

reduced-price meals and for the 2009-2010 year the number increased to 49.0%. Similarly, a

16.4% increase in the number of children on Individual Education Plans (IEP) from 1997-1998

to 2009-2010 has changed the nature of special education services.

The diversity in Oregon schools has been increasing, especially during the past decade.

Students identified as non native English speakers increased by 387.2% between 1997-1998 and

2009-2010, suggesting the need for a marked shift in instruction to accommodate the increase of

English language learning students. The percent of students identified as White dropped by

16.2% in the same period while the percent of students identified as Hispanic increased by

151.3% and the percent of students identified as Asian/Pacific Islander increased by 35.1%.


These changes in student demographics call attention to the need to support Oregon teachers and

schools as they work with increasing numbers of students from diverse backgrounds.

Geography and Access to Training: Oregonians live in a unique place geographically

which has a substantial impact on the educational infrastructure. Oregon has almost ninety-six

million square miles of land area and a population density of approximately 36 persons per

square mile (U.S. Censure Bureau). However, multiple mountain ranges divide the state. There

are a few densely populated urban areas but the vast majority of the state remains sparsely

populated. The high mountain ranges and the expansive high desert plateau result in a multitude

of small rural communities, often isolated and far from services and training opportunities. The

Department is working on ways to overcome this disparity, however, in the interim, the smallest

districts (most of whom are already faced with the biggest resource challenges), are presented

with additional challenges associated with traveling great distances to attend a training and

finding the appropriately trained and available substitute teachers for the periods of absence.

Economic and Budget Challenges: Oregon also faces severe economic challenges that

differ from other states. According to the Oregon Employment Department, for the past two

decades the state‘s rapid population growth, coupled with average to below-average income

growth has resulted in an increasing disparity in per capita personal income for Oregon residents

as compared to the rest of the nation. This growing decline in personal income is of great

concern to state policy makers and economists. In 2008 Oregon's personal income showed the

largest deficit when compared to other states since 1929 when such statistics first began to be

documented. Oregon's unemployment rate has long been above the national rate, regardless of

the status of the national economy. In 2009, Oregon's average unemployment rate was 11.1

percent, compared to 9.3 percent for the nation and Oregon has a lower employment-to-


population rate ratio relative to the rest of the nation (N. Beleiciks, 2011 Oregon Labor Market

Information System).

Oregon‘s public school finance system is also complicated by several factors. Currently,

money to support public education in grades K-12 comes from state income taxes, lottery funds,

local revenues (primarily property taxes) and federal funds (2011 Oregon Blue Book).

Approximately two-thirds of the money needed to run Oregon‘s public schools comes from the

state with the remaining one-third from local property taxes.

State funding of education was accelerated with the passage of Measure 5 in 1990 which

required that strict limits be placed on school funding through local property taxes. In 1991, in

response to Measure 5, the Legislature passed a permanent K–12 equalization formula, which

determines how much money each school district will receive from the State School Fund to fill

the gap between the district‘s local revenue and its equalization target. In 1997, Measure 50

assigned a permanent tax rate for each taxing district and stipulated that a property‘s assessed

value cannot increase by more than 3% in any given year. The effect of these tax measures

shifted the bulk of public school funding from local property taxes to the state‘s general fund.

In times of high unemployment like these, the general fund is substantially reduced as it

relies almost exclusively on the state‘s income tax revenue. During the last several years, Oregon

has experienced the worst budget shortfalls since World War II. Financial support for education

in Oregon has faced an increasingly competitive budget environment as other state agencies

compete for general fund dollars (2011 Oregon Blue Book). Although the challenges described

are Oregon specific they must also be considered in the context of overall financial stress across

education systems in the U.S. According to Education Week‘s June 29 publication, ―The closest

parallel to today‘s weak economy is the recession of the early 1980‘s…but this recession is


deeper and broader… [and] far more is being asked of the schools, and they‘re getting less

money.‖

Initiative Overload: The state of Oregon is currently responding to district Superintendent

outcry from superintendents across the state related to what they perceive as an overload of

apparently disparate (and occasionally contradictory) activities occurring at the district level.

This ―initiative overload‖ at the district level can have multiple sources. It is sometimes a result

of multiple offices in the Oregon Department of Education responding to district needs without

the benefit of a coordinated state framework. Sometimes the initiatives are in districts a result of

self-initiated requests to vendors, contractors, or university researchers intended to locate

professional development opportunities specific to a population (e.g., special education), specific

to a content area (e.g. math), or specific to a particular instructional model or set of

interventions--such as Positive Behavioral Interventions and Supports (PBIS).

To further complicate district attempts to streamline these activities, in some cases the

―initiative‖ is attached to funding streams that restrict the district to only providing the support to

personnel serving specific populations of students. This can result in intra-district conflicts over

resources that were purchased for a specific population but desired for students not eligible for

the funding stream.

Currently, the state has responded to a district-initiated request to demonstrate unequivocally

how the three preferred state initiatives (originating in different offices) align in a district. The

three initiatives selected were selected as initiatives that were demonstrating success in districts

through professional development intended to facilitate improvement through the use of data,

tiered responsive interventions, and targeted coaching.


The three initiatives selected were:

Effective Behavioral and Instructional Support Systems (EBISS; 2006 SPDG),

Oregon‘s nationally recognized DATA (Direct Access to Achievement) project,

Oregon‘s Statewide System of Support (OSSS)

Personnel deficits: Oregon‘s 2006 SPDG project was designed to improve specific elements

of professional training for educators with a variety of program designs. Some of the funding

obtained from Oregon‘s 2006 grant was used to address personnel deficiencies of special

education service providers, including: orientation and mobility specialists (O&M), Speech

language pathologists (SLP), and highly qualified special education teachers. That objective

resulted in the recruitment and retention of more than 80 additional special education

professionals. However, despite those gains high priority personnel deficits in Oregon continue,

especially for SLPs.

According to a March 28, 2009 article by Ann Ereline published by The Oregonian, the

largest statewide newspaper, there is a continued need for speech language pathologists in

Oregon. Each year, about 50 SLP students graduate with master's degrees from two Oregon

universities. Although that number has grown within the last few years there are still not enough

candidates to fill SLP vacancies in Oregon's school districts (Ereline, 2009). Information from

EdZapp, a web based education employment service used by many Oregon school districts,

revealed that over the last five years an average of 88 SLP positions have been posted by Oregon

school districts each year. In 2006 there were 66 postings which had increased by 2010 to 106

postings (personal communication, June 28, 2011). While the number of applicants for these


positions has increased, it remains far below the average number of applicants for other special

education positions. To confirm the accuracy of these reports, the Department sent a survey in

June 2011 to special education directors in public school districts and Oregon‘s Education

Service Districts (ESD). The survey asked them to specify the district‘s top five related service

personnel hiring needs. Thirty of the 49 respondents (61%) to the ODE survey indicated that

speech language pathologists represent the primary personnel deficit in school districts. This was

true despite economic and employment difficulties that have resulted in fewer available jobs for

other education positions (Ereline, 2009).

In school settings SLPs evaluate students, develop treatment plans, provide direct treatment,

and assess progress of children who have communication disorders. They work with students

who have speech and language difficulties, students with autism, learning disabilities, or hearing

loss, and a host of other health conditions. Oregon‘s statewide shortage of SLPs contributes to

identification delays of children who have communication disorders (Ereline, 2009) resulting in

delays to the necessary services that allow students to be successful in school. In 2010-2011

there were over 30,000 Oregon children and students with an Individualized Family Service Plan

(IFSP) or an Individual Education Plan (IEP) with Communication Disorder listed as one of their

handicapping conditions (ODE, June 28, 2011). In addition, speech pathologists also serve

children and students with many other types of disabilities depending on the individual needs of

each child. There are more than 21,850 children in Oregon with Communication Disorder as

their primary disability, over 8390 children with autism, more than 27,280 students with Specific

Learning Disabilities, and more than 1100 children with Hearing / Impairment or Deafness


(ODE, June 28, 2011). These numbers reveal the critical need to increase the quantity of SLPs

providing services to students in Oregon‘s public schools.

Summary: Need for project activities

The factors described in the table and narrative of this section are intended to provide support

for four types of activities referenced throughout the remainder of this application:

A systematized approach to introducing the Common Core standards to all relevant

stakeholders

A solid evidence-base in any work undertaken

A scientific framework for efficient implementation

Targeted recruitment specifically designed to address areas of need

In the remainder of this application we intend to show how Oregon‘s prior SPDG activities,

coupled with lessons learned through recent technical assistance on Implementation Science,

(Fixsen, Naoom, Blase, Friedman, & Wallace, 2005) and research on professional development

places us in a unique position to create a sustainable professional development program that

begins to move beyond the confines of population, content, or initiative-specific model while

still allowing responsive systems that differentiate according to student need.

SIGNIFICANCE

The Significance of and the likelihood that the proposed project will result in system

change or improvement.

In 2008, as a result of the potential demonstrated by Oregon‘s Effective Behavioral and

Instructional Support System (EBISS) model, Oregon applied for and was accepted into the State

Implementation Scaling of Evidence-based Practices (SISEP) work being carried out by the


National Implementation Research Network (NIRN) at the University of North Carolina (UNC)

by Drs. Dean Fixsen and Karen Blasé. As Oregon‘s knowledge of Scaling (implementation

science) expanded, it became clear that this model can be used to provide all districts and early

childhood programs, access to the principles of the evidence-based practices that would enhance

the state‘s State Performance Plan (SPP) activities, improve on outcomes measured as part of the

state‘s Annual Performance Report (APR), and provide efficient services for systems and

students statewide. This is accomplished through five main principles that provide an efficient

framework for state work:

1) Approach implementation by defining geographic zones to build regional capacity and

support.

2) Approach implementation by using specific team structures for dynamic problem-solving

3) Ensure strong communication through feedback loops that connect classrooms to state-

leadership.

4) Ensure the promotion of evidence-based practice for all activities

5) Promote intentional action-planning for improvement and sustainability.

In spite of the numerous deficits described in the previous section, Oregon is confident in the

likelihood of successful outcomes moving forward for three principle reasons. First, the

recognition that the activities selected in 2006 and moving forward are supported by a strong

evidence-base; second, the demonstrated successes of the activities undertaken over the past five

years; and third, our confidence in new research in the area of implementation and evidence-

based professional development which has already provided Oregon with valuable lessons

learned (both technical and adaptive in nature) that have begun to transform our work.


The significance of the current application is based on our intent to improve on the modest

gains achieved through our previous SPDG by applying the principles of implementation science

to improve (1) the infrastructure associated with professional development, (2) increase the depth

of previous activities to include areas previously undeveloped, (3) increase breadth to new

districts in the state, and (4) to continue to support areas of professional deficit in the state by

recruitment and scholarship activities.

OBJECTIVE 1: Increase capacity of state-supported Implementation Coordinators who

are responsible for statewide professional development on evidence based practices to

convey the interconnectedness of state endorsed initiatives (CCSS, RTI, PBIS) and to

develop valid evidence-based materials for district use in tiered decision-making.

Objective 1 seeks to ensure that those individuals responsible for developing the

sustainable program for the state (those who will train so that routines become embedded, those

who will develop tools and measures that are replicable, and those who will develop manuals

that document the processes associated with successful outcomes) are themselves trained and

informed in a shared understanding of the evidence-base that supports the work. In support of

objective 1, implementation coordinators who will be responsible for sharing the information for

districts across the state will be trained in the core features across all relevant content to assure

alignment of training, and fidelity to the concepts.

In the state of Oregon, the likelihood of success at objectives 1 – 3 are based in large part

on the evidence-base of the activities at the center of the work. The initiatives that are at the heart

of Oregon‘s 2011 application are Effective Behavioral and Instructional Support Systems

(EBISS) which combines (a) Response to Interventions for academics (RTI) and (b) Response to


intervention for behavior (PBIS). Also at the heart of this application is the work associated with

the implementation of the Common Core State Standards (CCSS).

Evidence base

Academics: The academic tiered model uses the Response To Intervention (RTI) framework and

promotes access to high-quality core instruction and increasingly intensive and timely

interventions for students who struggle in core instruction (National High School Center,

National Center on Response to Intervention, and Center on Instruction, 2010). RTI provides

educators with systematic measures of student progress that yield data used to make important

educational decisions (Batsche et al., 2006) and provides states, districts, and schools with a

framework for allocating instructional services and resources in response to students‘ needs.

VanDerHeyden, Witt, and Gilbertson (2007) conducted an evaluation of an RTI model over

multiple years. The results of this comprehensive study indicated that application of the RTI

model reduced the number of students evaluated for special education services, essentially

eliminated the disproportional rate at which ethnic minority and male students were referred for

special education evaluations, and substantially reduced the amount of financial resources

dedicated to unnecessary special education evaluations. In light of this evidence, Oregon has

maintained a state-supported RTI initiative for the past 8 years with support from the leadership

of an exemplar school district (Tigard-Tualatin) whose organized efforts have trained 38 of the

state‘s 197 districts in these principles statewide.

Behavior: The tiered model for behavior is Positive Behavioral Interventions and Supports

(PBIS), a systems approach to discipline that emphasizes (a) prevention of problem behaviors by

providing proactive instruction of desired behavioral expectations, active reinforcement of


appropriate behavior, and monitoring and correction of problem behavior; (b) on-going

collection and use of data for decision-making; and (c) application of more intensive and

individualized support for students who do not respond to prevention efforts (Lewis & Sugai,

1999; Sugai & Horner, 2002). School-wide PBIS is currently implemented with local adaptations

in over 1,500 schools in 23 states. Schools implementing School-wide PBIS with fidelity report

20-60% reductions in office discipline referrals (Chapman & Hofweber, 2000; Colvin &

Fernandez, 2000; Horner & Sugai, 2000; Horner et al., in press; Lohrman-O‘Rourke et al., 2000;

Nakasato, 2000; Nersesian, Todd, Lehmann, & Watson, 2000; Sadler, 2000; Taylor-Greene &

Kartub, 2000), improved student satisfaction (Lewis-Palmer, Horner, Sugai, Eber, & Phillips,

2002), improved faculty/staff satisfaction (Taylor-Greene et al., 1997), and improved

administrator perceptions of school safety (Schneider, Walker & Sprague, 2000). Preliminary

results also indicate that effective behavioral systems melded with effective instruction are likely

to result in improved academic gains (Horner et al., in press; Kellam, Mayer, Rebock, Hawkins,

& Wesley, 1998). Since 2003, Oregon has supported varying models of PBIS implementation in

the state. These services have been provided by networks of Educational Service Districts

(ESDs) providing services to districts within their regions with support of a university-organized

group of researchers and practitioners (Northwest PBIS Network).

Common Core State Standards. On October 28, 2010 the State Board of Education

adopted the CCSS which represent K-12 learning expectations for students in English-language

arts and mathematics. In combination with the educational strategies promoted by the recently

adopted Oregon K-12 literacy Framework, the CCSS are intended to provide a foundation for

Oregon‘s Diploma and reflect the same high academic expectations for all students. ODE has

http://www.ode.state.or.us/search/results/?id=368


convened a steering committee of education leaders representing policy and higher education, to

ensure the appropriate implementation of the CCSS to educators statewide. The purpose of this

ODE governed committee is to provide policy guidance and the ongoing facilitation of a

statewide team of nominated education stakeholders who will be primarily responsible for

developing the implementation plan.

While the evidence-base of the newly adopted standards is in its early stages of being

gathered, the soundness of the standards‘ development to date provides initial support for their

validity. The Common Core state standards were designed by a selected group of educational

experts around the nation and subjected to significant rounds of review and feedback. The

implementation of these standards in 43 states most of which will start in the next year, will

generate a host of research studies designed to demonstrate the evidence-base of the standards.

The intent for the standards nationwide is that they are demonstrated as robust and relevant to the

real world, reflecting the knowledge and skills that students need for success in college and

careers.

Through Implementation Science and the presentations of Dr. Dean Fixsen, Oregon has

learned that activities developed in silos will ultimately remain siloed unless active steps are

taken to ensure that the work is integrated at the time of implementation. To this end, a critical

component of the current application is the assurance that implementation coordinators

responsible for training the state on RTI, PBIS, and EBISS, are also soundly trained on the

CCSS. In addition, implementation coordinators responsible for keeping parents of students with

disabilities informed on the activities undertaken by the state must also be trained on the CCSS.

And finally, all implementation coordinators must be trained on the principles of implementation


science to ensure that state-supported initiatives are implemented cohesively in support of a

whole child.

Implementation coordinators selected for this grant through Request for Proposals, or

through expertise as a sole source provider (many of whom have endorsed their support of this

application) will each contribute the content level expertise for these relevant trainings,

however, because the information presented will range from new to old, the same approaches to

adult learning embracing all 6 of the adult learning practices (introduce, illustrate, practice,

evaluate, reflect, and master) will be utilized in support of any trainings developed or provided

(Trivette, Dunst, Hamby, and O‘Herin, 2009).

The CCSS provide the content for instruction. The standards establish what students need

to learn, but they do not dictate how teachers should teach. With technical assistance from SISEP

and the evidence-based professional development practices ensuring successful adult learning

outcomes, the work of this grant will be able to develop a sound infrastructure of connections

between and among the expertise of RTI, CCSS, and other instructional strategies espoused by

the state such as those presented in the state‘s K-12 Literacy Framework. This approach will

ensure that content delivery for state trainings is sound and functionally integrated for end-users

(educators) in the field.

OBJECTIVE 2: Increase the capacity of Districts already versed in Action Planning to

develop and sustain a three-tiered model of behavior and academic support in areas not

previously targeted but identified as deficient by district leadership (Depth).

Although the EBISS initiative has improved the level at which schools in participating

districts implement a continuum of effective educational interventions and behavioral supports to


children and students with disabilities, many of the districts focused on either academics or

behavior, and limited district implementation to a constricted number of grade levels (i.e.

Elementary Schools). Feedback from district EBISS coaches, data analysis, and through new

implementation knowledge a pattern of identified next steps emerged related to increasing the

depth of EBISS implementation. In the current grant, we would provide personnel development

support to our 28 previously supported districts according action planning needs, to increase

depth of service to improve activities and services implemented in the prior five years. Training

for these personnel would focus on adult learning areas of reflection and mastery. Areas of depth

would include:

1. Extending levels of EBISS implementation to Pre-K or Secondary

2. Extending Implementation to strengthen activities in either RTI or PBIS.

3. Extending Implementation to enhance services for target populations (e.g., ELL).

4. Develop district procedures for using RTI processes for LD Eligibility.

5. Analysis and integration of Scaling Up Implementation Drivers.

Confidence in furthering depths for previously supported districts is based on successful

outcomes during the implementation of the 2006 SPDG. During the first two years of the 2006

SPDG EBISS state coordinators developed and refined a district self assessment tool called the

District Systems Support Plan or DSSP (see Appendix C). Shortly thereafter, further

modifications were made to develop a parallel tool for early childhood applications the

Interagency Systems Support Plan (ISSP) (See Appendix C). These tools allow district and

programs to identify, implement, and sustain essential features of effective systems-level


academic and behavior support. Essential implementation features of effective systems-level

academic and behavior support are operationalized in the DSSP:

district leadership and commitment,

action planning with EBISS schools,

coordination and coaching,

professional development and training capacity,

on-going assessment and evaluation systems,

visibility and political support, and

funding

Scores on these features from 2010-2011 show that across the seven key components,

district implementation averaged from 61% to 80%. The overall implementation average of these

components across all districts ranged from 22% to 99% reflecting no change from the previous

year, but reflecting an approximately 15% improvement over scores from 2008-2009.

OBJECTIVE 3: Increase the number of districts not previously engaged in state-supported

implementation activities in the use of three-tiered models of behavior and academic

support by using principles of implementation science to take action-planning awareness

for behavioral and academic strategies to all districts (Breadth).

Because the concept and evidence related to applying an implementation framework in the

field of education is not widely known and is in the process of being documented, we will

provide professional development to all districts across the state (197) on how to successfully

implement and scale programs so that every student can benefit will be a critical component of


this grant. In order to share the principles of implementation that support efficient practices,

participating districts for this objective will be districts who did not receive technical assistance

as a part of the 2006 SPDG. To alleviate geographical inefficiencies, these districts will be

trained annually by zone, so that appropriate adjustments can be made for the travel associated

with training. Districts in regions prone to weather-related travel delays, will be scheduled to to

minimize challenges.

Individuals attending these trainings will have a highest need for trainings ranging from

introduction and illustration to planning and application and will be designed to introduce and

clarify the concepts around building a district infrastructure for sustainability, action-planning,

and implementation using the example of an EBISS model. Scaling consists of developing a

teaming structure within state system to support the full and effective uses of evidence-based

approaches to education for all students. Implementation capacity across all districts in the state

is necessary to assure that each teacher and staff person is ready, willing, and able to provide the

most powerful and efficient education practices every day for every student.

OBJECTIVE 4: Increase the number of highly-qualified special education specialists in

Oregon by providing certification incentives to decrease state personnel deficits relative to

special education.

The fourth component of Oregon‘s State Personnel Development Grant (SPDG) is focused

on increasing the number of highly-qualified specialists who are working in Oregon schools. In

Oregon‘s 2006 SPDG, this goal was successfully accomplished through the recruitment of

Oregon school district employees who agreed to complete university-level coursework that

would result in a degree and licensure. Through this ―grow your own‖ concept, districts focused

http://sisep.fpg.unc.edu/glossary/term/24


on retention of current employees with proven records to become highly-qualified special

education teachers, administrators, and other service providers.Through the new SPDG, Oregon

proposes to continue offering scholarships for speech language pathologists by developing

additional cohorts of scholarship recipients. Scholarships are awarded to individual school

districts on behalf of the specific scholarship recipient who is selected for the degree program.

To provide the coursework required for the speech language pathology cohort program and

licensure, five years ago, ODE and Oregon partners developed a hybrid SLP Cohort program

with Nova Southeastern University in Florida, this model will serve as the basis for future

cohorts. Districts and scholarship applicants apply for admission into the Oregon SLP Cohort

through ODE. In a parallel application, the educator applies for admission to Nova Southeastern

University. Once both applications are approved, the individual is admitted to the Oregon Speech

Language Pathologist Cohort. Coursework is presented online and also by Nova on-site in

Oregon twice per term. During this three-year program, aspiring SLPs will continue to work full-

time in their districts, complete internship and externship placements, and finish a clinical

fellowship year in their school districts. After licensure, scholarship recipients will remain in

their districts for a minimum of two years. However, in the past, many districts required their

scholarship recipients to remain for as many as five years post-licensure, depending on the level

of additional financial support supplied by the district itself.

For administration purposes, four times per year, ODE would maintain contact with the

scholarship recipients, checking on their progress in their respective programs and monitoring

the support that people are receiving from their districts and degree program staff. Through this


continuous relationship with ODE, recipients‘ requests for additional support and problem-

solving expertise are met. Retention rates have been high.

Across eight cohorts supported 92 of the 95 individuals (97%) are currently still working in

Oregon schools. Of the 66 individuals working toward licensure, as of this date 56 (85%) have

earned the intended licensure. All SLP cohorts to date have received scholarships through both

federal and state funding sources.

In summary: Across the four objectives identified for the accomplishment of this work, the

likelihood that the work will be successful is based on data and experiences from prior

implementation; our reliance on an evidence-base for all programs selected for implementation;

and our confidence in the practices promoted as a result of our technical assistance from SISEP.

QUALITY OF THE PROJECT DESIGN

The goal of Oregon‘s State Personnel Development Grant (SPDG) is to improve the capacity of

educators in the state to respond to the differentiated educational needs of all students using

evidence-based reading and behavior instructional strategies to ensure that adequate resources

are available to students with the greatest need for educational support. As noted, this outcome is

proposed by pursuing four objectives:

Objective 1: Increase the capacity of state-supported implementation coordinators who are

responsible for statewide professional development on evidence-based practices to convey the

interconnectedness of state endorsed initiatives (CCSS, RTI, PBIS) to develop valid evidence-

based materials for district-use in tiered decision-making.


Method: Ensure that all providers of technical assistance funded or partnered by IDEA

initiatives have access to training and resources around the implementation of Oregon‘s

adopted Common core standards. This work is also submitted here in fulfillment of our

intent to meet the grant‘s Competitive Priority expectation

Objective 2: Increase the capacity of districts already versed in action planning to develop and

sustain a three-tiered model of behavior and academic support in areas not previously targeted

but identified as deficient by district leadership.

Method: Increase the depth of previous district activities by using guided design

(Trivette, Dunst, Hamby, and O‘Herin) to train district personnel to use the principles of

Implementation Science to infuse specific improvements into areas of need.

Objective 3: Increase the number of districts not previously engaged in state-supported

implementation activities in the use of three-tiered models of behavior and academic support by

using principles of implementation science to take action-planning awareness for behavioral and

academic strategies to all districts.

Method: Increase breadth by using coaching and guided design (Trivette, Dunst, Hamby,

and O‘Herin) to train personnel in districts beyond those reached in previous plans to use

the principles of Implementation Science to prepare systems for the implementation of

evidence-based practices.

Objective 4: Increase the number of highly qualified special education specialists in Oregon – by

providing certification incentives to decrease state personnel deficits relative to special

education.


Method: Impact specialized areas in need by recruiting and train professionals in the

most under-filled special education areas as identified by the state.

(i) The extent to which the goals, objectives, and outcomes to be achieved by the proposed

project are clearly specified and measurable.

The activities in each of these focus areas are propelled by objectives and goals that are

measureable on multiple levels. Within objective 1 (training on Common Core), successes of

outcomes will be measured by evaluating learner outcomes including the evaluation of materials

developed for use in initiative-specific trainings.

Within the objectives 2 and 3 (depth and breadth), successes of outcomes will be measured using

variables such as:

The number of districts/individuals impacted by the Professional Development activity

The number of action planning goals developed, addressed, or achieved

The specific outcomes on action planning goals

Evaluation of learner outcomes

SISEP supported implementation activities in Oregon to date have also generated several

outcome measures that will be incorporated into this work (see Evaluation Section) to measure

success on implementation activities toward both depth and breadth and include:

A district capacity assessment

A regional capacity assessment

A state-level capacity assessments

A Stage-Based Measure of Implementation


The function of these assessments is to determine the degree to which infrastructure supports at

each level are in place, and serve as Action Planning tools. For the purpose of our work, these

supports are referred to as ―Drivers‖ and include staff selection, training, coaching, performance

assessments, systems intervention, facilitative administration and decision support data systems.

In each level of the capacity assessments listed above, all Drivers are analyzed in relation to the

other to ensure an integrated system for infrastructure building and sustainability. A more

detailed description of the Drivers and their role in evaluating activities related to our application

can be found in the Evaluation section of this proposal.

Within the focus area related to certification successes of outcomes will be measured in variety

of ways. We will collect data related to:

The number of SLP scholarship recipients

The number and percentage of recipients that completed training within a specified time

The number and percentage of recipients still working in Oregon school districts at the

end of the grant.

* Although change in student outcomes will be an outcome that is monitored in this work,

change in student outcomes is not considered a direct outcome measure for any objective.

(ii) The extent to which the design of the proposed project is appropriate to, and will

successfully address, the needs of the target population or other identified needs.

Because Oregon‘s Personnel Development Goal is to enhance state and district implementation

capacity toward improved student outcome, the target population intended for this grant is

educational personnel with responsibilities for student outcomes through systems improvements.


Target personnel are intended to be both general and special educators and administrators.

However, roles vary according to each of the four focus areas specified by this grant. Students

ultimately impacted by this work will represent a broad range of needs ranging from specific

needs in communication and learning disabilities (supported by SLPs and districts using RTI to

determine eligibility) to all students.

Objective 1 (common core):

The primary personnel targeted by weaving a Common Core focus into this work will be

university personnel, district personnel, and expert consultants who are, have been, or will

potentially be contracted by the state to provide professional development on the implementation

of evidence-based practices to district personnel across the state. These individuals (whose

qualifications and credentials are provided in the Adequacy of Personnel section) are considered

―experts‖ in their areas and are responsible for either conducting original research or are

otherwise nationally recognized. These individuals are accustomed to accessing training for their

personnel to maintain currency with research and training content related to evidence-based

practices and associated professional development, however, they stand to be on the outside of a

state-hosted roll-out of Common Core implementation activities. The funds from this grant will

ensure that these individuals have access to the ongoing professional development and resources

related to statewide implementation of the Common Core standards. Aside from registering to

attend as participants at the national CCSSO initiative, at http://spaces.ccsso.org, in order to meet

the needs of this group for deep understanding of this material, access will be provided to state-

level activities and will include inclusion in state-supported implementation activities.

The table below provides a framework for providing training on the Common Core.

https://owa.oregoneesc.org/owa/redir.aspx?C=ae2bc025da144de2be1c9afb495856c8&URL=http%3a%2f%2fspaces.ccsso.org


Common Core Content for Initiative Leaders

Target Audience Implementation Coordinators

Content of live trainings (2 live

trainings)

Familiarity with CCSS expectations.

Familiarity with CCSS resources.

Outcome of live trainings Fluency in CCSS

Measure of success Ability to utilize knowledge to inform evidence-based

practice application

Content of pre-scheduled web

sessions (2 web-based seminars)

Membership, function and capacity development

Introduction to Scaling concepts

Outcome of pre-scheduled Web-

based seminars

Familiarity with purpose of the group

Familiarity with Scaling concepts

Measure of Success Ability to inform stakeholders and other initiative

personnel on the function of the (consortium)

Trainings listed in the table include two ―live‖ trainings and two pre-scheduled web-based

seminars. The live trainings will focus on the Common Core State Standards and will develop

participants‘ facility with the content, as well as provide time for group activities for application

to the specific contexts that the participants are representing. The sessions will allow attendees to


develop knowledge as well as the professional relationships that permit connections among the

state initiatives. The pre-scheduled web-based seminars will extend the learning from the on-site

sessions by providing a variety of contexts for the application of the ―live training‖ content and

provide enhanced activities to analyze successes, barriers and challenges to the integration of

multiple initiatives. The trainings will include research based professional development

techniques that we have had an opportunity to practice through our work with SISEP. These

include activities such as linking new information on the implementation of CCSS to their own

experiences to increase relevance, applying newly learned CCSS concepts to the familiar context

of their representative innovation, as well as providing opportunities for self-assessment and

reflection so integration of new knowledge to existing, known frameworks can occur (Donnvan,

Bransford and Palligrino, 1999). In accordance with best practices in Professional Development,

evaluation will include: Participant Reaction, Participant Learning, Organizational Support and

Learning, Participant Use of New Knowledge and Skills, and Student Learning Outcomes

(Guskey 2002).

Objective 2: Increasing Depth

The primary target group for improving the depth and breadth of implementation activities

across all of Oregon‘s 197 districts over the 5 years of this grant will be district personnel.

Districts that have been involved in receiving technical assistance on the implementation of

evidence-based practices in academics and behavior through the 2006 SPDG and who are

therefore moving into greater depths with the activities proposed in this grant already have

organized teams of individuals who will be targeted for this enhanced professional development.

These teams consist of the Special education director, Curriculum director, Title 1 director, and


the district literacy and behavior coaches; all will benefit from the professional development

proposed by these grant activities. The needs of these individuals have been defined by their

existing Action plans. DSSP data from 2010-2011 show that across the seven key components

district implementation averaged from 61% to 80% in the areas of Leadership & Commitment

(75%), Action Planning (65%), Coordination & Coaching (80%), Professional Development

(80%), Assessment & Evaluation (72%), Visibility & Stakeholder Support (61%) and funding

(63%). Overall, scores from 2008 to 2010 increased from 57% implementation to 72%

implementation. (See Appendix D for additional data). In addition to these data, state

implementation coordinators have identified five general areas of district deficit that will be

specifically addressed via this professional development application: 1. Extending levels of

EBISS implementation to Pre-K or Secondary; 2. Extending Implementation to strengthen

activities in either RTI or PBIS; 3. Extending Implementation to enhance services for target

populations (e.g., ELL); 4. Develop district procedures for using RTI processes for LD

Eligibility; 5. Analysis and integration of Implementation Drivers.

To meet the needs of this group of personnel for a deep understanding of decision-making in

action planning, these trainings will be designed with embedded team-time for individuals to

work as a member of their district team and to reflect on the sources of the deficits they

identified as a need in their districts by engaging in critical self-assessment activities as well as

self-assessment techniques that will enable then to address scaling concepts with the context of

the chosen EBP.

The trainings listed in the following table include three ―live‖ trainings and a total of 15 pre-

scheduled web-based seminars. Both the live and web-based seminars will focus on the


integration of initiative-specific, and scaling concepts. Each district will also choose one area of

―depth‖. Each of the five ―depth‖ topics has three associated web-based seminars. The web-

based seminars and will more fully develop participant‘s facility with the content, as well as

provide scenarios for group activities for application to the specific District contexts that the

participants are representing. The pre-scheduled web-based seminars will extend the learning

from the on-site sessions by providing a variety of contexts for the application of the ―live

training‖ content and provide enhanced activities to analyze successes, barriers and challenges to

the integration of activities related to enhancing depth of implementation.

Increasing Depth

Target Audience Districts who have previously completed action

plans (EBISS)

Content of live trainings (3 live trainings) Identify areas in need of more intensive

implementation strategies. Facilitated

discussions toward areas of deficit.

Outcome of live trainings Knowledge of areas of deficit, Action planning,

use of data, identification of barriers, selection of

evidence-based practices

Cross-team problem solving

Measure of success As identified by district in collaboration with

implementation coordinator.


Content of 15 pre-scheduled web sessions Specific to one of five topics of depth

Outcome of 15 pre-scheduled web sessions Action planning for increased depth of services

Measure of Success As identified by district in collaboration with

implementation coordinator based on action

planning goals.

Content of 5 requested Web-based sessions Specific to one of five topics of depth

Outcome of 5 requested Web-based

sessions

Trouble shooting or additional depth of

implementation

Measure of Success As identified by district in collaboration with

implementation coordinator

The guided-design trainings will include research based professional development

techniques, including activities such as linking new information on the elements of ―deep‖

implementation to their own experiences to increase relevance, applying newly learned concepts

to the familiar context of their current implementation, and providing opportunities for self-

assessment and reflection so integration of new knowledge to existing, known frameworks can

occur (Donnvan, Bransford and Palligrino, 1999). In accordance with best practices in

Professional Development, evaluation will include: Participant Reaction, Participant Learning,

Organizational Support and Learning, Participant Use of New Knowledge and Skills, and

Student Learning Outcomes (Guskey 2002).


Objective 3: Increasing Breadth

Districts who did not receive technical assistance as a part of the 2006 SPDG will be

integrated into the grant activities beginning in year 2. Each year, we will invite approximately

50 new Districts (by regional zone) to participate so that by the end of the grant, every district n

the state will have participated in sustained guided design trainings on the topic of

implementation. Each District will be encouraged to bring at a minimum a similar team of

individuals as the ―Depth‖ districts to participate in the activities designed to generate their

awareness of implementation activities.

To meet the needs of these individuals for skills around planning and/or application, these

trainings will be designed to coach on and guide trainees through introductory concepts of

action-planning for sustainability, and will use concepts specific to the combined behavioral and

academic model known as EBISS to illustrate the application of action-planning for scaling and

sustainability in districts.

The trainings listed in the following table include three ―live‖ trainings and a total of 15

pre-scheduled web-based seminars. Both the live and web-based seminars will focus on the

concepts related to Scaling and Implementation Science (Fixsen, Naoom, Blase, Friedman, &

Wallace, 2005). The three live trainings will immerse the participants in the concepts related to

the broader Oregon Scaling Up plan, and build knowledge of the integrated initiatives that the

state is sponsoring. The 15 pre-scheduled web-based seminars include one general overview of

Implementation Science and the Scaling Up work being conducted in Oregon. Of the remaining

web-based seminars, two each will be devoted to the seven Drivers. These trainings will also be

presented according to the research based professional development techniques.


Increasing Breadth

Target Audience Districts who have not previously completed

action plans

Content of 3 live trainings Awareness of evidence based practices and

resources, decision-making, action-planning,

self-assessment

Outcome of 3 live trainings Facility with principles of action planning

Measure of success Use of action planning for Evidence Based

practices in district

Content of 15 web-based seminars Specific to one of seven drivers

Outcome of 15 web-based seminars Action planning

Measure of Success Use of action planning in district

Content of 5 web-based seminars Specific to one of seven drivers

Outcome of 5 web-based seminars Trouble shooting

Objective 4: Licensure and certification

The fourth component of Oregon‘s State Personnel Development Grant (SPDG) is focused on

increasing the number of highly qualified specialists who are working in Oregon schools. In-


service training systems are supported with district scholarships to ensure that additional

personnel are prepared annually to enter areas of the field with the greatest personnel deficits.

The design of this program will be consistent with the expectations of the program of study

selected to provide this training.

(iii) The extent to which the proposed activities constitute a coherent, sustained program of

training in the field.

All trainings will be presented to the field as part of a cohesive system of training. Individuals

will be evaluated on outcome data, indicated by Guskey‘s Five Levels of Evaluation (2002) in

each of the aforementioned sections, as well as the action plans they develop, to ultimately

move into the deeper understanding strands being presented to the EBISS districts. Once all

districts are trained in action-planning, the use of data, and have had some experience with the

deeper understanding of action-planning to a specific area of deficit, districts will be left with a

self-sustaining model of decision-making with which to tackle future areas of deficit and to

address and adjust based on data and staff changes. The logic model of the grant activities is

included in Appendix A. The model gives a clear depiction of the activities proposed by this

grant as a single plan.

To carry out a plan of this magnitude with an expectation for sustainability, the state needs to be

prepared with an infrastructural system that supports the activities occurring in the field. The

state needs to be prepared to address and respond to issues that we encounter or create during the

implementation of well-intended activities. To accomplish this, the implementation activities

supported by Dr. Dean Fixsen et al at NIRN provide support for and guidance around the

formation of five distinct entities that each have a role in the successful implementation of this


work. The teams are: The state management team, The State Transformation Team, State

Transformation Specialists, the Regional Implementation Team, and the District Leadership

team.

State Management Team (SMT)

The membership of the SMT includes the Chief State School Officer and his or her

directors of general education, special education, information technology, school improvement,

finance and administration and others accountable for student outcomes at the state level. The

State Management Team‘s role in capacity development is in working closely with the State

Implementation and Scaling up Evidence‐based Practices (SISEP) center to develop knowledge,

skills, and abilities related to increasing implementation capacity and scaling up evidence‐based

practices (EBP) statewide. The SMT will also work closely with the State Transformation

Specialists (STS) and the State Transformation Team (STT) to promote alignment of systems as

major decisions on the scale‐up of evidence‐based practices.

State Transformation Team (STT)

The membership of the State Transformation Team (STT) will include at least two State

Transformation Specialists (STS), selected members of the Regional Implementation Team

(RIT) and representatives from the State Management Team (SMT). The State Transformation

Team‘s role in capacity development is in ensuring that the Regional Implementation Team‘s

capacity and competency is built, sustained, and improved through the State Transformation

Team‘s efforts. The STT is also accountable for facilitating the communication among and

between the Districts, the RITs and the SMT. By building state capacity to effectively align


systems and policies, the STT helps to ensure that a hospitable environment is created along with

a sustainable and improving infrastructure.

State Transformation Specialists (STS)

Two State Transformation Specialists (STS) will represent general education and special

education, and will work closely with the State Implementation and Scaling‐up of

Evidence‐based Practices (SISEP) Center to develop their knowledge, skills, and abilities related

to implementation and scaling‐up. In terms of capacity development, the STSs provide two

critical overarching functions in developing implementation and scaling‐up capacity for EPBs:

Purposeful, multi‐level communication to promote problem‐solving and alignment and

leadership in developing the capacity to help assure the success of the overall implementation

infrastructure development in the State. They train, coach and assess RIT members to assure

their success and Initiate and serve as core members of the State Transformation Team

Regional Implementation Team (RIT)

The Regional Implementation Team (RIT) will consist of three to five members who are

currently working in, or are familiar with, the districts they will support. The RIT will receive

training and support from the STSs to acquire expertise in the following areas: Implementation

requirements and content knowledge related to identified evidence‐based practices (EBP) (using

the EBISS Systems Coach manual Appendix E); Implementation science and best practices;

Improvement cycle processes and organization change strategies and Systems transformation

approaches. In terms of building systems capacity, the RITs develop the implementation capacity

in Districts in order to make full and effective use of one or more EBP. Regional Implementation


Teams also are a conduit for information that needs to flow back to the State Management Team

(SMT) and the State Transformation Team (STT) so that more hospitable environments (e.g.

policy, funding, regulations, etc.) are created to promote and sustain EBP use. To build

implementation capacity and facilitate alignment, the RITs engage in a wide variety of activities

focused on: Implementation best practices, organization change strategies and system

transformation approaches.

District Leadership and Implementation Team (DLIT)

The District Leadership and Implementation Team (DLIT) will include members

currently working in the districts. Members will include the district superintendant and lead

district and building administrators. Through the support of its Regional Implementation Team

(RIT), the DLIT will develop implementation capacity by learning and applying the following

critical elements: Selecting evidence‐based practices (EBP) and/or evidence‐informed

innovations (EII) and understanding their core features; understanding the current infrastructure

and available resources to support the District Improvement planning process; Implementation

science and practice and Improvement cycles. The key function of the members of the DLIT is to

ensure that implementation capacity is developed at the school level in all schools, and that

collectively the schools and District build the infrastructure needed for high fidelity

implementation. DLIT would be accountable for assisting schools in developing their Building

Leadership and Implementation Teams (BLIT) that would serve the function of ensuring high

fidelity implementation of the selected EBPs and/or EIIS resulting in improved student

outcomes.


The current project‘s professional development plan incorporate best practice adult learning

strategies that include intensive and sustained practice with inclusion of on-going technical

assistance designed to develop Initiative and District leader‘s skills in the key components of the

CCSS, RTI and Implementation Science. Because we will ultimately train each district in the

State, technology will be used to ensure that Professional Development for districts in diverse

geographical regions is consistent with the training all other sites receive. Many districts are

more than 100 miles away from the three main cities where trainings are traditionally held.

School districts participating in the project and all districts in the state will have access to Web-

based information to continue their learning. This grant will help to fund the creation of

technology-based training, allowing a mode of dissemination that will be available to all districts

in the state for training after the grant is completed.

Objective 4: Increasing personnel in specialized areas

Through the new SPDG, Oregon proposes to continue offering scholarships. The table below

provides a proposed timeline for recruitment.

Proposed Timeline for Oregon SPDG Scholarship Program 2012 – 2016

Personnel Deficit Program Year Implementation

9/2012

SLP Cohort IV Masters degree through NOVA

Southeastern University

2012 - 2016 Pursue partnerships with Oregon institutions of


SLP

higher education

3/2014

SLP Cohort V Masters degree through Oregon

institution of higher education

9/2015

SLP Cohort VI Masters degree through Oregon

institution of higher education

Through Oregon‘s current SPDG, the Oregon Department of Education (ODE) has

developed partnerships with school districts and higher education in order to administer the work

of recruitment and retention. Oregon proposes to continue with this same model, as described

below. In exchange for the scholarship funding, a district employee signs a formal agreement,

pledging to remain in the district throughout the duration of the degree program and for a number

of years beyond licensure, based on the formalized agreement between the individual and the

district, and approved by ODE.

Higher Education

For the speech language pathology cohort program and licensure, the model has been

developed because of the severe shortage of higher education programs in SLP within the state

university system and the nation. Therefore, five years ago, ODE and Oregon partners

developed a hybrid SLP Cohort program with Nova Southeastern University in Florida. Districts

and scholarship applicants apply for admission into the Oregon SLP Cohort through ODE.

Coursework is presented online and also by Nova on-site in Oregon twice per term. During this


three-year program, aspiring SLPs continue to work full-time in their districts, complete

internship and externship placements, and finish a clinical fellowship year in their school

districts.

(iv) The extent to which the design of the proposed project reflects up-to-date knowledge

from research and effective practice.

Embedded in this proposal are researched based Professional Development techniques that

address both session activities as well as best practice approaches to evaluation of professional

development sessions.

(v) The extent to which the proposed project will establish linkages with other appropriate

agencies and organizations providing services to the target population.

The state of Oregon enjoys partnerships with a variety of professional entities responsible for

serving educators across the state. Each of these entities has been engaged in implementation or

dissemination activities with the state over the course of the 2006 SPDG.

The Oregon Parent Training Institute has been a partner of the Oregon Department of Education

for over 10 years. Oregon PTI is a statewide parent training and information center that serves

parents of children with disabilities by educating and supporting parents, families and

professionals and by building partnerships that meet the needs of children and youth with the full

range of disabilities, ages birth to twenty-six. ORPTI follows a regionalized model that has staff

living in the area they serve.

The following linkages demonstrate collaborative efforts between the Office of Student Learning

and Partnerships (OSL&P) and its partners:


School Improvement (Continuous Improvement Planning): Collaborative effort with Title I,

Office of Student Learning and Partnerships, and Office of Educational Improvement and

Innovation on personnel development and school improvement activities.

Common Core State Standards: Collaborative effort with Common Core Stewardship team,

Title I, Office of Student Learning and Partnerships, and Office of Educational Improvement and

Innovation on personnel development and school improvement activities.

Scaling Up: Collaborative effort with Office of Student Learning and Partnerships, ODE State

Management Team and University of North Carolina‘s State Implementation & Scaling Up of

Evidence-Based Practices (SISEP) center.

Reading: Collaborative effort between Center for Teaching and Learning, Title I, the Office of

Student Learning and Partnerships, IHE‘s, State Interagency Coordinating Council

Middle and High School Reading: Tigard-Tualatin School District

Response to Intervention: Collaborative effort with Reading First, Title I-A, Tigard-Tualatin

School District, OR-PTI

Positive Behavior Supports: Collaboration with the PBIS Statewide Network, IHEs, and the

Department of Human Services

Individualized Behavior Supports: Collaboration with University of Oregon and the PBS

Statewide Network and Early Childhood Social Emotional Work Group.

Recruitment and Retention: Collaboration between Teacher Standards and Practices

Commission, IHEs, ODE, and the Teaching Research Institute R& Retention Project

Early Childhood: Head Start, EI-ECSE, Even Start and Early Head Start programs in the State,

the University of Oregon (PBS training), Western Oregon University, Portland State University


(the Oregon EC Registry); Southwestern Oregon Community College (lower division overlay

credit), and the Governor‘s Office.

(vi) The extent to which the proposed project is part of a comprehensive effort to improve

teaching and learning and support rigorous academic standards for students.

Oregon has been taking strides to improve student outcomes via a variety of methods across a

range of state and local entities. These activities came about in gradual response to concerns that

Oregon students were in need of more rigorous instruction and expectations in order to keep pace

with global expectations. According to overall results on the National Assessment of Educational

Progress (NAEP), the average score of fourth-grade students in Oregon in 1998 was 212, but

following incremental annual gains, 11 years later scores had only gone up by 6 points to 218.

These numbers indicate that from 2007 to 2009, our strategies for educating students in Oregon

either did not change, or that the benefit of a shift in strategies produced no discernable results.

This is even more evident when we note the scores of students At Basic, Below Basic and

Proficient or Advanced:

In 1998, the percentage of students scoring at Below Basic was 39%; in 2009 the

percentage of students scoring at Below Basic was 34%

In 1998, the percentage of students scoring at Basic was 33%; in 2009 the percentage of

students scoring at Basic was 34%;

In 1998, the percentage of students scoring at Proficient was 23%; in 2009 the percentage

of students scoring at Proficient was 24%

In 1998, the percentage of students scoring at Advanced was 5%; in 2009 the percentage

of students scoring at Advanced was 7%


[http://nces.ed.gov/nationsreportcard/pdf/stt2009/2010460OR4.pdf]

In order to address this lack of improvement, Oregon has taken great strides to ensure that

rigorous academic expectations are considered the norm for all students as evidenced by the

following activities:

Oregon adopted more rigorous math (2010) and science (2009) content standards that

resulted in increased achievement standards.

On October 28, 2010 the State Board of Education adopted the Common Core State

Standards (CCSS) for English-language arts and mathematics.

The Oregon State Board of Education also adopted the Oregon K-12 Literacy Framework

in December 2009, as a tool for the state, districts, and schools to support the Essential

Skill of Reading, a requirement of the Oregon Diploma. These essential skills are process

skills that cross academic disciplines and are embedded in the content standards.

In January of 2007, the State Board of Education voted to adopt new high school

graduation requirements. These new requirements are designed to better prepare each

student for success in college, work, and citizenship. A phase-in schedule (2007 – 2014)

has been created to allow students, families, schools and teachers to adequately prepare to

meet these new requirements.

Oregon Department of Education (ODE) has been engaged in CCSS work since the

initiative‘s inception in spring 2009. ODE staff reviewed each iteration of the standards and

provided feedback throughout the development process. In alignment with Oregon‘s

commitment to the Common Core State Standards, Oregon is also a lead state in the SMARTER

BALANCE Consortium (SBAC) which is a national effort toward the development of a

http://www.ode.state.or.us/search/results/?id=368


nationally oriented assessment based on the Common Core standards and designed to assure

students are assessed on nationally benchmarked standards that promote international

competition.

Summary: The design for Oregon‘s work on the 2011 SPDG application is based on a strong

operational objectives, a strong sequential design based on research and practice, invested

partnerships, frequent evaluation and measurement, and a cohesive plan of professional

development intended to impact systems for multiple populations.

QUALITY OF PERSONNEL

The Oregon Department of Education adheres to Oregon‘s Administrative Rules (OAR 105-

040-0001), which outline policy on equal employment opportunity and affirmative action in

keeping with federal Equal Opportunity Employment regulations. These regulations encourage

the submission of employment applications from individuals who are from traditionally

underrepresented groups. All of the partners included in this application are subject to the same

administrative rules and are monitored for compliance with these regulations. Partners from

Oregon institutions of higher education represented in this application employ rigorous

standards for non-discriminatory hiring including targeting associations and conferences that

represent diverse stakeholders to distribute announcements regarding new positions.

The State‘s efforts exceed those that are simply non-discriminatory and are approached

proactively and purposefully to ensure the adequate representation of underrepresented groups.

All job announcements contain the affirmative action/equal employment opportunity statement.

There is active recruitment of individuals with disabilities. In addition, diverse representation on


interview panels increases the likelihood diverse applicants will be interested in new positions.

Non-discriminatory practices are in place to protect all personnel at ODE. Resumes of Key

Personnel are included in Appendix F of this application.

(i) The qualifications, including relevant training and experience, of key project personnel.

Nancy Latini, Ph.D., State Personnel Development Grant Director: Assistant Superintendent

for the Oregon Department of Education past eight years, Dr. Latini‘s role as the head of the

Office of Student Learning and Partnerships, include the management, and full oversight of the

Part B and Part C IDEA expectations for the state of Oregon. She has worked in the field of

special education for over 30 years as a district administrator, teacher trainer, and as a classroom

teacher.

Dianna Carrizales-Engelmann, Ph.D., State Personnel Development Grant Director: Dr.

Carrizales-Engelmann also holds a Master‘s degree in Special Education. Dianna is currently

director of the Monitoring, Systems, and Outcomes unit activities for three of the 5 years in the

Office of Student Learning and Partnership. In this role, she provides active oversight and

guidance to state IDEA-funded activities, and oversees state monitoring activities for Part B.

Dianna previously held a School Psychologist license in the state of Oregon.

Martha Buenrostro, Ph.D., State Personnel Development Grant Coordinator: Dr. Buenrostro

has managed the programmatic components of the SPDG since she joined the department in

October 2010. Martha has been licensed as a School Psychologist in two states, and worked as

a school psychologist in two Oregon School districts prior to her position at ODE. Martha is


currently licensed as a School Psychologist in the State of Oregon, and has held positions in the

field of social work and human services.

Catherine Heaton Grant Specialist. Catherine has extensive experience in early learning

transitions, specifically transitions into Kindergarten from pre-school and school readiness from

Kindergarten to first grade. Catherine has worked at the department serving in multiple areas in

the Office of Student Learning and Partnerships (OSLP) for almost nine years. Catherine is also

responsible for managing special education reporting for OSLP and has coordinated office

stakeholder activities for over 5 years.

(ii) The qualifications, including relevant training and experience, of project consultants

or subcontractors.

Dean Fixsen, Ph.D. Director SISEP Scaling Activities Dr. Fixsen is co-director, along with

Karen Blase, of the National Implementation Research Network and of the State

Implementation and Scaling up Evidence-based Practices Center. He has spent his career

developing and implementing evidence-based programs, initiating and managing change

processes, and working with others to improve the lives of children, families, and adults. In

addition to co-authoring over 100 publications, he has advised federal, state, and local

governments. Dean is a Senior Scientist at the Frank Porter Graham Child Development

Institute, University of North Carolina - Chapel Hill. Dr. Fixsen has been providing Technical

Assistance to the state of Oregon for three years.

Rob Horner, Ph.D., Advisor on Statewide Systems Change Activities: Dr. Horner is professor

of Special Education at the University of Oregon. Dr. Horner brings a 30-year history of


research, grants management and systems change efforts related to school reform and positive

behavior support to this project as its advisor on systems change. Dr. Horner has published over

150 professional papers and 6 texts. He has directed over $20 million dollars in federal grants,

and currently co-directs the OSEP Technical Assistance Center on Positive Behavioral

Interventions and Supports and OSEP Research and Demonstration Center on School-wide

Behavior Support.

Kimberly Ingram-West, Ph.D. State Transformation Specialist Kimberly currently serves as

one of two State Transformation Specialists for Oregon‘s Scaling Up activities. Kimberly has

previously managed SPDG activities as an Educational Specialist in the Office of Student

Learning and Partnerships. Kimberly‘s expertise with evidence-based practices is based on

multiple professional experiences over the course of her educational career and through her

background as a consultant with SISEP and as a PBIS coordinator.

Marick Tedesco, Ph.D. State Transformation Specialist. Marick currently serves as one of two

State Transformation Specialists for Oregon‘s Scaling Up activities. Marick has previously

served as a restructuring service provider for several schools in the State of Hawaii using

evidence-based instructional programs to restructure schools in need of improvements. Marick

has provided educational consulting for multiple entities over the course of her career.

Erin Chaparro Ph.D. Project Director for EBISS. Dr. Chaparro is a Research Associate at the

University of Oregon's Center on Teaching and Learning. Dr. Chaparro is currently the co-

principal investigator and project director of the Effective Behavior and Instructional

Schoolwide Systems (EBISS) program. Dr. Chaparro has worked with state departments of


education across the country (i.e. South Dakota, Utah, New Mexico, Arizona, Alaska,

Wyoming, and Colorado) on the topic of effective and explicit instruction and assessment for

at-risk learners. Dr. Chaparro has worked as a school psychologist, a special education teacher,

and a school counselor.

Chris Borgmeier, Ph.D. Is currently an Associate Professor at Portland State University in the

area of Special Education. Chris is actively involved in state-funded initiative activities and has

partnered with the state on the Coaches Task Force and as a member of the Northwest PBIS

Network. A prolific researcher, Chris recently received a federal award related to the highly

qualified status of special educators.

David Putnam, Ph.D. Dr Putnam is currently director of Oregon‘s Response to Intervention

technical assistance center (ORRTI). Dr. Putnam has led this work as a contractor for the state

of Oregon for over a year. David was previously an active School Psychologist and professor

and currently holds an Oregon license in School Psychology as well as an Initial

Administrator‘s license.

Lori Lynass Ed. D. Dr. Lynass is currently Executive Director of the Northwest PBIS

Network, a network organized to provide PBIS supports to schools and districts in the Pacific

Northwest by providing links to research and practice. Since 1998, Lori has served in a number

of educational leadership capacities across three states.

Janice Roberts, Executive Director, Oregon Parent Training and Information Systems: Ms.

Roberts has been the executive director of ORPTI for the last ten years. Janice is also the parent

of a child with disabilities and she will continue to ensure that the parent perspective is


represented in all activities of the project, and will oversee grant activities and participate in

project evaluation.

ADEQUACY OF RESOURCES

(i) The adequacy of support, including facilities, equipment, supplies, and other resources,

from the applicant organization or the lead applicant organization.

Oregon remains dedicated to leveraging all of our potential resources to ensure that highly

qualified educators are serving students with disabilities and that data-driven decision making

continues its trajectory of successes toward becoming a practical reality for the state. To this end

Oregon will be continue to combine and streamline the use of existing resources at the

Department of Education (including recently accrued intellectual capacity), with additional

resources from the State‘s university partners, technical assistance groups, and any additional

resources that we are awarded as a result of this grant application.

Intellectual Resources: Aside from the wealth of intellectual capital in the form of partners at

the University of Oregon, ODE also enjoys relationships and partnerships with individuals and

groups in the broader University system including the University of Oregon, Western Oregon

University and Portland State University. In addition to our ability to access local research on

evidence-based practices such as Response to Intervention (RTI, Gerald Tindal), Positive

Behavioral Instruction and Supports (PBIS, Rob Horner), Effective Behavioral and Instructional

Support Systems (EBISS, Scott Baker) from the University of Oregon, and Evidence-based

professional development practices (Chris Borgmeier) from Portland State University, Oregon‘s

reputation for openness to innovation and improvements through intentional change, has also

earned us the attention of universities at the national level. As a result of this attention, we now


enjoy a close working relationship with innovators at the University of North Carolina (UNC,

Dean Fixsen) whose expertise in implementation has become a valued state asset.

Dr. Dean Fixsen and colleagues have synthesized over ten years-worth of research related to the

―how to‖ elements of putting evidence-based practice into effect (Fixsen, Naoom, Blase,

Friedman, & Wallace, 2005). The term associated with this work is ―Implementation Science‖

and it provides large-scale entities such as state governments with the capacity to develop a

systematic framework for fulfilling, performing, carrying out, or putting good ideas into effect.

In the last three years, through regular seminars, trainings, and the intellectual exploration

associated with the occasional misstep, Oregon has honed its infrastructural resources to a

promising state of readiness for large-scale implementation. Dr. Dean Fixsen‘s monthly technical

assistance has resulted in capacity in several areas that we now consider valuable state resources.

1) Over the course of three years, Dr. Fixsen‘s group out of UNC--State Implementation for

Scaling of Evidence-based Programs (SISEP) has formally trained three State

Transformation Specialists (STSs) for Oregon. STSs are individuals (educational experts

in their own right) who are trained to approach improvement at a system‘s level by

developing and operating systems that allow the state to manipulate a specific set of

features (drivers) to achieve improvements in evidence-based practices. Following a

retirement, two of the three STSs now serve the state with the knowledge and skills

acquired through this science.

2) Dr. Fixsen provides monthly ―seminars‖ on the principles and expectations of

Implementation Science to Oregon‘s Management Team and selected state personnel

using implementation examples gathered from participating EBISS districts in the state.

The Management Team members (including the State Superintendent of Public


Instruction, Susan Castillo) use this opportunity to problem-solve the state implications of

these relevant district-level issues. This practice has become an embedded structural

element at the monthly meetings and has infused systems-change well beyond the

activities scheduled on the original 2006 SPDG (EBISS) implementation agenda.

3) Dr. Fixsen‘s SISEP team also meets monthly with 8 district representatives and three of

the state‘s primary ―initiative‖ leaders representing evidence based practices for PBIS,

RTI, and EBISS. These 11 representatives, referred to as a Regional Implementation

Team (RIT) based on the role they may ultimately play for the state, are routinely

included in the training and Scaling Up developments for the state and have in turn

become a valuable state implementation resource and vehicle for future implementation.

These human resources, the documentation that supports their activities, and the consensus

generated as a result of the state‘s new knowledge around implementation are supplemented by

a vast implementation resource repository. Drs. Dean Fixsen and Karen Blasé along with their

colleagues at the National Implementation Research Network have developed a repository of

training and research resources that states can access as they move through the stages of

implementation (see table below).

Stages of Implementation District Activities/Indicators

Exploration Stage Establishing the needs for change

Installation Stage Gathering resources

Initial Implementation Stage Begin training on the innovation


Full Implementation Stage Continued training, coaching and refinements

Innovation Stage Capitalizing on lessons learned through implementation

Sustainability Stage Building capacity

The NIRN Resource repository housed at http://www.fpg.unc.edu/~nirn/resources/default.cfm

allows entities fluent in implementation science (and individuals interested in learning more) to

access an array of tools (typically evaluation tools), critical questionnaires, stage-based

assessments, and protocols to facilitate discussions. Documents are available both in a final

stage, and as documents developed by other states to be refined and adapted for use by others.

Tools (a small sample of which are included in this application), are provided to assist states and

districts in their self-organization towards efficiency by providing action-planning templates

around the seven drivers (see table below).

Driver (Type) Activity

Recruitment and Selection

(Competency)

Evaluate methods for recruiting and selecting

practitioners with identified characteristics

Training (Competency) Train practitioners to use new approaches and new

skills.

Coaching (Competency) Trained advisor provides guidance specific to the

innovation

Performance Assessment Assess practitioner performance for fidelity to the

http://www.fpg.unc.edu/~nirn/resources/default.cfm


(Competency) construct

Decision Support Data

Systems (Organizational)

Use data to assess overall performance of the

organization

Facilitative Administration

(Organizational)

Leadership uses data to make decisions and maintain

focus

Systems Intervention

(Organizational)

Ensure the availability critical resources

Technical and Adaptive

(Leadership)

Create common vision, problem-solve

Technological Resources: Since Oregon applied for SPDG support in 2006, the Oregon

Department of Education as a whole has invested significant resources into developing its

already nationally renowned technical resources and capacity to include an integrated statewide

data system to enhance student and teacher data collection. In 2010, ODE applied for and

received a grant from the US Department of Education to create a single operational data ―store‖

capable of meeting the needs of researchers, policy-makers, parents, students, teachers,

principals, Local Educational Agency leaders, community members and other stakeholders. As a

result of work under this grant (Project ALDER) Oregon‘s existing capabilities are being

enhanced to ensure:


Pre-school to age 20 linkages including additional data such as attendance (including

pre-school and kindergarten data), diagnostic test data, and other specific data such as the

Child Find data elements required under IDEA.

Interoperability of systems to allow student information to be matched and consolidated

from multiple sources using specific unique identifiers.

Teacher student link that links teachers to course content and students.

Teacher licensure and preparation data expansion to link pre-service and in-service

training elements to staff collections to enrich the effectiveness of teacher education

programs.

Continuous improvement and timely reporting capacity will ensure explicit and valid

linkages between formative data and summative data to facilitate decision-making

Efforts to influence improvements in data quality throughout all phases of the data

life-cycle (from collection to reporting to auditing and archiving)

General improvements in data storage and retrieval for higher quality data reporting

In order to achieve these outcomes, the project will develop twelve (12) system capabilities

intended to address and improve system capacity. While all 12 system capabilities will impact

the objectives of the existing SPDG application, three of these elements link directly:The

capacity to communicate with higher education and to facilitate data-sharing about an individual

student, including education and career goals. Information will be shared by utilizing a system

used by colleges and university partners including schools and higher education in neighboring

Washington State.Student level test data that can be used to expand the state‘s use of data from

growth models to further inform instruction and to provide additional instructional information

by adding additional formative assessment data.


1) Teacher identifiers that allow communication around the qualification status of teachers

between the Oregon Department of Education and the Teacher Standards and Practices

Commission (TSPC) to assure fluidity of communication and reports to relevant

stakeholders.

In addition to these intellectual and technological resources, the educational field in Oregon is in

an unprecedented state of readiness due in large part to the (not always perfectly) coordinated

activities of several offices in the department all toward the end of training on data awareness.

Primarily, as a state the activities proposed in this grant will benefit greatly from the work of

Oregon‘s NCES funded DATA project (Direct Access to Achievement) which was promoted

statewide as a means of strengthening the structure of the longitudinal data system

through providing stakeholders with comprehensive training and informed access to educational

data and which has succeeded in greatly enhancing the field‘s awareness of and comfort with

data and by increasing their interest in pursuing further opportunities to self-educate on this

topic.

(ii) The relevance and demonstrated commitment of each partner in the proposed project

to the implementation and success of the project.

Aside from the technical capacity of the state the Oregon Department of Education continues

to work collaboratively with a variety of advisory committees that provide guidance and assist

with state level decision-making. Letters of support from multiple partners are included in

Appendix G. In spite of changes in state governance, both the State Interagency Coordinating

Council, (SICC) and the State Advisory Council for Special Education (SACSE) continue to

allow diverse stakeholders in the field of early childhood (Part C) to inform and interact with the

department on critical issues that impact children of all ages. The Oregon Parent Training and


Information Center (OrPTI) closely partners with the State to ensure that the components of

professional development provided by the State that are applicable to parents may be

disseminated effectively, reaching parents of students with disabilities throughout the state.

Letters of support from these entities for the state to continue its focus on and activities toward

Evidence-Based practices can be found in the appendices.

LEA Partners. OSL&P continues to have positive collaborative relationships with districts

across the state that have been further enhanced by a strong IDEA monitoring system. The

monitoring model used in Oregon is a web-based application that allows districts to self-evaluate

on their IDEA expectations and remain proactive and responsive to IDEA expectations. The

design of the system allows adult learners in the state to understand and respond to the

consequences of their decisions immediately.

ODE also has the support of specific districts that have benefited from the large-scale

implementation of evidence-based practices. Letters of support from the Tigard-Tualatin school

district (both a participating district in the previous SPDG and the contracted state provider for

RTI services), the Springfield school district, and a joint letter signed by leadership representing

the 8 districts who have received technical assistance and support from SISEP show the level of

commitment ODE‘s OSLP can anticipate from participating and interested districts.

In addition, in spite of Oregon‘s current state restructuring related to the governance of early

childhood programs, ODE‘s OSLP continues to develop all improvement activities to ensure the

inclusion of early childhood partnerships, in demonstration of our commitment to continue to

include early childhood into our improvement activities we have solicited and received letters of

support from Early Childhood early intervention program heads (such as Early Childhood Cares)


who were participants under the previous SPDG and who continue to encourage the furthering of

this work.

(iii) The extent to which the budget is adequate to support the proposed project.

The budget proposed for the current grant is proportioned approximately as follows.

Activities

(proportions

y1%/y2-5%) Recipient(s)

Year 1

Year 2-5

Infrastructure and

product (44/34%)

CCSS $20,000 $8,000

Scaling Contract (STS) 1.4FTE (year

1)/.7 FTE (year 2-5) $130,000 $65,000

Evaluation contract $70,000 $50,000

District funds for RIT members $72K X3

[subgrants] $216,000 $216,000

Depth Action

Planning with 18

districts (29/30%)

District/program $5K * ~18 (year 1)

/$10K*18 (year 2 – 5) [subgrants]

$90,000

$180,000

Depth contract

$200,000

$120,000

Breadth 50 districts

per year starting

year 2

(13/22%)

District/program $2000 * ~50 year

[subgrants] (year 2 – 5) $100,000

Breadth Contract $130,000 $120,000


State: Focus group

/Meetings state

hosted trainings

Web/travel/ etc.

(5/5%)

State $38,000 $38,000

USDE/state travel $12,000 $12,000

Educator

certification

Scholarships and

review contractor

(5/5%)

Educators $5,000 X ~8 [subgrants] $40,000 $40,000

Certification tracking contract $14,000 $14,000

Direct Total $960,000 $963,000

Indirect Rate 20.0% 20.5%-22%

Indirect Amt $36,800 $35,260

GRAND TOTALS $996,800 $998,260

The highest percentage of funds (44% in year 1/34% in years 2-5) will go toward ensuring the

replicability of the infrastructure for this work based on lessons learned. These funds will be

devoted primarily to developing embedded routine, replicable measures, and to the

―manualization‖ of the improvement activities for use both within and beyond the time of the

grant. Districts with demonstrated success have manuals with expectations and trouble-shooting

guides. The next highest percentage of funds (29% year 1/30% years 2-5) will be used to support

districts who are directing their previous training toward areas of educational improvement that


will require additional resources in terms of research and intensity. Revelations from these

districts will ultimately be used to inform districts across the state.

Districts who will be gaining skills based on previously ―manualized‖ practices (implementation

skills related to evidence-based practices) will have the benefit of state-resources and

experiences and so will only need (13% year 1 and 22% in years 2 – 5).

Work in the area of certification (5%/5%) will provide incentives for 8 individuals a year (40

practitioners over the course of 5 years). In that same time activities including focus groups and

facilitated discussions will be occurring to ensure links of this work from classroom to policy

level. State funds will support the activities under Absolute priority two (5%/5%) as well as the

facilitation of meetings toward continued infrastructure development.

The intent of the previous grant was to assist districts in developing infrastructures specific to

academic and behavioral supports. These infrastructures would sustain practice beyond the

timeline of unpredictable grant funding sources. Districts used previous SPDG funds to use time

to meet as teams to review student and systems-level data, attend conferences, attend trainings,

purchase databases and associated technology as well as access ongoing technical assistance

toward their academic and behavioral system needs. In the process of developing this work,

districts were able to establish routines, develop and replicate tools, and manualize processes

using infrastructure level data. Data, such as those provided as a result of data from their District

Systems Support Plans (DSSPs) to evaluate their ongoing needs across the variables that, similar

to seven drivers in implementation science, were considered critical indicators of system-health

for implementation.

As part of the 2006 SPDG EBISS activities, scores on these variables led to the development of a

district-level action-plan designed to inform their system needs and provide information on their


growth toward capacity. In the current proposal, districts who have previously developed action

plans on these variables will continue to be supported with funds from this grant to continue to

improve their practices with less contractor/vendor support, but with more focus on internal

coaching and a redirection to the seven Implementation Science drivers. Under the 2011 award

districts would receive a comparable amount of funds to focus on internal coaching efforts and

on areas identified as being deficient but requiring additional sustained planning beyond that

previously identified. Districts who were not originally part of the 2006 SPDG will receive

funds to gain access to the resources necessary to develop their systems-readiness for

implementation science and to develop routines around sustainability of evidence-based

practices. The budget indicates the general areas of expenditure for the proposed activities.

(iv) The extent to which the costs are reasonable in relation to the objectives, design, and

potential significance of the proposed project.

All the costs associated with these activities are reasonable as based on two dimensions: (1) As

compared to past costs; (2) As matched to the intended intensity of the focus by proportion.

Because we have added breadth and implementation science to these activities, all areas

(included contract awards) are now less than were budgeted in the previous SPDG.

Activity

(Proportion)

Link to OSEP Objective

(design)

Significance

Funds toward

infrastructure and

evaluation

(44/34%)

Sustainability

Improvement over time

(Contracts with implementation

Personnel supporting district

implementation routines and

developing implementation

materials for replication and use


coordinators, RIT membership)

Funds to districts

to scale work to

additional depths

(29/30%)

Sustainability


Support identified competencies

(Training and Coaching)

Districts able to problem-solve

issues that fall outside the

mainstream of previously

developed implementation

resources. Manualize and share.

Funds to districts

to scale toward

additional breadth

(13/22%)

Sustainability


Support identified competencies

(Training and Coaching)

Districts developing infrastructure

toward typical (previously

researched) implementation

activities for EBP.

Funds toward

investing in

certification/licen

sure (5%)

Highly qualified and retained

for two years after initial

activities

State primary personnel under-fill

is attacked.

Funds to the state

(5%)

Activities related to priority 2.

State sustainability

State provides support to

implementation activities.

(v) The potential for continued support of the project after Federal funding ends,

including, as appropriate, the demonstrated commitment of appropriate entities to this

type of support.

The routines, documents, and lessons learned between 2006 and 2011 during the implementation

of the previous SPDG, provide evidence of the potential for sustainability intended by this work.


The 28 districts and 6 programs involved in the previous 5 years continue to operate using all

three elements of sustainability (embedded routines, replicable tools, manualized processes) in

their day-to-day operations. Districts previously identified as well as the partners previously

invested in this work remain invested in continuing the work that was begun and to take it to new

depths by investigating new areas of need using routines, documents, and lessons learned in the

previous five years.

QUALITY OF THE MANAGEMENT PLAN

(i) The adequacy of the management plan to achieve the objectives of the proposed project

on time and within budget, including clearly defined responsibilities, timelines, and

milestones for accomplishing project tasks.

The Oregon Department of Education will use funds from this grant to support several

professional development activities that will be coordinated by staff in the Office of Student

Learning and Partnerships. To facilitate the delivery of efficient professional development

sessions over the course of the grant life and beyond using principles of effective adult

instruction, the coordination of the professional development activities will be managed with a

major focus devoted to ensuring parity of audience need. This approach allows clear messages

to be directed to audiences at the appropriate instructional (or needs-based) level. To ensure

alignment with current research and federal expectations, funds will be designated for ODE

staff to attend the annual Project Director Meeting in Washington, DC.

In this application, individuals responsible for the development and delivery of the professional

development for this evolution of Oregon‘s SPDG when not referred to individually are referred


to by the term ―Implementation Coordinators‖. This term is intended to include state employees,

contract holders, and university consultants or technical assistance providers.

Contract holders will be selected for their expertise in relevant evidence-based practices,

implementation science, or other content identified as relevant to this grant (such as Common

Core standards, University level pre-service providers). Contract holders and other consultants

will be selected according to agency policy and, as such, may be solicited either through a

Request for Proposal (RFP) process, or through their demonstration as a sole-source provider

for the evidence-based program or relevant content. In some cases, contract holders already

providing these services in the state will be referenced as a potential provider for the next

iteration of this service delivery. Individuals responsible for ensuring the Common Core State

Standard expertise of the State‘s Implementation Coordinators will be selected in consultation

with the Chief Council of State Officers (CCSSO) .

The work conducted by the state will occur over the course of four phases that roughly align

with the state‘s four objectives. The overall success of the work undertaken by the 2011-2016

SPDG activities will be evaluated according to its sustainability (and/or potential for

sustainability) as evidenced by: (1) Valid, evidence-based practices demonstrated in embedded

routines, (2) replicable tools/measures, 3) the support of a manualized process.

Phase I: Developing Infrastructure (see table)

Objective 1: Increase the capacity of state-supported Implementation Coordinators (who are

responsible for statewide professional development on evidence-based practices) to convey the

interconnectedness of state-endorsed initiatives (CCSS, RTI, PBIS) and to develop valid


evidence-based materials for district-use in tiered decision-making.

In this phase, Implementation Coordinators will use the Fall of 2011 to develop the state‘s

comprehensive plan of professional development for the four groups of adult learners that are the

focus of the grant‘s activities:

Educational experts in need of CCSS content knowledge (Fall 2011 - ongoing);

Practiced districts exploring new depths of learning (Winter 2012 - ongoing);

Willing districts (by zone) to achieve new breadth of state delivery (Fall 2012 - ongoing)

Professionals interested in pursuing career paths in the field of Speech Language,

Pathology (Winter 2012 - ongoing).

Table 1: Developing Infrastructure

What Who When Replicable

Product

Submit any relevant contract-

requests for approval

Buenrostro/Carrizales Upon award Contracts with

providers for

Depth, Breadth,

Evaluation,

Certification

tracking, STS


contract

Develop training format and

schedule

Common Core CCSS Coordination

Committee

Fall 2011 Training plan

Depth Ingram-West/Tedesco

STSs


Breadth (based on zones) Ingram-West/Tedesco

STSs


Certification Heaton Fall 2011 Certification

plan

Develop training content (modules)

Common Core State Identified CCSS

provider (national

contract)

Fall 2011-

ongoing

Module and

evaluation plan

Depth Contract holder (EBP

implementation expert

or exemplar district)

Fall 2011 3/18 training

modules


Breadth Contract holder (EBP

implementation expert

or exemplar district)

Fall 2011 3/18 training

modules

Certification Contract holder (local

or national university)

Fall 2011 Certification

plan and

reporting

schedule

Develop Website Buenrostro/Heaton Fall 2011-

Spring 2012

Website

Develop Individual and

consolidated Communication

Schedules and timeline

Buenrostro/Carrizales-

Engelmann

Fall 2011 Communication

plans and

reporting

timelines for

contract-

holders*

Common Core Buenrostro Fall 2011 *(see above)

Districts exploring depth Buenrostro/Ingram-

West/Tedesco

Fall 2011 *(see above)

Districts exploring breadth Buenrostro/Ingram-

West/Tedesco

Fall 2011 *(see above)


(based on zones)

Certification/licensure

program

Buenrostro/Heaton Fall 2011 *(see above)

Consolidate implementation

documentation and materials

Ingram-

West/Tedesco/Contract-

holders

Fall 2011 Trainer

resource

repository

Phase II: Implementation, Infrastructure, Certification, Enhancing Depths (see table)

Objective 2: Increase the capacity of districts already versed in action planning to develop and

sustain a three-tiered model of behavior and academic support in areas not previously targeted

but identified as deficient by district leadership.

Objective 4: Increase the number of highly-qualified special education specialists in Oregon –

by providing certification incentives to decrease state personnel deficits relative to special

education.

Phase II of this work will begin in Winter of 2012 and will be conducted by contract holders of

evidence-based practices, with the consultation and oversight of ODE personnel (Buenrostro,

Carrizales-Engelmann, Latini) and SISEP supported State Transformation Specialists (Ingram-

West, Tedesco). In this phase, professional development plans will be extended to the

development, adaptation, and preparation of materials to support the ongoing training of the first


18 districts in Zone 1 exploring depth. In the past this and similar work has been conducted by

either the Center for Teaching and Learning [CTL] at the University of Oregon (Chaparro,

Baker) or by exemplary district Tigard-Tualatin (Putnam); with support from the PBIS Network

of Oregon (Borgmeier, Lynass).

Three live general meetings for these advanced districts, will be supplemented by 15 specific

web-based seminars in five possible areas of depth over the course of the year. In addition to

these five, as-needed/upon request web-based seminars for specific areas of technical assistance

will also be provided. The five possible areas of depth are:

1. Extending Implementation EBP to Pre-K or Secondary levels

2. Extending Implementation to enhance EBP focus in either academics or behavior

3. Extending Implementation to enhance services for target populations (For example,

ELL).

4. Extending the Multi-Tier Process to develop district procedures for infusing RTI data and

procedures into the process for LD Eligibility.

5. Analysis and integration of Scaling Up Implementation Drivers.

In addition, during this phase work will be conducted by partner universities providing pre-

service education for educators receiving grant-sourced scholarships. This work will follow the

typical timeline of university semesters or terms, the typical evaluation process requiring

passing grades for licensure and certification, and will include a grant-funded tracking and

communication contract holder who will maintain access to the scholarship recipients by

managing their data as a cohesive cohort of students. In the past this work has been supported

by Nova. During this phase the state will also align SPDG activities with other state activities

in higher education (e.g. Project ALDER) to ensure the alignment of pre-service licensure and


certification activities with state needs and current research on evidence-based instructional

practices.

Table 2: Implementation, Infrastructure, Certification, Enhancing Depths

What Who When Product

Prepare materials and conduct Depth

training live

Contract holder

(EBP

implementation

expert or

exemplar

district)

Winter 2012 –

Spring 2016

3 annual

Trainings

completed.

Proficiency

scores.

Evaluations.

Action planning

documents.

District

indicator data

Prepare materials and conduct web-

based seminars toward 5 areas of depth

(3 each)

Contract holder

(EBP

implementation

expert or

exemplar

Winter 2012 –

Spring 2016

15 annual

Trainings

completed.

Action planning

documents


district)

Prepare materials and conduct Depth

training in areas of focus as-needed

areas using web-based seminar format

Contract holder

(EBP

implementation

expert or

exemplar

district)

Winter 2012 –

Spring 2016

5 annual

trainings

Develop response mechanism for TA

Provide TA

Buenrostro /

contract holder

Winter 2012-

Spring 2016

Technical

Assistance plan

and log

Develop data-oriented communication

schedule for districts exploring depths-

- including SISEP coined formats for

communication feedback loops

Buenrostro/

Carrizales-

Engelmann/ All

Implementation

coordinators/

SISEP

Winter 2012 Communication

schedule

Ongoing CCSS training State Identified

CCSS provider

(national

contract)

Fall 2011 –

Spring 2013

Proficiency

outcomes of

trainees


Develop fidelity measures for

Implementation Coordinators to

include CCSS in EBPs

State Identified

CCSS provider

(national

contract)

Fall 2011-

Spring 2013

Fidelity tools

Develop ODE (agency-wide

communication on implementation

activities)

Carrizales-

Engelmann/

Latini

Fall 2011-

Spring 2016

Bi-annual

memos

Begin implementation of certification

tracking, plan focus groups around

aligning state certification needs to

state needs

Heaton/Contract

Holder

Fall 2012 Tracking logs

Phase III: Implementation, Infrastructure, Certification, Enhance Depth, Introduce Breadth

(see table)


implementation activities in the use of three-tiered models of behavior and academic support by

using principles of implementation science to take action-planning awareness for behavioral and

academic strategies to all districts.

Phase III of this work will begin in Fall of 2012 and will continue the infrastructure activities

that began in Fall of 2011, the depth and certification activities that began in Winter of 2012,


but will now include the final adult learner group (willing districts by zone to achieve new

breadth of state delivery). Like the trainings for districts exploring depths, this level of

professional development will comprise three live general trainings and 15 web-based seminar

sessions intended to provide further professional development on improvement activities around

the seven drivers, as well as five specialized Webinars. During phase three and continuing into

Phase IV, additional infrastructural components (of evaluation and coordination) will be

developed and managed by the state to ensure that the objectives are measured cohesively and

that learning in one area of focus is able to inform and improve other areas of focus.

Table 3 Implementation, Infrastructure, Certification, Enhancing Depths


Develop coordinated schedule

involving Depth and Breadth districts

Buenrostro/

Carrizales-

Engelmann

Fall 2012 Schedule of

events and

outcomes for

breadth districts

Conduct Breadth Trainings Contract holder

(EBP

implementation

expert or

exemplar district)

Fall 2012 –

Spring 2016

3 annual

Trainings

completed.

Proficiency

scores(pre/post).


Evaluations.

Action planning

documents.

District

indicator data

Create and utilize breadth measures

(fidelity and evaluation)

Ingram-West

/Tedesco /SISEP

Spring 2013 Tools for

evaluating

success of

breadth training

adult learner

outcomes and

ultimately

learner data.

Consolidate data across

implementation activities (CCSS,

Depth, Breadth, Certification)

Implementation

Coordinators

Fall 2013 Cohesive

tracking

document for

outcomes

Evaluate embedded routines and

replicable tools/measures and

manualized processes

Latini/State

Management

team/ SISEP/

evaluation

Spring 2014 Sustainability

report


contractor

Develop web-based resource

repository

Buenrostro/

Heaton / Ingram-

West /Tedesco

Fall 2013 District

accessible

repository

Phase IV: Ongoing Evaluation, Review, and Adaptation (see table below)

Phase IV ensures that state retains awareness of the ongoing impact and implications of this

work and is in a position to utilize the principles of Implementation Science to ensure

adaptations along the way.

Table 4: Ongoing Evaluation, Review, and Adaptation


Ongoing data evaluation and

adaptation

Buenrostro/

Carrizales-

Englemann/ Latini

Implementation

Coordinators

Spring 2012 –

Spring 2016

Quarterly

Meetings

logged

(ii) How the applicant will ensure that a diversity of perspectives are brought to bear in

the operation of the proposed project, including those of parents, teachers, the business


community, a variety of disciplinary and professional fields, recipients or beneficiaries

of services, or others, as appropriate.

As part of the process for reviewing Oregon‘s Annual Performance Report (APR), updating

Oregon‘s State Performance Plan (SPP), and providing input on specific indicator targets and

activities, OSLP gathers a group of relevant stakeholders annually to provide input on topics

related to successful outcomes for students with disabilities. This group has provided ongoing

input for SPDG (and previously SIG) activities and will continue to be involved in activities

related to the sustainability of these programs or to any continuing award. Approximately 100

stakeholders representing school districts, education service districts, higher education, parent

groups, charter schools, private schools, service organizations, and other state agencies provide

input during the meeting and may be solicited at critical decision points within the academic

year. Members of the State Interagency Coordinating Council (SICC), the State Advisory

Council for Special Education (SACSE), the Dispute Resolution Committee, the Secondary

Transition Advisory Committee, and the Personnel Development Committee also participate.

In addition to the formal stakeholder processes, OSLP solicits feedback from the broad range of

special education administrators who attend the state‘s annual fall conference. Over the past 5

years, OSLP has used this conference as a forum to present the special education field with

updates and information on the specific SPDG and IDEA funded initiatives that have been

supporting the state‘s special education activities.

In addition to these formal and informal opportunities for diverse perspectives, OSLP enjoys

ongoing relationships with the numerous partners represented by the 26 letters of support

submitted with this application. These letters along with many unlisted, represent both formal


and casual partnerships and relationships with entities who support OSLP‘s many efforts to

positively impact students with disabilities.

OSLP will continue to utilize the communication structures and practices taught by SISEP to

keep all of these numerous relevant stakeholders actively engaged via data-oriented

communication around implementation outcomes, professional development-oreinted adult

outcomes, and student achievement and behavior outcomes. This transparency in communication

around outcomes will be achieved through regularly published memos, news announcements,

web-updates and as updates preceding web-based seminars. The state personnel have an

enormous role in ensuring the management of this plan remains transparent and accessible.

QUALITY OF THE PROJECT EVALUATION

Methods of evaluation are feasible, and appropriate to the goals, objectives, and outcomes

of the proposed project. The table below gives an overview of the evaluation plan for our

proposal. There will be two evaluation teams. The internal evaluation will be conducted by

ODE‘s project coordinators. The external evaluation will be conducted by a contracted entity

who will evaluate the project (a) according to the specifications outlined in this document and (b)

to ensure that comparisons can be made across iterations of the grant work. Both qualitative and

quantitative data will be collected to provide feedback to the Oregon Department of Education

(ODE), to OSEP, and to collaborating agencies. The rationale and the context for the evaluation

model are below.

Objective 1: Increase the capacity of state-supported implementation coordinators who

are responsible for statewide professional development on evidence-based practices to


convey the interconnectedness of state endorsed initiatives (CCSS, RTI, PBIS) to develop

valid evidence-based materials for district-use in tiered decision-making.

Evaluation Questions:

1) If Initiative leaders are trained, do they have awareness of national content

expectations for CCSS?

2) If Initiative leaders are trained, are they able to develop Professional Development

products for integration CCSS and their work?

3) If Initiative leaders are trained, do they have knowledge of the critical expectations for

implementing CCSS?

Objective 2: Increase the capacity of districts already versed in action planning to

develop and sustain a three-tiered model of behavior and academic support in areas not

previously targeted but identified as deficient by district leadership.

1) If professional development is provided, do districts implement the practices?

2) If districts implement, do they do so with fidelity?

3) If districts implement with fidelity, do they sustain the practice(s)?

4) If districts sustain the practice(s), what is the impact on student outcomes (school,

group, individual)?


implementation activities in the use of three-tiered models of behavior and academic

support by using principles of implementation science to take action-planning awareness


for behavioral and academic strategies to all districts.

1) If professional development is provided, will districts develop an Action Plan for

Infrastructure Building?

2) If professional development is provided, do districts implement the Action Plans?

3) If districts implement, do they do so with fidelity?

4) If districts implement with fidelity, do they sustain the practice(s)?

5) If districts sustain the practice(s), what is the impact on student outcomes (school,

group, individual)?

Objective 4: Increase the number of highly qualified special education specialists in

Oregon – by providing certification incentives to decrease state personnel deficits relative

to special education.

1) If scholarships are provided, will professionals apply?

2) If scholarship recipients receive funding, do they complete the program?

3) If scholarship recipients complete the program, do they take positions in their new

field?

4) If scholarship recipients take positions in their new field, do they do so in Oregon?

5) If scholarship recipients take positions in Oregon, do they maintain employment in

Oregon for an extended period of time?

ODE will evaluate the activities proposed in this grant with respect to the training content (e.g.,

CCSS, Breadth, Depth and licensure), District settings (e.g., Cohort 1, Cohort 2), and the


beneficiaries of the training (e.g., SPDG providers, Initiative providers, District Leaders,

Parents). We will build capacity at the state level to provide (a) CCSS Training and

access to national resources , (b) Specialized Decision-making training and

Action planning refinement supports for program implementation across multiple Districts, and

(c) Generalized Decision-making training and Action planning refinement supports for program

implementation across multiple Districts, and (d) supports for ongoing program evaluation to

inform sustainability efforts. Thus, ODE will document the content and fidelity of the CCSS

integrated training provided to district and school staff, the fidelity of district implementation of

practices (RTI), changes in District capacity for Implementation, and districts‘ ability to sustain

practices.

The evaluation will address these objectives with qualitative and quantitative data sources

and a variety of methodological procedures. Following guidelines developed by the Science

Panel on Interactive Communications and Health (Henderson, Noell, Reeves, Robinson, &

Strecher, 1999; Robinson, Patrick, Eng, Gustafson, 1998), the evaluation of the activities in this

proposal will include formative, process, and outcome approaches. Formative evaluations will

provide information aimed at improving the development of products. Process evaluations, will

provide necessary information for Regional team development, tests for functionality and

quality. Outcome evaluations test the program under controlled circumstances (i.e. Society for

Prevention Research, 2004) and in actual schools (i.e., effectiveness). Through the development

and implementation of training, Stufflebeam‘s (1997, 2000, 2002) context, input, process, and

product (CIPP) evaluation model will be used as the framework for the internal and external


evaluations of the proposed activities. Context, input, process, and products evaluation types are

shown in Table 5.

Table 5. The CIPP Evaluation Model by Type with Evaluation Purposes

Evaluation

Type

Decision

Type

Question

Answered

Evaluation Purpose

Internal-Formative External-Summative

Context Planning What should

we do?

Guidance for setting

objectives and priorities

Comparison of

objectives to needs

Input Structuring How should

we do it?

Guidance for planning

programs and other

services

Comparison of plans to

alternatives

Process Implementin

g

Are we doing

it as planned?

If not, why

not?

Guidance for delivery of

services and

implementation of

programs

Record and appraisal of

implementation

Products Recycling &

Outcome

Did it work? Guidance for recycling-

continuation-improvement

decisions

Comparison of

achievements to needs,

objectives, and priorities


Each of the CIPP evaluation types will be used for both formative and summative (outcome)

purposes, where internal evaluators will conduct the formative evaluations and external

evaluators will conduct the summative evaluation. Table 5 also shows the purposes of each

evaluation type for internal-formative and external-summative evaluation. ODE will conduct

formative evaluation during development and initial training. Due to the need to provide annual

progress reports to OSEP, summative evaluations will be conducted after each phase of

implementation.

The internal evaluation provides ODE with the opportunity to refine and improve

professional development content, before offering training to districts statewide. The internal,

formative evaluation will also assist ODE in identifying training needs and assessing

development plans, implementation, and outcomes. Data will be used to create internal progress

reports for staff developers and collaborating agencies. Internal evaluation data will include

interviews, focus groups, and surveys with administrators, K-12 teachers, and districts personnel

about (a) Context, e.g., the training needs, objective-setting, and priorities, (b) Input, e.g., how

best to implement training, (c) Process, including initial reviews of implementation quality, and

(d) Products, e.g., measures of knowledge of critical expectations for CCSS integration.

Table 6 describes the key evaluation elements by evaluation type for the internal evaluation

with example measures and data collection methods. Details about measures and analysis

techniques are discussed separately, later in this program evaluation section.

Context. For the context analysis, ODE will address the question, what should we do? The

needs assessment will include information about districts‘ goals, current implementations of

Evidence Based practices, the materials available, and the measures used within districts for


making decisions with respect to the academic and behavioral outcomes of students. Student

performance and behavior data will also be collected to establish the training context and ensure

that grant activities addresses all needs. During training, information will be collected from

school teams, such as the DSSP and Level of Implementation assessments.

Input. The input analysis will examine the scope of the training program, budget, and

implementation feasibility. The results of this input analysis and the context analysis will guide

the development of grant activities. Training, budget, and implementation data for the formerly

―non‖ integrated Initiatives will provide a basis for comparison.

Process. The process analysis will monitor the training programs provided to districts and

their implementation of action plans. For training, the process analysis will include measures of

participants‘ knowledge gains, intentions to use the skills and practices taught, and satisfaction

with training. For the implementation of action plans within districts, the assessment will include

measures of implementation fidelity, such as whether core elements of a chosen EBP have been

extended to more diverse populations.

Methods of evaluation examine effectiveness of the implementation strategies.

Products. The products analysis will include an assessment of impact, effectiveness,

sustainability, and transportability evaluations (Stufflebeam, 2002). After training, the products

analysis will determine whether district staff, teachers, administrators, and other staff learned the

intended skills and intend to continue to use them. That is, the products of training include a

cadre of educators within the districts that will initiate the integration of research-based practices

and CCSS. The evaluation will assess whether the appropriate staff attended, the costs of


attending, and so on. This will also include an examination of the costs and benefits of training

based on the web-based seminars for professional development that is technology and site based.

Table 6. Internal Evaluation Using the CIPP Model with Elements and Measures

Evaluation Element or Objective Measures and Collection Methods

Context Analysis

Personnel awareness of national content

expectations for students

Personnel trained to identify program

areas with implementation deficits

Personnel awareness of implementation

principles

Personnel in-training and

Stakeholder awareness

Interviews with districts and school personnel.

Focus groups with key informants.

District performance data (e.g., OAKS).

Action Planning goals.

Input Analysis


Applicability of existing programs to

wide-scale implementation.

Use of research-based practices.

Scope of the project and budget.

Plan implementation feasibility.

Interviews.

Focus groups.

Analysis of action plan progress.

Process Analysis

Training implementation quality.

District implementation fidelity

Surveys of school and district staff satisfaction.

District Team meeting notes.

RIT conducted site observations.

Implementation surveys with teachers,

administrators, district staff.

Products Analysis

Training participant satisfaction,

knowledge of critical expectations for

implementing CCSS or RTI, and

intentions to continue to use the practices

learned.

Intended recipients (e.g., did the districts

Focus groups with key informants.

Surveys of satisfaction, knowledge, and intentions.

Implementation Capacity Assessments: (e.g., DCA,

RCA)


benefit from RIT activities and support?).

Action planning outcomes.

Development of implementation

infrastructure.

For the integration of CCSS and Innovations, the products analysis assesses efficacy, whether

personnel knowledge and capacity for implementation improved and whether those districts who

improved represented the intended populations, such as districts with schools under sanctions or

districts serving a large number of students with special needs. It will also address sustainability

through continued focus groups, interviews, assessment of implementation, and RIT

performance. The products analysis will examine transportability within districts, through

assessments similar to those for sustainability but for districts that may not have been part of the

initial cohorts and implementation, and across districts, through trainings of a second and third

cohorts of districts.

In the internal evaluation, these data will be used to determine if components can be repeated

without modification, how to modify specific training components, or which components might

need considerable revision (Stufflebeam, 1997). The internal evaluation, then, represents the

entire development process where ODE will select in-place training materials, repeatedly revise

and test trainings for districts, and work to improve the training materials over time.

External Evaluation. After completion of program development and trainings for a

component (e.g., extending depth of RTI to develop District policy for evaluation of students for

special education), the external evaluation team will conduct summative evaluations. External


evaluators will provide information to ODE about whether to retain, revise, or amend the staff

development model. Summative evaluations differ very little from formative evaluations in their

criteria and in the information collected (Stufflebeam, 1997), so a separate table for the CIPP

model that describes the external evaluation will not be presented.

Context and input. The context analysis will compare the needs of districts to the objectives

of the grant activities. In the input analysis, comparisons will be made between the programs

provided and available alternatives. These two types of evaluation are intended to assure ODE

that target activities addressed the needs of educators throughout Oregon and that ODE provided

the best alternatives.

Process. The process and products analysis are key to the external evaluation, as they address

the implementation and outcomes of grant activities. The external process evaluation assesses

and analyzes two levels of implementation fidelity. First, the quality of training provided by the

state will be assessed using measures of school and district staff satisfaction, knowledge, and

their intentions to use the principles and techniques they learned. Again, this is similar to the

internal evaluation, but with the appraisal of the final version of the grant activities.

Second, the quality of implementation of district capacity to build and sustain infrastructures

for EBPs will be evaluated with standard implementation measures. That is, after trainers have

taught districts staff how to respond to specific implementation needs, the external evaluators

will examine the fidelity of the action plan implementation of action plan within districts. This

will help identify the sources of any potential weaknesses in the model to change action planning

and implementation activities.


Products. The products analysis of the external evaluation will examine whether sustainable

products were developed in the form of embedded routines, replicable tools, and manualized

processes in relation to stage of implementation. This phase represents the final outcome analysis

of the educator training, and also includes their implementation of action plans (embedded

routines), the use and accessibility of appropriate tools (minimized re-development), and the use

and accessibility of manualized processes for ensuring fidelity of application of program

components.

In addition, this type of evaluation also refers to the assessment of whether grant activities

reached the intended audiences for each training component and whether those participants

benefited from the implementation of the training content. Teachers and other staff—including

those who did not attend training but helped implement the action plans—will be assessed with

respect to their satisfaction, knowledge, and awareness of resource accessibility. Learner

outcomes will be measured by comparing knowledge before and after trainings using the

statistical procedures defined below. Finally, hypotheses about moderation and mediation will

also be tested when improvements in outcomes have been detected.

Evaluation uses objective performance measures related to the intended outcomes.

Measures. Key measures for internal and external evaluations for each year of the project

with their sources are listed below. Details about key measures and instruments are discussed on

following pages. Examples of select key measures and instruments can be found in Appendix H.

District Capacity Assessment (DCA). Similar to the DSSP (described in previous sections

and included in Appendix C) and used as a measure of EBISS implementation, this document is


designed to help districts assess their current capacity to support the full and effective uses of

evidence-based academic and behavior practices and other innovations in schools and

classrooms . This tool can be used to assess capacity along several dimensions: 1) District

Leadership Team, 2) District Implementation Team, 3) District Evaluation Measures, 4)

Visibility and Political Support, 5) Funding, and 6) Action Planning with Schools. To complete

these items, Regional Implementation Team members meet with district leaders and stakeholders

to review each item. This discussion is intended to be informative regarding how districts can

support effective education in schools, and evaluative regarding current district capacity to

provide such supports. For items identified as ―Partially In Place‖ or ―Not In Place,‖ needed

actions can be discussed and specific plans can be made to work toward having the item ―Fully

In Place. Space is included for future progress checks. Thus, this tool serves to support district

capacity activities (by serving as a framework for conducting a given activity) as well as assess

district capacity activities. Regional Implementation Teams and ―District‖ Leadership and

Implementation Teams are encouraged to use this as a working document. Districts with many

items requiring action can review and update the plan each month. Districts with many items ―in

place‖ can focus on sustainability and update the plan at least twice a year (e.g. October and

April).

Stages of Implementation Analysis. This tool provides the planning team with the

opportunity to plan for and/or assess the use of stage-based activities to improve the success of

implementation efforts for evidence-based programs or other innovations. The tool can be used

to assess current stage activities (e.g. ―We are in the midst of Exploration‖) or past efforts related


to a stage (e.g. ―We just completed most of Installation? How did we do? What did we miss?).

For activities scored as ―Not Yet Initiated‖ the planning team may wish to:

a) Examine the importance of the activity in relation to achieving success

b) Identify barriers to completion of the activity

c) Ensure that an action plan is developed (sub-activities, accountable person(s) identified,

timeline, evidence of completion) and monitored

A ‗strength of stage score‘ can be computed for each stage to help guide action planning.

Regional Implementation Team Capacity Assessment (RCA). This document is designed to

help Regional Implementation Teams (RITs) assess their current capacity to support the full and

effective uses of evidence-based academic and behavior practices and other innovations in

schools and classrooms. This tool can be used to assess RIT capacity to interact with ―District‖

Leadership and Implementation Teams along several dimensions: 1) Exploration Stage

Activities, 2) Installation Stage Activities, 3) Initial Implementation Stage Activities, and 4)

Evaluation Activities. To complete these items, State Transformation Team members and the

State Implementation and Scaling up of Evidence-based Practices staff member meet with

Regional Implementation Team (RIT) members and stakeholders to review each item. This

discussion is intended to be informative regarding how RITs can support effective education in

districts and schools, and evaluative regarding current RIT capacity to provide such supports.

For items identified as ―Partially In Place‖ or ―Not In Place,‖ needed actions can be discussed

and specific plans can be made to work toward having the item ―Fully In Place. Space is

included for future progress checks. Thus, like the DCA this is an assessment that can be used to

support RIT capacity development and to assess RIT capacity development.


State Capacity Assessment (SCA). The purpose of this measure is to track the development

of a State's implementation capacity. This measure will be piloted within the first two years of

funding. The SCA will be administrated by two members of the National SISEP team for the

first year of the use of the SCA for action planning purposes. For this first year of administration,

the intended target group are all of the State Transformation Specialists (STS) in the State, the

State RITs. SISEP will meet with the State Management Team (SMT) for separate

administrations of the SCA.

Implementation Drivers Best Practices Analysis and Action Planning Tool. RITs, with

members who know the intervention well, can use this tool as a way to discuss the roles and

responsibilities at the building, District, State and TA level they are responsible for guiding.

Engaging TA providers and program developers in this process with those who are charged with

successful implementation at the school and District level can yield a useful and enlightening

discussion that will not only impact program quality but also programmatic sustainability

through action planning. The Implementation Drivers are processes that can be leveraged to

improve competence and to create a more hospitable organizational and systems environment for

an evidence-based program or practice (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005). A

pre-requisite for effective use of the Implementation Drivers is a well operationalized, evidence-

based intervention, program or practice. The more clearly the core intervention components are

defined and validated through research (e.g. fidelity correlated with outcomes; dosage and

outcome data), the more clearly the Implementation Drivers can be focused on bringing these


core intervention components ―to life‖ and sustaining and improving them in context of

classrooms, schools, and Districts (Blase, Van Dyke, Duda & Fixsen, 2010).

Schoolwide Evaluation Tool (SET and PreSET). The School wide Evaluation Tool (SET)

(Horner, et al., 2000) was developed to measure whether school personnel are implementing the

practices and systems associated with school wide positive behavior support. Data to complete

the 28 items of the SET are collected by on-site observers who review school documents,

examine physical spaces, interview staff and the administrator, and interview a sample of

students. Each SET item is scored as ―in place,‖ ―partially in place,‖ or ―not in place.‖ The SET

produces an implementation average summary score (0-100%) and seven subscale scores: a)

school wide behavioral expectations are defined; b) expectations are taught; c) rewards are

provided for following the behavioral expectations; d) a continuum of consequences for problem

behavior is in place; e) data on problem behavior are collected and used for decision-making, f)

an administrator actively supports PBS, and g) the school district supports School wide PBS.

When the SET results indicate a summary score > 80% and an ―expectations taught‖ subscale

score > 80%, we consider it as having met our criterion of ―successful implementation of

Schoolwide PBS‖. We will assess each school with the SET during the fall and spring of each

year in the study and as a baseline measure used to exclude schools that have already

implemented PBS. The SET implementation average, and the seven subscales above, will all be

used as continuous measures for analysis purposes. The Early Childhood Programs will use the

PreSET developed recently at the U of O under the direction of Dr. Rob Horner. Districts

advanced in PBIS may also provide Benchmarks of Quality (BOQ) data.


Effective Behavior Support Self-Assessment Survey. This survey measures the extent and

quality of implementation of each of the five facets of school wide discipline as they relate to the

whole school, non-classroom settings, and classrooms (Sugai, Lewis-Palmer, Todd, & Horner,

2000. It has been validated in research on dissemination of PBS (Horner et al., 2004).

Office Discipline Referrals. The School wide Information System (SWIS) is a web-based

information system designed to help school personnel use office referral data to design school

wide and individual student interventions. SWIS provides PBS schools with a) an efficient web-

based system for gathering and reporting information, and b) a practical process for reporting

information for decision-making. These elements give school personnel the capability to evaluate

reports on individual student behavior, the behavior of groups of students, behaviors in common

areas, and behaviors during specific times of the school day. It will be implemented only in PBS

treatment schools, not in control schools. In order to establish a consistent ODR recording

method, both treatment and control schools will use paper and pencil recording. The form will

include all of the required elements of the www.swis.org database including the date of the

referral, student name/ID#, referring teacher/ID#, problem behavior, location of the referral, and

time of day. After PBS implementation, schools will enter the information into the SWIS

database from the paper and pencil form. Before implementation of PBS, referrals will be

entered into a separate database.

Surveys of participants in ODE training. Surveys of districts and school personnel will

include basic demographic and background variables, such as gender, ethnicity, years of teaching

experience, current position (e.g., principal, 2nd-grade teacher), hours of training on reading or

behavior management not provided by ODE, and school climate. Questions will be added to aid


the evaluation during and after context and to assess the likelihood of applying the concepts

taught in their school settings. Satisfaction measures will be developed by ODE and outside

evaluators to determine facility with concepts and materials.

Planning and Evaluation Tool (PET-R). The PET-R was developed by Oregon Reading

First for building teams to assess school strengths and areas of improvement in beginning

reading. Building teams rate their capacity on a scale of 0 (Not in place) to 2 (Fully in place) on

goals, assessment, instructional programs and materials, instructional time, differentiated

instruction / grouping/ scheduling administration / communication, and professional

development. Teams complete the PET-R at their initial training and then update it annually to

self evaluate their capacity in each of the target areas.

Statistical Analysis Approach. For objective outcome measures, the analyses will include

comparisons to established benchmarks or standards. In evaluation of the implementation of

school-wide positive behavior supports, for example, a score of 80 on the School-wide

Evaluation Tool (SET) continues to be acceptable level of implementation (Horner, Todd,

Lewis-Palmer, Irvin, Sugai, & Boland, 2004). For acceptable training, 75% of schools are

expected to achieve an 80 on the SET in the year following training. Similarly, for reading,

schools will be expected to reach established benchmarks for achievement on DIBELS and the

OSA.

We expect that training participants will rate the program for stimulation, comprehension,

acceptability, relevance, and persuasiveness, as described in the measures section, with a mean

rating of 4 on a 1-5 scale. As a post-only outcome for intervention participants, we will not


analyze this data in the same manner as other outcomes. We will, however, attempt to

demonstrate that usage and satisfaction are associated with outcomes on other measures.

The external evaluation will test for improvement from before to after training on multiple

measures of knowledge, implementation, and outcomes for both trainings of district staff and

program implementation within districts. Changes in measures will be tested with paired t-tests

or repeated measures analyses that account for the non-independence between pretest and

posttest measures from single individuals or schools (Gall, Borg, & Gall, 1996). For measures

associated with ODE training of district personnel, the tests will be applied to the individual

administrators, teachers, and other staff who attend the training. Measures will include surveys of

knowledge, intentions, and satisfaction with the training. When evaluating the implementation of

programs within districts, these tests will be conducted on district-level means. Such measures

include the SET, EBS Surveys, the proportion of students who achieve above the 40th percentile

on the OSA, the proportion of students who attain the grade-specific benchmark for DIBELS

oral reading fluency, and so on. These analyses will use two-tailed tests (Fleiss, 1981).

When reductions in outcome measures have been identified in the above analysis, regression

analysis will be conducted to identify moderation or mediational effects (Baron, & Kenny, 1986;

Cohen, Cohen, West, & Aiken, 2003). Moderation effects are represented by interactions and

indicate whether an outcome change varies in conjunction with demographic and other variables,

such as years of teacher experience. Tests of interactions are straightforward and will be

conducted as specified by Jaccard and Turrisi (2003). Mediational models are more complicated

(Baron, & Kenny, 1986) and represent interesting hypotheses. For example, we expect variation

in EBP implementation outcomes to be a function of implementation fidelity. Thus, relationship


between reading outcomes and implementation fidelity will be tested. Similarly, measures of

implementation fidelity should be associated with survey data about knowledge and intentions

from the participants of training. If these relationships attain statistical significance as well as

increases in knowledge and intentions after ODE training, then we have shown a link between

ODE training and student outcomes. While this result is not a causal link, and would not be as

strong as one in an experimental trial, it would provide some evidence that grant activities led to

improved student outcomes. Tests of these and similar mediational hypotheses will follow the

advice of MacKinnon, Lockwood, Hoffman, West, and Sheets (2002) who recommend different

tests of mediation to minimize Type I and Type II errors depending on the specific hypotheses

and assumptions of the effect in question.

Methods of evaluation will provide performance feedback and permit assessment of progress

toward achieving intended outcomes. The RIT model will provide for direct feedback to district

teams and personnel. The results of all implementation measures and outcomes measures will be

presented to districts in a form that they can understand and use to make modifications in

programming. In addition, most of the measurement tools being used (e.g., DCA) are intended

to be administered by the districts on at least a yearly basis even after the project is complete.

The professional development component of this grant and the technical assistance from the RIT

are both focused on teaching district administrators to use data to inform action planning for

building district capacity for implementation of EBPs. Accordingly, the data will provide a

means for ODE to evaluate their efforts and the technical assistance provided by contracted

entities.


State Reporting. As per GRPA, ODE is required to collect and report five performance

measures for the State Personnel Development Grant (SPDG). ODE will track and report (a) the

percent of personnel receiving professional development through the SPDG based on integration

of CCSS and scientific- or evidence-based practices, (b) the percentage of SPDG projects that

have implemented personnel development or training activities that are aligned with

improvement strategies identified by the State Performance Plan, (c) the percentage of

professional development or training activities provided through the SDPG based on increasing

―depth‖ of evidence-based practices, (d) the percentage of professional development or training

activities provided through the SPDG based on scientific- or evidence-based implementation

practices, provided through the SPDG, that are sustained through on-going and comprehensive

practices (e.g., mentoring from RIT), and (e) the state-wide percentage of highly qualified

special education teachers in state-identified professional disciplines (e.g., teachers of children

with emotional disturbance, deafness, etc.) who remain teaching after the first three years of

employment.

Summary. Oregon has the need, the interest, and the capacity to implement Evidence Based

Practices in diverse districts throughout the state. In addition to the project evaluation results,

which will provide important descriptive data regarding implementation of, and infrastructure for

integration of CCSS and EBPs, Oregon will share the tools, procedures, and policies that are

created for the State and district levels. A strong base of support has been built for coordination

of multiple initiatives in Oregon. The proposed project will allow our initial work to be taken to

scale to the benefit of students with disabilities and all students who need more than the basic

level of support.

Date post:	02-Sep-2018
Category:	Documents
Upload:	truongdien
View:	213 times
Download:	0 times