School Improvement Facilitators Network May 7, 2014 1.

School Improvement Facilitators Network

May 7, 2014

1

Target #1 •Progra

m Evaluation Tool

Target #2

•Data Discovery

•Determine Focal Points

•Discover Facts/Hypothesis

Target #3

•Interpret Proficiency Targets

•Beginning Plan Development

Agenda

• 8:30-8:40 Introductions and review process• 8:40-10:00 Progress Monitoring with Program Evaluation

Tool• 10:00-10:10 Break• 10:10-11:30 Data Discovery• 11:30-12:00 Facts/Hypothesis• 12:00-12:30 Lunch• 12:30-1:00 Goal Building• 1:00-3:00 Facilitated Work Time for Goal Building• 3:00-3:30 Evaluation, Feedback and Next Steps

3

Working Agreements

• Participate Fully• Press for Clarification• Collaborate• Share your Thinking

Calling back to whole group with hand signal

4

5

Continuous District Improvement Facilitators Network Meetings

Vision/Mission of CSIF/CDIF Network –

It is the mission of the Jackson County School Improvement Consortium to

support a community of collaboration using a Continuous School

Improvement Process in order to increase student achievement.

6

Continuous School Improvement Facilitators Network Meetings

School Improvement Support

September 25 – Planning for Implementation

November 13 – Got SIP now what?

January 22 – School Process Rubrics

March 26 – School Data Profile / Analysis

May 7 – School Improvement Plan

The Big Picture

MDE Program Evaluation ToolLearning Outcomes for Schools

• Understand the role of program evaluation within the continuous improvement process

• Understand the purpose, benefit, and process of program evaluation

• Learn about the MDE Program Evaluation tool and resources to help to evaluate a strategy/program/initiative/reform strategy

At the top of an index card, identify a hobby, sport, or activity in which you enjoy participating.

Then identify the following:

1. What would you have to do to be ready to participate?

2. What knowledge and/or skills would you need?

3. What opportunities would need to be present?

4. How would you know if you were carrying out the activity in the way it was intended?

5. What would be the result if you were skilled at the activity?

Making Connections

Activity: Why, How, What?Why:• Why is it important to strategically implement, monitor, and evaluate

the strategy/program/initiative/reform strategy?

How:• How will we communicate the plan to all stakeholders so that they

clearly understand and own their roles in implementation?

• How will we ensure implementation with fidelity?

• How is the implementation and impact of your plan monitored and evaluated?

What:• What will each strategy/program/initiative/reform strategy look like

when it is implemented with fidelity?

• What is the expected impact on student achievement?

https://www.youtube.com/watch?v=qp0HIF3SfI4

https://www.youtube.com/watch?v=qp0HIF3SfI4

Why is MDE Requiring Program Evaluation?To positively impact student achievement and close gaps for the subgroups

To ensure that high quality planning, implementation and evaluation are part of the Continuous Improvement Process

To ensure ongoing engagement of multiple stakeholders (students, teachers, parents/community, administrators) in the planning and evaluation process

To maximize the use of resources to impact student learning

To provide documentation of program implementation to inform future decision-making

To meet state and federal requirements

State and Federal Requirements

❑ Annual evaluation of the implementation and impact of the School Improvement Plan

❑ Modification of the plan based on evaluation results

❑ Annual evaluation of all federal programs—effectiveness & impact on student achievement, including subgroups

❑ Modification of the plan based on evaluation results

MICHIGAN FEDERAL

ISDs/RESAs are required by PA25 to provide technical assistance to schools and districts to develop annual evaluations. ESEA requires annual evaluations of programs funded by the federal programs such as Title I, Part A, C, D; Title II and Title III.

Program Evaluation Timeline

District/School Improvement Plans for 2014-2015: • Include program evaluation activities to support Program Evaluation as part of the

Continuous Improvement Process• Implement Program Evaluation activities throughout the 2014-2015 school year

Summer 2015 and Beyond• Sustain professional learning to discuss successes, challenges, and dandy necessary

follow-up training materials and support systems

June 30, 2015 Program Evaluation submitted in ASSIST• A completed program evaluation using the MDE Program Evaluation Tool will be

required for submission of the Consolidated Application for 2015 – 2016.

What to Evaluate?

Schools are required to select one:

• strategy/reform strategy• program• initiative

that would have the greatest impact on student achievement and close the achievement gaps.

What to Evaluate?

Districts are required to select one:

• strategy/reform strategy• program• initiative

that would most impact on student achievement and close the achievement gaps.

Where is program evaluation in the Continuous Improvement Process?

Are the RIGHT PEOPLE…

Doing the RIGHT THINGS…

In the RIGHT WAY…

At the RIGHT TIME…

…for the benefit of students?

Where is program evaluation in the Continuous Improvement Process?

20

Did your plan include activities to monitor and evaluate?

➢ Monitor Implementation➢ Evaluate Implementation Adult Focused

Student Focused➢ Monitor Impact➢ Evaluate Impact

LEADERSHIP

COM

PET

ENCY

ORGANIZ

ATION

TRAINING

COACHING

SELECTION

PERFORMANCE ASSESSMENT

TECHNICAL ADAPTIVE

DATA-DRIVEN DECISION MAKING

LEADERSHIP SUPPORT

SYSTEM SUPPORT

MDE PROGRAM EVALUATION

What is/was the readiness for implementing the program/ initiative/strategy or activity?

Do/did the participants have the knowledge and skills to implement the program?

What is/was the program’s impact on students?

Is/was the program implemented as intended?

Is/was there opportunity for high quality implementation?

Planning: How will we ensure ….. ?

Plan forward…

Get Ready1. Readiness? 2. Knowledge/Skills?3. Opportunity?

Implement4. Implement With Fidelity?

Monitor/ Evaluate5. Impact on Students?

Evaluation:To what extent was there/did we…..?

…to evaluate impact.

Here now!

23

Leadership and Learning Center 2010

24

Baseline

PM 1, Formative

PM 2, Formative

PM 3, Formative

PM 4 = Program Evaluation;Summative

Progress Monitoring → Program Evaluation

Questions for Evaluation

Impact on students?

Knowledge and skills?Readiness?

Opportunity?

Implemented as intended?

The MDE Program Evaluation Tool

When to Use the MDE Evaluation Tool

MDE Tool

Planning

BeforeIdentify and avoid gaps,

pitfalls, and potential barriers; shape program

planning

During Make mid-course corrections

Evaluation After Learn why results turned out as they did

Plan forward…

Get Ready1. Readiness? 2. Knowledge/Skills?3. Opportunity?

Implement4. Implement With Fidelity?

Monitor/ Evaluate5. Impact on Students?

Planning: How will we ensure ….. ?

Evaluation:To what extent was there/did we…..?

…to evaluate impact.Here now!

30

AdvancED ASSIST

PLATFORM

The Program Evaluation Tool has 5 Sections& a Set of Conclusions

IMPACT: What is the IMPACT of the STRATEGY/ PROGRAM/ INITIATIVE ON STUDENT ACHIEVEMENT?

1. What is the READINESS for implementing the strategy/ program/initiative?

2. Do participants have the KNOWLEDGE AND SKILLS to implement the program?

3. Is there OPPORTUNITY for implementation?

4. Is the program IMPLEMENTED AS INTENDED?

32

ImpactIMPACT: What was the impact of the strategy/program/initiative on student Achievement?IN AN IDEAL STRATEGY/PROGRAM/INITIATIVE, the school’s achievement results on state or district wide assessments meet proficiency standards. Achievement gaps between each of the relevant subgroups and their counterparts have been narrowed as proposed in the School Improvement Plan’s measurable objectives. Interim assessment results indicate progress toward proficiency for all students to the satisfaction of all stakeholders.

a) What is the evidence and what does it show regarding achievement of the measureable objective for all students when compared to baseline state and local data?

b) What is the evidence and what does it show regarding achievement of the measureable objective for subgroups and their counterparts when compared to baseline state and local data?

c) What is the evidence and what does it show regarding stakeholder (staff, parents, students) satisfaction with the results?

Now What?If Objectives were met

Conclusion

Were objectives

met?

Determine if the strategy/program

/initiative should be continued or institutionalized

Analyze further using the other 4

questions

Yes

No

Conclusion

Were objectives

met?



Complete the conclusion

section & analyze further using the other 4 questions

Determine if the strategy/program/ initiative should be continued or institutionalized

& use the 4 questions for further study



Now What?If Objectives were met

Conclusion

Were objectives

met?




questions

Yes

No

Conclusion

Were objectives

met?







Determine if the strategy/program/ initiative should be

continued or institutionalized


If Objectives were metConclusion: If the objectives were met, should the strategy/program/initiative be continued or institutionalized?

• What is the evidence and what does it say regarding whether this was the right strategy/program/initiative to meet your needs?

• What is the evidence and what does it say regarding whether the benefits of the strategy/program/initiative are sufficient to justify the resources it requires?

• What adjustments if any might increase its impact while maintaining its integrity?

• What is needed to maintain momentum and sustain achievement gains?

• How might these results inform the School Improvement Plan?

Now What?

Conclusion

Were objectives

met?





Yes

Complete the conclusion section

AND


questions

No

If Objectives were not metConclusion: If the objectives were met, should the strategy/program/initiative be continued or institutionalized?

• What is the evidence and what does it say regarding whether this was the right strategy/program/initiative to meet your needs?

• What is the evidence and what does it say regarding whether the benefits of the strategy/program/initiative are sufficient to justify the resources it requires?

• What adjustments if any might increase its impact while maintaining its integrity?

• What is needed to maintain momentum and sustain achievement gains?

• How might these results inform the School Improvement Plan?

39

1. Readiness: What was the readiness for implementing the strategy/program/initiative?

Each section begins with adescription of an ideal program

1. What is the READINESS for implementing the strategy/initiative/program? IN AN IDEAL PROGRAM, stakeholders are well-prepared to implement the program. They have read and can articulate the research foundation, and regularly use the terms in conversation with each other, students, and with parents. Staff, students and parents express a high level of interest in, support for and commitment to the program. Specific concerns have been identified and solutions have been planned/ implemented. Staff is able to seamlessly integrate the program within the context of other building/district initiatives.

Each section has 3-5 sub-questions thatask for relevant evidence

1. What is the READINESS for implementing the strategy/initiative/program?

a) What evidence do you have that stakeholders can articulate and believe the research behind the decision to implement the program?

b) What evidence do you have that stakeholders are committed to the program with both hearts and minds?

c) What evidence do you have that stakeholder (staff, parent, student) concerns about the program have been identified and addressed?

d) What evidence do you have that staff are able to integrate this program with other existing initiatives?

Each section suggests possible data sources1. What is the READINESS for implementing the

strategy/initiative/ program?a) What evidence do you have that stakeholders can articulate

and believe the research behind the decision to implement the program?

b) What evidence do you have that stakeholders are really committed to the program with both hearts and minds?

Possible Evidence:

•Professional Development materials•Conference/Workshop attendance•Data collection plan•Suggestion box ideas collected•SI team agendas•Focus group interviews

•Data analysis work•Meeting agendas/minutes•Books/papers about the program•Staff surveys•Stakeholder survey results•SI Plan elements

Finally, the section asks for a self-rating and for“ action steps” the data suggest

1. What is the READINESS for implementing the strategy/initiative/ program?

Stakeholders are fully prepared.

Support and commitment are generally high, but some concern or work remains.

Some promising signs are mixed with major gaps in knowledge or confidence.

Interest and/or commitment are low so far.

Next Steps: What action steps are needed to increase readiness to undertake the program?

When is the Evaluation Submitted?

Required to

Complete the Tool by

Spring 2015

Avoid These PitfallsEvaluating federally funded programs separatelyInclusion of many strategies/unclear on strategySelecting ‘weak’, non-robust action steps Not addressing questions 1-4 when the initiative did

not meet the objectiveNo evidence to support high self rating on scaleList of assessments rather than actual data with pre-

post analysis supporting progress or lack of progressUnclear, contradictory or confusing conclusionsConfusion regarding subgroups

At the top of an index card, identify a hobby, sport, or activity in which you enjoy participating.

Then identify the following:

1. What would you have to do to be ready to participate?

2. What knowledge and/or skills would you need?

3. What opportunity would need to be present?

4. How would you know if you were carrying out the activity in the way it was intended?

5. What would be the result if you were skilled at the activity?

Making Connections(Reference to first activity)

Michigan Proficiency Targets

85% by 2021-2022

Where to Find Your Targets http://www.michigan.gov/mde/0,1607,7-140-22709_25058---,00.html

• Google: Michigan School Scorecard– Scroll to Tools– Click on

List of Michigan District and School Proficiency Targets for the 2012-13 School Year

– FAQs

http://www.michigan.gov/documents/mde/Michigan_Proficiency_Targets_413516_7.xls


Data Discovery

Data Targets

Determine or focus on the content area requiring most attention.

Determine what our summative/available data strengths and weakness are.

Determine our GAP areas of focus.

The Big Picture

Professional educators honor the privacy of student, staff and family information.

FERPA Compliance

I pledge to honor the privacy and confidentiality regarding data and discussions involving students, staff and other school-related issues. I understand that the data and discussions shared at this retreat are for professional school improvement purposes.

I will not divulge this confidential information to any persons outside of the professional education

arena.

Data Retreat® is a registered service mark of CESA 7, WI 54

Steps-

1.Data Discovery2.Facts/Hypotheses3.Proficiency targets4.Plan Development5.Put into ASSIST6.Progress monitoring plan /

evaluation7.Rollout to stakeholders

Monitoring Implementation

Demographics

Student Outcomes

Perceptions

School Processes

What data to use?

What data do you use and why?

• Formative• Interim• Summative• Perception• Demographic• Process

Data Mining Points to consider:Reading Writing Math Science Coordinated

School Health

Letter/Sound Id, Rigby, DRA, Fountas & Pinnell, QRI,STAR, EXPLORE, MEAP, MEAP Access, MI Access, PLAN, ACT, MME, COMPASS,CWCA, NWEA

Writing Prompt, MEAP, MEAP Access, MI Access, Writing Prompts, Grammar Quizzes, PLAN, ACT, MME, CWCA, NWEA

Addition/Subtraction, CWCA, MEAP, MEAP Access, MI Access, Quarter Exams, PLAN, ACT, MME, COMPASS, NWEA

MEAP, MEAP Access, MI Access, Physical Science Assessments, PLAN, ACT, MME, NWEA

Bully Prevention, Nurtured Heart, Parent Surveys, MiPhy

58

Reading-Overall Performance

59

Reading-Grade A Gender

60

Reading-Grade B Gender

61

Reading-Grade A ED

62

Reading-Grade B ED

63

Reading-Cohort Group

43.1%

Data Interaction

©2012 AdvancED65

Select Student Testing

©2012 AdvancED66

Select Desired Assessment on top

tabs

©2012 AdvancED67

Choose filters in each category

Choose filters in each category

©2012 AdvancED68

Toggle between snapshot and trend

Facilitated Work Time


Data DiscoveryStep #1– Choose at least 5 of the most relevant

data points from state summative data in each content area at each grade level.

(For example: Gender, Economically Disadvantaged, SWD, 30%>ethnicity, ELL, Gap-top/bottom)

– State Assessment Data (MISchooldata.org)

– Record the facts/hypothesis discovered on the chart as your groups view data

Data Tables

Fact Finding & Contributing Cause

for the Gap:

Hypothesis & Reason for Gap:

Student Goals & Measurable

Objective Statement:

Strategy Statements:

Activities & Action Plan : Roll Out & Sustainability

Monitoring and Evaluation (Checkpoints):

Primary Concerns: Primary Issues:

Graphic Representation – Gap Statement

Gathering Data and Team Readiness

11:45-11:50Where are we in the process?


Step #2-Facts – Hypothesis Chart

.

Hypothesis

Fact Finding/Gap Statement:

1. Observations based on studentdata and patterns.

2. Can’t be disputed.

3. Should include numbers such as percentages.

4. STATE THE FACTS! (Avoid words like “only” or “just”)

*Example15% of the group “all students” performed at a Level 1 on the

2013 MME Math.

*NOTOur textbook series and parents are just

bad.

Facts1. What are we doing that might contribute to these Results? (Possible reasons / causes for data patterns

observed).2. Explanations should come from school and classroom based factors and are about practices that can be altered.

3. Explanations should NOT be regarding characteristicsof individuals or UN-alterable factors.

*ExampleWe … statements

Ex: 1. We as a district don’t have a well rounded understanding of the GLCEs and

HSCE taught in other grades/building. 2.We do not have the opportunity to

team for strategy intervention support.

NOT - Ex: 1. Too many of our students are economically disadvantaged.

2. That year of data was from a low groupof students.

3. Our students have a poor home life.


Step #2-Facts – Hypothesis Chart

.

Hypothesis

Fact Finding/Gap Statement:

1. Observations based on studentdata and patterns.

2. Can’t be disputed.

3. Should include numbers such as percentages.

4. STATE THE FACTS! (Avoid words like “only” or “just”)

*Example15% of the group “all students” performed at a Level 1 on the

2013 MME Math.

*NOTOur textbook series and parents are just

bad.

Facts1. What are we doing that might contribute to these Results? (Possible reasons / causes for data patterns

observed).2. Explanations should come from school and classroom based factors and are about practices that can be altered.

3. Explanations should NOT be regarding characteristicsof individuals or UN-alterable factors.

*ExampleWe … statements

Ex: 1. We as a district don’t have a well rounded understanding of the GLCEs and

HSCE taught in other grades/building. 2.We do not have the opportunity to

team for strategy intervention support.

NOT - Ex: 1. Too many of our students are economically disadvantaged.

2. That year of data was from a low groupof students.

3. Our students have a poor home life.


Data Discovery

Step #3:

-Narrow your focus to the top student concerns that emerged from your fact finding/observations/discussions

-Determine the concerns that will serve as the basis for GOAL building for your next year School Improvement Plan.

Lunch

Goal StatementSet Measurable

Objectives

Building Goals in ASSIST

Strategy Statement

Activities

What is a goal vs. objective vs. strategy vs. activity?

What is it? What is it’s purpose What does it look like?

Goal Statement Data-based, general statement linked to student learning needs—either Academic or Organizational

“All students will be proficient in _______.”

Measurable Objective Specific target that is linked to the goal. Should be “S.M.A.R.T.”

“What proportion of who will do what by when as measured by ___.”

Strategy Specific, research-based practice that guides educator to observable actions

“Teachers will . . . _______”

Activities Specific observable actions that a student is guided to perform.

“Students will . . . ______”

Academic

Types of Goals

Organizational

All students will (insert content-focused goal here)

All students will be proficient in mathematics

Increase or improve the (insert goal here)

To increase the number of partnerships the school district has with businesses in the community

Setting Goals

3- 5 GoalsNOT

Title 1 Schools

Must address any content area where school does not meet proficiency targets

Title 1 Schools Must address areas identified on CNA

Single goal may address more than one content area

http://www.michigan.gov/documents/mde/Michigan_Proficiency_Targets_413516_7.xlshttp://www.michigan.gov/documents/mde/ProficiencyTargetsFAQ_413494_7.doc



http://www.michigan.gov/documents/mde/ProficiencyTargetsFAQ_413494_7.doc

http://www.michigan.gov/documents/mde/ProficiencyTargetsFAQ_413494_7.doc

Your turn . . .

In the Goal Planner template, write a Student Goal Statement at the top.

Each goal should be:1. Student focused2. Stated broadly3. Based need identified by data

analysis

SMART Measurable Objectives

Specific

Measurable

Achievable

Realistic

Time-bound

SMART

Setting Measurable Objectives

What proportion

?

Who? What? When?Measure

dby?

87% Limited English

Proficient 1st and 2nd graders

demonstrate a proficiency in problem solving in

math

3/14/2013 measured by the MEAP.

http://extranet.advanc-ed.org/assist_resources_and_tools/docs/ASSIST_GoalsPlansGuide.pdf



Design of Measurable Objective

• Proportion? (%)• Who? (subgroup or all)• What? (content &

skill/strand)

• Measured By? (assessment/season/year)

• By When? (date)

100% of all students will demonstrate a proficiency in number fluency in math by moving from 46 % to 49% as measured by the State Summative Assessment by June, 2015.

State Summative—MEAP, MME, alt.

Local Assessment data

Norm Referenced—ACT, Explore, Plan, etc.

Data MeasurementTriangulation

What data is in your Measurable Objective?

• Plan to develop strategies around those targets• Plan to monitor that data in

coming school year• Plan to bring current SY data to

the June, 2014 Retreat

Your turn . . .

In the Goal Planner template, record any measurable objectives for each goal

Each M.O. includes 1. Proportion? (%)2. Who? (subgroup or all)3. What? (content & skill/strand)4. Measured By? (assessment/season/year)5. By When? (date)

The Big Picture

Selecting Best Practices

Is this the right thing

to do?

Can we do it the

right way?

Right Thing?


to do?

•Does the practice align to our need?•Will the practice be a good

fit with other district and school needs and priorities?

Is This the Right Thing to Do?

http://media.mivu.org/mvu_pd/micsi/RightThingToDoHandout.pdf

Right Way?

Can we do it the

right way?

Can we implement with fidelity?


to do?

Methodology

FrequencyFidelity

Non

Negotiabl

e

Right Way?

Can we do it the

right way?

• Similar to research situation?• Build commitment and readiness?• Develop technical and adaptive skills in

leadership? • Put the right people in the right place? • Build the necessary capacity, knowledge, and

skills and provide on-going support? • Put the required structures in place?• Commit the necessary financial resources?• Monitor fidelity?

How Do We Decide?

Is this the right thing to

do?

Choose a different best

practice

Can we do it the

right way?

Proceed to Plan

stage and develop

an action plan

Choose a different best

practice

Yes

Yes

No

No

Strategy

• What staff members will do instructionally to help students achieve the measureable objective.• This also includes the research &

MTSS Tier I, II, or III identification.

Strategy-Research

• What Works Clearinghouse• Data 4 SS• Mel.org• Michigan Department of Education• Research Works

• Ask your Regional SI Coordinator for additional supporting research.


Next Steps-JUNE SESSION




Who/What•Who are you bringing in June?

•Why are you bringing them?

•What additional data are you bringing in June?

Data Mining Points to consider:Reading Writing Math Science Coordinated

School Health

Letter/Sound Id, Rigby, DRA, Fountas & Pinnell, QRI,STAR, EXPLORE, MEAP, MEAP Access, MI Access, PLAN, ACT, MME, COMPASS,CWCA, NWEA

Writing Prompt, MEAP, MEAP Access, MI Access, Writing Prompts, Grammar Quizzes, PLAN, ACT, MME, CWCA, NWEA

Addition/Subtraction, CWCA, MEAP, MEAP Access, MI Access, Quarter Exams, PLAN, ACT, MME, COMPASS, NWEA

MEAP, MEAP Access, MI Access, Physical Science Assessments, PLAN, ACT, MME, NWEA

Bully Prevention, Nurtured Heart, Parent Surveys, MiPhy


Next Steps-SEPTEMBER SESSION



Facilitated Work Time for Goal Building

Questions

Questions/Comments?Please contact:

Susan Townsend ([email protected]) Maeghan McCormick ([email protected]) Sandy White ([email protected]) Steve Doerr ([email protected]) Jennifer Fox ([email protected])

Or visit the MDE - School Improvement website

mailto:[email protected]





http://www.michigan.gov/mde/0,1607,7-140-28753_38959---,00.html

Date post:	11-Jan-2016
Category:	Documents
Upload:	amelia-page
View:	212 times
Download:	0 times

School Improvement Facilitators Network May 7, 2014 1.

Documents