1

Data Retreat Refresher

Judy K. SargentDirector, CESA 7 School Improvement ServicesMay 2008

2

Main Points

1. The Full Cycle of School Improvement

2. Setting Up for Success

3. Data Collection

4. Team Readiness

5. Data Analysis Steps

6. Sequencing and Pacing

7. Primary Concerns

8. Aspiring Goals

9. Crafting the SIP

Q & A and Discussion Throughout

3

The Full Year Cycle of Improvement

LT

Steps 3-5Data Retreat®

Workshop

LTLT

Steps 7 & 8Commitment Retreat

Measures, Monitoring & Roll-Out

Steps 5 & 6Clarifying Retreat

From Issues to Objectives & Strategies

LT

= Student Assessment

= Mini-Data Retreat

LTSteps 1-2Prep Work

= Monitoring Checkpoint

LT

LTLTLT

LT

LT

LT

LT = Leadership Team Meeting

LT

Annual School Improvement Plan

Improving Planning Cycle

Reflection Retreat

4

Setting Up for Success: Vision, Purpose & Process Vision

Establish a vision of a data-driven, forward-thinking professional learning community that puts kids’ learning central to all work

Purpose Clarify purpose to collectively uncover student

performance patterns that inform professional practices by school staff; for the development of an annual school improvement plan

Process Understand that the Data Retreat is the catalyst for a

year long commitment to ongoing improvement work led by the school improvement leadership team

5

Setting Up for Success: LeadershipReference to: Rick DuFour, Fall 1999 Journal of Staff Development

Principal’s Leadership Responsibilities (DuFour)

At theData Retreat

“Loose” on teachers’ strategies AND “tight” on vision and collective commitments

Set the vision clearly with the Leadership Team (LT); ground rulesHonor the strategies they suggest

Create collaborative structures that focus on teaching and learning

Inform the LT of their role all year and of other collaborative teams to support the goal

Pose questions to keep the focus on issues of teaching and learning

Guide the focus on student data observationsGuide teachers’ explanations to “hypotheses of practice”

Provide staff with training & parameters to make good decisions

Have a clear agenda for successUse the Data Retreat guides for guiding questions and processes

6

Setting Up for Success- Logistics

Focus Reading, writing, math, behavior?

Time One full day (8:00 to 3:00) per focus area; lunch on site

Location NO INTERRUPTIONS; copier, internet

Stuff Computer & LCD Projector; Power Point presentations to guide work Copies of data Flip charts & easel Supplies – highlighters in 4 colors; flip chart markers’ post-it notes;

calculators Refreshments and lunch

7

Data Collection

STUDENT DATA Local spring data in focus

area Examples: leveled reading

data; attendance data; failure rates; common Algebra test results, etc.

State test reports from past 3 to 5 years

STAFF DATA Perceptions data

All student data must be disaggregated by SUBGROUPS

Organize data so that ALL

team members can

view it (binders, etc)

8

Data Collection: Organize Data

In paper form In “data binders” –

so all members can view and get their hands on the data

Organize by data source so data are easy to locate

Hill School 2008 Data

Sta

te Te

st

MA

P T

est

9

Team Readiness: Leadership Teams

Necessary To drive all school

improvement efforts Integral

Embedded in the culture of the school as a Professional Learning Community

Collaborative Shared leadership

where teachers partner with the administrator to carry out school improvement efforts

1. Attend the Data Retreat

2. Shape the improvement plan direction and details

3. Meet monthly all year

4. Review progress data

5. Recommend adjustments based on data

10

Team Readiness-Steps Prior to the Retreat

Choose “stars” and “worker bees” Meet prior to the retreat

AGENDAFull School Improvement CycleRoles as Leadership Team MembersSchedule for summer retreat AND work all yearGround Rules for Effective TeamsSet focus area for retreat Identify data sources AND data collection

process (who will do it and organize the data)

11

Final Set Up

Set Up Data Retreat room ahead of timeBe fully prepared – all materials and

equipment set upShow the commitment and seriousness

of this process

12

Data Analysis Steps

Data Analysis Task Sequence

1. Data table

2. Graphic representation

3. Observations, discussion and documentation

4. Hypotheses

5. Classroom connections

13

Task 1: Data Table

Having a table Making a table Posting a table Using templates &

posters

GOAL – getting participants close to the numbers

3 4 5

2003-04 54 60 50

2004-05 65 55 55

2005-06 78 68 54

14

Task 2: Graphic Representation

Graph the dataOR

Highlight patterns

AYP Projections

61.0% 61.0% 61.0%

67.5% 67.5% 67.5%

74.0% 74.0% 74.0%

80.5%

87.0%

93.5%

100%

37.0% 37.0% 37.0%

47.5% 47.5% 47.5%

58.0% 58.0% 58.0%

68.5%

79.0%

89.5%

100%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

50.0%

55.0%

60.0%

65.0%

70.0%

75.0%

80.0%

85.0%

90.0%

95.0%

100.0%

School Year

% a

t Pro

fici

ent

& A

dva

nce

d

Reading Math

Reading 61.0% 61.0% 61.0% 67.5% 67.5% 67.5% 74.0% 74.0% 74.0% 80.5% 87.0% 93.5% 100%

Math 37.0% 37.0% 37.0% 47.5% 47.5% 47.5% 58.0% 58.0% 58.0% 68.5% 79.0% 89.5% 100%

2001-02

2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13

2013-14

Percent P & A in Math by Grade Level 2005-06

66

78

63

5755

53

60

64

68

0

10

20

30

40

50

60

70

80

90

100

Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9 Gr. 10 Gr. 11

Grade

% P

& A

3 4 5

2003-04 54 60 50

2004-05 82 55 55

2005-06 78 68 54

15

Task 3: Observe, Discuss, Document

Observe data patterns

Discuss what is observed

Write data findings on the flip chart

Observations

16

Task 4: Pose Hypotheses of Practice

What is it that we aredoing that might contribute to

these results?

What is it that we aredoing that might contribute to

these results?

17

Task 4: Pose Hypotheses of Practice, continued

Hypotheses should:Be explanations that come from school and

classroom factors.Be explanations about practices that can be

altered.

Hypotheses should NOT:Be regarding characteristics of individualsBe explanations about unalterable factors

18

Task 5: Suggest Connections to Practice

While looking at data and posing hypotheses – it’s natural for educators to think about solutions.

At this step – allow only a few minutes to jot down any immediate thoughts of strategies that might improve the findings.

CAUTION – don’t spend much time here – this is not the time to jump into the improvement plan. We are only acknowledging a few immediate ideas, if they arise.

Connections

19

Small Group Activity—Analyzing Proficiency Find your proficiency data On the flip chart at the top, note …

Name of test Grade levels Subject Number of years

Determine the % of (all or subgroup) students at proficient & above.

Task 1-Make a data table on your flip chart. Task 2-Graph the data.

20

Analyzing proficiency, continued

Task 3-Observe, discuss & document data findings on the flip chart.

Task 4-Pose hypotheses – note on the flip chart

Task 5-Suggest strategies

21

What You Have Observations

of student performance

Hypotheses

Connections

Go to GOALS

Go to IMPROVEMENT STRATEGIES

22

Other Student Data Analysis

Local Assessment – can you follow the same 5 steps?

1. Data table2. Graphic representation3. Observations4. Hypotheses5. Classroom connections

Behavior data Perceptions data

23

Prioritize Primary Concerns

Situation Appraisal – After all student data has been analyzed

Teams review all observations recordedTeams use group process (voting or

consensus) to determine their primary concerns for students (1 to 3 concerns)

24

Primary Concerns

What to watch for … Drifting away from data

observations Not focusing on concerns for kids

CRUCIAL STEP – Goes to GOALS

Do not move on until the teams get this.

Primary Concerns for Students

1,

2.

3.

25

Pacing – Set the pace early!

Keep teams on track through … Power point slides that show tasks Overhead or projector timer Keeping up with recorded findings on flip charts Post the steps or agenda and point to where you are

in the process.

Time Wasters

No recorder at the flip chart

If no data patterns, move on

Off topic

Not following agenda

Time Wasters

No recorder at the flip chart

If no data patterns, move on

Off topic

Not following agenda

26

About Recording Work

Flip charts – post-it paper works best Encourages conversation Gets folks close to the data and findings Creates group ownership

Use pre-made poster charts Computer recording – okay, but not

IN PLACE OF posted work. Helpful for the team during follow-up

work.

27

Teach the Process

Once teams learn the steps in the process – they can “fly” and analyze their own local data.

Remind teams to document findings.

Discuss – what local data would you expect to see at a retreat?

28

Analyzing Professional Practices

Lack of dataMay have arisen in discussions,

hypotheses and connectionsAvailable tools in the Data Retreat ®

materialsNote Literacy Practices Surveys

(Beginning on page 70.)

29

Practice Surveys Using the

Participant’s Workbook -- let’s go through the surveys.

Take a few minutes to peruse the surveys in the workbooks.

Discuss the use of tools like this at your table.

•Data Collection Survey – p. 22

•Technological Data Systems – p. 32

•Technological and Human Systems of Data Management – p. 33

•Grade Book Inventory – p. 34

•PK Literacy – p. 70

•K-2 Literacy (25) – p. 73

•K-2 Literacy (45) - p. 75

•3-5 Literacy – p. 79

•6-12 Literacy – p. 82

For Students:

•What I think about Reading – p. 90-92

•What I think about Writing – p. 93-94

•Data Collection Survey – p. 22

•Technological Data Systems – p. 32

•Technological and Human Systems of Data Management – p. 33

•Grade Book Inventory – p. 34

•PK Literacy – p. 70

•K-2 Literacy (25) – p. 73

•K-2 Literacy (45) - p. 75

•3-5 Literacy – p. 79

•6-12 Literacy – p. 82

For Students:

•What I think about Reading – p. 90-92

•What I think about Writing – p. 93-94

30

More Practice Surveys

Elementary Math (3-5) - p. 121 Secondary Math – p. 124Student Surveys:

What I think about Math and Science – p. 132-135

Observed Student Morale – p. 149 Respectful Learning Environment – p.

150 Systems of Support – p. 152 Behavior Policy and Procedures – p.

154 Analyzing Rigor and Relevance – p.

176 Instructional Strategies in Spec Ed –

p. 194 Guiding Principles Belief Survey – p.

211 Curriculum Review – p. 225

Elementary Math (3-5) - p. 121 Secondary Math – p. 124Student Surveys:

What I think about Math and Science – p. 132-135

Observed Student Morale – p. 149 Respectful Learning Environment – p.

150 Systems of Support – p. 152 Behavior Policy and Procedures – p.

154 Analyzing Rigor and Relevance – p.

176 Instructional Strategies in Spec Ed –

p. 194 Guiding Principles Belief Survey – p.

211 Curriculum Review – p. 225

Teacher Professional Development – p. 232

Parent Involvement – p. 234 Standards & Curriculum for ELLs

– p. 251 Checking Our Beliefs and Values

– p. 252

Teacher Professional Development – p. 232

Parent Involvement – p. 234 Standards & Curriculum for ELLs

– p. 251 Checking Our Beliefs and Values

– p. 252

31

Analyzing Interventions (RtI)

Highly recommended reflection activity to analyze … Adequacy of universal options

Interventions Screening assessments

Adequacy of selected options Interventions Progress monitoring

Adequacy of targeted options Interventions Progress monitoring

32

Universal Instruction(includes differentiation)

Special education servicesSelected

Interventions

Supports Universal

Classroom (does not supplant)

Targeted Interventions

Small group

s flexible

Highly Focused on Skill in Need

Taught by

Expert

Progress Monitoring

at least monthly

Progress Monitoring

at least twice/mont

h

1:1Taught

by Expert

Intensely Focused on Skill in Need

Supports Universal(does not supplant)

33

Reviewing Improvement Goals

Improvement goals are the bridge from data analysis to improvement planning.

Data Analysis Improvement Planning

GOALS

34

ASPIRING GOALS

A Assessable

Measurable with a defined assessmentprocess or tool

S Specific Intensity of focus for students

P Purposeful Stretches with a measurableacceleration of improvement

I Inclusive Includes all students in the school

R Reinforcing

Reinforces and aligns withdistrict strategic action plans

I Involving Involves all staff as “doers” of the goal

N Now States the target data as acurrent annual goal

G Gaps Addresses equity in success

35

Sample Aspiring Goal

We, all teaching staff at Elm Creek Middle School will improve skills in analyzing expository text in all content areas of all 6th, 7th, and 8th grade students so that 85% show proficient and advanced levels, while accelerating the performance of students with disabilities so that 67% show proficient or advanced levels on the 6th, 7th and 8th grade reading benchmark spring assessments by May, 2009.

36

A Assessable

Measurable with a defined assessmentprocess or tool

A. ASSESSABLE- measurable with an assessment process or tool

Names the Assessment – Names the specific tool(s) or process to be used to measure the goal.

Incomplete Naming – Suggests the process without explicitly stating what it is.

No Assessment – Does not list the measuring device(s) or process for the goal.

… as measured by the 5th grade 4th quarter reading comprehension benchmark assessment.

… on the kindergarten numbers exit checklist.

… as assessed with the Elm Creek District writing process 8th grade rubric for persuasive writing.

37

S Specific Intensity of focus for students

S. SPECIFIC-intensity of focus for students

Sharp – This goal has a sharply intensified focus on specific student learning or behavior skills.

Directed – This goal has a sense of direction on certain student skills.

Broad – This goal is a statement of improvement in a broad or ambiguously stated area of skills.

… in reading fluently fiction and nonfiction text in all content areas …... in drawing inferences from literal and informational text …… in word meaning on grade level targeted vocabulary in the content areas, so that ……in literal and analytical comprehension of technical and nonfiction text, …

38

P Purposeful Stretches with a measurableacceleration of improvement

TEACHERS: Teach like your hair is on fire!

PRINCIPALS:Lead improvement like your hair is onfire!

Accelerate, Stretch & Intensify!

Lower and

Slower

P. PURPOSEFUL-stretches with a measurable acceleration of improvement

High Expectations – This goal shows that the trajectory of improvement is accelerated enough to make significant, but reasonable, measurable gains for students.

Solid Growth – This goal shows solid growth in measured skill proficiency of students.

Inches Along – This goal shows conservative increments of improvement.

39

Stretch, for example2005-06 2006-07 2007-08

61% at proficient/advanced on MAPs

67% at proficient/advanced on MAPs

80% at proficient/advanced on MAPs

57% at grade level reading benchmark

60% at grade level reading benchmark

85% at grade level reading benchmark

81% meeting targeted growth on MAPs

76% meeting targeted growth on MAPs

95% meeting targeted growth on MAPs

72% proficient/advanced on WKCE

75% proficient/advanced on WKCE

95% show lexile growth

40

I Inclusive Involves all students in the school

I. INCLUSIVE-involves all students in the school

Inclusive – This goal will have a positive impact on the learning of all students; and also may have a sharper impact on a specific group of students.

Majority – This goal will involve a majority of the students.

Few – This goal involves a subset of students.

… so that …85% of all kindergarten students …; 89% of all 1st grade students; 91% of all 2nd grade students

… 81% of all 6-8 grade students and 75% of all studentswith disabilities …

41

R Reinforcing

Reinforces and aligns withdistrict strategic action plans

R. REINFORCING-reinforces and aligns with district strategic action plans

Fully Aligned – The goal is directly aligned to achievement of a district strategic goal.

Partially Aligned – This goal is partially aligned to a district’s strategic goal.

Not Aligned – This goal is not aligned with any of the district’s strategic goals.

Check that the student outcome in the goal is aligned with district strategic goals.

42

I Involving Involves all staff as “doers” of the goal

I. INVOLVING-involves all staff as “doers” of the goal

All Staff – Explicitly states that all staff in the school will be involved and responsible for implementing the goal.

Suggests Staff – Implies which staff will be implementing the goal.

No Staff – Does not state which staff are involved or responsible for implementing the goal.

•We, all staff of Willmar Middle School, will …

•We, all educators in Heritage Elementary school will collaborate to improve the …

•All teaching pupil services staff of Bayport High School will work as a professional learning community to …

43

N Now States the target data as acurrent annual goal

N. NOW- states the target date as a current annual goal

Specific Date – Specifies the date as an annual target date by which the goal will be achieved.

Window of Time – Suggests a period of time for the goal to be accomplished that may extend beyond the current timeframe.

No Date or Extends Beyond – Lacks a due date to accomplish the goal or sets a due date that is far beyond “now”.

The due date is written for the goal to be measured and accomplished within the current school year.

•… by May, 2009.

•… on the spring, 2009 assessments.

44

G Gaps Addresses equity in success

G. GAPS-addresses equity in success

Equity-Oriented – Focuses on closing gaps and toward equitable success between underperforming student groups and their comparison groups.

Suggests Equity – This goal suggests closing gaps between groups – and is not explicit.

Misses Equity – This goal has little or no reference to closing gaps or meeting individual needs.

If any gaps exists, they must be addressed in the goal.

… at least 85% of all students meet targeted RIT growth, while the median RIT gap between students with disabilities and nondisabled students decreases by 10 RIT points… 99% of boys and girls with below grade level reading benchmark in grades k-4 reach grade level benchmark while boys and girls at grade level benchmark increase by 2 levels by …… 100% of boys in grades 9-12 increase their Lexile levels from fall to spring, and 90% of girls increase their lexile levels from fall to spring

45

Categorizing Issues

Universal Classroom Instruction to Engage All Learners

Universal Classroom Instruction to Engage All Learners

Interventions & Enrichments

Interventions & Enrichments

Collaborative Culture of ImprovementCollaborative Culture of Improvement

Student Success

School Climate

Student Success

School Climate

Assessment and Data UseAssessment

and Data Use

Strategic Family and Community Partnerships

Strategic Family and Community Partnerships

Professional LearningProfessional Learning

46

Study Best Practices

Universal Instruction

Strategies for high engagement; differentiation; high rigor to achieve the goal for all students

Interventions & Enrichments

Strategies for effective interventions for struggling learners and effective enrichments for advanced learners

Assessment & Data Use

Strategies for periodic assessments and classroom formative assessments FOR learning; effective data management

Collaborative Culture

Strategies for frequent professional learning communities to do the work of improvement toward achieving the goal

Student Success Climate

Strategies that eliminate failure and provide success for every student; grading reform; positive school climate with support

Family & Community

Strategies to involve parents positively in their children's’ learning; to involve the community in school-wide efforts

Professional Learning

Strategies for relevant and effective professional development of all staff to support achievement of the goal

47

Quest for Fidelity

Team Task: How do we maintain fidelity to our purpose?

Fidelity to …

Desired Student Outcomes

Stated in the Goal

Desired Student Outcomes

Stated in the Goal

Tasks to be Completed

Stated in the Improvement Plan

Tasks to be Completed

Stated in the Improvement Plan

Periodic Measures of Student Progress

Aligned to outcome measures

Monitoring Checkpoints of Task Completion

Aligned to tasks and objectives

48

The Full Year Cycle of Improvement

LT

Steps 3-5Data Retreat®

Workshop

LTLT

Steps 7 & 8Commitment Retreat

Measures, Monitoring & Roll-Out

Steps 5 & 6Clarifying Retreat

From Issues to Objectives & Strategies

LT

= Student Assessment

= Mini-Data Retreat

LTSteps 1-2Prep Work

= Monitoring Checkpoint

LT

LTLTLT

LT

LT

LT

LT = Leadership Team Meeting

LT

Annual School Improvement Plan

Improving Planning Cycle

Reflection Retreat