Organizational Learning Through

Data ReviewILT

January 12, 2015

Outcomes Understand the district approach to evaluating

effectiveness of curriculum and instructional delivery through data review processes

Appreciate the potential collective impact of small individual improvements in STAAR performance

Recognize the district CBA data analysis process

Organizational Learning• The detection and correction of error (Argyris & Schön, 1978)• Organization-wide continuous improvement process that

enhances the collective ability to accept, make sense of, and respond to internal and external change

• Systematic integration and collective interpretation of new knowledge gained through risk-taking as experimentation resulting in organizational action

Variable Inputs

Predictable

Quality Outputs

DSA

PResponsive Processes

Argyris, C., & Schön, D. (1978). Organizational learning: A theory of action perspective. Reading, MA: Addison Wesley.

The district journey

STAAR Results District Data Review Tier 1 Priorities

CBA Results CBA Analysis Protocol

District data review process

Improved Alignment of Instructional

Delivery

CBA Production Errors Process Analysis Improvements to

Production Process

Variability in processes limits our ability to effect change

2012 2013 2014 201560

65

70

75

80

85

90

95

100STAAR Reading Performance (English)

R5R3R8R4R7R6E2E1

Year

Perc

enta

ge M

et L

evel

IIThe current state

What is the appropriate response to data such as these?

2012 2013 2014 201560

65

70

75

80

85

90

95

100STAAR Math Performance (English)

M5A1M3M7M6M4M8

Year

Perc

enta

ge M

et L

evel

IIThe current state

Why is there less variability in performance in 2015 than in previous years?

2012 2013 2014 201560

65

70

75

80

85

90

95

100STAAR Performance (English)

Bio

US

Sc5

Sc8

W7

W4

So8

Year

Perc

enta

ge M

et L

evel

IIThe current state

Tier 1 Priorities

Vocabulary of the Standards

Workshop Model

Guided Reading

Three Types of Thinking(Close Reading)

Tightening instructional delivery to improve student learning

POWER2Theof

Get 5

POWER2Theof

POWER2Theof

1 44 2 6 1312 9 3419 8 381143 4 422041153526 7 3216 5 3933403727281417 3 36102430232218292125310

10

20

30

40

50

60

2015 STAAR Grade 5 Science

Item Number

Diffi

culty

(per

cent

age

inco

rrec

t)

Level I Level II Level III

4 pt rubric x 2 scorers = 8 points/essay

8 points/essay x 2 essays = 16 raw score pts for essays

1 extra point/essay = 1 pt x 2 scorers x 2 essays = 4 pts extra

Grade 4 Writing

PlusOne impact on overall score

Essays

1 pt x 2 scorers x 2 essays = 4 pts

1 pt x 2 scorers x 2 essays x 2 = 8 pts

Grade 4 Writing

Grade 7 Writing

1 pt x 2 scorers x 1 essay x 3 = 6 ptsEnglish I and English II

Calibrating instruction and assessment using district CBAs

Lessons

Unit 1

Common assessme

ntCA

Assessments

CA CA

Regroup, Reteach, Redesign

Unit 2 Unit 3 Unit 4

PLC

District CBA

PLCEvaluate alignment between classroom instruction, common assessments and district CBAs

CurriculumCoordinators

District Data ReviewCBA

Analysis Protocol

Suggestions for Reteach/ReviewCurriculum Improvements

PLCs have a responsibility to maintain quality control over instructional delivery

The current state

2012 2013 2014 20150

10

20

30

40

50

60

70

80

Fall CBA Performance - Reading

2nd4th7th5th8th6th3rdE1E2

Year

Perc

enta

ge M

et S

atisf

acto

ry S

tand

ard

(70%

cor

rect

)

The current state

2012 2013 2014 20150

10

20

30

40

50

60

70

80

Fall CBA Performance - Math

2nd

7pap

5th

8th

4th

A1ms

6th

3rd

7th

A1hs

Year

Perc

enta

ge M

et S

atisf

acto

ry S

tand

ard

(70%

cor

rect

)

The current state

2012 2013 2014 20150

10

20

30

40

50

60

70

80

Fall CBA Performance - Science & Social Studies

Sci 5

Soc 8

Sci 8

Bio

US

Year

Perc

enta

ge M

et S

atisf

acto

ry S

tand

ard

(70%

cor

rect

)

CBA central data review process Identify 3 items with the lowest performance on two tests under

the coordinator’s purview Determine what students should know and be able to do to

correctly answer the questions Use item answer distribution to determine common errors in

thinking or problem solving Ascertain where in the scope and sequence students had an

opportunity to learn the necessary content or skills Celebrate data indicating performance improvement or mastery of

standards Based on OFIs identified, develop materials for review or reteach

and make updates to curriculum documents

TEKS 4.5B. Predict the changes caused by heating and cooling such as ice becoming liquid water and condensation forming on the outside of a glass of ice water4.5B (Unit overview, 1st 6 weeks):Students might think that condensation on a glass comes from inside the glass, rather than from the outside air; be sure to clarify that water vapor in the air condenses on the glass. See note for 4.5B in Resources below.4.5B ResourcesAIMS Kool Kups; STEMScopes Engage; STEMScopes Explore Part 3, etc.

60%20%11%8%

Data review take-aways Recognition that we need to be finding solutions, rather than

making excuses Find ways to address gaps both moving forward this year, and by

improving curriculum and delivery for next year Currently in the learning curve to determine the most efficient

and effective way to review CBA data and respond in a manner conducive to improving instruction Meeting with HEB coordinators Beefing up review process for February CBAs Implementing expectations for similar reviews at the campus

level next year