Getting Serious about School Reform:

Three Critical Commitments Districts and Schools Should Make

40 Years

Same Problems

The problem isn’t at the classroom teacher level.

The problem is at the systems level.

Teacher School District Student

P50 P50 P50 P50

Tentative Findings Reading

Teacher School District Student

P50 P50 P50 P50

P84 P50 P50 P60

Tentative Findings Reading

Teacher School District Student

P50 P50 P50 P50

P84 P50 P50 P60

P98 P50 P50 P70

Tentative Findings Reading

Teacher School District Student

P50 P50 P50 P50

P84 P50 P50 P60

P98 P50 P50 P70

P50 P84 P84 P57

Tentative Findings Reading

Teacher School District Student

P50 P50 P50 P50

P84 P50 P50 P60

P98 P50 P50 P70

P50 P84 P84 P57

P50 P98 P98 P63

Tentative Findings Reading

Teacher School District Student

P50 P50 P50 P50

P84 P50 P50 P64

P98 P50 P50 P76

P50 P84 P84 P59

P50 P98 P98 P67

Tentative Findings Mathematics

Districts and schools must find ways to ensure effective practices occur in every

classroom, but still provide flexibility to the classroom

teacher.

For decades we have operated from the belief that this is

impossible in U.S. education.

Weick (1976,1982)Tightly Coupled Organizations…

• Clearly defined unit of accountability

• Clearly defined criterion for success

• Clear understanding of what produces success

• Entire system mobilizes when criterion for success is not met.

Weick’s Conclusions

Districts and schools are not tightly coupled regarding individual student achievement as the unit of analysis for success.

Weick’s Conclusions

Districts and schools are not tightly coupled regarding the

criterion of student achievement,

BUT THEY ARE TIGHTLY COUPLED REGARDING

THE BUS SCHEDULE AND PAYROLL.

Based on the notion that schools are inherently loosely-coupled we began operating

from a “hands offs” pespective at the classroom level.

Everybody is equal.

The classroom teacher is an autonomous, independent

contractor.

Within a loosely coupled district or school, there will be

individual points of light but no possibility of a strong, unified

focus that can eradicate the 40 year old problems inherent in

the system.

Bellamy and Colleagues (2005)Highly Reliable Organizations (HROs)

Bellamy and Colleagues (2005)Highly Reliable Organizations (HROs)

…• Clearly defined unit of accountability

• Clearly defined criterion for success

• Clear understanding of what produces success

• Entire system mobilizes when criterion for success is not met.

Bellamy’s Conclusions

Districts and schools are not tightly coupled (HROs) regarding individual student achievement as the unit of analysis for success.

Bellamy’s ConclusionsDistricts and schools are not tightly coupled regarding the

criterion of student achievement,

BUT THEY CAN BE IF

THEY ARE WILLING TO COMMIT TO SPECIFIC ACTIONS THAT ARE DISTRICTWIDE (OR

SCHOOLWIDE).

Three Critical Interventions (COMMITMENTS)

• A system of individual student feedback on learning goals at the classroom, school, and district levels (standards-based grading using formative assessments)

• Ensuring effective teaching in every classroom

• Building background knowledge for all students

All three can be approached at the school level but are more powerful at the district level.

Three Critical Interventions(COMMITMENTS)

• A system of individual student feedback on learning goals at the classroom, school, and district levels (standards-based grading using formative assessments)

• Ensuring effective teaching in every classroom• Building background knowledge for all students

Phase I: Track student progress using a formatively-based system.

This draws on the research from formative assessments.

Feedback from classroom assessments should provide students with a clear picture of their progress on learning goals and how they might improve.

# of studies Characteristic of Feedback from Classroom Assessment

Percentile Gain/Loss

Bangert-Drowns, Kulik, Kulik, & Morgan,1991

6 Right/wrong -3

39 Provide correct answers 8.5

30 Criteria understood by student vs. not

understood

16

9 Explain 20

4 Student reassessed until correct

20

Feedback from classroom assessments should provide students with a clear picture of their progress on learning goals and how they might improve.

# of studies Characteristic of Feedback from Classroom Assessment

Percentile Gain/Loss

Fuchs & Fuchs 1988

89 Displaying results graphically

26

49 Evaluation by rule

[uniform way of interpreting results of

classroom assessments using a tight logic)

32

49 Evaluation by rule

[uniform way of interpreting results of classroom assessments using a tight logic)

32

You can’t rely on the 100-point scale.

C. Item 15–16

Two items that asks for application in novel situations that go beyond what was explicitly taught

Total for section =

Total for section =

Total for section =

A. Items 1–10

Ten items that require recall of important but simpler content that was explicitly taught

B. Items 11–14

Four items that ask for application of complex content that was explicitly taught AND in situations similar to what was taught.

Total /100

Total /100

Total for section=

Total for section=

Total for section=

/40

/20

/40

A. Items 1–10

Ten items that require recall of important but simpler content that was explicitly taught

B. Items 11–14

Four items that ask for application of complex content that was explicitly taught AND in situations similar to what was taught.

C. Item 15–16

Two items that asks for application in novel situations that go beyond what was explicitly taught

+

+

Total for section =

Total for section =

Total for section =

All correct

Two correct

None correct

A. Items 1–10

Ten items that require recall of important but simpler content that was explicitly taught

B. Items 11–14

Four items that ask for application of complex content that was explicitly taught AND in situations similar to what was taught.

C. Item 15–16

Two items that asks for application in novel situations that go beyond what was explicitly taught

Total /100

/40

/20

/40

Total /100

+

+

Total for section =

Total for section =

Total for section =

40/40

20/40

0/20

All correct

Two correct

None correct

A. Items 1–10

Ten items that require recall of important but simpler content that was explicitly taught

B. Items 11–14

Four items that ask for application of complex content that was explicitly taught AND in situations similar to what was taught.

C. Item 15–16

Two items that asks for application in novel situations that go beyond what was explicitly taught 60

4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class

3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught

2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes

1 With HELP, a partial knowledge of some of the simpler and complex details and processes

0 Even with help, no understanding or skill demonstrated

Scale

4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class

3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught

2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes

1 With HELP, a partial knowledge of some of the simpler and complex details and processes

0 Even with help, no understanding or skill demonstrated

Scale

4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class

3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught

2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes

1 With HELP, a partial knowledge of some of the simpler and complex details and processes

0 Even with help, no understanding or skill demonstrated

Scale

4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class

3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught

2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes

1 With HELP, a partial knowledge of some of the simpler and complex details and processes

0 Even with help, no understanding or skill demonstrated

Scale

4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class

3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught

2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes

1 With HELP, a partial knowledge of some of the simpler and complex details and processes

0 Even with help, no understanding or skill demonstrated

Scale

4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class

3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught

2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes

1 With HELP, a partial knowledge of some of the simpler and complex details and processes

0 Even with help, no understanding or skill demonstrated

Scale

4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go beyond what was taught in class

3.5 In addition to exhibiting level-3 performance, partial success at in-depth inferences and applications that go beyond what was taught in class

3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught

2.5 No major errors or omissions regarding any of the simpler information and/or processes and partial knowledge of the more complex information and processes

2 No major errors or omissions regarding the simpler details and processes BUT major errors or omissions regarding the more complex ideas and processes

1.5 Partial knowledge of the simpler details and processes, but major errors or omissions regarding the more complex ideas and processes

1 With help, a partial knowledge of some of the simpler and complex details and processes

.5 With help, a partial knowledge of some of the simpler details and processes but not of the more complex ideas and processes

0 Even with help, no understanding or skill demonstrated

Scale

C. Item 15–16 (SCORE 4.0)

Two items that asks for application in novel situations that go beyond what was explicitly taught

ALL CORRECT

2 OF 4 CORRECT

NONE CORRECT

A. Items 1–10 (SCORE 2.0)

Ten items that require recall of important but simpler content that was explicitly taught

B. Items 11–14 (SCORE 3.0)

Four items that ask for application of complex content that was explicitly taught AND in situations similar to what was taught.

RUBRIC SCORE= 2.5

This represents a shift from an emphasis on isolated classroom

activities for which points are assigned to an emphasis on student progression

toward specific learning goals.

This approach is to be used as another measurement tool in a system of

inherently imprecise measurement tools.

Cizek (2007)State test for a large Midwestern state: rel = .87

Subscales: Estimation and mental computation, geometry, measurement, number and number relations, patterns, algebra, problem solving

Subscale reliabilities = .33 to .57

Cizek (2007)State test for a large Midwestern state: rel = .87

Subscales: Estimation and mental computation, geometry, measurement, number and number relations, patterns, algebra, problem solving

Subscale reliabilities = .33 to .57

Reliability of differences scores = .015

…It …might be that the dependability of conclusions about differences in sub-area performance is nearly zero.

In many cases, a teacher who flipped a coin to decide whether to provide the pupil with focused intervention in algebra (heads) or measurement (tails) would be making the decision about as accurately as the teacher who relied on an examination of sub-score differences for the two areas.

Phase II. Design learning goals in all subject areas and redesign reporting system (i.e. report cards).

Reporting Topics for Language Arts• Word recognition and vocabulary• Reading comprehension• Literary analysis• Spelling• Language mechanics and conventions• Research and technology• Evaluation and revision• Listening comprehension• Analysis of oral media• Speaking applications

Many districts like to organize reporting topics into strands.

Language Arts Strands and Topics

READING– Word recognition and

vocabulary– Comprehension– Literary analysis

WRITING– Spelling– Language mechanics

and conventions

– Research and technology

– Evaluation and revisions

LISTENING & SPEAKING– Listening

comprehension– Speaking applications

3.00 – 4.00 = A2.50 – 2.99 = B2.00 – 2.49 = C1.50 – 1.99 = DBelow 1.50 = F

3.50 – 4.00 = Advanced2.50 – 3.49 = Proficient1.50 – 2.49 = BasicBelow 1.50 = Below Basic

3.00 - 4.00 = A = 95%2.50 - 2.99 = B = 85%2.00 - 2.49 = C = 75%1.50 - 1.99 = D = 65%Below 1.50 = F = 60%

This represents a shift from an approach in which grades are

interpretable only in the context of individual teachers’

classrooms to an approach in which grades can be interpreted

uniformly from teacher to teacher.

Phase III. Implement in a staged fashion.

At this point you’re pretty serious.

If you really want to get serious then….

If you really want to get serious then….replace the time-based

system with a performance-based system.

The time based system is over 100 years old.

The Requirements

of a Time-Based System

1. Must be in-class the entire year.

2. Must behave appropriately.

3. Must complete the work assigned them.

4. Must figure out what each individual. teacher expects of them.

5. Can be passed on without adequate knowledge if they are successful at 1–4.

Students:

Some Unintended Consequences of a Time-Based System

1. Students succeed if they can figure out and are willing to follow the rules.

2. Those students who do not figure out the rules or are unwilling to follow them get so far behind relatively quickly that they have a high probability of dropping out.

The Requirements of a Performance-Based System

1. Must demonstrate competence in important content knowledge.

2. Must behave appropriately if they wish to be involved in classes.

3. Must take some responsibility for their own learning.

4. Cannot be passed on until they demonstrate competence in important knowledge.

Students:

Some Intended Consequences of a Performance-Based System

1. Students do not have to figure out the rules from teacher to teacher.

2. To catch up or move ahead at an accelerated pace, students do not have to spend a specific amount of time in class. Rather, they must demonstrate competence in important content.

3. There are fewer dropouts and more students completing graduation.

Level 1

Level 2

Level 9

Level 10

NO GRADE LEVELS

K-2

3-5

LOWER DIVISION (9-10)

UPPER DIVISION (11-12)

GRADE LEVEL BANDS(Keep Course Structure)

6-8

Three Critical Interventions(COMMITMENTS)

• A system of individual student feedback on learning goals at the classroom, school, and district levels (standards-based grading using formative assessments)

• Ensuring effective teaching in every classroom

• Building background knowledge for all students

Phases of Ensuring Effective Teaching in Every Classroom

• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.

• Phase I: The school/district develops a common “language of instruction” or model.

• Phase II: Teachers systematically interact about effective teaching using the model.

• Phase III: Teachers systematically observe master teachers and each other using the model.

• Phase IV: The school or district monitors the effectiveness of individual teacher’s instructional styles as a form of teacher feedback.

Phases of Ensuring Effective Teaching in Every Classroom

• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.

Phases of Ensuring Effective Teaching in Every Classroom

• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.

• Phase I: The school or district develops a common “language of instruction” or model.

Start with someone else’s model and adapt it to your own needs.

The Art and Science of Teaching as a beginning place.

Q1: What will I do to establish and communicate learninggoals, track student progress, and celebrate success?Q2: What will I do to help students effectively interact withnew knowledge?Q3: What will I do to help students practice and deepentheir understanding of new knowledge?Q4: What will I do to help students generate and testhypotheses about new knowledge?Q5: What will I do to engage students?

Q6: What will I do to establish or maintain classroom rules and procedures? Q7: What will I do to recognize and acknowledge adherence to and lack of adherence to classroom rules and procedures?Q8: What will I do to establish and maintain effectiverelationships with students?Q9: What will I do to communicate high expectations for allstudents?Q10: What will I do to develop effective lessons organizedinto a cohesive unit?

Observing a lesson looks very different from the perspective of

The Art and Science of Teaching.

Fundamental Segments of a Classroom Instruction

• Segments that are routine components of every lesson

• Content specific lesson segments

• Segments that must be enacted on the spot

What do you look for as routine components of every lesson?

• Rules and procedures (Q 6)

• Communicating learning goals (Q1)

• Tracking student progress (Q1)

• Celebrating success (Q1)

What do you look for as routine components of every lesson?

• Reviewing important rules and procedures• Reviewing learning goals• Reviewing student progress• Celebrating success

Fundamental Segments of Classroom Instruction

• Segments that are routine components of every lesson

•Content specific lesson segments

• Segments that must be enacted on the spot

• Critical input experiences (Q2)

• Knowledge practice and deepening activities (Q3)

• Hypothesis generation and testing tasks (Q4)

Identify the type of content segment that is the focus of the

lesson.• Does this segment involve critical input

experiences (Q2) ?

• Does this segment involve knowledge practice and deepening activities (Q3) ?

• Does this segment involve hypothesis generation and testing tasks (Q4) ?

If the segment involves a critical input experience what do you

expect to see?

If the segment involves a critical input experience what do you

expect to see?• Previewing activities

• Info presented in small chunks

• Students processing each chunk in small groups

• Students summarizing and taking notes after content has been introduced

• Students reflecting on their learning

If the segment involves knowledge practice and deepening activities

what do you expect to see?

If the segment involves knowledge practice and deepening activities

what do you expect to see?• Brief review of content• Activities involving similarities and differences• Activities involving identifies errors in thinking• Activities involving massed and distributed

practice• Homework possibly used as an extension of

these activities

If the segment involves hypothesis generating and testing tasks what

do you expect to see?

If the segment involves hypothesis generating and testing tasks what

do you expect to see?• Brief review of content• Students working individually or in groups on

long term tasks• Teacher acting as facilitator and resource

provider

Fundamental Segments of Classroom Instruction

• Segments that are routine components of every lesson

• Content specific lesson segments

•Segments that must be enacted on the spot

• Engagement activities (Q5)

• Consequences regarding rules and procedures (Q7)

• Relationships (Q8)

• Expectations (Q9)

What do you look for regarding segments that must be enacted on

the spot?

What do you look for regarding segments that must be enacted on

the spot?• Engagement activities when student lose focus• Acknowledge of rules and procedures being

followed or not being followed• Behaviors that forge positive relationships with

students• Attention to behaviors that communicate high

expectations for all students

1. Learning Goals and Feedback2. Interacting with New Knowledge3. Practicing and Deepening4. Generating and Testing Hypotheses5. Student Engagement6. Establishing Rules and Procedures7. Adherence to Rules and Procedures8. Teacher-Student Relationships9. High Expectations

SupervisingThe Art and Science of Teaching

Learning Goals and Learning Goals and FeedbackFeedback

Rules and ProceduresRules and Procedures

INVOLVES ROUTINES

ENACTED ON THE SPOTStudent EngagementStudent Engagement

High ExpectationsHigh Expectations

T

eac

her

/Stu

de

nt R

ela

tion

ship

sT

eac

her

/Stu

de

nt R

ela

tion

ship

s Ad

here

nce to

Ru

les a

nd P

roce

dures

Ad

here

nce to

Ru

les a

nd P

roce

dures

Generating/ Testing

Hypotheses

Practicing and

Deepening

Interacting with New

Knowledge

Supervising

The Art and Science of Teaching

ADDRESSES CONTENT IN SPECIFIC WAYS

Phases of Ensuring Effective Teaching in Every Classroom

• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.

• Phase I: The school or district develops a common “language of instruction” or model.

• Phase II: Teachers systematically interact about effective teaching using the model.

Phases of Ensuring Effective Teaching in Every Classroom

• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.

• Phase I: The school or district develops a common “language of instruction” or model.

• Phase II: Teachers systematically interact about effective teaching using the model.

• Phase III: Teachers systematically observe master teachers and each other using the model.

At this point you are pretty serious.

If you really want to get serious then….systematically monitor how much students learn in

teachers’ classrooms.

Phases of Ensuring Effective Teaching in Every Classroom

• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.

• Phase I: The school/district develops a common “language of instruction” or model.

• Phase II: Teachers systematically interact about effective teaching using the model.

• Phase III: Teachers systematically observe master teachers and each other each using the model.

• Phase IV: The school/district monitors the effectiveness of individual teacher’s instructional styles as a form of teacher feedback.

Teacher ID

Q1 …… Q10 Ave pre/postgain

Ave studenteffort

Avestudent learning

39946 2.5 3.0 .76 2.11 3.11

bill01 2.0 2.0 1.33 3.23 3.06

S0S3 1.5 1.5 .43 1.98 2.11

SchoolAVE

2.45 2.36 .53 2.33 2.56

Three Critical Interventions (COMMITMENTS)

• A system of individual student feedback on learning goals at the classroom, school, and district levels (standards-based grading using formative assessments)

• Ensuring effective teaching in every classroom

• Building background knowledge for all students (particularly those who do not come from advantaged backgrounds).

Phase I: Establish a districtwide or schoolwide list of critical terms and accompanying model of vocabulary instruction.

angle

area

average

bar graph

congruent

difference

estimation

hexagon

length

width

octagon

parallel

perimeter

product

rhombus

square

symmetry

triangle

vertical

horizontal

3rd-Grade Mathematics

predict

compare

observe

investigate

analyze

amplify

frequency

vibration

Newton

properties

pitch

food web

life cycle

living organism

structure

inherit

reproduction

environment

characteristics

hypothesis

3rd-Grade Science

actor

animation

audience

cause & effect

commercial

conclusion

cue

detail

directions

drama

ending

facial expression

humor

minor character

main character

plot development

role playing

central idea

mood

myth

3rd-Grade Language Arts

prairie

Oregon Trail

lariat

oxen

expedition

territory

homestead

settlement

pioneer

farm/ranch

Native American

legacy

stampede

fertile

wagon trails

cabin

cattle

seasonal dwelling

plateau

frontier

3rd-Grade Social Studies

Step 1: Provide a description, explanation, or example of the new term.

Step 2: Ask students to restate the description, explanation, or example in their own words.

Step 3: Ask students to construct a picture, symbol, or graphic representing the term or phrase.

Step 4: Engage students periodically in activities that help them add to their knowledge of the terms in their notebooks.

Step 5: Periodically ask students to discuss the terms with one another.

Step 6: Involve students periodically in games that allow them to play with terms.

A Six-Step Process for Teaching New Terms

Mutualism

The interaction of organisms within an ecosystem in a manner that significantly benefits both, although the resulting relationship is not critical to the continued existence of either.

Term, phrase

Category: (standard, unit, alphabetical…)

Picture/graphicDescription

Related terms, phrases

Phase II: Implement the vocabulary program district-

wide or school-wide.

At this point you are pretty serious.

If you really want to get serious then….have students keep track

of their progress using vocabulary notebooks.

Three Critical Interventions

• A system of individual student feedback on learning goals at the classroom, school, and district levels

• Ensuring effective teaching in every classroom

• Building background knowledge for all students

Leadership for Incremental Change

• Emphasize relationships.• Establish strong lines of communication.• Be an advocate for the school.• Provide resources.• Maintain visibility.• Protect teachers from distractions.• Create culture of collaboration.• Look for and celebrate successes.

High above the hushed crowd, Rex tried to remain focused. Still, he couldn’t shake one nagging thought: He was an old dog and this was a new trick.

Leadership for Second-Order Change

• Shake up the status quo.• Expect some things to seem worse.• Propose new ideas.• Operate from strong beliefs.• Tolerate ambiguity and dissent.• Talk research and theory.• Create explicit goals for change.• Define success in terms of goals.

“Hey! They’re lighting their arrows! . . . Can they do that?”

“This ain’t gonna look good on our report, Leroy.”

Thank You!

Robert J. Marzano