1

Milwaukee Mathematics Partnership

Program EvaluationYear 5 Results

Carl HanssenHanssen Consulting, LLC

Cindy WalkerUniversity of Wisconsin-Milwaukee

October 2008 MTL Meeting

The Milwaukee Mathematics Partnership (MMP) is supported with funding from the National Science Foundation.

2

Evaluation Goals

Help the MMP better serve its constituents and improve its effectiveness

Serve the broader mathematics education community through documentation and dissemination of MMP activities

3

MMP Evaluation Logic ModelStudent

Achievement

Teacher Content& Pedagogical

Knowledge

Math FacultyInvolvement

Learning TeamEffort

SchoolBuy-in

TeacherInvolvement

NewCourses

DistrictBuy-in

MPA Ownership

MATCBuy-In

UWMBuy-In

ClassroomPractice

MMPActivities

ProximalOutcomes

DistalOutcomes

4

Agenda

1. Student Achievement

2. Learning Teams

3. Distributed Leadership

4. Conclusions

5. Next Steps

5

1. MMP Impact on Student Achievement

StudentAchievement

Learning TeamEffort

SchoolBuy-in

TeacherInvolvement

ClassroomPractice

6

MMP Impact on 2006 Student Achievement

Are student achievement gains greater in schools that have more fully embraced MMP principles?

7

Sep 04

Sep 05

Sep 06

Sep 07

Sep 08

MMP Online SurveySpring 2005

WKCEFall 2005

WKCEFall 2006

WKCEFall 2007

MMP Online SurveySpring 2006

MMP Online SurveySpring 2007

2004

-200

5S

cho

ol Y

ear

2005

-200

6S

cho

ol Y

ear

2006

-200

7S

cho

ol Y

ear

2007

-200

8S

cho

ol Y

ear

Data Collection Timeline

MMP Online SurveySpring 2008

8

Two cohorts of students

Cohort A Approximately 3,000 students Same school in grades

3-5 from 2005-2007

Cohort B Approximately 2,800 students Same school in grades

6-8 from 2005-2007

9

HLM Analysis

Consistent curriculum +Teachers working together +PD perceived as valuable

Predicts MathFocus

MathFocus

Predicts % of students proficient

MathFocus

Used as school-level predictorfor HLM analysis

10

HLM Results

Math focus was a predictor of initial math achievement scores AND of student learning rates

If your school scores 1 point higher on math focus, you can expect

Cohort A students to start 15 points higher and grow at a rate 7 points higher

Cohort B students to start 18 points higher and grow at a rate 7 points higher

11

HLM Results

Time

Sco

re

15-18 pt gapto start

Expected growth may be 30 pts

Growth would be30 + 14 points

School X: math focus score 3.5

School Y: math focus score 2.5

Yr 1 Yr 2 Yr 3

12

2. Learning Teams

StudentAchievement

Learning TeamEffort

SchoolBuy-in

TeacherInvolvement

ClassroomPractice

DistributedLeadership

13

Twelve Case Study Schools

Diverse set of schools Grade levels

9 schools with K-5 students 6 schools with 6-8 students 2 high schools 10-12

Diverse demographics Minority students (30-90%) Special education (11-32%) Free/Reduced lunch (41-95%)

14

Case Study Data Collection

22 learning team or math department meeting observations

42 classroom observations

MKT Assessment for math teachers

SNA Survey for mathteachers and administrators

15

Key Trends from Learning Teams

Overall, meetings tended to focus more on ‘administration’ than ‘learning’ though higher performing teams retain more of an emphasis on learning

Contrast between LT meetings and math department meetings Math meetings more focused Math departments less team oriented

Transition to released MTL model Seamless transitions Identification of replacement teacher No release

16

Characteristics of High &Low Rated Learning Teams—Team Functioning

Less focus on administration

Positional authority is less important

Multiple views are represented and heard

Multiple segments of the school are represented

Written agenda, note taker, facilitator

Explicit action items Participants have high

knowledge and skill levels

Focus on administration Principal does all the

talking A few individuals

dominate the discussion No agenda or team is

easily distracted from the agenda

Little follow-through on assignments

No clear actions

High Low

17

Characteristics of High & Low RatedLearning Teams—MMP Issues

Consistent curriculum Math is addressed

alongside and in combination with other subjects

Coherent within grades and across grades

Math is discussed irrespective of presence of MTL

Reference to MMP work courses including formative assessment, descriptive feedback, benchmarks

Teachers operate autonomously

Math not addressed at the meeting

No clear math leader—MTL may be unsure of role

Confusion about the MMP and CMF

High Low

18

3. Distributed Leadership

Teachers and administrators in each school were asked to name individuals with whom they communicated about mathematics

This is a key indicator of distributed leadership

19

Mathematics Distributed Leadership Continuum

High Low

Tight NetworkMTL CentralMany Links to MTLMTS InsideMany Links to MTSLeadership is shared

among many

Loose NetworkMTL Not Central

Few Links to MTLMTS Outside

Few Links to MTSLeadership

responsibility of few

20

Low

Student Achievement:2007: 66% Proficient3-year trend: -9%

School n Total Named

Network density

Density in school

MTL Role--In Degree

MTS Role--In Degree

K 11 57 3.8% 13.7% 7.6 0.4

Average 23 66 4.8% 10.7% 16.5 3.4 SD 9.3 24 1.7% 2.5% 6.4 4.1 Median 23 64 4.1% 11.0% 15.0 1.4

21

Medium

Student Achievement:2007: 47% Proficient3-year trend: +15%

School n Total Named Network density

Density in school

MTL Role--In Degree

MTS Role--In Degree

A 22 38 7.8% 12.7% 28.4 0.7

Average 23 66 4.8% 10.7% 16.5 3.4 SD 9.3 24 1.7% 2.5% 6.4 4.1 Median 23 64 4.1% 11.0% 15.0 1.4

22

High

Student Achievement:2007: 53% Proficient3-year trend: +3%

School n Total Named Network density

Density in school

MTL Role--In Degree

MTS Role--In Degree

G 32 79 5.0% 12.1% 16.3 5.4

Average 23 66 4.8% 10.7% 16.5 3.4 SD 9.3 24 1.7% 2.5% 6.4 4.1 Median 23 64 4.1% 11.0% 15.0 1.4

23

Evolution of Distributed Leadership

1. MTL is active within the school

2. Teachers begin extensive collaboration

3. MTL & Teacher collaboration extends outside school (MTS may become heavily involved in the school)

4. MTL is used primarily as a resource

5. Teachers assume math leadership

24

4. Overall Conclusions

There is support for the argument that schools that have more fully adopted MMP principles are demonstrating stronger results.

There is tremendous variability in MMPadoption and progress across the district—though MMP impactappears more pervasive.

25

Overall Conclusions

Important considerations for sustaining MMP work

High levels of math focus have been shown to be related to higher student achievement. What are the indicators of math focus?

Creating Distributed Leadership in a school takes time—and communication is criticalbut helps promote math focus

26

Overall Conclusions

Important considerations for sustaining MMP work

MTL role may be shifting from focal point to facilitator—we see a shift in the perception of who is responsible for helping the school focus on improving mathematics teaching and learning

MTL release model presentsbenefits and challenges for sustaining MMP work

27

5. Evaluation 2008-09

MMP Online survey in May 2009 Continue HLM analysis for student

achievement 25-30 case study schools to participate

over 3 years—Sign up now! LT/Math meeting Observations SNA

Focus on different MTL release models Goal to implement SNA in most

schools across the district

28

Focus Question

What message will you be taking back about…

Your ongoing work with teachers to improve math in your school?

How your learning team can bemost effective?