Improving Teacher Quality Grants, Cycle 6:

External Evaluation ReportHighlights and Spotlights

December 3rd, 2009University of Missouri

Evaluation Team

Evaluation Team

Principal InvestigatorsSandra AbellFran ArbaughMark EhlertJohn LanninRose Marra

Graduate Research Assistants

Ya-Wen ChengMark GagnonMichele LeeDominike MerleS. Rená Smith

Context of the Evaluation

• Improving Teacher Quality Grant program, Cycle 6, 2008-2009– Required 50% or more participants from high-need

schools – Funded 8 professional development projects (2

from previous cycle)– Science and mathematics, grades K-12– Formative and summative evaluation

Funded ProjectsProject Title Higher Education Institution;

Principal InvestigatorGrade Level;

Content FocusProject

Year

Professional Development for Enduring Understanding of Science via Inquiry & Literacy

Lincoln University ; Gouranga Saha Grades K-9; Science 1 of 3

Science Education and Quantitative Literacy: An Inquiry-Based Approach

Missouri University of Science and Technology; V.A. Samaranayake

Grades 2-7; Mathematics and Science

1 of 3

Science and Mathematics Achievement from Rural Teachers

Missouri State University; Lynda Plymate

Grades 4-10; Mathematics and Science

3 of 3

Quality Elementary Science Teaching University of Missouri; Deborah Hanuscin Grades K-6; Science 1 of 3

Physics for Elementary and Middle School Teachers: Constructing an Understanding of Physics

Rockhurst University; Robert Hegarty Grades 4-8; Science 3 of 3

Connect 9 Math Three Rivers Community College; Mary Lou Brown

Grades 3-8; Mathematics 1 of 3

gecKo mathematicsFoundation for Mathematical Proficiency Truman State University; Janice

Grow-MaienzaGrades K-6; Mathematics 1 of 1

Teacher Enhancement for Active Middle School Science in Kansas City

University of Missouri-Kansas City: Jerzy Wrobel Grades 6-8; Science 1 of 1

Participant Summary• 264 participants: 252 teachers, 8 pre-service teachers,

and 4 administrators;  • More taught science at the end of Cycle 6 than at the

beginning;• Most were elementary or middle level teachers;• 58.7% were new to the ITQG program;• Taught in 61 different Missouri school districts, and 6

private schools; • Directly impacted 15,523 students in the 2008-2009

school year.

Percentage of Participants from High-Need Districts

59.5%

40.5%

% participants from high-need districts % participants from non high-need districts

• 5 projects met or exceeded 50% goal

• 3 projects involved fewer than 50% (2 funded in Cycle 4)

• Overall greater % than in Cycles 3, 4, or 5

ITQG Objectives– Improve student achievement in mathematics

and/or science– Increase teachers’ knowledge and

understanding of key mathematics and/or science concepts

– Improve teachers’ pedagogical knowledge and practices in inquiry-based instruction

– Enhance teachers’ use of assessment to monitor effectiveness of instruction

– Impact the preparation of pre-service teachers

Teacher Content Knowledge Pre- and Posttests Show Gains

23

3846

24

49

79

35

53 56

72

87 89

41

89

0

10

20

30

40

50

60

70

80

90

100

Project 1 -Science

Project 3 -Math

Project 3 -Science

Project 4 -Science

Project 5 -Science

Project 6 -Math

% C

orre

ct

Pretest Posttest Posttest #2

Curriculum Design and Content Knowledge: Teachers as Learners

2.3

2.4

2.5

2.6

2.7

2.8

2.9

3.0

All Projects MU QUEST Rockhurst

Improved Confidence in Content Knowledge from PD

End of Summer End of Project

Self rated Improvement in Content Knowledge Confidence 1 = A Little to 3 = Very Much

Curriculum Design and Content Knowledge: Teachers as Learners

7.57.67.77.87.9

88.18.28.38.4

Overall Rockhurst MU QUEST

Contribution of Content Knowledge Component to Professional Growth

Self rated Contributionof Content Knowledge to Professional Growth 0 = None to 10 = Very Much

Student Achievement

42 39 49 53

31 37

64 72 76 77 75

49

0

10

20

30

40

50

60

70

80

90

Project 1 -Science

Project 2 -Math

Project 2 -Science

Project 3 -Math/Science

Project 5 -Science

Project 6 -Math

% C

orre

ct

Pretest Posttest

Student Achievement

8.7

8.3

-4.5

-3.0

11.0

5.5

-0.3

-0.4

2.7

-1.3

5.3

4.8

-1.1

-0.9

2.9

0.3

16.3

1.8

-0.1

-1.3

0.7 0.7 4.

4

2.4

-25.0

-20.0

-15.0

-10.0

-5.0

0.0

5.0

10.0

15.0

20.0

25.0

3 4 5 6 7 8

Inde

x P

oint

s

Grade

Changes in MAP Mathematics Index Scores

HN - PD Partic. HN - Not PD Not HN - PD Partic. Not HN - Not PD

5.7

5.3

-2.2

-1.5

4.1

3.8

0.5

-0.3

1.9

-1.6

2.4 2.9

0.1 0.

8 3.3

2.1

8.4

3.0

0.1

-0.8

0.9

-0.2

1.9

1.6

-10.0

-8.0

-6.0

-4.0

-2.0

0.0

2.0

4.0

6.0

8.0

10.0

3 4 5 6 7 8

% P

rofic

ient

or A

dvan

ced

Grade

Change in MAP Mathematics Proficiency Rates

HN - PD Partic. HN - Not PD Not HN - PD Partic. Not HN - Not PD

Student Achievement

-5.9 -2.8 -2.6

4.6

-10.3

3.5

0.0

2.1

-13.0

-8.0

-3.0

2.0

7.0

12.0

5 8

Inde

x P

oint

s

Grade

Changes in MAP Science Index Scores

HN - PD Partic. HN - Not PD Not HN - PD Partic. Not HN - Not PD

|------------ Grade 5 ---------------| |------------ Grade 8 ----------------|

-2.7 -2.4 -1.3

1.5

-7.6

1.5

0.0

1.1

-8.0

-6.0

-4.0

-2.0

0.0

2.0

4.0

% P

rofic

ient

or A

dvan

ced

Grade

Changes in MAP Science Proficiency Levels

HN - PD Partic. HN - Not PD Not HN - PD Partic. Not HN - Not PD

|------------- Grade 5 ------------| |------------ Grade 8 ---------------|

Teacher Knowledge and Practice of Inquiry

Objective 3 from the Cycle 6 RFP:

“To improve teachers’ pedagogical knowledge and practices that utilize scientifically-based research findings and best practices in inquiry-based instruction.”

Teacher Knowledge and Practice of Inquiry

Teacher Knowledge and Practice of Inquiry

Teacher Assessment Knowledge

Objective 4 from the Cycle 6 RFP:

“To enhance teachers’ use of assessment data to monitor the effectiveness of their instruction.”

Teacher Assessment Knowledge

Impact on Higher Education• Objective 5 from the Cycle 6 RFP:

“Impact the preparation of pre-service teachers through improvement to existing coursework or the design of new mathematics and/or science content and/or pedagogy courses.”

• Outcomes:– 2 projects included preservice teachers– 2 new graduate content courses– New program of study—minor in physics– Changes to existing courses

Spotlight on Best Practices

– Related to Higher Ed/K-12 Collaboration– Related to views of teaching and learning– Related to design features

Spotlight on Collaboration

Effective PD “links with other parts of the education system” (Loucks-Horsley et al., 2003, p. 44)

– UMKC and Kansas City Schools

Spotlight on Collaboration

The summer institute/PD project was relevant to my teaching assignment (for the coming school year)

End of Summer Mean

All Projects 2.7/3.0

UMKC 3.0

Spotlight on Views of Teaching and Learning

“effective PD is driven by a well-defined image of effective classroom learning and teaching” (Loucks-Horsley et al., 2003, p. 44)

• Learning cycle: Rockhurst and UMKC• 5E: Lincoln and MU• Seamless assessment and universal design: MU

Spotlight on Views of Teaching and Learning

Teacher Satisfaction with Projects: “Instructors modeled good practice”

Project End of Summer Mean

End of Project Mean

All Projects 3.6/4.0 3.7/4.0

Lincoln 3.9 4.0

Rockhurst 3.9 3.9

MU 4.0 4.0

UMKC 3.8 3.7

Spotlight on Design Features– TRCC math coach in schools / classrooms

– MU science camp for children and opportunity for teachers to plan, deliver, and review lessons with kids (mean value on contribution of Summer Institute = 10.0)

6.5

7

7.5

8

8.5

Summer institute School-year call backs Other school-based activities

Contribution of PD Components to Professional Growth

All TRCCSelf rated Contribution of PD Components to Professional Growth 0 = None to 10 = Very Much

Questions/Comment

Copies of the Cycle 6 Report and Executive Summary available at:

www.pdeval.missouri.edu