New Teacher Project Report on HISD

Preliminary Findings from Analysis of Human Capital in Houston Independent

School District

April 29, 2010

2© The New Teacher Project 2010

Contents

Background and Methodology

The Importance of Great Teachers

Findings

Next Steps


Sources of Data for this Analysis:

HISD Human Resources, compensation, and achievement data, including records on teacher appraisal, hiring, separation, and measured impact of student growth.

Online surveys administered to all HISD teachers and principals in March and April 2010 examining topics including appraisal, professional development, compensation, and working conditions.

Project Background and Methodology

Surveys yielded responses from:

6,279 current

teachers

55% response rate

144 principals

56% response rate

In December 2009, The New Teacher Project (TNTP) partnered with the Houston Independent School District (HISD) to conduct an analysis of current human capital policies and practices that will help HISD give all of its students great teachers.

This report contains preliminary findings from TNTP’s analysis. Feedback on these findings will guide the creation of recommendations that will appear in a final report this summer.


Contents



Findings

Next Steps


Research has shown that effective teachers are critically important

to student learning.

Dallas students who start 3rd grade at about the same level of math achievement…

57

55

0 20 40 60 80 100

Group 2

Group 1

Average Percentile Rank

27

77

0 20 40 60 80 100

Group 2

Group 1

Average Percentile Rank

End of 5th Grade

After 3 EFFECTIVE Teachers

After 3 INEFFECTIVE Teachers

…finish 5th grade math at dramatically different levels depending on the quality of their teachers.

Original analysis by the Education Trust.

Source: Heather Jordan, Robert Mendro, and Dash Weerasinghe, The Effects of Teachers on Longitudinal Student Achievement, 1997.

50


To realize sustainable improvement, effective teaching must be the

guiding concern behind all elements of a district’s human capital system.

Talent PipelineCreate supply of effective teachers to fill all vacancies.

CORE METRIC

Number and percentage of new

teachers who demonstrate

effectiveness above a target

threshold

Effectiveness ManagementOptimize effectiveness of teacher workforce.

CORE METRICS

Retention

rate of

top-quartile

teachers:

Retention

rate of

bottom-

quartile

teachers

Average improvement in

retained teachers’

effectiveness over time

Recruitment

Selection

Training /

Certification

Hiring /

Placement

On-

Boarding

Evaluation /

Prof. Dev.

Compensation

Retention /

Dismissal

Working

Conditions

School-

Level

Human Cap.

Mgmnt.

An effective teacherin every

classroom

Measures of student learning


Teacher Effectiveness in Improving Student Achievement

Boost effectiveness of all teachers through effective evaluation and targeted professional development.

Improve or exit persistently less effective teachers and replace with more effective teachers.

Retain and leverage most effective teachers.

5

2

4

1 Optimize new teacher supply by hiring from preparation programs whose teachers consistently achieve better student outcomes.

Prioritize effective teachers for high-need students.

3

Current teacher performance

Potential teacher performance

Dramatic improvements in student achievement cannot occur without

a sustained and strategic focus on teacher effectiveness.

5Strategies for Optimizing Teacher Effectiveness


Summary of Preliminary Findings

HISD’s teacher appraisal and development systems do not adequately

differentiate performance, identify improvement areas, or support teachers’

individual needs.

Teachers want appraisal and support processes that accurately identify their

individual needs and address those needs with targeted professional

development.

Teachers support including measures of student growth in their appraisals, but

they have concerns about the ability of current tools to fairly and accurately

measure a teacher’s impact on student learning.

HISD has missed opportunities to improve or remove ineffective teachers.

While HISD has made great strides towards rewarding its best teachers, the

district must find ways to retain these teachers at higher rates than less effective

teachers.

1

2

3

4

5




Findings

Next Steps

Contents

HISD’s teacher appraisal and

development systems do not

adequately differentiate

performance, identify improvement

areas, or support teachers’

individual needs.


Overview of Teacher Appraisal Systems in HISD

55%

• HISD currently uses two systems to evaluate teachers’ classroom performance—the Professional Development and Appraisal System (PDAS) and Modified Professional Development and Appraisal System (MPDAS).

• Both systems evaluate teachers on a four-point scale in eight domains.

• Term or continuing contract teachers that have been rated “proficient” or higher in each domain the prior year can choose to be appraised using MPDAS.

• Under MPDAS, teachers accept the previous year’s ratings as the highest possible ratings for five of the eight domains, and assessors gather data for those five domains using during walk-through observations rather than formal observations.

�

Source: 2009-2010 HISD Teacher Appraisal Implementation Manual


Many teachers do not believe that HISD’s teacher appraisal systems

accurately measure their effectiveness in the classroom.

Source: Survey of HISD teachers. PDAS n= 5806, MPDAS n=3168. Teachers participating in MPDAS processes may have responded twice.

MPDAS

44% 55%

PDAS

Teachers Who “Agree” or “Strongly Agree” That Systems Allow Appraisers to Accurately Assess Instructional Performance


According to teachers, current appraisal tools do an especially poor

job assessing performance at the secondary level.

Teachers Agreeing that PDAS Allows Appraisers to Accurately Assess Instructional Performance, by School Level

Source: Surveys of HISD teachers. Percentages indicate respondents who selected “Strongly agree” or “Agree”.


Principals share teachers’ concerns about the district’s current

appraisal systems.

Source: Surveys of HISD principals. Percentages indicate respondents who selected “Strongly agree” or “Agree”.

59%

of principals say that problems with the PDAS/MPDAS tools are a significant barrier to providing effective appraisal of instructional performance.

28%of principals say that PDAS/MPDAS allows them to accurately assess teachers’ instructional performance.


These concerns are not surprising given that both appraisal systems

rate almost all teachers as good or great.

Source: HISD Human Resources Records.

97%

PDAS/MPDAS Domain Ratings, 2005-06 Through 2008-09

Just 3.4% of teachers rated on PDAS/MPDAS between 2005-06 and 2008-09 had any domain rated “below expectations” or “unsatisfactory.”

�


Despite these results, principals and teachers report that there are

significant numbers of low-performing teachers at their schools.

Average Distribution of Teacher Performance, As Perceived by Principals and Teachers

Source: Surveys of HISD teachers and principals and HISD Human Resources records. Survey respondents were asked to identify thepercentage of teachers at their school performing in each of the four groups. Percentages here calculated by averaging all respondents’distributions.

Only 1% of PDAS/MPDAS domains were rated below “Proficient” between 2005-06 and 2008-09


Although principals sometimes use the top two ratings to

differentiate the performance of their teachers…

Source: HISD Human Resources and EVAAS Data. EVAAS percentile determined by two and three year EVAAS averages. 90th percentile or above n=1008 ratings (126 teachers), 11th to 89th percentile n=8064 ratings (1008 teachers), 10th percentile or lower n=1048 ratings (131 teachers).

PDAS/MPDAS Domain Ratings According to EVAAS Math Percentile, 2008-09

Only 2% of domains from teachers in the EVAAS 10th

percentile or lower were rated

“below expectations.”0% were rated “unsatisfactory.”


…even the district’s least effective teachers are rarely rated

“unsatisfactory” in any area.

Teachers in EVAAS 10th Percentile or Lower Who Were Rated “Proficient” or “Exceeding Expectations” in Every PDAS/MPDAS Domain, 2008-09

Source: HISD Human Resources and EVAAS Data. EVAAS percentile determined by two and three year EVAAS averages.


This is true even in the district’s lowest-performing schools.

97.0%97.2%

Of the 11 HISD schools rated “Academically Unacceptable” by the Texas Education Agency in 2008-09, only two rated any teacher unsatisfactory on any

domain.

of the 661 teachers appraised using PDAS at these 11 schools had any domain rated unsatisfactory.

Just 2

Source: HISD Human Resources Data.




Findings

Next Steps

Contents

Teachers want appraisal and support

processes that identify their

individual needs and provide

targeted professional development.


Teachers at all grade levels want an appraisal system that

accurately assesses their instructional performance and identifies

areas for improvement.

Source: Surveys of HISD teachers. Percentages indicate respondents who selected “Strongly agree” or “Agree”.

Teacher Support for Appraisal Goals, by School Level


Right now, many teachers do not receive professional development

that helps them improve their performance.

Source: Survey of HISD teachers.

“Professional development needs to match the needs of teachers. We are expected to provide differentiated instruction…The material is…just not relevant to my needs.” - Elementary Teacher

�

“…professional development is tailored according to the needs of my students.”

51% 45%

“…professional development adequately addresses my individual needs as an educator.”

43%

“…the PDAS process helps me improve my

instructional performance.”

HISD Teachers Who “Agree” or “Strongly Agree” That…


Changes to appraisal processes that address these concerns would

make teachers more confident in the credibility of the systems.

Source: Surveys of HISD teachers.

Percentage of Teachers Who Say the Following Factors Would Increase Their Confidence in the Appraisal System

68%Clear timeline and expectations for the appraisal process

66%Clear performance standards with appraisal rubrics and ratings aligned with those standard

65%Annual surveys of all teachers on their satisfaction with the appraisal process

64%Appraisers are themselves evaluated in a transparent process

63% Individualized professional development

59%Comprehensive training of appraisers and quality checking of their work

95% of teachers said that at least one of these six factors would increase their

confidence in the system

70% of teachers believe that opportunities to receive targeted professional development should be based on appraisal results.�


Teachers and principals believe that individualized supports are

more effective than school-wide programs at improving teachers’

instructional performance.

Source: Surveys of HISD teachers and principals. Percentages indicates respondents who selected “Highly effective” or “Effective”.

“Which the following types of professional development support are effective in helping you/your teachers improve instructional practice?”


Principals cite a lack of time and training as the primary barriers to

conducting effective teacher appraisals and development.

18 percent of principals have sufficient time to focus on teacher appraisal and development.

32 percent of principals are given adequate training and support on how to appraise and develop teachers’ instructional performance.

Source: Surveys of HISD principals. Percentages indicate respondents who selected “Strongly agree” or “Agree”.

47 percent of principals cite a lack of time as one of the two most significant barriers to comprehensive appraisal and effective professional development.


Principals say that HISD does not hold them accountable for helping

teachers do their best work.

78%

Source: Surveys of HISD principals. The top five responsibilities that principals most often cited as one of the three responsibilities they are held most accountable for were: “Producing student achievement gains,” (77%) “Completing all administrative tasks,” (62%) “Ensuring that the school operates smoothly,” (42%) “Ensuring school safety,” (35%) and “Assisting teachers with professional growth” (29%).

of principals say that assisting teachers with professional growth is one of the three responsibilities for which they are held most accountable by HISD.

29%


As a result, principals don’t spend as much time as they would like

supporting their teachers.

17%Visiting classrooms and observing teachers 32%

7%Providing teachers with feedback on instruction 14%

6%Planning and delivering professional development 7%

16% District required paperwork 5%

8%Reviewing student achievement data 9%

9%Attending staff or district

meetings 3%

7% Parent relations 6%

Principals’ Average Time Allocation

Actual Desired

of principals say that their time is not distributed in a way that best supports student learning

and growth.

78%

Source: Surveys of HISD principals. Average “Actual’ and “Desired” time allocation taken from principals who indicated that their time is not distributed in a way that best supports student growth.


Contents

Teachers support including

measures of student growth in their

appraisals, but they have concerns

about the ability of current tools to

fairly and accurately measure a

teacher’s impact on student learning.



Findings

Next Steps


Teachers support including many factors in their appraisals,

including measures of student learning and academic growth.

Source: Surveys of HISD teachers.


Although most teachers support the use of tests to measure student

growth, they have concerns about current value-added measures.

Source: Surveys of HISD teachers. “Test-based assessment” indicates teachers who selected at least one of the following: standardized test scores, standardized test score growth, or student growth based on school- or teacher-generated assessments.

“Which of the following should be used to measure teacher impact on student learning as part of the teacher appraisal process?”

18%

…but only 18 percent believe that HISD’s current measures of a their effect on students’ standardized test score growth, such as EVAAS, are fair and accurate.

87 percent of teachers believe that some form of test-based assessment should be used to measure their impact on student learning…


Teachers raise a number of concerns regarding the potential use of

student growth data for appraisals.

Source: Surveys of HISD teachers. 200 open responses were randomly sampled from the pool of 1,699 responses.

“What, if any, are your greatest concerns or questions regarding the use of student learning measures…as part of an appraisal system?”

�“Are the students being fairly and equally assigned to each teacher based on behavior issues,

attendance history, GT status, etc.[?]” – Elementary Teacher

“The statistical methods are so complicated…that I cannot be sure that there are not anomalies in the

appraisals.” – HISD Teacher


In the coming weeks, TNTP will continue to analyze root causes of

teachers’ concerns with EVAAS and identify possible solutions.

Does participating in HISD teacher training on EVAAS increase teachers’ belief

that it provides a fair and accurate measure of a teacher ‘s impact on student

learning?

What factors in the design or implementation of a teacher appraisal system that

includes measures of student growth would increase teacher confidence in that

system?

What additional findings about teachers’ concerns regarding EVAAS and other

measures of teacher impact on student learning can be obtained through

analysis of teacher focus group data collected by TNTP?

What additional trends emerge when the group of teachers who have

confidence in EVAAS are compared to those who lack confidence in it?

Areas of Further Research (EVAAS)



Human Capital Transformation Framework

Findings

Next Steps

Contents

HISD has missed opportunities to

improve or remove ineffective

teachers.


Before the 2010 staff review process, almost no probationary

teachers were nonrenewed for their performance in the classroom.

of principals say that a poorly performing probationary teacher at

their school received a term contract in the last five years.

Only 0.6% of all probationary teachers between 2005-06 and 2008-

09 were nonrenewed for performance reasons.

0.6%

100%

Source: Survey of HISD principals, HISD Human Resources Data. Termination and nonrenewal data provided by HISD Professional Standards Office.

56%


Just 50 term or continuing contract teachers have been terminated

or nonrenewed for performance reasons since 2005-06.

51% of all principals have not even attempted to nonrenew or terminate a poorly-performing term or continuing contract teacher in the last five years.

Source: Survey of HISD principals. Termination and nonrenewal data provided by HISD Professional Standards Office.

0.1%7 of 8482

0.1%12 of 8504

0.2%13 / 8459

0.2%18 / 8562

2005-06 2006-07 2007-08 2008-09

Percentage of Continuing or Term Contract Teachers Terminated orNonrenewed for Performance by School Year

70 percent of those principals say that the procedures to initiate a dismissal are too resource-

intensive.


Contents

While HISD has made great strides

towards rewarding its best teachers,

the district must find ways to retain

these teachers at higher rates than

less effective teachers.



Findings

Next Steps


While ASPIRE has helped boost the retention of high-performing

teachers, HISD retains low performers at a similar rate.

1 SAS Institute, “Analysis of a Teacher Pay-For-Performance Program: Determining the Treatment Effect and Overall Impact”2 HISD Human Resources and EVAAS Data. EVAAS percentile determined by two and three year EVAAS averages.

Retention Rates of Teachers by EVAAS Percentile in 2008-09 by Subject2

“Teachers who leave the district tend to be the teachers who did not receive an award. While 50.36% of the teachers did not receive an award based on their performance in 2007, a much

larger percentage of the teachers who were not in the data in either 2008 or 2009 did not receive an award in 2007: 59.17%.”

According to a recent analysis by SAS Institute, Inc. on EVAAS and the ASPIRE Award program1:


HISD’s highest performing teachers do not consistently plan to

remain in the district longer than lower performers.

Source: Survey of HISD classroom teachers, HISD EVAAS Data. EVAAS percentile determined by two and three year EVAAS averages.

Percent of Teachers by EVAAS Percentile Who Plan to Leave HISD Within Three Years, 2008-09


Teachers planning to leave cite a number of factors influencing their

decision.

Source: Survey of HISD classroom teachers. Reasons for exiting district coded from all open responses. “Personal” reasons represented 43% of responses.

Top Non-Personal Reasons that Teachers Plan to Leave HISD Within Three Years

25% School/District Management

14%Working environment / conditions

8% Support

8% Student discipline

7% Recognition

7% Emphasis on testing

7% Financial compensation

7% Evaluation system

“I …feel that professional development and potential for growth are inadequate. I do not feel like I will grow as a teacher or reach my future goals if I stay at my school in HISD. "

- Elementary Teacher

�

“I feel like I don't get to actually teach. I love teaching, I love my students…however, I feel like everything is so focused on testing and compliance that the focus on actual classroom instruction and learning is lost.."

- High School Teacher


Contents



Findings

Next Steps


Next Steps

TNTP will continue analysis of data from teacher and principal surveys, as well

as HISD Human Resources records and teacher focus groups. In addition to a

further examination of the topics covered in this analysis, TNTP will analyze

data regarding teacher placement/hiring, recruitment and selection, training

and certification, and compensation.

Complete findings from TNTP’s analysis will be presented to the HISD Board of

Education in early June in order to solicit feedback and guide creation of

recommendations.

To gather additional feedback, TNTP will brief stakeholders including teacher

organizations, school principals, and community and business groups on the

complete findings.

In July, TNTP will publish a final report with recommendations that will inform

the planning for HISD’s comprehensive human capital reform efforts that will

launch beginning in September 2010.


Questions?

For more information:

www.tntp.org

Contact: Vinh Doquang, Project Director, at [email protected]

Date post:	27-Apr-2015
Category:	Documents
Upload:	enm077486
View:	2,714 times
Download:	1 times

New Teacher Project Report on HISD

Documents