New Superintendents Induction Program … Superintendents Induction Program Evaluation Report...

New Superintendents Induction Program

Evaluation Report Findings from the first four years of program implementation

The New Superintendents Induction Program (NSIP) is a joint initiative of the Massachusetts Association of School Superintendents and the Massachusetts Department of Elementary and Secondary Education.

July 23, 2014

NSIP Evaluation Report – July 2014 Contents

UMass Donahue Institute Applied Research & Program Evaluation

Contents

Executive Summary ................................................................................................................. 1

Program Development and Refinement .......................................................................... 1

Program Impact .............................................................................................................. 2

Conclusion ...................................................................................................................... 3

Introduction .............................................................................................................................. 4

Program Overview .......................................................................................................... 4

Participation ........................................................................................................................ 5

Funding ............................................................................................................................... 5

External Evaluation ......................................................................................................... 6

Program Development and Refinement .................................................................................. 7

Program Impact ........................................................................................................................ 9

Impact on Superintendents ............................................................................................. 9

Impact on Districts ........................................................................................................ 11

Conclusion .............................................................................................................................. 13

Appendix A: Participating Districts ....................................................................................... 14

Appendix B: NSIP Program Feedback and Assessment Plan ............................................. 16

System Overview .......................................................................................................... 16

Entry Plan Rubric .......................................................................................................... 17

Entry Report Rubric ...................................................................................................... 19

Leadership Skills Assessment....................................................................................... 22

Rubric to Assess District Strategy ................................................................................. 26

Leadership Team Rubric ............................................................................................... 27

Superintendent Goals Report Rubric ............................................................................. 29

NSIP Engagement Rubric ............................................................................................. 31

Appendix C: MASC-MASS/NSIP Collaboration..................................................................... 34

NSIP Evaluation Report – July 2014 Introduction


1

Executive Summary

The Massachusetts Association of School Superintendents (MASS) and the Massachusetts Department of Elementary and Secondary Education (ESE) have formed an innovative partnership to develop strong district leadership and, by extension, enhance district capacity to support school improvement. The New Superintendents Induction Program (NSIP) seeks to enhance superintendents’ effectiveness through an induction process built on a framework of knowledge, skills, and habits of mind critical to effective school district leadership.

NSIP is conceptualized as a three-year induction program designed to meet the needs of the Commonwealth’s new superintendents1 with a cohort of newly hired, first-year superintendents entering the program each July. The program delivers its curriculum through a series of content days, and it supports implementation of key practices through related work assignments and intensive coaching by a team of former superintendents who participate in all content days with Year 1 and 2 superintendents. Coaches themselves receive extensive professional development in relation to the curriculum and the coaching skills required to support its implementation. The delivery approach applies important principles of adult learning, including choice, use of authentic work, and rubrics that make expectations clear and afford the opportunity for self-assessment. Throughout the curriculum NSIP introduces and provides opportunities for superintendents to practice with tools that they can bring back to use in their districts.

The first cohort of 25 superintendents entered to program in July 2010 of whom 20 continued on for the full three years and completed the program in June 2013. At this report’s writing, the fifth cohort of 32 is entering the program. Including cohort 5, NSIP has reached 97 of the Commonwealth’s 408 operating school districts.2 Approximately 130,000 (14%) of the state’s students are in districts led by NSIP graduates and 245,000 (25%) are in districts led by current NSIP participants. The participants represent all levels of the state accountability system, including 43% of the state’s Level 3 districts (the lowest performing 20% of districts), which appear to hire inexperienced superintendents at a disproportionately higher rate than Level 1 or 2 districts.

The University of Massachusetts Donahue Institute (UMDI) has served as NSIP’s external evaluator since the program’s inception. In the program’s first four years, the primary focus of the evaluation was supporting program design and delivery by generating formative feedback and assessing interim impacts through superintendent self-report and coaches’ perceptions. In FY14, UMDI launched a multi-year impact study focused on participants beginning with cohort 4. The study will draw heavily on data gathered through the embedded assessment system. Ultimately, the study holds the potential to demonstrate the impact of NSIP, which may serve as a model for induction programs across the country.

Program Development and Refinement NSIP was launched with a strong theoretical framework and a planned scope and sequence for curriculum delivery. Over the course of the first year, the specific approach to delivering that curriculum was defined and refined in an interactive process that was responsive to the perceived needs of superintendents. That development and refinement process has continued through the first four years of the program, including a significant revision of the curriculum in FY13. This ongoing reflection and adjustment process reflects the complexities of providing

1 For the purposes of NSIP, new superintendents are defined as individuals with no previous experience as a superintendent or experienced superintendents who are new to Massachusetts. 2 This is an unduplicated count of districts with cohort 1–3 superintendents participating for at least two years, cohort 4 superintendents completing one year, and incoming cohort 5 superintendents.



2

timely and effective support in the context of a highly demanding and variable education environment, as well as NSIP’s commitment to modeling the practice of a strong learning organization.

One critical characteristic of the NSIP program is that it offers some flexibility and adaptability within an established framework. From the outset, it was a clear that NSIP would be challenged to strike a balance between highly customized supports for individual superintendents and delivery of a core curriculum that establishes a common understanding of key concepts and builds specific skills that reflect a reform-focused approach to the superintendency. NSIP has responded by providing explicit opportunities for differentiated instruction within each of the content days. More significantly the program’s one-on-one coaching component in years 1 and 2 and group coaching in year 3 provide critical opportunities to tailor support to each superintendent’s starting point while maintaining a strong linkage to the program content.

Program leaders also demonstrate a solid understanding that new superintendents need to readily see the connection between the program and the somewhat overwhelming job in front of them. One very clear example of this effort to connect NSIP to real district challenges was the program’s emerging emphasis on supervision and evaluation—specifically implementing the new statewide Educator Evaluation system. Consistent with NSIP’s strategic orientation, the focus was on using Educator Evaluation as a lever for advancing the district’s improvement strategy by aligning educator goals with objectives and initiatives articulated in the district strategy. More recently, NSIP and ESE staffs have collaborated to more effectively integrate ESE planning requirements for Level 3 districts with NSIP strategy development practices. These approaches can serve as models for how NSIP can support ESE in rolling out complex new initiatives and support superintendents in effectively implementing and integrating those initiatives with their strategies for improvement.

Perhaps most importantly, the NSIP program has benefitted greatly from strong ownership on the part of the coaching staff. In addition to their direct support of superintendents, the coaching group plays a critical role in advising NSIP leadership on all aspects of the program including the overall approach, curriculum scope and sequence, agendas for each content day, and development and implementation of the NSIP assessment system. As a group, the coaches are strongly committed to improving their own skills and practices. It has also been a great benefit that the roster of coaches has been highly stable since the program’s inception. It does seem likely that there will be some turnover within the next few years. As such, it will be important to conceptualize a plan for recruiting, training, and integrating new coaches over time, perhaps with experienced coaches serving as mentors to those joining the team.

Program Impact Cross-cohort survey data suggest that NSIP engagement over a three-year period can have a positive influence on superintendents and their leadership of district improvement efforts. Most participants reported that NSIP has changed how they think about their role and that they spend most of their time focused on instructional improvement activities. Many reported that NSIP participation has substantially improved their ability to think strategically about district improvement. Most also acknowledged that the program provided tools and support that helped them to do their jobs in a way that was consistent with that philosophy.

Survey data show that superintendents from the first three cohorts reported that NSIP has had the most impact on their ability to think strategically about district improvement. As they exited the program, one-third of cohort 1 superintendents reported substantial improvement in their abilities to establish effective systems for supervision and evaluation, conduct root cause analysis of instructional problems, serve as an effective instructional leader, and create effective leadership teams. Superintendents from the first two cohorts also credited NSIP with relatively strong impact on focusing the leadership team on shared values and goals related to driving instructional improvement and student achievement. In addition, many cohort 1 superintendents indicated that NSIP had a strong influence on the leadership team’s clarity about its role and how decisions are made and implemented.



3

However, across cohorts less than one-fifth of the participants reported substantial improvement in their ability to manage resources in a strategic manner. Given that the work in NSIP’s second and third years is highly focused on strategy implementation, there is a natural opportunity to provide superintendents with additional guidance related to aligning resources (budget, personnel, time and political capital) in order to support those efforts. In fact, program leadership is currently working to restructure Year 2 and 3 content days to provide superintendents with more explicit support for strategy implementation.

The program’s impact on participants’ ability to forge collaborative relationship with key constituencies, particularly school committees and teachers’ unions, was similarly low. NSIP program staff and coaches have acknowledged that managing constituent relationships, particularly those with school committees and labor union leaders, are critical challenges for many superintendents and that the program curriculum has not effectively supported most in this regard. NSIP leadership and coaches are actively working to strengthen that aspect of the program both directly and through collaboration with the Massachusetts Association of School Committee’s (MASC) District Governance Support Project. As a result, MASC is offering a workshop on school committee-superintendent to all interested cohort 5 districts. NSIP has also launched discussions with the ESE-supported Massachusetts Education Partnership (MEP) for labor management collaboration to identify shared readings that will help to foster shared understandings among key stakeholders (district administration, school committee and union leadership) in participating districts.

Conclusion The New Superintendents Induction Program represents significant new leadership development capacity within Massachusetts’ public education system and is a key component of the state’s efforts to transform teaching and learning. It serves as a strong complement to other ESE support and improvement initiatives including the District and School Assistance Centers, the new Educator Evaluation system, implementation of the common core, and efforts focused on high need populations such as English language learners, special education and low income students.

Thus far the investment has produced a thoughtful three-year curriculum and delivery model supported by a cadre of well-prepared and dedicated coaches. Through the program, superintendents develop and begin to implement a district strategy with objectives and initiatives that target improving student learning and teaching quality. In addition, participating superintendents are forming valuable peer support networks and many of those relationships are persisting beyond the three-year induction timeframe. Many NSIP participants credit the program with substantial improvement in their ability to think and act strategically about district improvement, to establish effective systems for supervision and evaluation, and serve as effective instructional leaders. Furthermore, some members of district leadership teams report that superintendents’ use of program tools and resources have spread effective leadership practices within their districts.

Moving forward, it would be advantageous for ESE and MASS to revisit the original vision where NSIP was to part of a “3-legged stool” addressing effective district leadership. As originally conceived the other two legs were school committees and labor unions, each of which was addressed through its own Race-to-the-Top funded initiative. Although the three programs shared an overarching objective and engaged many of the same stakeholders they were developed largely in isolation from each other. As a result they operated in parallel, but generally not in concert. Although it may take some time to put the pieces in place, NSIP leadership has begun working with MASC and MEP to improve collaboration and further accelerate district leadership and instructional improvement.



4

Introduction

Program Overview The Massachusetts Association of School Superintendents (MASS) and the Massachusetts Department of Elementary and Secondary Education (ESE) have formed an innovative partnership to develop strong district leadership and, by extension, enhance district capacity to support school improvement. The New Superintendents Induction Program (NSIP) seeks to enhance superintendents’ effectiveness through an induction process built on a framework of knowledge, skills, and habits of mind critical to effective school district leadership. The framework has the following four essential elements, each with specific focus areas:

1. Focus the work of the district on improving learning and teaching: (a) Create a deep, shared understanding of effective instruction and the role every adult in the system has to support it and (b) Develop and implement a high leverage strategy that will accelerate learning.

2. Develop the capacity of leaders, teachers, and the organization: (a) Focus on adult learning, individual and organizational reflection, and continuous improvement; (b) Develop high-functioning teams, including the leadership team and school committee, that are deeply engaged in and “own” the district’s improvement work; and (c) Design an effective support and accountability system for the adults in the system aligned to district priorities.

3. Think and act strategically: (a) Build and sustain a culture of high expectations and mutual accountability based on district vision and core values, and (b) Address changes adaptively.

4. Further develop skills as leaders: (a) Assess and strengthen communication skills, political acumen, and self-awareness, and (b) Build habits of self-reflection and continuous improvement.

NSIP is conceptualized as a three-year induction program designed to meet the needs of the Commonwealth’s new superintendents3 with a cohort of newly hired, first-year superintendents entering the program each July. The program delivers its curriculum through a series of content days, and it supports implementation of key practices through related work assignments4 and intensive coaching by a team of former superintendents who participate in all content days with Year 1 and 2 superintendents. The coaches themselves receive extensive professional development in relation to the NSIP curriculum and the coaching skills required to support its implementation.5 The intended role of the coaches is primarily to support superintendents’ implementation of that curriculum. Coaching is structured so that superintendents receive the most direct support (eight hours per month) in their first year, decreasing to four hours per month in their second year. Third year superintendents no longer receive direct one-on-one coaching, but are expected to participate in regional consultancy groups, which meet six times during the year and are facilitated by a member of the NSIP coaching team.

The program’s foundational text, Strategy in Action, was written by Rachel Curtis and Liz City who were actively engaged in program development and delivery through the first four years. The delivery approach applies important principles of adult learning, including choice, use of authentic work, and rubrics that make expectations 3 For the purposes of NSIP, new superintendents are defined as individuals with no previous experience as a superintendent or experienced superintendents who are new to Massachusetts.

4 During the first 10–16 months, participants complete three public work documents that are designed to build district-wide commitment and capacity to accelerate student learning. Those documents are an Entry Plan, A Report of Entry Findings, and a District Strategy. 5 Each year coaches participate in seven full-day professional development sessions focusing on both NSIP content and coaching skills.



5

clear and afford the opportunity for self-assessment. Throughout the curriculum the program introduces and provides opportunities for superintendents to practice with tools that they can bring back to use in their districts including the Current Initiatives Activity, designed to identify fragmentation and lack of strategy; Root Cause Analysis to understand issues more deeply; After-Action Reviews to develop habits of rigorous assessment; and a Leadership Team Rubric to track team functioning over time.

Participation

The first cohort of 25 superintendents entered to program in July 2010 and was limited to districts in Level 3 or 4 accountability status. Of them, 20 continued on for the full three years and completed the program in June 2013. A second cohort of 26 entered the program in July 2011 with a three-year completion rate of 77%. The third cohort of 22 and fourth cohort of 23 entered the program in July 2012 and 2103, respectively. At this report’s writing, the fifth cohort of 32 is entering the program.

Including cohort 5, NSIP has reached 97 of the Commonwealth’s 408 operating school districts.6 Approximately 130,000 (14%) of students are in districts led by NSIP graduates and 245,000 (25%) are in districts led by current NSIP participants. The participants represent all levels of the state accountability system, including 43% of the state’s Level 3 districts (the lowest performing 20% of districts), which appear to hire inexperienced superintendents at a disproportionately higher rate than Level 1 or 2 districts. A roster of participating districts is provided in Appendix A.

Figure 1: District Accountability Level – NSIP compared to Statewide

Funding

Entering its fifth year, NSIP has an annual operating budget of $750,000 with the largest proportion of these resources (59%) devoted to the coaching component followed by the cost of content days (18%). The remaining funds are directed to administrative costs, such as program leadership, training and support, and program evaluation. The most significant source of funding is contributed by ESE, predominantly through state funds ($420,000), which have been supplemented with federal Race to the Top funding ($230,000). Those federal funds will not be available beyond FY15 and the NSIP partnership is making efforts to raise funds to replace them.

The remaining funding is provided by MASS ($50,000) and through individual participation fees. Fees are currently $3,900 for the first year and $2,900 for each of the next two years. Through FY14, districts in Level 3 or

6 This is an unduplicated count of districts with cohort 1–3 superintendents participating for at least two years, cohort 4 superintendents completing one year, and incoming cohort 5 superintendents.

0% 10% 20% 30% 40% 50% 60% 70%

Level 1

Level 2

Level 3

Level 4

3%

67%

27%

3%

24%

57%

16%

3%

State

NSIP



6

4 accountability status were exempt from those fees. However, beginning with cohort 5 the fees will be assessed to all participants. Those fees are typically covered through each district’s budget. Annually, one or two superintendents have been unable to secure funding to participate. In at least one instance the superintendent paid out of her own pocket. It remains to be seen whether districts with frequent superintendent turnover (the program is serving five districts for a second time and one district for the third time) will start to push back about being repeatedly assessed that fee, but thus far program staff do not report this to be an issue.

External Evaluation The University of Massachusetts Donahue Institute (UMDI) has served as NSIP’s external evaluator since the program’s inception.7 In the program’s first four years, the primary focus of the evaluation was supporting program design and delivery through generating formative feedback and assessing interim impacts through superintendent self-report and coaches’ perceptions. Evaluation data are collected through documentation, interviews, and surveys of both superintendents and coaches. In addition, program observation offered an opportunity to better understand how the program components function. Observation of coach training days in particular, provided exposure to reflective discussion and strategizing in relation to program implementation.

The NSIP program has been diligent about reviewing participant feedback, coaches’ input, and evaluation data and making program adjustments in response. During the 2012–2013 school year, NSIP’s third year of implementation, the staff drew on those resources and undertook a significant program revision. Those efforts resulted in a new articulation of the program framework and focus areas; refinement of the content day and coaching components of the program, including the introduction of “strands of study” and development of the regional consultancy groups for Year 3 superintendents; improved definition of expected program outcomes; and formalization of a more consistent coaching model. In that same year UMDI partnered with NSIP program staff to develop an embedded assessment system to address desired participant outcomes more systematically, including capturing the perspective of participants’ district leadership teams. Implementation of the newly developed NSIP assessment system will serve multiple purposes, including guiding coaches’ efforts to facilitate growth and development of individual superintendents, generating formative feedback to inform program improvement efforts, and providing key data and evidence to support ongoing program evaluation efforts. An overview of the assessment system and its key instruments are available in Appendix B.

In FY14, UMDI launched a multi-year impact study focused on participants beginning with cohort 4.8 The study will draw heavily on data gathered through the embedded assessment system. This will ensure that the measures used to gauge impact are closely aligned and calibrated to the program curriculum and expected outcomes. In addition, it should ensure strong compliance with evaluation reporting requirements, as most of the data collection will be tightly integrated with superintendents’ and coaches’ ongoing program participation. The impact study is focused on superintendent growth and development over time. Given that all new Massachusetts superintendents are invited to participate in NSIP, and to date most do, there is no appropriate comparison group against which to contrast their performance. As an alternative, the study will identify variations in individual-level engagement with NSIP and test the hypothesis that superintendents who are more actively engaged with the program will demonstrate more growth in relation to the defined outcomes than those who are less actively engaged.9 The impact study will also draw on extant data sources, such as Educator Evaluation results, district-level TELL Mass survey data, and superintendent tenure. Ultimately, the study holds the potential to demonstrate the impact of NSIP, which may serve as a model for induction programs across the country.

7 Previous NSIP evaluation reports can be found online at http://www.doe.mass.edu/research/reports/alphabetical.aspx?alphabet=N. 8 It was initially hoped that the impact study could be designed and launched with cohort 3. However, NSIP staff needed additional time to make final revisions to the Year 1 curriculum and better define the desired program outcomes against which impact could be assessed. 9 UMDI has been working closely with NSIP program staff and coaches to develop a program engagement rubric through which superintendents will be assessed.

http://www.doe.mass.edu/research/reports/alphabetical.aspx?alphabet=N

NSIP Evaluation Report – July 2014 Program Development and Refinement


7

Program Development and Refinement

NSIP was launched with a strong theoretical framework and a planned scope and sequence for curriculum delivery. Over the course of the first year, the specific approach to delivering that curriculum was defined and refined in an interactive process that was responsive to the perceived needs of superintendents. That development and refinement process has continued through the first four years of the program, including a significant revision of the curriculum in FY13. This ongoing reflection and adjustment process reflects the complexities of providing timely and effective support in the context of a highly demanding and variable education environment, as well as NSIP’s commitment to modeling the practice of a strong learning organization.

One critical characteristic of the NSIP program is that it offers some flexibility and adaptability within an established framework. In reality, there is significant variation among new superintendents in terms of skills, prior experience, and district context. From the outset, it was a clear that NSIP would be challenged to strike a balance between highly customized supports for individual superintendents (as might be found in a traditional mentoring model) and delivery of a core curriculum that establishes a common understanding of key concepts and builds specific skills that reflect a reform-focused approach to the superintendency (as might be found in a straight professional development model).

In the first two years of the program, experienced superintendents, and to some extent those who previously served as assistant superintendents, expressed frustration that, in their perception, the program assumed that everyone was brand new to their role. As a result, these superintendents found that some of the content addressed areas in which they felt they were already quite strong. In addition, a vocal minority of participants complained that some of the content was redundant with leadership development programs that they had attended in the past.10 NSIP responded by providing explicit opportunities for differentiated instruction within each of the content days. More significantly, the program’s one-on-one coaching component in years 1 and 2 and the group coaching in year 3 provide critical opportunities to tailor support to each superintendent’s starting point while maintaining a strong linkage to the program content.

Program leaders also demonstrate a solid understanding that new superintendents need to readily see the connection between the program and the somewhat overwhelming job in front of them. As designed, the NSIP curriculum follows a specific outline of topics to be covered in a particular sequence. In its first year, there was some disconnect between the timing of the NSIP curriculum and the cycle of district work. NSIP staff recognized that superintendents were not going to commit such a substantial amount of time to the program if it was seen as an additional responsibility that was not helping them with their day-to-day work challenges. This resulted in efforts to re-sequence the Year 1 curriculum to achieve better alignment and support learning by developing skills and providing tools that superintendents could readily bring back to their work in the districts.

This is yet another area where the coaching component is quite valuable, as coaches continually work to link NSIP content to their superintendent’s most pressing issues. Superintendents expressed how highly they value the opportunity to address other issues through coaching as a critical means of obtaining help dealing with concrete day-to-day issues that they are facing in their districts, but are not addressed as part of the formal content day curriculum. While maintaining their commitment to their primary role, which is to support understanding and implementation of the NSIP curriculum, coaches acknowledged the need to allocate some proportion of their time to address other issues of concern. They are working hard to frame their approach to such conversations in a way that supports superintendents’ development as strategic thinkers—particularly coaching the superintendents to

10 Many specifically cited the National Institute for School Leadership (NISL) Executive Development program.

NSIP Evaluation Report – July 2014 Program Development and Refinement


8

developing their own solutions and helping superintendents see the linkage between NSIP and their day-to-day work.

One very clear example of this effort to connect NSIP to real district challenges was the program’s emerging emphasis on supervision and evaluation—specifically implementing the new statewide Educator Evaluation system. This was introduced as a major theme of the Year 2 content day agendas, and evaluation data suggest that it made sessions feel more relevant for many participants. An initial step was providing superintendents with a strong orientation to the system and providing opportunities to discuss very practical considerations related to local design and implementation. However, consistent with NSIP’s strategic orientation, the real emphasis was on approaching Educator Evaluation as more than a compliance exercise and using it as a lever for advancing the district’s improvement strategy by aligning educator goals with objectives and initiatives articulated in the district strategy. More recently, NSIP and ESE staffs have collaborated to more effectively integrate ESE planning requirements for Level 3 districts with NSIP strategy development practices. These approaches can serve as models for how NSIP can support ESE in rolling out complex new initiatives and support superintendents in effectively implementing and integrating those initiatives with their strategies for improvement.

Perhaps most importantly, the NSIP program has benefitted greatly from strong ownership on the part of the coaching staff. In addition to their direct support of superintendents, the coaching group plays a critical role in advising NSIP leadership on all aspects of the program including the overall approach, curriculum scope and sequence, agendas for each content day, and development and implementation of the NSIP assessment system. The coaches are generally not shy about challenging the ideas presented by leadership or engaging in active dialogue to collaboratively improve program design and delivery. It is clear that the program leadership values this input greatly and is willing to take the time to thoughtfully engage in a slower, more deliberate program development and refinement process in order to take advantage of coaches’ insights.

As a group, the coaches are also strongly committed to improving their own skills and practices. In the first few years of the program, much of the coach development time was focused on learning NSIP practices, which represented a new approach to the superintendency. At that time, coaches and their superintendents were essentially learning the NSIP content together. Over time, the coaches have undoubtedly become more familiar and comfortable with the program content. But they aren’t stopping there. They are now taking the opportunity to focus more on improving their coaching practice by working together to develop common expectations and approaches for providing superintendent support. In addition to ongoing discussions, their regular coaching meetings frequently include time for consultancies where they can help each other develop plans to address challenging situations and relationships.

It has also been a great benefit that the roster of coaches has been highly stable since the program’s inception. In fact, there have been no new coaches since the second year of the program. Given that the current group of coaches are all retired superintendents, it does seem likely that there will be some turnover within the next few years as individuals decide that they are ready to fully retire. As such, it will be important for the program to conceptualize a plan for recruiting, training, and integrating new coaches over time, perhaps with experienced coaches serving as mentors to those joining the team.

NSIP Evaluation Report – July 2014 Program Impact


9

Program Impact

Annual survey data and interviews suggest that NSIP objectives are generally well aligned with superintendents’ priorities for district improvement. Participants expressed a high degree of satisfaction with the program overall, with particular appreciation of the coaching component. A strong majority indicated that they would recommend NSIP to other new superintendents as they enter the role. Cross-cohort survey data suggest that NSIP engagement over a three-year period can have a positive influence on superintendents and their leadership of district improvement efforts. There is also some indication that adjustments to the program curriculum and its delivery have resulted in improvements to superintendents’ understanding of key concepts and tools as well as their ability to incorporate those concepts into their practice.

Impact on Superintendents As an initial step toward transforming the profession, NSIP seeks to influence how participants think about the role of the superintendent. The desire is to shift the perception away from the traditional model of an administrator focused on managing operational complexity toward the notion of a superintendent as the key instructional leader in the district focused on developing and implementing a strategy to improve student achievement. A related indicator is the proportion of superintendents’ time focused on activities and initiatives that will have a direct impact on the quality of instruction in their districts. Most participating superintendents reported that NSIP has changed how they think about their role and that they spend most of their time focused on instructional improvement activities. Many reported that NSIP participation has substantially improved their ability to think strategically about district improvement.

As a result of the program, I have a better understanding of how the role of the superintendent has changed dramatically from managerial and reactive to strategic, targeted, and proactive. [Cohort 3 superintendent]

Several of the superintendents indicated that they had come into the program with a philosophy that was already well aligned with NSIP’s view. Most of them also acknowledged that, while their perceptions did not change, the program provided tools and support that helped them to do their jobs in a way that was consistent with that philosophy.

I always believed that my job as a superintendent was to bring a cohesive approach to addressing the challenges the district faces . . . What NSIP did was introduce me to new ideas, give me tools to help carry out my goals, and give me a much needed support system, especially in the first year, to keep things moving in the right direction. [Cohort 1 superintendent]

During their alumni interviews, cohort 1 superintendents noted that NSIP helped them navigate the early years of their superintendency and reminded them to focus on the big picture: their strategy and major goals. The program provided a research-based foundation for strategy and practice through its content and coaching. Cohort 1 alumni also noted that the program provided them with explicit time to focus and think strategically about communication planning. Many also valued the program’s teachings regarding being open to criticism.

The spring 2013 survey data presented in Table 1 show that superintendents from the first three cohorts reported that NSIP has had the most impact on their ability to think strategically about district improvement. As they exited the program, one-third of cohort 1 superintendents reported substantial improvement in their abilities to establish effective systems for supervision and evaluation, conduct root cause analysis of instructional problems, serve as an effective instructional leader, and create effective leadership teams.



10

Table 1: NSIP Impact on Superintendent Abilities (Self-report data from spring 2013 survey)

Percent Indicating Substantially Improved

Cohort 1 Cohort 2 Cohort 3 All Cohorts

Serve as an effective instructional leader 32% 23% 18% 24%

Think strategically about district improvement 63% 68% 41% 57%

Conduct root cause analysis of instructional problems

37% 27% 41% 35%

Manage resources in a strategic manner 11% 36% 9% 19%

Develop instructional leadership skills among school leaders

26% 36% 18% 27%

Create effective leadership teams 32% 41% 27% 33%

Forge collaborative relationships with key constituencies

22% 18% 18% 19%

Establish effective systems for supervision and evaluation

39% 23% 27% 29%

Across cohorts less than one-fifth of the participants reported substantial improvement in their ability to manage resources in a strategic manner. Given that the work in NSIP’s second and third years is highly focused on strategy implementation, there is a natural opportunity to provide superintendents with additional guidance related to aligning resources (budget, personnel, time, and political capital) in order to support those efforts. In fact, program leadership is currently working to restructure Year 2 and 3 content days to provide superintendents with more explicit support for strategy implementation.

The program’s impact on participants’ ability to forge collaborative relationship with key constituencies, particularly school committees and teachers’ unions, was similarly low. NSIP program staff and coaches have acknowledged that managing constituent relationships, particularly relationships with school committees and labor union leaders, are critical challenges for many superintendents and that the program curriculum has not effectively supported most in this regard. Superintendents who had the experience of working with difficult school committees suggested that the program spend more time on this topic and perhaps require school committee members to attend an orientation to the program or some type of supplemental training. NSIP leadership and coaches are actively working to strengthen that aspect of the program both directly and through collaboration with the Massachusetts Association of School Committee’s (MASC) District Governance Support Project. As a result, MASC is offering a workshop on school committee-superintendent to all interested cohort 5 districts. A brief description of that workshop is provided in Appendix C. NSIP has also launched discussions with the ESE-supported Massachusetts Education Partnership (MEP) for labor management collaboration to identify shared readings that will help to foster shared understandings among key stakeholders (district administration, school committee and union leadership) in participating districts.



11

Impact on Districts As shown in Table 2, superintendents from the first two cohorts credited NSIP with relatively strong impact on focusing the leadership team on shared values and goals related to driving instructional improvement and student achievement. In addition, many cohort 1 superintendents indicated that NSIP had a strong influence on the leadership team’s clarity about its role and how decisions are made and implemented. These results were reinforced by the comments of leadership team representatives who were interviewed as part of the cohort 1 alumni interview process who noted that their meetings generally focused on district goals, curriculum, and instruction rather than administrivia.

Table 2: NSIP Impact on Leadership Team Functioning (Self-report data from spring 2013 survey)

Cohort 1 Cohort 2

Strongly Agree

Strong Influence

Strongly Agree

Strong Influence

The team is driven by shared values and goals. It has a clear, widely understood purpose related to driving instructional improvement and student achievement and its work is related to this.

47% 56% 38% 43%

There is clarity about the team’s role and how decisions are made and implemented. 32% 44% 29% 19%

The team has norms and meeting agendas and follows them. There is a designated facilitator who skillfully manages the meeting. Minutes are generated for each meeting and referenced as needed.

32% 22% 67% 57%

Team members know when they are advising the superintendent and when they are equal partners in decision-making.

32% 22% 57% 33%

There are structures and processes in place to ensure transparency within the team. 32% 22% 52% 29%

Humor and mutual respect are part of the team experience. 74% 22% 86% 10%

Team members come to meetings prepared. They actively engage and focus on the agenda and tasks at hand. 58% 17% 38% 24%

Questions and concerns are raised in meetings. Decisions are consistently supported once made. 68% 28% 52% 24%

Note: The statements presented in this table represent the proficient-level of the NSIP Rubric to Assess Effectiveness of Leadership Team

Data presented in Table 3 show that, among superintendents with at least two years in the program, the substantial majority strongly agreed that their districts’ strategic objectives and initiatives targeted improving student learning and teaching quality, with about half indicating that NSIP had a strong influence in that area. Those cohorts also credited NSIP with relatively strong impact on focusing the leadership team on shared values and goals related to driving instructional improvement and student achievement, though fewer than half strongly agreed that their team demonstrated proficiency in this respect. They report the least influence in leveraging operations areas to support improving student learning and teaching quality. This is generally



12

consistent with ongoing feedback identifying a desire for additional support managing the day-to-day operational aspects of their jobs.

Table 3: NSIP Impact on Current work in the District (Self-report data from spring 2013 survey)

Cohort 1 Cohort 2

Strongly Agree

Strong Influence

Strongly Agree

Strong Influence

Strategic objectives and initiatives target improving student learning and teaching quality

79% 53% 67% 48%

Operations areas are leveraged to support improving student learning and teaching quality

32% 18% 33% 14%

Strategic objectives are tightly integrated, complementary, and high leverage

32% 53% 67% 48%

All key constituencies understand the strategy and talk about how their work relates to it

11% 29% 19% 29%

Decision making at every level of the system is aligned to the strategy

26% 53% 19% 33%

Note: The statements presented in this table represent an adaptation of the characteristics of an organization with a “high impact” strategy as described in the Signs of Strategy Rubric from Strategy in Action (Curtis and City, Harvard Education Press, 2009. p. 31).

NSIP Evaluation Report – July 2014 Conclusion


13

Conclusion

The New Superintendents Induction Program represents significant new leadership development capacity within Massachusetts’ public education system and is a key component of the state’s efforts to transform teaching and learning. It serves as a strong complement to other ESE support and improvement initiatives including the District and School Assistance Centers, the new Educator Evaluation system, implementation of the common core, and efforts focused on high need populations such as English language learners, special education and low income students.

Thus far the investment has produced a thoughtful three-year curriculum and delivery model supported by a cadre of well-prepared and incredibly dedicated coaches. Through the program, superintendents develop and begin to implement a district strategy with objectives and initiatives that target improving student learning and teaching quality. In addition, participating superintendents are forming valuable peer support networks and many of those relationships are persisting beyond the three-year induction timeframe. Many NSIP participants credit the program with substantial improvement in their ability to think and act strategically about district improvement, to establish effective systems for supervision and evaluation, and serve as effective instructional leaders. Furthermore, members of district leadership teams report that superintendents’ use of program tools and resources have spread effective leadership practices within their districts.

To date, nearly one-quarter of Commonwealth’s 408 operating school districts have been reached by the program and the vast majority of participating superintendents indicate that they would recommend the program to other new superintendents as they enter the role. Indeed, NSIP enjoys a strong reputation, which has resulted in an exceptionally high participation rate with more than 95% of eligible superintendents choosing to join. Among the first two cohorts nearly 80% of participants have stayed on for three years and completed the program. As the fifth cohort joins NSIP, the program staff is working to secure stable funding to continue the program beyond the 2015-2016 school year.

NSIP program leadership has demonstrated a strong commitment to program improvement and it is likely that the curriculum and delivery model will continue to evolve over time. Moving forward, it would be advantageous for ESE and MASS to revisit the original vision where NSIP was to part of a “3-legged stool” addressing effective district leadership. As originally conceived the other two legs were school committees and labor unions, each of which was addressed through its own Race-to-the-Top funded initiative – ESE partnered with the Massachusetts Association of School Committees (MASC) for the District Governance Support Project and the Massachusetts Education Partnership (MEP) for the District Capacity Project, which was focused on labor-management collaboration. Although the three programs shared an overarching objective and engaged many of the same stakeholders they were developed largely in isolation from each other. As a consistent partner and funder in each project, ESE did not set explicit expectations for coordination or collaboration among them. As a result they operated in parallel, but generally not in concert. Although it may take some time to put the pieces in place, it is not too late to consider how the partners and their respective programs could collaborate to further accelerate district leadership and instructional improvement.

NSIP Evaluation Report – July 2014 Appendix A: Participating Districts


Appendix A: Participating Districts

Cohort 1 Cohort 2 Cohort 3 Amherst-Pelham Regional Arlington Ayer-Shirley Regional Beverly Bourne Canton Chelsea Dracut/Framingham* East Longmeadow Easthampton Greater Lawrence Vocational Tech Holyoke King Philip Regional Nantucket Newburyport Norwood Randolph Reading Tewksbury West Springfield

Amesbury Bellingham Gill-Montage/Berlin-Boylston* Cohasset Dartmouth Foxborough Gloucester Grafton Leominster Lowell Malden Marshfield Mendon-Upton Regional North Brookfield Sharon Sudbury Waltham Wayland Westfield Winthrop

Bedford Billerica Dracut** Hamilton-Wenham Regional Lincoln Melrose Narragansett Regional Norfolk North Andover Norwell Rockport Springfield Uxbridge Webster Wellesley Weymouth

* Superintendent changed districts during the program ** Repeat district

NSIP Evaluation Report – July 2014 Appendix A: Participating Districts


Cohort 4 Cohort 5 Assabet Valley Vocational Avon Braintree Brockton Dover-Sherborn Regional Falmouth Gill-Montague Regional** Hanover Holbrook Holyoke** Hopkinton Ipswich Leicester Marblehead Mashpee Monomoy Regional Monson Newburyport** Ralph Mahar Regional Randolph** Saugus Southern Berkshire Regional Wakefield

Acton-Boxborough Regional Ayer-Shirley Regional** Belmont Central Berkshire Regional Collaborative for Educational Services*** Dedham Dighton-Rehoboth Regional Dudley-Charlton Regional Easton Gardner Greenfield Groton-Dunstable Regional Harvard Hudson Lenox Lower Pioneer Valley Collaborative*** Lynnfield Millbury North Brookfield** Northboro-Southboro Regional Northbridge Oxford Randolph** Somerset South Hadley Southbridge Swampscott Wachusett Regional Ware West Springfield* Wilmington Wrentham

** Repeat district *** Collaborative participants are Executive Directors

NSIP Evaluation Report – July 2014 Appendix B: NSIP Assessment System


16

Appendix B: NSIP Program Feedback and Assessment Plan

System Overview

Assessment Tool Timing Purpose

Growth and Development Program Evaluation

Rubrics: Entry Plan and Report of Entry Findings

Fall (Entry Plan) and Winter (Entry Report) – Year 1

Coaches will facilitate these steps for each of the tools:

1) Coach and superintendent rate performance against the rubric or complete the survey

2) They discuss their ratings/survey responses and the evidence used, noting any discrepancies.

3) They establish a shared understanding of current performance and, if a repeated rating assess changes since the prior assessment.

4) They discern where the ratings and/or evidence suggest the need for more inquiry and/or skill building, and use the discussion as the basis to establish a focus for coaching as well as the superintendent’s plan for action.

NSIP Program evaluator uses aggregate ratings to evaluate program impact (no superintendent is every identified by name or district).

To address issues of inter-rater reliability for rubric measurement, the evaluator will use calibrated ratings.

1) NSIP Program evaluator uses initial ratings as baseline data.

2) NSIP Program evaluator uses subsequent ratings to assess growth.

Survey: Leadership Skills Assessment (Self and “360” assessment) Spring – Years 1, 2 and 3

Rubric to Assess District Strategy Spring – Year 1 and (if revised) Years 2 and 3

Leadership Team Rubric (Self and “360” assessment) Fall – Years 2 and 3

Rubric: Superintendent Annual Goals Report June – Years 1, 2 and 3

NSIP Program Engagement Rubric Spring – Years 1, 2 and 3

Note that program leadership is considering adding a Strategy Implementation plan and related rubric.



17

Entry Plan Rubric High Impact Moderate Impact Little or no Impact

What • Includes comprehensive review of documents that reflect the key work of the district, including district and school improvement plans and principal evaluations.

• Includes review of most, but not all, documents that reflect the key work of the district.

• Includes review of few, if any, documents that reflect the key work of the district.

• Uses open-ended interviews designed to generate new learning. Informants include school committee, union leaders, principals and other key stakeholders.

• Uses open-ended interviews with a limited set of stakeholders.

• Does not open-ended interviews as a data source.

• Data analysis that will examine trends in all key performance indicators

• Data analysis plans will examine trends in many, but not all, key performance indicators

• Analysis plans specify little or no examination of performance trends

• Analysis plan will utilize disaggregated data to examine the performance of all key population subgroups.

• Data analysis plans will examine the performance of most, but not all key subgroups.

• Analysis plans specify little or no examination of disaggregated data.

• Includes observations of classrooms, school functioning and culture, key district office functions and teams, as well as vision and values.

• Includes some observations, but missing key categories as specified under “high impact”

• Includes little or no observation data

Why • Clearly articulates and emphasizes the entry plan as an opportunity to hear multiple voices and gather a variety of data to understand the district, including vision, culture and core values.

• Articulates the purpose for the entry plan in a way that focuses primarily on compliance with NSIP program requirements

• Articulates no clear purpose for the entry plan.

• Communicates commitment to learn about the current conditions of the district in order to identify future needs.

• Communicates a commitment to learn about the current conditions of the district, but does not explicitly make the linkage to identifying future needs

• Does not communicate a commitment to learn about the current conditions of the district.

• Articulates goal of identifying areas of strength and areas for further study.

• Articulates other goals, but not identifying areas of strength and areas for further study

• Does not articulate goals



18

High Impact Moderate Impact Little or no Impact

How • Includes outreach to a broad group of internal and external stakeholders representing different levels within the system and variety of community members and groups

• Includes outreach to a narrow group of informants and doesn’t ensure diversity of perspectives or engagement of critical stakeholders.

• Includes few details about which stakeholders will be engaged in the process.

• Addresses how key internal and external stakeholders will be engaged, including a specific outreach plan for school committee members and union leaders

• Offers only a general description of how stakeholders will be engaged

• Provides little or no information on how stakeholders will be engaged.

• Mechanisms for collecting data are clearly outlined and include counting, seeing AND hearing data.

• Includes partial information about the mechanisms for collecting data and/or does not include counting, seeing AND hearing data.

• Does not specific the mechanisms for collecting data.

• Clearly defines the timeline for entry plan implementation.

• Vaguely defines the timeline for entry plan implementation.

• Does not define the timeline for entry plan implementation.

• Clearly defines the audience for the superintendent’s report of entry findings.

• Vaguely defines the audience for the superintendent’s report of entry findings.

• Does not define the audience for the superintendent’s report of entry findings.



19

Entry Report Rubric


What • Includes a rich mix of count, see and hear data. • Includes a limited mix of count, see and hear data and/or relies heavily on one kind.

• Includes very few data sources and/or relies almost exclusively on only one kind.

• Focuses on the instructional core. • Addresses aspects of the core, but does not place it at the center of the report.

• Little focus on the instructional core.

• Addresses organizational effectiveness. • Addresses effectiveness of some aspects of the district, but omits some key functions.

• Pays little or no attention to organizational effectiveness.

• Recognizes strengths with appropriate evidence.

• Articulates strengths with limited evidence and/or focuses on weaknesses.

• Little or no evidence presented for strengths and/or few strengths identified.

• Includes findings backed with multiple data sources and stakeholders’ input.

• Most findings are backed with some data sources and/or stakeholder input.

• Few findings are backed with data or stakeholder input.

• Organizes data into a manageable number of patterns, trends and major findings (generally 4 to 8 major findings)

• Identifies patterns, trends and major findings but either presents too many or too few major findings.

• Identifies few patterns, trends or major findings and/or presents a long list of findings.

• (OVERALL) Presents findings and the evidence linked to them in a way that provides a deep and nuanced understanding of key district strengths, areas for development, context, and the relationship of each to the instructional core.

• (OVERALL) Presents findings and the evidence linked to them in a way that shows understanding of some key district strengths, areas for development, context and the relationship of most to the instructional core.

• (OVERALL) Presents findings and the evidence linked to them in a way that shows little understanding of key district strengths, areas of development, context and the relationship of each to the instructional core.



20


Why • Responds to needs and interests of stakeholders and the report’s audiences.

• Shows evidence of responding to the needs and interests of some stakeholders.

• Shows little or no evidence of responding to the needs and interests of stakeholders.

• States core values clearly. • Hints at core values • Makes no mention of core values

• Presents findings of district needs in a way that signals the direction of prioritization and strategy without identifying priorities, strategies or actions to be taken.

• Report goes beyond signaling direction to identifying some key priorities, strategy and actions to be taken.

• Report does not signal direction OR it identifies all or most priorities, strategies and actions to be taken.

• Presents findings of district needs in a way that is likely to build understanding and ownership.

• Findings are presented in ways that will generally be easy to understand and reasonable to accept.

• Findings are presented in ways that will be difficult to understand and/or accept.

• Presents findings of district needs in a way that is persuasive.

• Presents most findings in ways that are generally persuasive.

• Presents findings in ways that are not persuasive.

• Links findings together and tells a story about the district and the relationship among various findings.

• Links most findings together to tell a story about the district and/or the relationship among the findings.

• Offers few links among findings.

• Clearly states the purpose of the report, focusing on ensuring instructional excellence and student achievement.

• States the purpose of the report, but does not focus on ensuring instructional excellence and student achievement.

• Does not state the purpose.

• Reflects deep understanding and respect for the district’s culture and stakeholders.

• Reflects some understanding and respect for the district’s culture and stakeholders.

• Report reflects little understanding and/or respect.



21


How • Communicates clearly next steps of strategy development and how stakeholders will be engaged.

• Mentions that strategy development comes next and stakeholders will be engaged, but offers few specifics.

• Makes no mention of next steps and/or how stakeholders will be involved.

• Demonstrates senior leadership team engagement in vetting findings and themes.

• Offers little description of how senior leadership team was involved in vetting.

• Makes little or no mention of senior leadership engagement in vetting.

• Includes data sources the methodology used to analyze data.

• Describes data sources and methodology for analysis but provides few specifics.

• Provides little or no information on data sources and methodology for analysis.

• Includes attention to key stakeholders as well as those traditionally overlooked.

• Includes attention to key stakeholders but not those traditionally overlooked

• Identifies few specific stakeholder groups.

• Balances clarity of direction with flexibility in outline of next steps to ensure responsiveness

• Makes future direction clear with some flexibility for modification

• Makes future direction clear without providing much, if any, room for change.

• Reports findings using a variety of media and methodologies that respond to the needs and interests of different audiences, including posting on the district website.

• Reports findings using at least a written report with details of evidence, perhaps in an appendix, and a shorter report or slides for public presentation and posting.

• Uses a “one-size-fits-all” approach to presentation of findings.



Leadership Skills Assessment Your superintendent is one of nearly 75 superintendents currently participating in the New Superintendent Induction Program (NSIP) sponsored by the Massachusetts Association of School Superintendents and the Massachusetts Department of Elementary and Secondary Education. One purpose of NSIP is to support superintendents’ efforts to strengthen their leadership skills. The NSIP Superintendent Leadership Assessment is designed to advance that purpose, as well as yield information to help determine the impact that NSIP is having across the state. We are asking principals and central office administrators to complete the assessment. Participating superintendents are also completing a self-assessment using the same instrument.

Your responses will be gathered anonymously by the UMass Donahue Institute, which serves as NSIP’s independent research partner. The Donahue Institute will not have access to your name, the name of your superintendent, or your district. Using the superintendent code number provided below, the Donahue Institute will compile the data for each superintendent and generate standardized reports of aggregate results. Those reports will be returned to NSIP staff for distribution to superintendents and their NSIP coaches. Reports will not be generated for teams with fewer than five members.

Please access the assessment through the following link: < insert link here>. The survey will be available through March 28th.

Your superintendent’s identification code is: #######. Please keep this code handy as you will need to enter it as part of your survey response.

We estimate that completing the survey will require about 10 minutes of your time. Due to the anonymous nature of the survey administration process, you will need to complete the entire survey in one sitting, so please plan accordingly. You will not be able to stop midway through and return to the survey at a later time.

If you have any questions about the survey (including any technical issues), please contact Jen Gordon of the Donahue Institute at [email protected] or 774-455-7376.

Your responses to this survey over the next several years will help your superintendent gain valuable information and insight about his/her leadership behavior and how it is perceived, as well as steps s/he can take to become an even more effective superintendent. The results will also be analyzed as part of the Donahue Institute’s external evaluation of NSIP. We thank you in advance for your participation.

The MA NSIP Superintendent Leadership Assessment has been adapted with permission of the North Dakota LEAD Center from the Superintendent Leadership Assessment © Copyright 2005. ND LEAD Center. All rights Reserved. Skill definitions and behavioral indicators adapted by ND LEAD Center with permission from the National Association of Secondary School Principals.

mailto:[email protected]



Superintendent Identification Code (Response Required)

The email invitation that you received for this survey included a superintendent identification code that enables the UMass Donahue Institute to collect and link responses anonymously. Please enter that code into the text box below. (You may copy and paste from the email.)

Superintendent Identification Code

Respondent

Which of the following best describes your current position in the district? (select one)

o Central Office Administrator o Building Principal o Other

Directions

This survey seeks to assess superintendents’ leadership skills in four key areas: communication, interpersonal leadership, political acumen and self-awareness. For each skill area, first read the general definition. Next read the behavior statements below the definition and rate the superintendent’s current skill level with regard to each.

Mark 1 (never) if you believe the superintendent never exhibits the behavior.

Mark 5 (always) if you believe the superintendent almost always exhibits the behavior

Mark 2 (rarely), 3 (sometimes) or 4 (often) if you believe that the superintendent’s behavior falls somewhere in between.

Mark N/A if you have no basis to judge the behavior.

Communication

Clearly communicates orally and in writing. Presents information in ways that are clear, easy to understand, and accessible to various audiences.

1. Articulates a clear, inspiring vision for the school system and its efforts

2. Communicates own core values and acts according to them

3. Shares information, experiences, and expertise

4. Tailors messages to meet the needs of unique audiences

5. Clearly presents thoughts and ideas in one-to-one, small group, and formal presentations

6. Demonstrates effective presentation skills, e.g., opening and closing comments, eye contact, enthusiasm, confidence, rapport, use of visual aids

7. Communicates clear rationales for decisions



Interpersonal Leadership

Perceives the needs and concerns of others. Deals tactfully with others in emotionally stressful situations or in conflict. Knows what information to communicate and to whom. Relates well to people of varying ethnic, cultural, and religious backgrounds.

8. Generates enthusiasm for improving student learning and teaching

9. Interacts professionally and tactfully with others

10. Elicits perceptions, feelings, and concerns of others

11. Voices disagreement without creating unnecessary conflict

12. Supports others to disagree constructively

13. Clearly articulates expectations regarding the performance of others

14. Acknowledges achievement and accomplishments of others

15. Communicates necessary information to the appropriate people in a timely manner

16. Expresses verbal and/or non-verbal recognition of feelings, needs, and concerns of others

17. Takes action to divert unnecessary conflict

18. Assists the team in maintaining the direction needed to complete a task

Political Acumen

Identifies internal and external stakeholders (staff, school committee, union, parents, community leaders, media, etc.) that impact the district’s work and discerns their relationships, interests and influence. Uses knowledge of the formal and informal organizational and governmental structures with which they work to advance district.

19. Is clear about decision-making processes and seeks consensus where possible

20. Takes into account political considerations, stakeholder interests and needs before taking action

21. Listens and collects relevant information when dealing with unfamiliar issues and individuals

22. Seeks additional information about issues that arise and unexpected events or outcomes before taking action

23. Encourages others to share their ideas

24. Generates enthusiasm and works to persuade others to work together to accomplish common goals

25. Seeks commitment of all involved to develop/implement a district strategy for improving student learning and teaching

26. Supports the ideas and views of team members to solve problems

27. Motivates others to change behaviors that inhibit professional and organizational growth



Self-Awareness

Understands personal strengths and weaknesses. Takes responsibility for improvement by actively pursuing developmental activities. Strives for continuous learning.

28. Understands how own behavior is perceived by others

29. Recognizes and articulates his/her own strengths

30. Recognizes and articulates his/her own weaknesses

31. Actively pursues personal growth through participation in planned developmental activities

Additional Comments

32. List the most prominent strengths of the superintendent that positively impact overall effectiveness.

33. Please list the most important thing(s) the superintendent could improve on to increase overall effectiveness.

34. Use the space below to make any additional comments related to the superintendent’s effectiveness.



26

Rubric to Assess District Strategy High Impact Moderate Impact Little or no Impact

Instructional Core

• All strategic objectives target improving student learning and teaching quality.

• Most strategic objectives target improving student learning and teaching quality.

• Few strategic objectives focus on improving student learning and teaching quality

• All strategic initiatives target improving student learning and teaching quality.

• Most strategic initiatives target improving student learning and teaching quality.

• Few strategic initiatives focus on improving student learning and teaching quality.

• This Strategy leverages operations areas to support student learning and teaching quality.

• Some operations areas are leveraged to support student learning and teaching quality.

• Few, if any, operations areas are leveraged to support student learning and teaching quality.

Focus, Coherence

&

Synergy

• Strategic objectives are tightly integrated and complementary.

• There are explicit and deliberate connections among and between most strategic objectives.

• The connections among strategic objectives are not made clear.

• Strategic initiatives are high leverage. • Most strategic initiatives represent high leverage priorities that, if successfully implemented, will have a major impact in achieving the district’s strategic objectives.

• Few strategic initiatives represent high leverage priorities.

Both

Visionary

&

Problem Solving

• Pursuing this Strategy will address many of the problems identified problem(s).

• Pursuing this Strategy will address some of the problems identified in the Report of Entry Findings.

• There is little connection between the problems identified in the Report of Entry Findings and this Strategy.

• Pursuing this Strategy is likely to lead to realizing the district’s vision.

• Pursuing this Strategy may lead to realizing the district’s vision.

• Pursuing this Strategy is not likely to lead to realizing the district’s vision.

• Pursuing this Strategy is likely to transform student learning results.

• Pursuing this Strategy may lead to transforming student learning results.

• Pursuing this Strategy is not likely to lead to transforming student learning results.



27

Leadership Team Rubric Highest Impact

(Assumes the characteristics of High Impact are in place)

High Impact Limited Impact Little or No Impact

Purpose and Clarity

The team sets the direction and tracks the most important work of the system. Team members behave in ways that demonstrate their understanding that their role is to ensure the system’s overall success.

Team is driven by shared values and goals. It has a clear, widely understood purpose related to driving instructional improvement and student achievement and its work is related to this. There is clarity about the team’s role and how decisions are made and implemented.

The team has a stated purpose but its work is not consistently aligned to it. Team members understand their role on the team but don’t consistently act accordingly. The team spends a lot of time focused on things unrelated to the instructional core. The team occasionally makes decisions of consequence.

Team members don’t understand the purpose of the team or their role on it. The team either doesn’t make decisions or doesn’t make decisions of any consequence.

Structures Team members informally move the agenda forward and regularly assess structures and make adaptations to support continuous improvement.

The team has norms and meeting agendas and follows them. There is a designated facilitator who skillfully manages the meeting. Minutes are generated for each meeting and referenced as needed. Additional meeting roles (secretary, timekeeper) are used as needed.

The team has norms and meeting agendas but does not consistently follow them. The facilitation of the meetings is inconsistent.

The team does not have norms. It neither follows an agenda nor documents meetings. It is unclear who the facilitator is.

Trust Team members feel comfortable voicing uncertainty and asking for support. The team has structures and processes in place to ensure transparency of its work with the larger community. Questioning and alternative views are commonplace, explicitly valued, and strengthen relationships.

Team members know when they are advising the superintendent and when they are equal partners in decision-making. There are structures and processes in place to ensure transparency within the team. Humor and mutual respect are part of the team experience.

It is unclear to team members how decisions are made. Unproductive tension and/or disengagement are sometimes observable in meetings. Participants regularly feel that they don’t know everything they need to be an effective team member. Topics discussed in sidebars in and outside of the meeting are not consistently discussed in the meeting.

Many team members say little or nothing at meetings. Decisions are made without broad input. Team members are regularly surprised by information that is not shared with the team in a timely and appropriate manner.



28

Highest Impact

(Assumes the characteristics of High Impact are in place)

High Impact Limited Impact Little or No Impact

Commitment Team members hold one another to high standards, focus on organizational needs and continuous learning and improvement, and drive the team’s performance. Decisions are made and their implications for work are clearly addressed and tracked. The team sees itself (and is seen) as united.

Team members come to meetings prepared. They actively engage and focus on the agenda and tasks at hand. Questions and concerns are raised in meetings. Decisions are consistently supported once made.

Team members usually come to meetings on time and prepared, but their active participation is inconsistent. Team members loosely identify with the team and demonstrate limited support for team decisions.

Team members come to meetings late and unprepared. Meetings are meandering and unfocused.

Accountability All team members hold one another accountable to the norms of the group, decisions made, and the team’s and overall organization’s success.

Every meeting is assessed (process and content) and recommendations for improvement are solicited and implemented. Decisions are revisited only when there is a compelling reason. There are mechanisms in place to track implementation of decisions, review implementation data, and adapt work accordingly.

There are mechanisms in place to assess meetings, but the information gathered is not used to drive improvement. Decisions are made, but they are not implemented or are revisited without good reason.

There are no mechanisms for assessing the effectiveness of team meetings (process or content). Decisions are either not made or made and not implemented.



29

Superintendent Goals Report Rubric High Impact Moderate Impact Little or no Impact

Strategic Alignment

&

Specificity

• The goals include district improvement, student learning and professional practice goals.

• The goals do not include one of the following types: district improvement, student learning, professional practice.

• The goals include only one type of goal.

• Each goal is consistent with the district’s Strategy and demonstrates a deep understanding of the building blocks of instructional improvement.

• Most goals are consistent with the district’s Strategy and demonstrate an understanding of the building blocks of instructional improvement.

• Few goals are consistent with the district’s strategy and/or demonstrate understanding of the building blocks of instructional improvement.

• Goals focus on outcomes rather than outputs.

• Most goals focus on outcomes rather than outputs.

• All or most goals focus on outputs rather than outcomes.

• Each goal, if met, is likely to have a positive impact on advancing the district’s Strategy.

• Most goals, if met, are likely to have a positive impact on advancing the district’s Strategy.

• Few goals, if met, are likely to have a positive impact on advancing the district’s Strategy.

• All goals are clearly written with enough specificity to determine whether or not they have been achieved.

• Most goals are clearly written with enough specificity to determine whether or not they have been achieved.

• Few, if any, goals are clearly written with enough specificity to determine whether or not they have been achieved.

Action orientation, Timing

&

Realism

• For each goal, key actions with timelines and appropriate delegation are identified.

• For most goals, key actions with timelines and appropriate delegation are identified.

• Few goals include key actions with timelines.

• There are a challenging yet realistic number of ambitious goals that can be implemented effectively in the time specified (generally 3-5)

• The goals are not challenging, realistic, and/or there are too many to implement effectively in the time specified.

• The goals are not challenging and there are too many to implement effectively in the time specified.

• The actions identify the steps essential to accomplishing the goal and are realistic given the timeframe specified.

• Key action steps are missing and/or those included are unrealistic given the timeframe specified.

• There are few, if any, action steps identified.

• Action steps reflect alignment of resources (time, money & people).

• Action steps reflect some evidence that resources are being aligned.

• There are few, if any, action steps identified.



30


Measurement,

Tracking

&

Results

• For each goal there are measures to assess progress (benchmarks) on the key actions (outputs) as well as progress on the key intended results (outcomes).

• Most goals have benchmarks for both outputs and outcomes

• Few, if any, goals have benchmarks for either outputs or outcomes.

• The measures drive manageable collection of evidence that helps maintain focus and make mid-course adjustments.

• Collecting and presenting data for most measures will be manageable and helpful.

• The measures are likely to result in in an unmanageable and unhelpful data collection and presentation process.

• For each goal there is a clear statement of what will be different as a result of the action(s) described.

• For most goals there is a clear statement of what will be different as a result of the action(s) described.

• It is unclear for most goals what will be different.

Implementation • The Report documents substantial progress on all or all but one goal.

• The report documents progress on most goals.

• The report documents progress on few goals.

• The report provides convincing supporting evidence of progress.

• The report provides convincing supporting evidence of progress for most goals.

• There is little or no supporting evidence for most goals.

• The report celebrates successes of students, teachers, principals and the district as a whole

• The report mentions the work of key stakeholders but without sufficient specificity to be credible and celebratory.

• The report makes little or no mention of the contributions of stakeholders to the district’s progress on goals.

• The report demonstrates on-going and forthright communication with the school committee about the district’s Strategy and the Superintendent’s goals.

• The report demonstrates some communication with the school committee about Strategy and goals.

• The report demonstrates little or no communication between the superintendent and school committee about the Strategy and goals.

• The report demonstrates active and on-going involvement of the Leadership Team.

• The report demonstrates some involvement of the Leadership Team.

• The report demonstrates little or no involvement of the Leadership Team.

• The report reflects explicit and intentional development of the district’s professional culture.

• The report reflects efforts to strengthen the district’s professional culture.

• The report makes little or no mention of professional culture.



31

NSIP Engagement Rubric


Content Days • Regularly attends, arriving on time and remaining to the conclusion of the day.

• Misses a day and/or occasionally arrives late or leaves early.

• Frequently misses content institutes, arrives late and/or leaves early.

• Is well prepared, having completed readings, reflections, and assignments.

• Is generally prepared. • Is often not prepared.

• Resists the temptation to check email or make calls except at breaks or in an emergency.

• Periodically checks emails and/or makes/takes phone calls during session.

• Frequently checks emails and/or makes/takes phone calls during session.

• Is an active participant in small and large group discussions.

• Generally participates in discussions. • Does not typically participate in discussions.

• Regularly demonstrates an appropriate balance between listening and speaking.

• Generally demonstrates an appropriate balance between listening and speaking.

• Frequently fails to demonstrate an appropriate balance between listening and speaking.

• Assumes facilitation responsibilities when asked or needed (self-initiates).

• Assumes facilitation responsibilities when asked, but does not self-initiate.

• Does not typically assume facilitation responsibilities.

• Demonstrates willingness to share his/her own challenges, considers feedback thoughtfully, and gives feedback with candor and sensitivity.

• Somewhat guarded in sharing his/her own challenges, considers and offers some thoughtful feedback.

• Does not typically share his/her own challenges or offer thoughtful constructive feedback to colleagues.



32


Work Products

&

Practice

• Completes the major NSIP work products in a thoughtful and timely manner, deviating from the general schedule only when local context requires them to do so. (Entry Plan, Report of Entry Findings, Strategy, 360-degree Leadership Skills Assessment, Leadership Team Rubric.)

• Completes the major NSIP work products, but not necessarily in a thoughtful or timely manner.

• Fails to complete one or more of the major NSIP work products.

• Uses major NSIP tools appropriately in their district work. (e.g., Leadership Team Norms, Making Sense of the Work, RCA, SWOT, Brutally Honest Truths)

• Makes some effort to use major NSIP tools in their district work.

• Makes little or no effort to use major NSIP tools in their district work.

Coaching

(Years 1 and 2)

• Shares responsibility for scheduling calls and coach visits to the district.

• Is generally responsive to scheduling initiated by his/her coach.

• Scheduling is challenging. May miss planned coaching sessions.

• Is well prepared for the coach’s visits and calls, having completed agreed-upon tasks in advance.

• Is generally prepared for the coach’s visits and calls.

• Is not typically prepared for the coach’s visits and calls.

• Actively engages with the coach, demonstrating willingness to reflect on NSIP content and his/her own practice.

• Engages with the coach, but not strongly focused on NSIP content and his/her own practice.

• Does not effectively engage with the coach in relation to NSIP content and his/her own practice.

• Demonstrates strong interest in and acts on feedback.

• Generally demonstrates interest in and willingness to act on feedback.

• Demonstrates limited interest in and willingness to act on feedback.



33


Network Meetings

(Year 3)

• Regularly attends, arriving on time and remaining to the conclusion of the meeting.

• Misses a session and/or occasionally arrives late or leaves early.

• Frequently misses content institutes, arrives late and/or leaves early.

• Is well prepared, having completed readings, reflections, and problems of practice, as established.

• Is generally prepared. • Is often not prepared.

• Resists the temptation to check email or make calls except at breaks or in an emergency.

• Periodically checks emails and/or makes/takes phone calls during session.

• Frequently checks emails and/or makes/takes phone calls during session.

• Is an active participant in discussions. • Generally participates in discussions. • Does not typically participate in discussions.

• Regularly demonstrates an appropriate balance between listening and speaking.

• Generally demonstrates an appropriate balance between listening and speaking.

• Frequently fails to demonstrate an appropriate balance between listening and speaking.

• Assumes facilitation responsibilities when asked or needed (self-initiates).

• Assumes facilitation responsibilities when asked, but does not self-initiate.

• Does not typically assume facilitation responsibilities.

• Demonstrates willingness to share his/her own challenges, considers feedback thoughtfully, and gives feedback with candor and sensitivity.

• Somewhat guarded in sharing his/her own challenges, considers and offers some thoughtful feedback.

• Does not typically share his/her own challenges or offer thoughtful constructive feedback to colleagues.

NSIP Evaluation Report – July 2014 Appendix C: MASC-MASS/NSIP Collaboration

School Committee-Superintendent Collaboration, Beginning the Work Together, July 2014

Appendix C: MASC-MASS/NSIP Collaboration

School Committee – Superintendent Collaboration

A MASC-MASS/NSIP COLLABORATION

BEGINNING THE WORK TOGETHER

Objectives The objectives of the workshop are to begin building a partnership between the superintendent and the school committee, to help the superintendent learn about the district and help the committee learn about the NSIP, and to plan what the team would like to accomplish in the first year. Operating Protocols The first part of the workshop focuses on building common understandings about how the group will work together and how it will communicate with each other and with other district stakeholders. Through establishing or reviewing existing Operating Protocols, the team can discuss the many grey areas that can exist in carrying out its responsibilities. Developing agreements about the “how” of the work enables the team to concentrate on “what” needs to be done to drive student achievement in the district. Goal Setting Discussing the current context is a very important part of the goal setting discussion. Through this discussion, the new superintendent can gain an understanding of the committee’s perspective on the current issues and priorities in the district, as well as the status of existing goals and plans. In addition, the superintendent may gain perspective on the committee’s expectations for his/her first year. This will contribute to the information gathered by the superintendent in developing an entry plan. Conversely, the committee will learn about the work the superintendent will be engaged in as part of the New Superintendent Induction Program. Members will learn about the content days and coaching aspects of the program as well as the work products that the superintendent will be producing for the district. The governance team will have the opportunity to discuss how participation in the NSIP will coincide with and inform the work of the superintendent during his/her first years in the district. The superintendent and the committee can decide how the committee can stay informed about the superintendent’s progress as s/he develops the Entry Findings and the District Improvement Plan. With these common understandings, the team can begin to craft goals for the first year of the new superintendent’s tenure. In addition, the team can discuss how the work of the first year can lay the groundwork for district progress in future years. Monitoring Progress After discussing goals, the team will discuss how progress will be monitored by deciding how and when the superintendent goals for the first year will be finalized. In addition, the team will begin developing a year-long agenda to monitor progress and to help build the evidence toward accomplishing the goals. Facilitators The workshop will be led by an MASC facilitator and supported by the NSIP coach. The facilitator and coach will consult with the superintendent and the chair prior to the workshop to gain a basic understanding of district context, enabling them to lead a workshop tailored to the district.

NSIP Evaluation Report – July 2014 Appendix C: MASC-MASS/NSIP Collaboration

School Committee-Superintendent Collaboration, Beginning the Work Together, July 2014

School Committee-Superintendent Collaboration

SAMPLE WORKSHOP AGENDA

1. Ice-breaker/Get-to-Know Exercise & Workshop Preview (20 minutes)

2. Operating Protocols (60 minutes) Roles & Responsibilities (10 minutes) Contrast Roles of School Committee & Superintendent Context What are the formal/informal protocols already in place? What works well? Where is there need for change/improvement? Grey Areas What are the grey areas in this district? In addition, touch on: Access to superintendent, staff Access to schools (visits) Information requests Constituent communication/Chain of communication E-mail responses Confidential information Scope of Chair Role/Responsibility Commit to common expectations (Operating Protocols)

3. Goal Setting (60 minutes)

Context What documents are already in place?

What are the top student achievement issues in the district? What does the committee see as current priorities? Why did you hire this superintendent?

What did you hope s/he would bring to the district? Where do you want to be a year from now?

Year One – Laying the Groundwork for Future Years What does the Superintendent want to accomplish in Year one? How will the NSIP work inform/affect Year One work? How will the committee stay informed of NSIP work? How will the superintendent and committee ensure coordination?

4. Monitoring Progress (30 minutes)

How & when will Superintendent Goals for Year One be finalized? What evidence/reporting for monitoring progress will occur? Build Year-long agenda

5. Sustaining Progress (10 minutes) Mid-Year Check-in Annual Workshop

Date post:	21-Apr-2018
Category:	Documents
Upload:	doquynh
View:	220 times
Download:	2 times

New Superintendents Induction Program … Superintendents Induction Program Evaluation Report...

Documents