Download - Beyond Buy-In: Partnering with Practitioners to Build a Professional Growth and Accountability System for Denver's Educators

7/28/2019 Beyond Buy-In: Partnering with Practitioners to Build a Professional Growth and Accountability System for Denver's Educators

1/32

Beyond Buy-In:Partnering with Practitioners to Build A Professional Growth

and Accountability System for Denvers EducatorsBy Craig D. Jerald | June 2013


2/3202 The Aspen Institute Educat ion & Society Program

The Aspen Institute is an educational and policy studies organization based inWashington, DC. Its mission is to oster leadership based on enduring valuesand to provide a nonpartisan venue or dealing with critical issues. The Institutehas campuses in Aspen, Colorado, and on the Wye River on Marylands EasternShore. It also maintains o fces in New York City and has an international networko partners.

The Aspen Education & Society Program provides an in ormed and neutralorum or education practitioners, researchers, and policy leaders to engage inocused dialogue regarding their e orts to improve student achievement, and to

consider how public policy changes can a ect progress. Through our meetings,analysis, commissioned work, and structured networks o policymakers andpractitioners, the program, or nearly 30 years, has developed intellectualrameworks on critical education issues that assist ederal, state, and localpolicymakers working to improve American education.

Copyright 2013 by The Aspen Institute

The Aspen Institute

One Dupont Circle, N.W.

Suite 700

Washington, DC 20036

Published in the United States of America in 2013 by The Aspen Institute

All rights reserved

Printed in the United States of America

Publication Number: 13/019


3/32


and Accountability System for Denvers Educators

By Craig D. Jerald | June 2013


4/32

Acknowledgments Thank you to the Denver Public Schools teachers, principals, and central o ce sta who contributedtheir valuable time and expertise to this study. Special thanks to Heidi Harris or shepherding thestudy through DPS, and to Heidi and Jenni er Stern or providing invaluable context and eedback

throughout the dra ting and revision o the report. Thank you also to Rachel Curtis o Human CapitalStrategies or Urban Schools and to Ross Wiener o the Aspen Institute or their guidance andinsight ul input at every stage o the project.

Thank you to partners who supported this work: The Bill & Melinda Gates Foundation, JPMorganChase & Co., the Colorado Legacy Foundation, the Colorado Department o Education, and theJoyce Foundation.

The ndings and conclusions expressed in this publication are those o the author and do not

necessarily refect the views o these individuals, organizations or the oundations.

About the AuthorC D. J d is the president o Break the Curve Consulting, which provides technical assistanceand strategic advice to organizations working to improve education or all students. Be ore oundingBreak the Curve, Craig served as a principal partner at The Education Trust and as a senior editor at

Education Week . His other recent publications include Leading for Effective Teaching: How School Systems Can Support Principal Success (Bill & Melinda Gates Foundation, 2012) and Movin It and Improvin It: Using Both Education Strategies to Increase Teaching Effectiveness (Center or American Progress, 2012).


5/321 Beyond Buy-In: Partnering with Practitioners to Build A Pro essional Growth and Accountability System or Denvers Educators

Overview Although teacher buy-in has become a buzzwordin the national conversation about new approachesto evaluating teachers, Kendra Wilhelm recalls beingcorrected when she used that phrase during an

interview or the position o LEAP program manager orDenver Public Schools (DPS) last year. LEAP, whichstands or Leading E ective Academic Practice, isDenvers new system or developing and evaluatinge ective classroom teaching. Instead o designingLEAP in the central o ce rst and then using targetedcommunication strategies to encourage educatorsto buy into the new system, DPS made a seriouscommitment to deeply engage teachers and principalsin every step o LEAPs ongoing design, development,and rollout. It might seem like a technical distinction,but it represents a major shi t in philosophy, says

Wilhelm, who was hired in July 2012.

The process began early in 2009, when the Bill & Melinda Gates Foundation invited DPS to compete ora large multi-year grant to build a comprehensive talentmanagement system or ensuring an e ective teacherin every classroom. The competition required districtleaders to cra t their plans in partnership with localteachers associations, a stipulation o ering DPSa decided advantage stemming rom a success ulearlier collaboration with the Denver Classroom

Teachers Association (DCTA) on a groundbreakingper ormance compensation plan adopted in 2004. InJanuary 2010, the Gates Foundation awarded DPS a$10 million, three-year accelerator grant to supportthe districts e orts to develop a shared de nition o e ective teaching, along with an accountability andsupport system to assess teachers per ormanceagainst that de nition and to acilitate their e orts toimprove over time.

Figure 1. DistriCt Context

dIstrIct Facts*

Schools 155

Teachers 5,237

Students 66,814

student demographIcs

American Indian 1%

Asian 3%

Black 15%

Hispanic 58%

White 20%

Other 3%

Eligible or Free orReduced-Price Lunch 72%

* Excluding charter schools, which do not participate in LEAP.

Since then, DPS has leveraged a wide variety o strategies to ensure that LEAPs ongoing developmentis deeply in ormed by educators own experiences andideas. The district immediately carried its partnership

with DCTA into the initial design phase beginningin April 2010. Henry Roman and Carolyn Crowder,DCTAs president and executive director, agreed toserve with superintendent Tom Boasberg and otherdistrict leaders on a ve-member Steering Committeethat unctions as the core decision-making body orwork related to LEAP. In addition, Pam Shamburg, ahighly respected middle school teacher, was appointedby DCTA to work on special assignment as a member


and Accountability System for Denvers EducatorsBy Craig D. Jerald | June 2013


6/322 The Aspen Institute Education & Societ y Program

o the districts LEAP Leadership Team. In the ull-timeposition o DCTA Liaison, Shamburg works hand-in-hand with central o ce leaders to help manage everydetail o LEAPs ongoing development.

The composition o both the Steering Committee andthe LEAP Leadership Team ensures strategic cross-unctionality across the central o ce, along with strongteacher voice, which enables LEAP to be continuouslyaligned with policies related to curriculum andinstruction as well as human capital management. TheLEAP Leadership Team includes central o ce leadersresponsible or teacher leadership and development,student assessment, and talent management broadly.

As such, the work o designing and developing LEAPis shared across the central o ce rather than within asingle, siloed team within DPS.

DPS ormally launched LEAPs design phase byengaging an outside organization to acilitate 23stakeholder ocus groups during a three-week periodin April 2010, allowing more than 225 teachers,principals, and other stakeholders to provide earlyinput about what was working with the currentsystem and what guiding principles shouldin orm a new system. The district then ormed vepractitioner-led Design Teams to make more detailedrecommendations or LEAPs design based on theocus group ndings. DPS and DCTA jointly appointeda teacher and a principal to co-chair each Design

Team, and those co-chairs then selected several otherteachers and principals to ll out the diverse teams.

Based on Design Team recommendations, DPSdeveloped a customized Framework for EffectiveTeaching during the all o 2010, introducing a sharedde nition o e ective teaching that would becomethe oundation not only or LEAP but or aligning all o Denvers talent management policies. Early in 2011,DPS began piloting several LEAP components thatwere ready to test on the ground, including initialversions o the Framework , the classroom observation

and eedback cycle, and student perception surveys. A 16-school pilot conducted that spring enabled morethan 500 classroom teachers and school leaders toprovide eedback on those components, leading tosigni cant improvements in LEAPs initial design. Italso contributed greatly to the creation o pro essionaldevelopment resources aligned with the Framework .

DPS then invited all schools to test the re nedcomponents through a ull-year pilot during the 2011-12 school year. To ensure committed participation inthe pilot, DCTA and DPS agreed that school acultiesshould vote on whether or not to participate in thepilot. Based on a majority vote o teachers in eachbuilding, 94 percent o Denver schools volunteered toparticipate.

DPS adopted a multi- aceted strategy to ensure thatLEAP would continue to be developed and re nedwith signi cant input and eedback rom practitionersthroughout the yearlong pilot. The LEAP Teamexpanded to include a second ull-time DCTA positioncalled Outreach Manager, enabling music teacherZachary Rupp to visit nearly every school during theyear to talk with teachers about LEAP and to solicitace-to- ace eedback. Superintendent Boasberg

and Chie Academic O cer Susana Cordova alsodiscussed LEAP during the Superintendent FacultyMeetings they hold in each school every year.

Thousands o teachers responded to ormal onlinesurveys administered by the research organizationMcREL a ter each o our classroom observationwindows, and McREL also convened ocus groupso teachers several times during the pilot. Finally,hundreds o teachers and principals took advantage o the LEAP websites anonymous eedback unctionality.In a survey conducted during spring 2012, ully 80percent o teachers reported that they had provided

eedback on LEAP.

In addition, the lead-up to the 2011-12 district-widepilot provided opportunities or hundreds o teachersto go beyond providing eedback and take ownershipo helping their peers better understand LEAP. Prior tothe building-level vote on participation, more than 60teachers rom the original 16 pilot schools volunteeredto visit other schools in order to discuss their personalexperiences with LEAP and to answer questions aboutparticipation in the pilot. And, rather than conductingcentralized training or thousands o teachers on LEAP,

DPS asked teacher leaders and principals to conducttraining or educators in their own schools using a seto adaptable turnkey materials.

DPS learned valuable lessons about how to partnerwith practitioners at every stage o the LEAPprocess, rom initial design to large-scale piloting.

Above all, Denvers experience with LEAP shows



that it is possible or school systems to design andimplement major initiatives with signi cant involvementrom teachers and school leaders. I Denver is any

indication, educators are hungry or opportunities to bemeaning ully involved in large-scale re orm initiatives,and many will gladly spend signi cant amounts o time and energy on such e orts i they believe theircontributions will be taken seriously.

BackgroundIn March 2010, Denver Public Schools (DPS) published

The Denver Plan 2010, a comprehensive strategicblueprint or improving student outcomes. The Planplaced a high priority on ensuring a highly e ectiveteacher in every classroom and outlined a set o strategies to support that commitment, includingdevelopment o a shared de nition o e ective teachingand a multi-measure system to provide eedback and pro essional development to help teachersimprove their practice. That set o work, dubbedEmpowering Excellent Educators, would be supportedby a $10 million grant to de ne and support e ectiveteaching, which the Bill & Melinda Gates Foundationhad awarded DPS that January. In addition, DPS

planned to build on its track record o success ulcollaboration with the Denver Classroom Teachers

Association (DCTA), beginning with a ve-leaderSteering Committee that included Superintendent TomBoasberg, DCTA president Henry Roman, and DCTA Executive Director Carolyn Crowder.

Among large urban school districts, Denver enjoysan especially rich history o labor-management

collaboration to improve human capital policies. In1999, union and district leaders agreed to pilot anambitious pay- or-per ormance initiative eventuallyknown as the Denver Pro essional CompensationSystem, or ProComp. The agreement created aour-person Design Team to oversee the designand implementation o a two-year pilot, with thedistrict and union each appointing two members. In2001, DPS established a larger Joint Task Force on

Teacher Compensation, including ve teachers andve principals, to design a ull-scale version urtherpiloted and eventually approved by a majority voteo union members in 2004. According to a 2007book co-written by a union-appointed member o the original Design Team, teacher members o the[task orce] made critical contributions to ProCompsdesign, rendering it workable, air, and likely the mostprogressive teacher pay system in the United States 1

at the time.

A number o educators continued to play signi cantleadership roles in ProComp as it was urther re nedand implemented, including Henry Roman, who waslater elected DCTA president in 2009. Roman recallssome surprised reactions when he sent a personalizedletter to principals and assistant principals explainingthe amount o per ormance compensation theyquali ed or under ProComp. Here is the guy romthe teachers union explaining this to principals, saysRoman. You get to a certain level o trust i you do

this or a long enough time.

Susana Cordova, Denvers Chie Academic O cer(CAO), agrees that the ProComp experience laid ertileground or continuing collaboration with educatorsto establish new ways o working. She also recallsleaders very intentionally engaging in a listening tourto talk with educators about how district re orms wereworking and not working a ter she was recruited to

join the central o ce in 2002, ollowing a set o airlytop-down initiatives prior to that. By now we havea pretty long history o collaborating to gure out

what makes the most sense and, in act, a pretty highexpectation that thats the way you do things, shesays. I dont think that having a small group o peopledesign something and then say go run with it wouldwork here.

Denvers experience with LEAP shows that it is possible or school systems to design and implement major initiatives with signi cant involvement rom teachers and school leaders.

1 Gonring, P., Teske, P., & Jupp, B. (2007). Pay-for-Performance Teacher Compensation: An Inside View of Denvers ProComp Plan . Boston, MA:Harvard Education Press.



That historical context contributed to a productivecollaborative experience when DPS and DCTA agreedto respond to the Bill & Melinda Gates Foundationsinvitation to compete or a grant to support newways to de ne and improve teacher e ectivenessin 2009. And it suggested several strategies orengaging teachers in the work supported by thegrant, including a Steering Committee with signi cantDCTA representation and opportunities or teachersto pilot and help re ne the new system prior to ullimplementation.

However, while earlier experiences provided a valuableoundation or collaboration, with LEAP Denverwent ar beyond the level o educator engagementseen in any previous initiative. The reason had to dowith the undamental purpose o the new system.LEAP is a system o pro essional growth and

support with an evaluative component, explainsSuperintendent Boasberg. You would never just callLEAP an evaluation system. Because growing as apro essional requires a greater investment o time andenergy than merely being evaluated as a pro essional,it stood to reason that LEAP would need to bedesigned with signi cant involvement rom the veryeducators who would be expected to success ullyleverage it or that purpose. We knew we neededto get a lot o engagement rom our educators inorder to design the most thought ul system possible,recalls Boasberg.

Designing LEAP(January 2010 to January 2011)

Be ore diving into designing its new system o teachergrowth and accountability, DPS rst had to decideon an internal structure or managing that work, onethat would support signi cant participation rompractitioners while ensuring strategic alignment withthe districts broader goals. (See Figure 2.) To thatend, DPS determined that LEAP should be managedby a cross- unctional group o leaders rather thanscattering responsibilities across existing o ces, onthe one hand, or isolating all responsibility within anew or existing o ce, on the other. In addition to aclassroom teacher who serves as the ull-time DCTA Liaison, the LEAP Leadership Team includes centralo ce leaders with broad strategic responsibilities or

diverse policy areas ranging rom student assessmentto talent management to pro essional developmentand teacher leadership. You have to think hardabout your internal structure, says Jenni er Stern,Denvers Executive Director o Talent Management,because what youre building has to connect with thebroader work and strategy o the entire school district,which requires engagement with lots o other internalstakeholders.

DPS intended the composition o the SteeringCommittee to likewise acilitate strategic cross-unctionality along with strong teacher voice. Byincluding both the districts Chie Academic O cer,Susana Cordova, and its Chie Human ResourceO cer, Shayne Spalten, the district ensured thatLEAP would be developed in alignment with strategicinitiatives impacting the curriculum and instructionin classrooms along with DPSs e orts to recruit,reward, develop, and retain talented teachers in everyclassroom. (See Figure 2.)

To ensure that input rom educators and otherstakeholders would guide the systems development

rom the very outset, DPS engaged Civic Canopy, aDenver-based nonpro t organization, to acilitate aseries o ocus groups during April 2010. Civic Canopyorganized the ocus groups as an iterative processtaking place over a three-week period, allowing ordeeper probing o participants viewpoints. The rstround explored what participants believed to be theshortcomings o the current system, contextualizingthe question with data on student achievement and

DPS determined that LEAP shouldbe managed by a cross- unctional

group o leaders rather thanscattering responsibilities acrossexisting o ces, on the one hand, orisolating all responsibility within anew or existing o ce, on the other.



steerIng commIttee

Oversight committee or Empowering Excellent Educators responsibleor ongoing strategic direction and decision making. Members include:

Tom Boasberg, DPS Superintendent Susana Cordova, DPS Chie Academic O cer Shayne Spalten, DPS Chie HR O cer Henry Roman, DCTA President Carolyn Crowder, DCTA Executive Director

LEAP Communications LEAP Operations Team DCTA LEAP Outreach Manager LEAP Program Manager

A cross- unctional team o DPS sta members dedicated to thedesign and development o LEAP. The team includes cen tral o iceleaders who oversee major aspects o human capital, instructional,and pro essional development strategy. It also includes a teacheron a special assignment who services as the Denver Classroom Teachers Associations liaison or LEAP design and development.Current members include:

Tracy Dorland, Deputy Chie Academic O cer, Teaching and Learning Jenni er Stern, Executive Director, Talent Management Debbie Hearty, Executive Director, CAO School Supports Pam Shamburg, DCTA Liaison Theress Pidick, Director, Teacher E ectiveness Erin McMahon, Director, Teacher Talent Management Executive Director, Assessment, Research and Evaluation (Vacant)

d it

d it

d it

d it

d it

Practitioner-chaired teams o teachers and school leaders ormed to makerecommendations or LEAPs design based on ndings

Figure 2. organizational struCture For Managing leaPs Design anD DeveloPMent

Leap LeadershIp team other Leap staFF



teachers current evaluation ratings. Subsequentrounds built on that oundation to explore ideas aboutwhat a better approach might look like and then testedemerging ideas to ensure con dence in the ndings.(See Figure 3.) More than 225 principals, teachers,district sta members, and students participated in theprocess, and Civic Canopy wove their collective inputinto a set o guiding principles that were then usedguide LEAPs design moving orward. (See Figure 4.)

Teachers who participated in the ocus groups saythey appreciated the opportunity to provide input at

such an early stage. It empowered teachers to helpget the conversation going, explains Keith Roybal,a high school science teacher who participated inthe ocus groups and who later agreed to serve asthe LEAP Teams DCTA Outreach Manager or 2012-13. When youre making such a big paradigm shi t,you have to give people the opportunity to digest itand to think about it right rom the start, and thoseocus groups were antastic or that purpose. Roybalespecially valued the opportunity to refect on betterapproaches to pro essional development as a critical

Figure 3. guiDing Questions For aPril 2010 FoCus grouPs

Week 1

What about the current teacher per ormance and evaluation system plays an e ective role inincreasing student achievement and promoting teacher growth?

What about the current system does not play an e ective role in increasing student achievement andpromoting teacher growth?

Based on what is working and not working in the current system, what values do you see as mostimportant in a more ideal system?

Week 2

What are your responses to the core values that we heard rom participants in Week 1?

What jumps out at you?

What might be missing?

Based on these values, what would you want the Design Teams to take into account as they build theirindividual element o the system?

What should it look like? What is your hope?

What shouldnt it look like? What might be a concern?

Ideas/suggestions to help move it in the right direction.

Week 3

Based on your experiences with the current systemboth positive and negativeis there anything

you would want to be included in an ideal system that is not contained in the core values ormulatedduring Weeks 1 and 2?

How important, on a scale o 1-5, do you think each o these values is?

In Week 2, participants brainstormed what the Design Teams need to abide by as they build theircomponent o the system so that they create something that teachers and principals can support(small groups reviewed these lists):

Are these the right parameters?

Is there anything missing? Does something need changing?

O er your best set o Design Principles to pass o to the design teams.



component o the new system. The acilitators wouldask things like, Do you think you have received use ulcoaching rom instructional leaders? So it really gotus thinking about a whole new model o evaluation andpro essional development, which was exciting or me.

Tracy Dorland, Denvers Deputy CAO or Teachingand Learning, believes it was especially importantor DPS to engage a strong, independent thoughtpartner committed to transparency in the ocus groupprocess. She recalls attending one meeting aboutthe written report where representatives rom Civic

Canopy discussed including certain statements thatshe ound potentially problematic. I told them I wasntsure about putting that in the report, and they said,Then were not giving you the report because youhired us to produce an honest report about what yourstakeholders are telling you, recalls Dorland. Sothey were an excellent thought partner, to the point o pushing us to be completely transparent even when itwas uncom ortable.

Figure 4. guiDing PrinCiPles eMerging FroM sPring 2010 stakeholDer FoCus grouPs

rooted In proFessIonaL expertIse The de nition o e ective teaching needs to be based on the best research and is co-constructed byteachers themselves. Administrators and other evaluators must have the background and expertisenecessary to accurately and airly assess the quality o the teaching they are charged with observing.

muLtIpLe sources oF data The system o assessment should bring together various points o data (including principal observation,peer observation, student growth, sel -refection, and other in ormation)to identi y areas o strength and to set clear, speci c targets or growth.

contInuous FeedBack The system should provide requent and ongoing eedback about practice, rather than one-shot data points.Constructive eedback is the li eblood o improvement, providing in ormation about areas o strengthand areas or growth, and it should fow through all aspects o the system to ensure each element romclassroom practice to pro essional developmentis achieving the desired results.

consIstency WIth FLexIBILIty The system should set clear standards o e ective practice and apply them aith ully and airly across thedistrict, but allow enough fexibility to set goals or improvement and pro essional development based onthe levels o experience and unique needs o each educator.

accountaBILItyWhile the system should aspire to help everyone improve their practice, it must also distinguish betweenvarious levels o per ormance, and hold people accountable or reasonable results. Improvement plansmust be ollowed and have consequences. The measurement system should change rom a binarysatis actory/unsatis actory to a continuum o per ormance with speci cally de ned levelso pro ciency.

a cuLture oF LearnIng The system must support and encourage learning and innovation at all levelsin students, in educators,and in administratorsinstead o being punitive or just rewarding compliance. Growth must be the endgame or all members o the system. The district as a whole, as well as individual schools, must be intentionalabout ostering a culture that supports everyone to learn.

reWard eFFectIveness The system should reward e ectiveness, linking nancial rewards to the evaluation system as well as non-nancial rewards such as recognition and unique pro essional opportunities. It should reward e ectivenessregardless o years o experience.



DPS next convened ve practitioner-led Design Teams to make recommendations or ashioningseveral key components o the new system, guidedby the ocus group ndings. (See Figure 5.) Giventhat ewer than 50 o the districts more than 5,000teachers and school leaders could serve on a Design

Team, DPS took care to cra t an open, democratic,and competitive selection process that would con era high degree o credibility to the teams. First, allteachers and principals were openly invited to apply

via an online orm asking applicants to answer severalessay questions about why they wanted to participateand how they would improve the current system, aswell as to provide a peer recommendation.

Then, rom more than 100 applications received, DCTA proposed a teacher to co-chair each o the ve teamswhile DPS proposed a principal to co-chair eachteam. Leaders rom DCTA and DPS met to discusstheir proposed co-chairs and to jointly approve thenal selections. Frankly, we were a little worried thatthere would be some tension about those decisions,recalls Dorland. And we did have some very honestconversations, but that ultimately helped us tocontinue to build the relationship moving orward.

Finally, the new co-chairs met in pairs to collaborativelyselect several teachers and several principals (or

assistant principals) or the remaining slots on eachteam. DPS provided a protocol to help co-chairsreview the applications, but nal selections rested withthe co-chairs themselves. Ive seen Design Teams insome districts that didnt have credibility because theyappeared to be sta ed with hand-picked teachers,says Stern. I think the process we used made clearthat this was a serious e ort to give a real voice toeducators, and that really mattered.

Given that ewer than 50 o the districts more than 5,000 teachers and school leaders could serve on a Design eam, DPS tookcare to cra an open, democratic,and competitive selection processthat would con er a high degreeo credibility to the teams.

Figure 5. leaPs PraCtitioner-leD Design teaMs

d i t TeacherEfectiveness

PeerObservation

Assessment& StudentOutcomes

Pro essionalDevelopment

PrincipalEfectiveness

F De nitiono e ectiveteachingembedded ina ramework or observingclassroompractice

Process or peerobservation andeedback

Ways o aligningthe districtsstudentassessment planwith LEAP

Ways to connectpro essionaldevelopmentopportunities toLEAP

Initial dra to SchoolLeadershipFramework;policiesto enableprincipalsto supporte ectiveteaching,including waysto evaluateschool leaders



Another actor contributing to the success o theDesign Teams came rom ollowing through on thecommitment to be guided by the ocus group ndings.Having that ocus group data was really valuable,because there are times you eel outnumbered asa teacher, recalls Spanish teacher Noah Geisel,who served on the Teacher E ectiveness team.When disagreements arose, I was able to remindeveryone that we were charged with being aith ul towhat teachers wanted in the system. So the ocusgroup data were really valuable to me as a membero the team to make sure those voices were beingrepresented.

Lori Nazareno, the teacher co-chair or that team,strongly agrees that the ocus group ndings o ereda critical oundation during the initial design phase.The members o the Teacher E ectiveness team were

always going back and rechecking to make sure wewere staying true to what the original ocus groupshad said, she says. Had that not happened, it wouldhave just been a waste o everybodys time, an earlypoint to drop the ball and lose credibility.

According to Dorland, it also was important toestablish a clear process and scope o responsibilityor the Design Teams be ore they convened. Eachteam began by reviewing research and promisingmodels rom around the country and then workedtoward making high-level recommendations or

designing relevant components, along with a rationaleor each recommendation. The central o ce LEAP

Team then feshed out design proposals basedon those recommendations. A ter updating andconsulting with the Steering Committee, the LEAPLeadership Team brought unresolved issues back to the Design Teams or urther discussion. Thatdivision o labor permitted practitioners on Design

Teams to substantively guide the work (the whatand the why) without getting bogged down intechnical nuances (the how) they would not havehad su cient time or expertise to tackle. And it

enabled Dorland and her colleagues to coordinateacross Design Team recommendations to ensure thatthe process resulted in a coherent system rather thana set o ill-matching components.

The Design Teams pushed the district to ensure thatthey were given the fexibility they needed to createa system that they believed met the intentions o

the ocus groups. In July, the Teacher E ectivenessteam strongly recommended that DPS develop itsown ramework describing e ective teaching ratherthan simply recommending which nationally availableclassroom observation instrument the district shouldadopt, as had been the districts original goal orthat team.

The team presented a persuasive rationale or itsrecommendation, arguing that none o the instrumentsit had reviewed was su ciently keyed to the speci cneeds o Denvers teachers and students. First, noneadequately addressed the needs o English LanguageLearners, who comprise 35 percent o Denvers

student population. Second, according to teammembers, no instrument ocused enough attention onobserving student behaviors during a classroom lessonin addition to teacher actions. Finally, none providedexplicit and detailed enough per ormance descriptorsto adequately speci y and distinguish teacher andstudent behaviors across a broad spectrum o teacherdevelopment, particularly or teachers working in anurban context.

That unexpected recommendation presented districtleaders with an extremely di cult decision. Designing

a customized ramework would require much moretime and labor than they had originally anticipated,especially i educators were to continue to haveconsiderable input, a real concern given the districtsplan to begin piloting classroom observations byearly 2011. Moreover, given that the main purpose o LEAP is to enable teachers to improve their practice,the district had committed to providing pro essional

Te Design eams pushed thedistrict to ensure that they were given the exibility they needed to create a system that they believed met the intentionso the ocus groups.



development resources aligned with whateverramework was adopted or the pilot. Delivering thoseresources in time would be much easier i DPS simplyadopted a nationally available ramework rather thandeveloping a customized one.

But practitioners on the Design Team elt very stronglyabout the recommendation; or some, the districtsresponse would signal whether leaders were trulyserious about giving practitioners a meaning ul voicein LEAPs design. The teachers and administratorson the team agreed very strongly that we had signedup or a design team, not an adoption team, recallsNazareno. This was a very, very hard decision or thedistrict, but or us it carried a lot o water in terms o building trust.

Ultimately, district leaders agreed with the teams

rationale or making the recommendation, eventhough it would entail extensive and intensive work at a breakneck pace to meet the deadline or the pilot.You have to decide what you have the stomach orbe ore you embark on this kind o process, saysCAO Susana Cordova. To what degree will youallow the system to evolve in unexpected directionsversus adhering to a set o expectations that arenon-negotiable? My most important advice oranother district considering this would be not togo into a partnership with practitioners i yourenot willing to partner.

As a result, Denver now has a homegrown DPSFramework for Effective Teaching in ormed byresearch-based tools but developed with extensiveinput and eedback rom Denvers own teachersand principals through more than 80 revisions. I this is really something a district wants to do withteachers, then you need to invite them to be a part o the process and empower them, says Design Teammember Geisel. Had the district not agreed with us,I think wed be in a very di erent place right now interms o teachers and even administrators eeling like

they have had a real voice in this.

Although the central o ce LEAP Team took responsibility or the day-to-day work o dra ting theFramework , teachers continued to be involved in manyways. Members o the Teacher E ectiveness Design

Team reviewed and provided eedback on dra tsthrough the teams regular meetings. As the ull-

time DCTA Liaison on the LEAP Team, social studiesteacher Pam Shamburg played an elbows-deep rolein the dra ting process and also reached out to otherteachers or in ormal input on dra ts, ensuring thatteacher voice helped drive Framework developmenton a daily basis. Finally, during the all o 2010,DPS conducted an in ormal pre-pilot o classroomobservations by principals and peer observers in twoschools, enabling 28 teachers to react to the dra tFramework in use as an instrument or observinglessons and providing eedback to help teachersimprove their practice.

By the end o the summer o 2010, the otherour Design Teams also had submitted ormalrecommendations or designing various LEAPcomponents, and their recommendations too showeda keen desire or the system to meet the needs o

practitioners. For example, the Assessment andStudent Outcomes team proposed multiple bucketso student assessments to comprise the studentoutcomes measure in LEAP, going well beyondstandardized state and district tests to includeassessments that would be designed by teachersthemselves, both individually and in teams, at thebuilding level. The Pro essional Development teamrecommended that classroom observations provideteachers with targeted eedback in two especiallyrelevant ocus areas, a schoolwide growth areaand an individual area selected by each teacher. The

sidebar on page 11-12 describes the undamentalcomponents o the LEAP system as it stands todayollowing signi cant piloting and re nement beginningin 2011.

Piloting and Re ningLEAP (January 2011 to Present)

By 2011 several components o LEAP were readyto be more ormally piloted, including the dra tFramework ; a classroom observation and eedback process by principals and peer observers based onthe Framework ; the Student Perception Survey; and aninitial set o online pro essional development resourcesaligned with the Framework . Based on applicationsrom principals in Title I schools and a survey to gauge



the interest and support o teachers in those buildings,DPS selected 16 schools to take part in a pilot to beheld during the spring semester, allowing more than500 teachers to provide eedback based on rst-handexperience with LEAP.

In addition to gathering eedback in ormally rom thoseteachers throughout the spring pilot and encouragingthem to submit comments via the LEAP website, DPSalso engaged McREL, a national nonpro t researchgroup headquartered in Denver, to conduct severalormal surveys and ocus groups. From Februarythrough May, McREL held three ocus groups orteachers as well as one or principals and one orpeer observers. It also surveyed teachers a ter theirsecond classroom observations, using a mix o closed-and open-ended items to gather nuanced input tore ne LEAP. For example, one teacher wrote, TheFramework and the process itsel have been positiveand have made me refect on my practice. [However],I am concerned with the 100% o students engagedand 100% o students sitting on the edge o their seatstypes o comments on the ramework. Engagementlooks di erent in various situations.

Using what it learned rom the pilot, DPS immediatelybegan to make signi cant revisions to all o the LEAPcomponents. For example, the LEAP Team revisedthe Framework to remove language related to 100percent o students and added several indicators

ocused on English Language Acquisition. Givenconcerns rom teachers that observers could not seeeverything necessary to assess the indicators in theFramework during classroom observations, DPS alsodecided to increase the length o observations andto provide more fexibility in scoring indicators usingN/A. Moreover, a ter some teachers expressedconcerns about the length and complexity o the

Tripod student surveys, especially or younger studentsand English Language Learners, DPS also embarkedon a project to streamline and customize the survey towork in the DPS context.

Elementary school teacher Ian McIntire says heappreciated the opportunity to learn about and pilotLEAP early in its development because he elt it gavehim a signi cant voice in molding it. It was reallyvaluable to me to be involved early on with that pilot,says McIntire. Given how use ul the spring pilot turnedout to be both or educators like McIntire and or

unDerstanDingleaPLeading E ective Academic Practice(LEAP) is Denvers new growth andaccountability system or ensuringe ective teaching in all o its schools.LEAP provides educators witha multiple-measure view o theirteaching practice as well as withaccess to pro essional developmentresources to support continualgrowth.

ClassrooM observations. Teachers receive observations and eedback multiple times per year rom their schools principalor another school leader. Most teachers also areassigned a peer observer, a ellow teacher workingon ull-time special assignment or the school districtwho has experience in the same content area.Both kinds o observers collect evidence and scorethe lessons they have observed using the DPSFramework or E ective Teaching, a customizedobservation instrument developed with input romthousands o Denver teachers. Based on extensivepiloting and eedback rom practitioners, Denversobservation system now incorporates severaldi erent types o observations by school leaders,including ull observations, partial observations, andwalk-throughs. Following each ull observation, theobserver meets with the teacher to hold a refectiveeedback conversation ocused on improvingclassroom instruction and student learning. Theconversation identi es areas o strength to ampli yin uture lessons and areas or growth in whichthe teacher can make ocused e orts to improve.(DPS encourages school leaders to share eedback ollowing partial and walk-through observations aswell, either in person or via e-mail.)

ProFessionalisM. LEAPs Pro essionalism measure, which is de nedunder its own domain in DPS Framework orE ective Teaching, examines how well teacherscontribute to a positive school climate and culturethat osters student learning. The domain includesan additional indicator related to teacher leadershipor those who have, or plan to, take on a ormalleadership role in their school buildings.


16/32



importance o providing eedback on the new system.Importantly, rather than providing messaging trainingor the outreach teachers, the LEAP Team encouragedthem to answer questions and to tell their own storiesin their own words. Outreach teachers were paid astipend or their time, and were asked to report back on the conversations they held at the schools andhow they thought teachers were likely to vote. I hadoutreach teachers e-mail me to say, Wow, it was re-engine red when I walked in that room, but Im prettysure a ter we talked they were yellow to green andsee some value in the pilot, recalls Amy Skinner, whoserved as Senior Communications Manager on theLEAP Team until last year.

When the votes were tallied, ully 94 percent o schoolshad voted to participate. According to district leadersand educators, the outreach rom pilot teachers

contributed greatly to that very high rate o voluntaryparticipation in the ull-year pilot. Other actorsincluded vocal support rom most o the districtsprincipals; clear communication about the value o participation rom DCTA representatives; and tangibleevidence that DPS already had begun to make majorchanges to the LEAP design based on eedback it hadreceived rom educators during the 16-school pilot.

Expanding Practitioner Feedback to Improve LEAP

To ensure that LEAP would continue to be signi cantly

shaped by the voices o educators during the yearlongpilot, Skinner and her colleagues dra ted a 2011-12LEAP Communication and Teacher Engagementplan, which outlined multiple vehicles or teachersand principals to provide input. To begin with, thedistrict engaged McREL to continue to conduct ormalsurveys and ocus groups o practitioners at key

stages in the pilot. For example, ollowing each o three observation windows, McREL sent e-mails toteachers inviting them to respond to an online survey,and McREL also surveyed teachers who participatedin an in ormal observation using a heavily revisedversion o the Framework . The number o teachersresponding to surveys conducted during each o thethree observation windows ranged rom 1,286 to2,039, in some cases more than hal o the teacherswho were e-mailed a survey orm.

DPS also strongly encouraged all teachers andprincipals to take advantage o other avenues orproviding written or verbal eedback during the pilot,including an online eedback orm, an e-mail address,and a phone number on the LEAP website. (SeeFigure 6.) During 2011-12, the LEAP Team loggedmore than 500 individual pieces o eedback provided

over the website, and operational specialists on theteam ensured that anyone who submitted a writtencomment received an acknowledgment along withanswers to any questions asked. For Brenda Kazin,principal o Denvers Place Bridge Academy, thewebsite was an important way or me to let the LEAP

Team know when something wasnt working, andI used it a lot. For example, I signaled when someaspects o the Framework didnt make sense orobserving teachers and students at an Early ChildhoodEducation level.

Leaders also had begun to realize the high valueo ace-to- ace conversations with teachers aboutLEAP. Meeting with teachers in their own schoolswas proving to be more success ul than askingthem to travel to o -campus events. As the DCTA Liaison to the LEAP Team, Pam Shamburg wasideally positioned to reach out to teachers in theirown buildings. However, by this time, Shamburg wasplaying ar too signi cant a role in LEAPs day-to-daydevelopment to begin spending a signi cant amounto time traveling to schools. There ore, district andunion leaders agreed to add a second ull-time teacher

liaison position to the LEAP Team, DCTA OutreachManager, with an explicit goal o holding ace-to- acemeetings with teachers in all 128 pilot schools at leasttwice during the year. Zachary Rupp, a music teacherat Columbian Elementary and the Mathematics andScience Leadership Academy, agreed to take a leaveo absence rom the classroom to play the new role.

Meeting with teachers in their

own schools was proving to bemore success ul than asking themto travel to of-campus events.



Finally, Superintendent Boasberg and CAO Cordova

requently discussed LEAP with teachers during theSuperintendent Faculty Meetings they hold in eachschool every year. Rather than simply making a stock presentation and answering a ew questions aboutLEAP, Boasberg and Cordova made an intentionale ort to hold a rank two-way dialogue with teachersabout district priorities, including LEAP, during thosemeetings. Tom is not happy at a aculty meeting i somebody hasnt complained about something, says

Cordova. When he says, I want to hear whats not

working, hes very authentic about that. That givesteachers a way to express concerns directly to seniorleadership, though o course not every teacher iscom ortable doing it. Amy Skinner o ten attendedthe meetings to take notes on the conversations.She and other members o the LEAP Team ensuredthat eedback received via all channels was enteredinto a LEAP Feedback Log , managed via an Excelspreadsheet and now in Microso t Sharepoint.

Figure 6. theory oF aCtion anD button For ProviDing FeeDbaCk on leaP Website



Taken together, these strategies resulted in anunprecedented amount o highly detailed eedback rom practitioners during the 2011-12 pilot, which inturn in ormed a second round o major improvementsto LEAP. For example, as early as November o 2011,it had become increasingly clear that teachers andprincipals considered the Framework too long, toorepetitive, and simply too cumbersome to supportaccurate classroom observations and ocusedeedback to improve teaching practice. There ore,working with the Teacher E ectiveness Design Teamand other educators, the LEAP Team signi cantlyrevised the Framework in time to test a new version ina ourth round o classroom observations even be orethe pilot ended in spring 2012. The new versioncondensed the number o indicators rom 21 to 12 andintegrated English Language Acquisition, technology,and 21st century skills throughout the Framework

rather than breaking them out as separate indicators.

Additionally, in response to concerns rom manypractitioners that the Framework did not su cientlycapture nuances o teaching practice in areas suchas Early Childhood Education and the arts, DPSbegan developing appendices that o er supplementalguidance to principals and peer observers oraccurately observing classroom lessons in specializedareas. The LEAP website now o ers such appendicesin 18 separate areas, ve o them in speci c areas o special education.

Practitioner eedback also prompted major changes tothe observations o classroom practice component o LEAP. In 2011-12, school leaders were expected toconduct three ull observation and eedback cyclesor each teacher, with each classroom observationlasting 45 minutes. Teachers and school leadersexpressed similar concerns with that model. Teacherswanted more requent observations to support theirgrowth over the course o the school year. Principalsbelieved that shorter, more requent observation andeedback cycles would better enable them to osterbest practices in classrooms. As a result, in 2012-

13, school leaders are using a combination o ullobservations, partial observations, and morein ormal walkthroughs.

Because o the major changes to LEAP based on the2011-12 pilot, as well as the number o signi cantdesign decisions remaining, DPS decided to pilotLEAP again during 2012-13, this time with all schoolsparticipating automatically. Components being piloted

this year include the revised Framework , includingan updated rubric and eedback process or thePro essionalism domain; a new model or conductingobservations by principals and peer observers; are ned Student Perception Survey; and severalaspects o LEAPs student outcome measures. Inparticular, the 2012-13 pilot has a orded DPS anopportunity to develop and re ne student assessmentsin content areas or which such assessments werepreviously unavailable, as well as to collect data toin orm how LEAPs multiple measures will ultimatelycome together into a comprehensive system. Thedistrict also has continued to develop and pilot LEAPspro essional development resources, including aseries o Closer Looks that provide teachers within-person training and online tools aligned with speci cFramework indicators.

Connie Casson, Denvers ormer Executive Directoro Accountability, Research, and Evaluation, says the2012-13 pilot is especially critical or building out a airand accurate set o student outcome measures. Weare starting rom a place where theres a lot we dontknow, and no district or state has got this nailed, sowe have got to involve teachers in this process andgure it out together, explains Casson. In additionto collecting continuing input rom the Assessmentand Student Outcomes Design Team, Casson hasbeen convening open houses o practitioners toprovide ongoing input. Moreover, she says Roybal

was a great choice to be the LEAP Teams DCTA Outreach Manager or 2012-13 given his backgroundas a science teacher in a high-needs high school,especially given that science is not currently a subjecttested in state-level assessments. I we can measurethe per ormance o his students airly, we are gettingmost o the way toward our goal, so he is both a greatadvocate and a great partner or helping get romour theory o student outcomes to how they willactually work.

However, even though 2012-13 is technically another

pilot year, LEAPs mandatory implementation in allschools this year marks a signi cant milestone in itsestablishment as the districts o cial new teachergrowth and evaluation system. With LEAP taking rootand becoming more rmly established, the level o practitioner eedback has naturally tapered o , andDPS has adjusted its strategies or collecting sucheedback. Educators are still encouraged to submitquestions and comments via the LEAP website,



and the current DCTA Outreach Manger, KeithRoybal, is visiting schools on a regular basis to holdconversations with teachers about LEAP. DPS is nowhandling the eedback survey process internally andhas phased out externally-managed ocus groups.Boasberg and Cordova nd they still get somequestions and comments about LEAP during thisyears Superintendent Faculty Meetings, but teachersdo not bring up LEAP as a topic during those meetingsas much as they did last year.

Benefits and Challenges of DiverseFeedback Streams

Denvers experience o ers valuable lessons orleveraging multiple streams o practitioner eedback or other school districts planning to implementnew systems like LEAP. First and oremost,

leaders and educators alike point to the value o o ering practitioners a wide variety o avenues orcommunicating their eedback, even to the point o erring on the side o potential redundancy. The ideathat we could give eedback in a lot o di erent wayshelped teachers eel like there really was an attemptby the district to make sure their voices were beingheard, says elementary school teacher McIntire.There would have been value to having lots o waysto give eedback even i only ve teachers used one o those ways.

Remarkably, however, the multiple eedback channelsdid not prove to be redundant in practice. Askedwhether, in hindsight, DPS might have orgone one o the eedback channels or the sake o cost-e ciency,Stern says she believes all o them yielded highly use ulin ormation, though teachers o ten said similar thingsabout LEAP in the during the Superintendent FacultyMeetings and the McREL- acilitated ocus groups.

Teachers say it was especially important to o erways o providing eedback that o ered a credibleguarantee o anonymity. While DPS enjoys an

unusually productive relationship with its local teachersunion, the school system is not immune to the kind o distrust between building-level educators and centralo ce leaders that research shows to be endemicin urban school districts. For example, accordingto Molly Bendor , a ourth grade teacher at Green

Valley Elementary, some teachers worried that i theyexpressed concerns about LEAP they might receivelower results on uture classroom observations.

Moreover, some teachers were skeptical that theireedback would remain anonymous as promisedi they responded to McRELs online surveys orsubmitted unsigned comments to the LEAP website.I still hear teachers say, no, they can trace your IPaddress, Bendor explains.

Geisel, the teacher who served on the TeacherE ectiveness Design Team, points to such concernsas one o the many bene ts o having a DCTA Liaisonand a DCTA Outreach Manager on the LEAP Team.To the districts credit, having those positions gaveevery teacher a way to give eedback in a very sa eenvironment, which was really smart, says Geisel. Idont know how many people took advantage o it,but I knew that i I had something that I didnt eel likeputting in a survey, I could just e-mail Pam or Zach.

Another clear lesson cited by DPS and DCTA leaders alike is the importance o holding live, ace-to- ace conversations with educators. Real-timeconversations back and orth, with questions andexplanations and more questions, have been whatsreally use ul in the pilots, says Roman, the DCTA president. He believes that while technology can beuse ul or collecting eedback, its more power ul

when you have a ace-to- ace conversation. A corollary to that lesson is that more teachers engagein such conversations when they take place in theirown buildings rather than in another location requiringtravel. While they were time and labor intensive, theschool visits by DCTA liaisons like Rupp and Roybaland district leaders like Boasberg and Cordova provedwell worth the e ort.

Denvers experience ofersvaluable lessons or leveraging multiple streams o practitioner eedback or other school districts planning to implement new systems like LEAP.



Figure 7. FloW anD ProCessing oF PraCtitioner FeeDbaCk During 2011-12 Pilot

cLassroom teachers and schooL Leaders

Submit writtencomments or

questions via theLeap W b i (can

be done anonymously)

LEAP Operations Team enters eedback

into an ongoingF b L

LEAP team and othersuse eedback to re ne

and improve LEAP

LEAP Team analyzes eedback toidenti y

McREL producesperiodic w i

i o data rom surveys

and ocus groups

Communications Teamreviews eedback

and cra ts responsesto clari y LEAP andaddress concernsvia website and

newsletter

DCTA OutreachManager describes

how LEAP is evolvingand ollows up withteachers on speci cconcerns or issuesvia e-mail, phone,

or in person

Speak with or e-mailthe dcta Li i orthe dcta om or attend

meeting at own schoolhosted by the DCTAOutreach Manager

Attend thes iF l m i

held in each schoolonce per year

Respond tof l or participate in a

ormal f administered byresearch partner

McREL


22/32



districts designing systems similar to LEAP mightrespond; without comparative data there is no wayto know or sure. On the other hand, it is undeniablylower than what leaders had hoped to see in responseto that question. Asked to explain the gure, leadersand educators point to two possible reasons.

First, despite the vigorous e orts o the LEAP Teamto communicate how eedback was shaping LEAPsdevelopment, central o ces in large districts acereal obstacles in communicating such in ormationto teachers. For example, in interviews conductedor this case study, many teachers were unawarethat DPS had revised and signi cantly streamlinedits Student Perception Survey in direct response toeducators eedback. Roybal points to data rom aMcREL survey question asking where teachers getin ormation about LEAP, which showed that ar ewer

teachers used the website (50 percent) comparedwith talking to peer observers (60 percent) andschool leaders (65 percent) or relying on in-schoolpro essional development (75 percent). Shamburgbelieves that the only way to be absolutely surethat teachers understand how changes have beenin ormed by educators eedback is to make suchconnections completely explicit or them, change bychange, though she acknowledges that is di cult toaccomplish outside o ace-to- ace conversations.

Second, some teachers who disagreed that teachervoice was heard throughout the pilot process mightpersonally have provided eedback that could not leadto changes in LEAP or any number o legitimate legal,technical, nancial, or other reasons. For example,Colorados Senate Bill 10-191, a state law passedin May 2010, mandates that local teacher evaluationsystems include certain non-negotiable eatures suchas basing 50 percent o a teachers evaluation onmeasures o students academic growth and learning.Moreover, some eedback conficted with the guidingprinciples that emerged rom the stakeholder ocusgroups or would have undermined the technicalintegrity o a LEAP component or measuring teachinge ectiveness. Finally, on a purely practical basis, DPScannot respond to every individual request to changeLEAP because teacher voice is ar rom monolithic.

We did a good job explicitly communicating back to teachers, You told us this, so we did this, andthat was hugely important, says Kalpana Rao, whoserved as senior manager or Teacher Per ormance

Assessment during 2010 to 2012 be ore becomingan assistant principal at a Denver turnaround school.But now that Im a practitioner, I realize we didntcommunicate as much about eedback that wecouldnt respond to with changes or some goodreason. Being more intentional about that might not

Figure 8. hanDling FeeDbaCk FroM MultiPle sourCes: MaJor lessons learneD

1. Provide multiple avenues or educators to o er eedback, including ones that o er an unquestionableguarantee o anonymity. Understand that no single communications mechanism will reach all educators.

2. Hold ace-to- ace conversations with educators in addition to collecting written eedback, visitingeducators in their own schools i possible rather than asking them to travel to another location.

3. Invest in a process to track and analyze eedback on an ongoing basis.

4. I collecting written eedback through a website, ask commenters to select a topical category rom a pull-down menu when they submit a question or comment, as that can save sta time on the back end.

5. Feedback can sometimes be misleading i interpreted at ace value, so take extra time to dig deeper andget to the root cause o educators concerns.

6. Communicate regularly with educators about trends and themes in the eedback the district is receivingrom them and how the district is responding by making improvements.

7. Clearly communicate when eedback cannot lead to changes, explaining any legal, technical, or practicalbarriers the district aces in responding to educators concerns or requests.



necessarily make people eel better, but it would helpthem know they were heard.

DPS already has begun to act on that importantlesson learned. For example, this year, LEAPsta recognized that some o the eedback rompractitioners suggested changes to LEAP based onmisunderstandings about the system. In response,the Communications Team incorporated contentinto the LEAP e-newsletter that addresses suchmisunderstanding and helps to clari y importantaspects o LEAP, including a new section calledKeiths Report Card and another called MythBusters.

Leveraging Teacher

LeadershipDenvers experience also illustrates the value o givingpractitioners opportunities to play authentic leadershiproles in the process o designing and piloting a newsystem or teacher development and evaluation. Yes,educators had lots o ways to weigh in on LEAPthrough everything rom surveys to ocus groups,explains Stern, but it also was important to us tointentionally look or ways that teachers could becomeowners and advocates o the LEAP work. Denveraccomplished that goal in two ways. First, the districtleveraged its existing commitment to ostering teacherleadership by enabling teachers who already wereplaying ormal leadership roles to take ownership o critical aspects o the LEAP pilots. Second, DPScreated several entirely new kinds o positions throughwhich teachers could play ormal leadership roles inLEAPs ongoing design and development.

In 2010 Denver launched a new initiative called the Teacher Leadership Academy (TLA) with the ambitiousgoal o training and empowering accomplishedteachers to take on critical aspects o instructionalleadership in every school. By the time the LEAP work was launched, the TLA already had trained hundredso teachers to take on such leadership roles. Districtleaders recognized that TLA-trained teachers would bean important resource or ensuring that LEAP helpedteachers develop their practice rather than simplymeasuring their per ormance, especially as expertsin the vision or e ective teaching at the heart o the

new Framework . Moreover, because TLA-trained

teachers would be central to the districts CommonCore State Standards strategy, they also could providea critical linchpin or ensuring that teachers understoodhow the Common Core and LEAP should unction ascomplementary supports or great teaching rather thanas competing demands on teachers valuable time.

There ore, early in 2011, the district organized a serieso Get to Know the Framework sessions or TLA teachers to help them learn about the dra t Framework and to solicit their eedback or improving it. DebbieHearty, Denvers Executive Director o CAO School

Supports, recalls care ully planning and acilitatingthose sessions to align with the vision or teacherleadership in the TLA initiative. The key message inthe TLA was that this wasnt about implementing adistrict initiative or being a mouthpiece or the district,she says. So when it came to the sessions on theFramework , we told them, You are our leaders, sowe want you to be at the ore ront o this and also togive us your eedback on it. Then later we made sureto go back and show them how their eedback wasin orming the process.

Those sessions laid the groundwork or TLA teachersto play an especially important leadership role whenthe time came to conduct LEAP training or teachersin the 128 Denver schools that voted to participatein the ull-year pilot. When 94 percent o schoolsdecided to participate, all o the sudden we reacheda point where the LEAP Team couldnt be responsibleor every aspect o rolling out the pilot, recalls

Denvers experience also illustratthe value o giving practitionersopportunities to play authentic leadership roles in the processo designing and piloting a new system or teacher development and valuation



Cordova. We were using the RACI [Responsible, Accountable, Consulted, In ormed] model, whichhelped us recognize that opportunity, too. It wasreally, really, really important or us to think across thephases o large-scale implementation and how to buildownership across the di erent parts o the system thatneeded to own it. As a consequence, district leadersdecided to ask the TLA-trained teacher leaders andtheir principals to conduct the LEAP training or theirown buildings.

Over the summer, the LEAP Team held a three-day session or principals and teacher leaders romparticipating schools, both to help them learn aboutthe Framework and LEAP and to prepare them todeliver the training or rollout. Principals and teacherleaders then spent two days planning the LEAPtraining or their buildings and deciding how best to t

that training into the rest o their kick-o work or thenew school year, and they submitted their plans to thedistricts Instructional Superintendents or input andnal approval. The LEAP Team also made sure to beavailable to answer any questions and provide advice,though not every school needed additional help.

To ensure a baseline level o consistency in the trainingteachers received across the 128 pilot schools, theLEAP Team developed a set o adaptable turnkeymaterials or principals and teacher leaders to use,including Word documents, PowerPoint decks, and

video segments. The video segments ensured thatteachers would receive some in ormation expressedin exactly the same ways, while the PowerPoint slidesand Word documents allowed principals and teacherleaders to adapt the training to suit the speci c needso their own colleagues. That worked out to be anice way to ensure teachers were getting consistentand reliable in ormation, says Hearty, while alsobeing able to hear rom people they knew and alreadytrusted as opposed to someone rom downtownwhom they had never met. I really think we were ableto deliver the best o both worlds.

Principals and teacher leaders say they appreciatedthe opportunity to adapt the training or their ownschool contexts rather than simply delivering it in amechanical way. At Skinner Middle School, whichhad participated in the earlier 16-school pilot, principalNicole Veltze and her team decided the trainingneeded to go a step deeper in helping teachers reach

a more sophisticated understanding o the Framework at that point in their experience with LEAP. Thetraining protocols included an opportunity or teachersto observe and score a video-recorded lesson usingthe Framework , but Veltze and her team built ingreater opportunities or the kind o ormal inter-raterreliability calibration training that DPS was providingor principals and peer observers. While it was a lot o work, says Veltze, My teacher leaders enjoyed takingownership o driving LEAP and not just having theadministrators presenting it.

DPS leaders say the strategy was highly success uloverall and demonstrated a power ul approach orrolling out similar large-scale initiatives in the uture. Inparticular, it gave the TLA-trained teachers a chanceto establish themselves as knowledgeable aboutthe Framework in order to be credible sources o

expertise on LEAPs vision o teaching e ectivenessor their peers moving orward. We ended up invitingprincipals and teachers to speak to the school boardabout that experience, recalls Dorland. They saidthey had never elt so much a part o something thatwe were rolling out across the district. They told theboard, We elt very empowered.

In addition to leveraging its existing Teacher Leadership Academy strategy, DPS also created several newkinds o leadership roles or teachers to play in LEAPsongoing design and development, including theteacher co-chair position on the ve LEAP Design

Teams and the ull-time DCTA Liaison and DCTA Outreach Manager positions. Even though the LEAP

o ensure a baseline level o

consistency in the training teachers received across the128 pilot schools, the LEAP eamdeveloped a set o adaptableturnkey materials or principalsand teacher leaders to use


26/32



He also had taught in a school that conducted peerobservations, a LEAP component about which manyteachers had questions and concerns, and he hadcalled on that experience as a member o LEAPsPeer Observation Design Team. Just as important,Rupp also had served on the DCTA bargaining teamthat hammered out an agreement with the districtor ormally piloting LEAP. That gave me a lot o credibility in terms o establishing a sa e environmentor teachers to give me authentic eedback on theirexperiences, says Rupp, who also always made sureto wear a shirt or pin sporting the DCTA logo wheneverhe visited a school.

As Outreach Managers, Rupp and Roybal have had tond time to visit schools while also spending enoughtime in the central o ce to become extremely amiliarwith LEAP as the system is being continuously re ned.

One o the things we ound out early on is that ananswer o , We dont know, doesnt go over verywell when teacher have questions about LEAP, saysShamburg. You can tell them, We havent decidedyet, and here are the things were talking about basedon what weve heard rom the eld. But not justthat, We dont know. So Zach and Keith have hadto make sure to attend enough internal meetings tounderstand the strategic decisions being made.

Shamburg believes the two positions have o ered acritical leverage point or making DCTAs role in LEAPs

development truly collaborative rather than merelyreactive. Henry was the genius behind this, sherecalls. He said the union should be working on thisday in and day out, as opposed to the district buildingsomething and then coming to the union or a yesor no reaction. They also unctioned as a power ulvehicle or continuously strengthening the collaborativerelationship between the union and the district. Thatsthe value o actually being on sta so you can work together, Shamburg explains. You get to know eachother as people and to understand each other aspeople, which builds trust. You move beyond were

the district and were the union and start to solveproblems together.

Even so, maintaining that good working relationshiprequires ongoing and very deliberate work.Sometimes the district would orget that its supposedto be a partnership, and they had to be reminded,recalls Shamburg. She says its important or both

PraCtitioner FeeDbaCk. The cost o an outside research contractorto conduct surveys and ocus groups can beconsiderable, but a district could use ree onlineso tware such as Survey Monkey, or conductits own ocus groups, to collect similar targetedeedback during piloting. In act, DPS has adoptedboth o those lower-cost alternatives during thecurrent school year. When it came to logging andanalyzing eedback rom multiple sources, DPSused low-cost tools such as Excel spreadsheetsand, more recently, SharePoint so tware. LEAPOperations Team members advise districts toinclude a pull-down menu o topical categories whensoliciting Web-based comments or questions rompractitioners, as that can greatly reduce sta time onthe back end.

teaCher leaDershiP.Like Denver, districts already investing in a teacher-leadership strategy can ask teachers who alreadyhold such positions to play important roles inpiloting and implementing a system such as LEAP.Moreover, the cost o unding a position such asDCTA Liaison equates only to salary and bene ts orone replacement teacher or each year the Liaisonserves on special assignment; smaller districts mightconsider making the position a part-time one.

training viDeos. While Denver initially produced LEAP training andpro essional development videos with assistancerom an external contractor, the district has sincebegun to develop videos entirely in-house using

Adobe Captivatea so tware package costing onlyseveral hundred dollars.



organizations to assume good intentions and to ocuson shared values when discussing or debating detaileddesign decisions. Sometimes you need to veryintentionally take that step and say, Here is the realvalue or the real concern behind what were saying.

As weve done this work weve o ten discovered thatwere closer together than we had originally assumed.

Roybal or one would like to see more leadershippositions that enable teachers to serve other teachersthrough such temporary assignments in the centralo ce, perhaps on uture large-scale initiatives likeLEAP. Historically, there always seems to haveexisted this disconnect leading to an us versusthem mentality between people who work in schoolbuildings and people who work downtown, Royballsays. But this job has really taught me that it doesnthave to be that way. Ive also learned a lot about how

the central o ce works that I would never have beenable to in a school building, and that knowledge givesme an even stronger voice as a teacher.

ConclusionDenvers experience o ers valuable lessons or otherschool systems interested in engaging educators moredeeply in the design and rollout o major initiativessuch as LEAP.

Involve the union from the outset and make its rolecollaborative rather than merely reactive.

DPS took deliberate steps to ensure that DCTAs rolein LEAPs development would be collaborative (wedesign it together) rather than merely reactive (yourespond to our proposals). First, the district ensuredstrong DCTA participation on the Steering Committeeestablished to provide high-level guidance on LEAPsdevelopment and to acilitate strategic alignmentwith other district policies. Second, it includedPam Shamburg, a highly respected middle schoolteacher, as a key member o its cross- unctional LEAPLeadership Team. In the ormal role o DCTA Liaison,Shamburg works closely with central o ce leadersto solve problems o LEAP design and developmenton a day-to-day basis. Finally, the district includeda second teacher on its broader LEAP sta to playthe dedicated role o DCTA Outreach Manager,establishing an important bridge between building-level

educators and central o ce leaders responsible ordesigning and re ning LEAP.

Engage educators early and often through a widevariety of avenues.

DPS ensured that practitioners had many di erentkinds o opportunities to participate in LEAPs designand ongoing development. More than 150 teachersparticipated in an early series o stakeholder ocusgroups that produced guiding principles or LEAPsdesign. Several dozen educators served on Design

Teams that feshed out key proposals or designingLEAP. Hundreds o teachers participated in aninitial, 16-school pilot during the spring o 2011.Sixty teachers rom those pilot schools conductedoutreach to colleagues in other schools to sharetheir experiences and answer questions prior to a

school-by-school vote determining participation in asecond yearlong pilot. Hundreds o teachers whohad received training through the districts TeacherLeadership Academy partnered with their principalsto conduct training on LEAP in the 128 schoolsvolunteering to pilot LEAP in 2011-12. Finally,thousands o teachers and principals provided ocusedinput and eedback on LEAP during that 2011-12 pilot.

Solicit feedback from practitioners throughmultiple channels during piloting.

During the yearlong pilot in 2011-12, DPS usedmultiple channels to ensure it captured eedback oncritical aspects o LEAP rom as many practitioners aspossible. Teachers could submit written commentsvia the LEAP website, e-mail address or phone line;respond to ormal surveys or participate in ormalocus groups; participate in webinars to hear updatesor share input; attend the school aculty meetingduring which Denvers superintendent and CAO visitedto talk about LEAP; meet with the DCTA OutreachManager when he visited the building; or reach out tothe DCTA Liaison or DCTA Outreach Manager directly.

Engagement continues this year as the LEAP Teamcollects eedback, shares new in ormation, respondsto questions, and provides clari cation as needed.

These multiple channels allowed DPS to collecteedback that was both open-ended (e.g., throughcomments submitted to the website) and eedback ocused on particular aspects o LEAPs design and



unctioning (e.g., through ormal surveys). Just asimportantly, the multiple channels gave teachers andprincipals options or submitting eedback accordingto their own pre erences and com ort levels. Denversteachers say they especially appreciated thatDPS o ered several channels to provide eedback anonymously, including online surveys, a eedback orm on the LEAP website, and speaking with ore-mailing the DCTA Liaison or Outreach Manager.

Focus on professional growth and development

from the outset.

Any school system that plans to seek educator inputshould anticipate hearing rom practitioners that theywant the system to be designed to support teachersgrowth and development over time, not merely tomeasure their current per ormance. As a result,districts should plan to ocus on that goal immediatelyand continuously throughout the design anddevelopment process, rather than simply promisingeducators that such concerns will be addressed atsome later point.

Denver ocused on teacher growth and developmentat every single stage o the process described in thiscase study, rom the initial ocus groups held in April2010 all the way through LEAP piloting and re nement.Dedicating one o the ve LEAP design teams to thetopic o pro essional development sent a strong earlysignal to educators that LEAP would take that goal

very seriously, and it provided a way to obtain criticalpractitioner advice about how to make it happen.Given time constraints, DPS leaders had to work very hard to create aligned pro essional developmentresources in time or the spring 2011 pilot, but ailing todo so was never even considered to be an option. Asa result, LEAP o ers a wide variety o resources andopportunities to help teachers analyze and improvetheir practice guided by the vision or e ective teachingat the heart o LEAPa set o supports developedwith signi cant input rom educators to ensure thatthey work or educators. (See the sidebar on page11-12 or examples.)

Communicate clearly with practitioners about whatcan and cannot be changed, and why.

DPS has taken pains to ul ll its promise to take

practitioners input seriously, even when the adviceproved to be unexpected and logistically challenging,such as the recommendation by the TeacherE ectiveness Design Team that Denver develop itsown Framework or E ective Teaching rather thanadopting or adapting a nationally available one. Alongthe way, the district has made e orts to communicatewith educators about how LEAP was being shapedand re ned in response to educator eedback.However, DPS also has ound that in many cases itcould not change LEAP in response to practitionereedback or very compelling legal, technical, or

practical reasons. The district is evolving how itcommunicates with educators about updates andeedbackwhich aspects o LEAP can and cannot bechanged and what to anticipate on the horizonandbalancing that with the in ormation practitioners needto know now to take action.

Leverage teacher leadership in design and rollout.

DPS intentionally looked or ways that teachers couldgo beyond providing advice on LEAP and take onleadership roles that allowed them to become owners

and advocates o the new system. When it cametime to roll out LEAP in 128 schools or the yearlong2011-12 pilot, DPS trained principals and teacherleaders in each school to conduct aculty trainingrather than relying on central o ce sta to provideit. As a result, building-level educators are now seenas the experts on LEAP and the vision or e ectiveinstruction embodied by the new Framework. In

Any school system that plansto seek educator input should anticipate hearing rom practitioners that they want thesystem to be designed to support teachers growth and development over time, not merely to measuretheir current per ormance.



addition, DPS worked with the teachers union to createspecialized new leadership roles such as DCTA Liaisonand DCTA Outreach Manager that have proven to becritical lynchpins or ensuring strong teacher voice inLEAPs ongoing development.

Above all, Denvers experience with LEAP showsthat it is possible or school systems to design andimplement major initiatives with signi cant involvementrom teachers and school leaders. I Denver is anyindication, educators are hungry or opportunities to bemeaning ully involved in large-scale re orm initiatives, andmany will gladly spend signi cant amounts o time andenergy on such e orts i they believe their contributionswill be taken seriously.


31/32

www. i i . / i


32/32