Educator Effectiveness:State Frameworks and

Local Practice(??)

CCSSO Annual Conference, June 2012

Juan M. D’BrotExecutive Director of Assessment and Accountability

West Virginia Department of Education

Assessment Continuum in WV

• Promoting a balanced assessment system against WV State Standards

Formative DiagnosticsFormative Diagnostics

Interim DiagnosticsInterim Diagnostics

Interim BenchmarksInterim Benchmarks

Summative AssessmentsSummative Assessments

Formative, Interim, and Summative Assessments

• Summative – WESTEST 2– Online Writing Assessment (PT)

• Interim– Benchmarks in Math, ELA, Science, Social

Studies• Formative– Acuity Item Banks in Math, ELA, Science, Social

Studies– West Virginia Writes: Practice Writing Program– Other District Purchased Supports– Teacher Developed Assessments

Major Question: Purpose? • Purpose of Summative

– Evaluative against a set of standards– Least flexible: long-cycle– Broad sense of the educational state of the state or

district• Purpose of Interim

– Instructional,– Evaluative, and/or– Predictive– Medium-cycle

• Purpose of Formative – Process to adjust teaching and learning – Instructional diagnostics– Typically classroom specific– Very short-cycle

Examining the Use of Formative and Interim

• High Use vs. Effective Use– Emerging case study of a district and school

• Context– Targeted professional development– Dedicated school leadership and staff– State-level support around the use of

formative and interim assessments

• Early Results:

Grade 9 School A and B

Grade 10 School A and B

Grade 11 School A and B

WESTEST 2 Writing: At or Above Proficiency

Examining the Use of Formative and Interim

• Strong support for use of writing short-cycle assessments

• Currently examining use of other short-cycle (formative) and medium-cycle (interim) assessments on summative assessment results

• Implications on teacher evaluation systems looking for triangulated evidence of student learning

Multiple Use Concerns

• Tension between Compliance (accountability) and Continuous Improvement (short-cycle assessments)

Adequate Yearly Progress Improvement and Growth

Teacher Evaluation Systems Student Learning

Conflicting?

Multiple Use Concerns• Tension between Compliance (accountability)

and Continuous Improvement (short-cycle assessments)– Summative Assessment

• Gaming• Cheating• “Teaching to the Test”

– Interim/Formative Assessment• Fabricated student learning gains? • Student sorting to highlight teacher “gains”? • What does the research say? Not impact/effect, but

valid link to teacher input/output?• Cognitive vs. Non-Cognitive measures? Scratching

the surface

The West Virginia Case

• A transition from nothing to something– Old: 5 years experience evaluation “only

when the principal determines it to be necessary…”

– New: Yearly evaluation for all educators, principals, and evaluators using research-based professional standards and evidence of student learning

Developing the System• Goal: To develop a supportive system

around valid data to drive improvement

• Access: A variety of data to support self-reflective practice:– Summative Growth Data– Interim Benchmarks– Formative Diagnostic

Developing the System• Context: – Politics: National vs. State Pressure– Policy: Accountability– Pragmatics: Using actionable and valid data

• Development: Stakeholder driven– Board of Education – SEA Staff– LEA staff– Teacher Unions– Legislators– Community Members

Category Input Output

5 Professional Teaching Standards 80%

14 Critical Standard Elements – 4 pt. rubric

5.7%/CSE

Student Learning Outcomes 20%

Summative Growth Data•Math & ELA Only

5%

Other Student Learning Evidence•Any Content 15%

Professionalism (Mandatory) N/A N/A

The System• System Components

The System• 20% Student Learning Outcomes– Minimum 5% Student Assessment Data• School-wide Growth – Student Growth

Percentiles

– A host of evidence– Agreed upon with principal input– Multiple measures• Acuity (Math, ELA, Science, Social Studies)• WV Writes (Writing)• Other evidence of student learning• Teacher Developed Assessments

The System• Overall Ratings

Category 100 Point Scale

Distinguished 88-100

Accomplished 54-88

Emerging 33-54

Unsatisfactory <33

Implementation

• 2011-2012 – Concluded 1st pilot year in a small number of schools – Self-ratings– Accomplished across the board…

• 2012-2013 – 2nd pilot year with schools in every district represented

• 2013-2014 – Full statewide implementation for all schools in the state

Implications for a “Young System”

• Embedded Program Evaluation – WVDE Office of Research small scale

implementation study and program evaluation

– Formative data has resulted in changes to the system and program

– Exploring third-party consulting for enhancements to guidance, system, and process (post-revision)

Lessons Learned• Politics vs. Pragmatism– Representative inclusion vs. Over/under-

representation of certain views– Political support to push a bill to law

• Capacity and Foresight– In-house development and reinventing the

wheel? (…more politics)– Building a system for the present vs.

building a system for the future– Revising the re-invention

Thank You

Juan M. D’Brot(jdbrot@access.k12.wv.us)

Executive Director

Office of Assessment and Accountability

West Virginia Department of Education