How I Reported “The Test Generation”…

and What I’ve Learned Since

By Dana GoldsteinSchwartz Fellow, New America Foundation

Puffin Fellow, The Nationwww.danagoldstein.net

dana@danagoldstein.net

Premises of the Piece

Race to the Top incentivizes evaluating teachers according to how much they grow student achievement; half of all states responded by changing their laws

Colorado’s SB191, “The Great Teachers and Leaders Bill:” 51 percent of every teacher’s evaluation linked to student “growth”; end of tenure as we know it

Reformers like Sen. Mike Johnston acknowledge Harrison District 2 as one model for potential statewide SB191 implementation

Union folks pointed to a different school as a model: The Math and Sciences Leadership Academy in Denver

What Does SB191 Require?

“Student data that is monitored at least annually to ensure the correlation between ‘student academic growth’ and outcomes with educator effectiveness ratings.”

The “Colorado Growth Model” does NOT factor in student SES; it is based on CSAP score projections

CSAP affects only some teachers (reading, writing, math, science in grades 3-10)

Therefore, state must devise new tools to measure growth in non-tested subjects/grades, since SB191 applies to ALL teachers

Before I Visited Colorado, I Asked

What IS student growth-based and value-added-based teacher assessment, and what does the research say about its effectiveness?

The Best Papers I Read

Douglas N. Harris, “Clear Away the Smoke and Mirrors of Value-Added.” Phi Delta Kappan

Thomas Kane and Douglas Staiger, “Volatility in School Test Scores: Implications for Test-Based Accountability Systems.” Brookings Papers on Ed Policy, No. 5, 2002

Jonah Rockoff, “Information and Employee Evaluation: Evidence from a Randomized Intervention in Public Schools.”

Rockoff, “Subjective and Objective Evaluations of Teacher Effectiveness: Evidence from New York City.”

Institute for Education Sciences, DOE, “Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains.”

Economic Policy Institute, “Problems with the Use of Student Test Scores to Evaluate Teachers.”

But I Also Heard from Sources…

“Economists are too dominant in the teacher-quality debate”

Other disciplines, like sociology, psychometrics, and social psychology have contributed valuable research on teaching and learning

And So I Spoke To and Read

Edward Deci, social psychologist and expert on motivation: paper on “The Paradox of Achievement: The Harder You Push, the Worse it Gets”

Daniel Koretz, Harvard psychometrician, author of Measuring Up

Linda Darling-Hammond, “Evaluating Teacher Effectiveness,” Center for American Progress paper, Oct. 2010

I Called Colorado Educators

Michelle Rappoport, former TFA corps member in Denver, strong supporter of SB191

Christina Jean, Denver teacher-recruiter, cautiously supportive of SB191 and active with Center for Teaching Quality

Zachary Rupp, music teacher at the CTU’s favorite “model” school, the Math and Sciences Leadership Academy in Denver, which uses peer evaluation. Skeptical of 191.

Events on My 6 Day Colorado Schedule

Two days in Denver, visit to MSLA

Three days in Harrison District 2, Colorado Springs

Denver panel on “Waiting for Superman”

Pikes Peak Education Association monthly book club meeting – off the record

Sit-downs with Sen. Mike Johnston, Brenda Smith (DCTF), Stand for Children

Happy Accidents in Colorado Springs

“We rolled out a whole new set of tests last week in art, gym, and music! Everyone is buzzing about them!”

“We’re having a teacher committee meeting after school!”

“We’re also doing middle school classroom observations today!”

Incredibly open administration introduced me to many teachers and allowed me to speak to them privately

◦ They selected teachers who had performed well under the system—all but one openly skeptical of E&R

◦ There was a tone of resigned stress at the teacher committee meeting; I was able to speak privately to several more teachers there

◦ This all provided an important counterpoint to the off-the-record interviews, which had been set up for me by the union

Weeping Woman, Picasso

Grade 1 Art Assessment

“How do you feel when you look at this painting?”

“What is the woman in the painting doing that makes you feel that way?”

“Draw three different shapes Picasso used to show feeling or emotion.”

“Write three colors Picasso used to show feeling or emotion.”

“Draw three different kinds of lines Picasso used to show feeling or emotion.”

Grade 1 Art Assessment

“In the space below, sketch a happy or sad picture of yourself. Your sketch must include different lines, different shapes, and different patterns.”

“How would someone who looks at your picture feel? Why?”

Portrait of Woman in Purple, Matisse

Grade 1 Art Assessment

“The artist used many Elements of Art in this painting. Write a paragraph about how the artist used one Element of Art to show feeling or emotion.”

State Standards, Grade 1 Art

“Art is an emotional experience that is mindful of our personal perspectives and feelings. … Locate and discuss the characteristics and expressive features of art and design in a work of art that initiate specific feelings and/or emotions.”

“Art connects visual stories with literary stories.”

“Imagination can be used to see reality.”

Super’s Comments 1 Week Later

Superintendent Mike Miles:

“The prompt was probably too hard for first-graders. Next year it will be easier. … But for the first time, you have art teachers saying, ‘I’m going to have to teach to the standards, not just do coloring.’”

Grade 2 Phys Ed Assessment

“Draw a picture of how your hands look while they are catching a ball that is thrown above your head.”

“What are two rules students can follow so they do not run into others when running around in physical education class?”

“A friend of yours has asked you to help him or her become a better runner. In a well-developed paragraph, explain two ways your friend can improve his or her ability to run.”

State Standards, Grade 2 Phys Ed

“Demonstrate skipping, hopping, galloping, and sliding while transitioning on command.”

“Throw, catch, strike, and trap objects.”

“Move to even and uneven beats.”

“The more one performs physical activities, the more success and fun he or she will have doing them.”

How are these scores weighted?

8 student performance data points for each Harrison teacher

In art, 7 of the 8 are tests like these; 1 is a creative work of art

All 8 together count for 50 percent of evaluation score

8-16 “spot” observations – surprises, 10-15 minutes each

Several period-long, pre-scheduled observations

Teacher’s individual “Student Achievement Goal”

“Leadership” qualities

Evaluation at MSLA

40 percent of score: Teams of peer teachers observe each colleague for 5 instructional hours per semester

60 percent of score: Student achievement as measured by school-level assessments, portfolios of student work, and standardized test scores

Key Questions

If 51 percent of a teacher’s evaluation comes from evidence of student academic growth, how do we measure that growth?

Will the measurement tools be holistic (portfolios) or more quantitative (tests with wrong/right answers), or some combination of the two? How will the combo be weighted?

Are the tools developmentally appropriate?

And even if student assessment tools are high-quality, are they improving instruction, teacher quality, and teacher retention, or just leading to more test-taking time?

Since I Wrote the Piece…

Harrison Superintendent Mike Miles named a Broad fellow

Miles wins “Communicator of the Year” award from Colorado School Public Relations Association

But More Importantly…

We still don’t know exactly what role testing will play in evaluating teachers under SB191

EdNews Colorado: “The council is recommending that CDE develop state assessments for science and social studies, work with school districts to develop measurement and assessment methods for non-CSAP subjects, and to organize a state study of how to best measure academic outcomes for students in preschool through grade 2.”

The End!

Be in touch with me! dana@danagoldstein.net Twitter: @danagoldstein www.danagoldstein.net