Preparation vs. Production Program personnel initiate the evaluation Central admin or funding require the evaluation Internal team suggest an evaluation

Be familiar with the nature of the program Be familiar with the people served Be familiar with the goals and structure of the

program Why an evaluation is being considered

Meet with stakeholders Decide if useful Consider the timing, the manner, and the cost

Overview of Evaluation Models Rule: First ask the questions, then choose

the appropriate model The range of questions being addressed by

the evaluation or program setting often indicate the best model

Use approaches from several models to best meet the goals of the evaluation (back to the questions)

The Traditional ModelOverview of Evaluation Models

The Social Science Research Model

Overview of Evaluation Models

The Industrial Inspection Model

Overview of Evaluation Models

The Black Box Evaluation

Overview of Evaluation Models

The Objectives-Based Evaluation

Overview of Evaluation Models

The Goal-Free EvaluationOverview of Evaluation Models

The Fiscal EvaluationOverview of Evaluation Models

The Accountability ModelOverview of Evaluation Models

The Expert Opinion ModelOverview of Evaluation Models

The Naturalistic ModelOverview of Evaluation Models

The Empowerment EvaluationOverview of Evaluation Models

The Theory-Driven EvaluationOverview of Evaluation Models

An Improvement-Focused Model

Overview of Evaluation Models

Steps in Preparing to Conduct an Evaluation

If you were to conduct an evaluation of the Teacher Performance Appraisal….

Figure 6. The axial coding model of the use of the Ontario teacher performance appraisal and its contribution towards reflection and refinement of professional practice. (Barnett, 2006, p.181)

Barnett, J. (2006, November). Considering Growth in Teaching: Teacher Perspectives on the Ontario Teacher Performance Appraisal, Ontario Institute for Studies in Education of the University of Toronto, 2006.

Causal Conditions (Authentic)“the schema they come in with, the background they come in with is going to dictate their interpretation.” Section 20 teacher (Barnett, 2006)

“is very much like going to high school or even university where you have your profs. You write an essay for one they give you an A. If you wrote the same type of essay for another they give you a C, because it is not what they expect.”

Second Year teacher (Barnett, 2006)

“Do you judge them by what you see going on in the classroom in 20 minutes or do you judge him because he’s gone to the Sears’ Festival four years in a row and won?”Experience Secondary School Teacher (Barnett, 2006)

Central Phenomenon

Barnett, J. (2006, November). Considering Growth in Teaching: Teacher Perspectives on the Ontario Teacher Performance Appraisal, Ontario Institute for Studies in Education of the University of Toronto, 2006.

Content and Intervening Conditions Past Experience

Barnett, J. (2006, November). Considering Growth in Teaching: Teacher Perspectives on the Ontario Teacher Performance Appraisal, Ontario Institute for Studies in Education of the University of Toronto, 2006.

Strategies – Perceived Value

Barnett, J. (2006, November). Considering Growth in Teaching: Teacher Perspectives on the Ontario Teacher Performance Appraisal, Ontario Institute for Studies in Education of the University of Toronto, 2006.

Resultjust jumping through the hoop. A big hoop. A big hoop. It

was a lot of work. I don’t think I grew from it because I don’t think I had any sort of constructive feedback….I feel like it was sort of like a Seinfeld episode – a show about nothing. 6th year junior teacher

I really do think it has the potential to be a helpful tool….we could improve by trying some of their ideas or something like that. I just think maybe there are better ways of going about it and I don’t think this is necessarily, this government imposed option is perhaps not the best choice. And end of story right there. 4th year intermediate teacher

it wasn’t a judgemental process. It was more like a discussion. What are you interested in doing?...Are you happy with what you’re teaching? Is there anything we can do to help you? Do you have any concerns? 35th year French teacher

Barnett, J. (2006, November). Considering Growth in Teaching: Teacher Perspectives on the Ontario Teacher Performance Appraisal, Ontario Institute for Studies in Education of the University of Toronto, 2006.

Criteria and Standards Criteria

Reflect program intent Staff can influence Can be measured reliably Stakeholders participate in selecting

Goals How much agreement on goals is needed Different types of goals - implementation - intermediate - outcome * people are treated with respect, no discrimination, no negative side effects

Program Theory Is the program accepted?

The way we think about the organization in which we work influences what we consider to be possible within it and the actions we take…. In schools, novel approaches to leadership in general, and supervision in particular, can fly in the face of tradition and existing structure, and beget substantial resistance: “that is not the way we do things here.” However, such novel approaches can also lead to new ways of thinking about schooling and the professional growth of teachers.

(Mackinnon and Pynch-Worthylake, 2001, p. 6)

Is the program accepted?

Risk-taking is fundamental to teacher learning (Beerens, 2000).

Teachers need to feel there is a freedom in order to move into action (Zepeda & Ponticell, 1998).

The value of resisters, on the other hand, has been unappreciated. Resistant acts “often embody a form of good sense” (Gitlin and Margonis 1995).

Finding a way to reconcile positive and negative emotion is the key to releasing energy for change. (Fullan, 1997, p. 223)

Cunningham and Gresso (1993) contend: There are a number of personal and

organizational philosophies that work against employee innovation. One is the natural fear of newness…. There is a natural fear of trying something new even when there is ample evidence it works. This is part of the fear of making changes or even taking small risks. One becomes enamoured with security without realizing how insecure he or she is. (p. 151)

Schmoker (1999) warns “goal-setting that uses data to monitor progress can be a threatening endeavour” (p. 40).

Glickman (2003):

If I push too hard, alienate, and harden other people’s position, am I then part of the solution? If I clarify, help others understand my feelings – but then back off – am I then still part of the solution? If I take a walk because I’m opposed to the decision being made, am I now part of the problem? These issues of value stay with me long after meetings and workdays are over. (p. 339)