Alternative Strategies for Evaluating Teaching
How many have used end-of-semester student evaluations?
How many have used an alternative approach?
My comments are based in part on “Evaluating Your Own Teaching” by Dee Fink (published in Improving College Teaching by Peter Seldin (Ed.)
Bill BurkeProgram Manager for Educational DevelopmentTeaching and Academic Support Center
Alternative Strategies for Evaluating Teaching
Teaching evaluations can serve two purposes –
Formative feedback
Summative assessment
Doing good teaching evaluations is like doing good research
Need to identify the right questions to ask
Need to figure out how to get the data to answer them
Alternative Strategies for Evaluating Teaching
Student feedback Self-evaluation Recordings Student performance Outside evaluators
All sources have unique values and inherent limitations
Five sources of evaluation feedback
Use a multidimensional approach to evaluating what is a multivariable activity
A good evaluation program should be ongoing, strategic, comprehensive, multidimensional, integrated, and evolving
Alternative Strategies for Evaluating Teaching
Student Feedback
Can be done during the semester and/or end-of-semester
1) Questionnaires
+ Obtain feedback from whole class+ Anonymous+ Can provide quantitative and qualitative data
- Questions may not be appropriate or relevant or most informative- No chance to probe for further clarification
Alternative Strategies for Evaluating Teaching
Students don’t know enough to evaluate faculty
Those multiple-choice forms can’t be as meaningful as open-ended questions and interviews
Easy graders and entertainers get the high evaluations
Rigorous instructors will get low evaluations
Students may rate me low now, but they’ll appreciate me later
Misconceptions about student evaluations
Alternative Strategies for Evaluating Teaching
Fact: Student input on some items (holding office hours, returning papers in a timely manner) is reliable and the best source
Fact: High correlation between student ratings and other sources such as peers and administrators
Fact: Comparison of objective questions, written responses to open-ended questions, and group interviews yielded a correlation of 0.82
Fact: Conflicting findings exist on grading and evaluations (But can’t assume high grades & high ratings means no rigor)
Alternative Strategies for Evaluating Teaching
Fact: Teachers assigning more work and more difficult work tend to get higher evaluations
Fact: Students don’t change their opinions later
Alternative Strategies for Evaluating Teaching
Making student evaluations more meaningful
Have a clear idea about your teaching goals and learning outcomes
Do evaluations provide feedback on these?
If not, ask the students to comment on specific items (write them on the board)
Alternative Strategies for Evaluating Teaching
Making student evaluations more meaningful
Focus on specifics versus global items
Focus on teaching characteristics versus personal characteristics
Target one or two items to work on
Use more than one course (a variety of courses over several semesters)
Alternative Strategies for Evaluating Teaching
Making Sense of Written Comments
Harder to make sense of written comments
Some are contradictory
Need to impose some structure through systematic analysis
Alternative Strategies for Evaluating Teaching
Making Sense of Written Comments
Cluster evaluations according to overall course ratings
Similar comments between high and low raters?
Cluster according to comments
Do patterns emerge?
Use student demographic data when possible and appropriate
Alternative Strategies for Evaluating Teaching
Making Sense of Written Comments
Make a list of common positive and negative characteristics of teachers
Put checkmarks next to them based on student comments
Helps quantify the diversity of comments
Produces a visual as well as quantitative display
Alternative Strategies for Evaluating Teaching
Making Sense of Written Comments
Ask students to comment on –
What made you rate the course as you did?
What kept you from rating the course higher?
Alternative Strategies for Evaluating Teaching
Student Feedback
2) Interviews (by you or a third party)
Focus group
Whole class (e.g., Group Instructional Feedback Technique [GIFT])
+ Students identify unanticipated strengths and weaknesses+ Interviewer can probe
- Whole class may not be represented
Alternative Strategies for Evaluating Teaching
Self-Evaluation
Personal observations during class
Reflections / journal
Teaching portfolio
Teaching philosophy statement - personal and descriptive Evidence of putting philosophy into action Evidence of growth as a teacher You have a voice here Evaluators need a rubric
Evaluators need a rubric examples
Alternative Strategies for Evaluating Teaching
Self-Evaluation
+ Immediate and constant feedback+ Meaningful to you
- Subject to your biases, misconceptions, and delusions
Self-assess your perceived teaching strengths and weaknessesGet feedback to confirm or refute
Alternative Strategies for Evaluating Teaching
Recordings
Audio and/or Video
+ Provides objective information
- Information is true but may be meaningless by itself in determining impact on student learning
Alternative Strategies for Evaluating Teaching
Student Performance
Course assessments (exams, papers, etc.)
Pre / Post tests
Classroom research projects
+ Provides evidence of student learning
- Lack of an unequivocal causal relationship
Alternative Strategies for Evaluating Teaching
Outside Evaluators
+ Can offer positive and negative observations without a personal cost
+ Bring professional expertise in content and/or pedagogy
- Limited number of class visits (snap shot perspective)
Need an agreement on what is to be observed or reviewed and criteria for judgment
Observations of teaching
Review of materials
Alternative Strategies for Evaluating Teaching
Outside Evaluators
Conducted by -
Peers + Low political risk + Empathy - Limited experience and perspective
Senior faculty / Administrators + Experience - Political risk
Instructional specialists from a center + Objective with no political risk + Expertise in instructional strategies - Limited knowledge of subject matter
Alternative Strategies for Evaluating Teaching
Questions?
Comments?
Use a multidimensional approach to evaluating what is a multivariable activity
A good evaluation program should be ongoing, strategic, comprehensive,
multidimensional, integrated, and evolving
Alternative Strategies for Evaluating Teaching