23
Student Evaluations
Student Evaluations
What We’re Doing
• Joint Committee on Student Evaluation of Courses and Teaching was formed and had its first meeting in March of 2011 Per the Bylaws of the General Faculty
• Committee responsible for developing, reviewing and assisting in the conduct of the College program of systematic student evaluation of courses and teaching
• Committee evaluates the validity and reliability of the student evaluation document and its uses
• “It shall report its findings in detail to the General Faculty at least once every five years.” - Bylaws
2
Committee Composition
• Three faculty members selected by the Faculty Senate– Dahlia Remler, Larry Tatum, Stan Wine
• Four undergraduate students elected by the student government– Farhana Hassan, Mathew Melore, Prerak Paul, Luis Salguero
• Two graduate students elected by the Graduate Student Assembly– Chyrstle Greene, Patricia Vendikos
• VP for Academic Affairs or designee– Dennis Slavin
• The Deans of the several Schools or designees– Gari Hentzi, Jerry Mitchell, Don Schepers
3
Committee Composition (cont’d)
• VP for Student Development/Dean of Students or designee– Leslie Sutton-Smith
• Non-voting participants– Edward Adams (Registrar)
– John Choonoo (Institutional Research & Program Assessment)
– Denyse Ramkaran-Testing Office
– Ashok Vora (ZSB)
• Subject matter experts as needed
• Chair– Dennis Slavin
– Provost’s staff
4
Work Effort
• 5 full committee meetings
• 3 subcommittees each met one or more times
• Substantial work carried out via email
• Presentation by Brooklyn College Associate
Provost
• Surveyed practices of other CUNY colleges
5
Focus - uses of student evaluations
• Feedback to instructors about their teaching
• Information to colleagues and administrators to
be used in personnel decisions
• Information to students about the instructors
they might like to take courses with
• Additional goals
– provide a more informative alternative to sources
such as RateMyProfessor
– move from paper-based system to an electronic one
6
Survey• Created and reviewed a survey
• Conducted: Survey Fall 2011
• Sample Size: 642
• FT Tenured: 158, FT Untenured: 59
• Adjunct: 190
• Students: 218
• Second survey: Fall 2012
• Students: 1,470
• Open-ended and closed-ended questions
– Obtained significant feedback
7Following slides courtesy of Dennis Slavin and Prerak Paul
Findings:
Students:– Most unaware of the available results
– Find the results hard to understand
– Feel written comments more useful than numerical data
– Question reliability
• Most instructors receive favorable ratings
• Most students do not take the form seriously because they believe that the college does not take it seriously
– The information provided is not what the students are looking for
• Revisions to the questions are suggested8
Findings:
Members of the teaching faculty believe:
– They do not gain valuable feedback from the
numerical data
• Scores are usually 4-5
• Written comments are more useful
– The results should be available sooner
– Student interest is low
– Current evaluations are inadequate for admin
purposes
• Results are generally in the same range
• Difficult to compare scores 9
Findings:
Dep’t. Chairs and Admins believe:
– The evaluation is a useful instrument, but inadequate
by itself (Supplemented with peer evaluations etc.)
– Current scale is less reliable
• Scores are usually 4-5
– Dep’t. Chairs as well as instructors do not gain
necessary feedback to improve pedagogy
– Written comments are more useful than numerical
data
– Questions do not provide feedback about the
curriculum
– Inclusion of a single overall rating/summary helpful 10
RecommendationsHow can we make the evaluations more useful to students?• Revise the scale, 4 point instead of 5
• Faculty and students both suggested that we revise the current questions
– Include straightforward questions about:
• Fairness, learning
– Need to address areas such as:
• Instructor’s ability to communicate
• Appropriate level of difficulty
• Nature of the assignments, examinations, papers
• Instructor’s helpfulness
– Evaluation results should be available at registration
• Link provided along with the appointment time
– Results need to better publicized & easy to find 11
Recommendations (Cont’d)
• Faculty and students both
– indicated that written comments are a lot more informative than numerical data
• Students:
– should be allowed to view written comments
– want to expressing their opinions to other students
• why ratemyprofessors.com is popular
• Faculty:
– Senior FT opposed to making written comments available to students
– Untenured and Adjuncts less opposed
12
Recommendations (Cont’d)
How to make the evaluations more useful to
instructors?
• Faculty and students both suggested that the
current questions be revised
– Provide better feedback about the coursework
– More specific about instructor’s quality of teaching
• Suggestion:
• Include questions that will guide students towards
writing more useful comments
13
Prompts for Written Comments
• Baruch’s written comments section is vague
• Recommendations to make questions more
specific
– To prompt the students to write more feedback
– Possibly give instructors the option to customize
some questions
• A separate question which is shared with the
entire school community
14
Prompts for Written Comments
Sample Questions:
• Apart from the instructor, what are the
strengths of the course?
• How can the course be improved?
• Apart from the course, what are the strengths
of the instructor?
• How can the instructor’s teaching be
improved?
15
Source: Brooklyn College
Prompts for Written Comments
An additional question:
• What would you like to tell others about the
course?
To be shared with everyone in the Baruch
Community
16
Recommendations (Cont’d)
• Administer Mid-Term Evaluations
–optional and available only to the
instructor
• Timely processing of results
– enough time to make changes for the
following term
17
Recommendations
How to make the evaluations more useful to
Department Chairs and Administrators?
• Faculty criticisms:
– Questions do not effectively measure instructional quality
– Don’t provide dep’t. chairs and instructors opportunity to
learn about specific shortcomings
• The full-time teaching faculty suggested:
– The current rating format/scale needs to be revised, data
seems statistically unreliable
– Form does not provide sufficient detail about instructor
18
Recommendations (Cont’d)
Part-time faculty members:
– Expressed concerns about instructors diluting
grades and lowering class difficulty levels in order
to gain a favorable evaluation
– Suggested that the college should examine the
relationship between grading practices and
evaluation scores
19
Presentation of Data
Baruch:
20
Presentation of Data
• There are no departmental comparisons
• Difficult to read
• Difficult to navigate
• Improve the overall user interface
21
Presentation of Data
Individual vs. Department
22
Source: Brooklyn College
Presentation of Data
23
Source: Brooklyn College
