of 31
8/9/2019 Group One Writeup
1/31
Designing a New Course Evaluation System
For the Students, by the Students
Spring Term 2010
Group One
Michael Kiso, Aaron Bicknell, Alexsa Tucker,
Richard Bradley, Josh Counsil, Caroline Hilburn,
Matthew Nollar
Table of Contents
8/9/2019 Group One Writeup
2/31
The Current System byJosh
Counsil...Page 3
Other University Course Evaluation Systems by Matthew
Nollar....Page 6
Instructor Interviews by Michael
Kiso.. Page 9
Methods by Caroline Hilburn
.....Page 12
The Logic behind the Madness: explaining why we made the
new system the way we did by Richard Bradley
...............................................................................Page 14
A Proposed New System byAaron
Bicknell....Page 18
Student Views of the Course Evaluation Prototype byAlexa
Tucker....Page 21
Next Steps by Caroline Hilburn.
....Page 25
hem out as a student, as well as receiving them from students of my own. The one thing th
8/9/2019 Group One Writeup
3/31
The Current System Josh Counsil
How the System Works:
Currently, Oregon State University and many other schools around the
United States are using a paper Scantron system of teaching evaulations. At
the end of the term, teachers hand out evaluation sheets which they expect
students to fill out honestly, carefully and in full. They ask you questions
such as if you felt the class was good, and if the teacher was interested in
the students learning. It provides a very poor through exceptionalchoice system (basically a 1-5 system). The students then fill out these
sheets and individually place them in a folder while the teacher is not
present (so no tension is created between the teachers and the students).
The folder is then taken to its final destination by a student so the
information cannot be tampered with. These Scantron sheets are very
informative for teachers and administration. However, the students do not
feel that the system is effective for them. The current system could easily be
made much more cost efficient by providing it online. Also, it would be
beneficial to create a system that is accessible at the students digression
instead of wasting class time that is very valuable to a students learning.
8/9/2019 Group One Writeup
4/31
Fig. 1. Data collected by Group One. This chart reflects the opinions of OSU students
surveyed spring term, 2010
On our random surveys we asked the question do you feel like your
feedback is being heard? Over 50% of the students reply was Sometimes
(Figure 1). In my eyes, this means that the students do not fully understand
how the system is operating and what its benefits and downfalls are. When
interviewing staff at Oregon State, we found that most of them agree that
the course evaluation system is very teacher orientated. This means they
agree that the Scantron sheets system is not providing adequate feedback
for students.
Fig. 2. Data collected by Group One. This chart reflects the opinions of OSU students
surveyed spring term, 2010.
Why the Students Feel the System is flawed
As you can see, students at OSU feel that the current evaluation
system is obviously flawed. Over 98% of the student population that were
randomly surveyed felt the system at Oregon State is Good or less (Figure
2). That leaves only 2% of the student population that felt the system was
above average. Obviously there are issues at hand. Many may argue that
this teacher evaluation system is for the teachers and administration, but
why can it not benefit the students as well? We as students are not trying to
build a hostile environment by exposing the results of the teacher evaluation
system to the student body, but exposing the positives and negatives of
each teacher and the classes they teach will help students plan their
courses. We have the right to see how other students felt about the teachers
and their classes considering we are the ones paying their salaries. We as
students have the right to see the class difficulty, teacher evaluation, last
8/9/2019 Group One Writeup
5/31
terms class GPA, last terms syllabus, and hours of homework per week. This
is not because we are trying to expose bad teachers (even though I dont see
that as a bad thing) or pick that classes that are least difficult, but better yet
try to schedule classes accordingly. Personally, I will pick the classes with the
best teacher effectiveness because I am not the quickest learner. This could
greatly benefit many students at Oregon State and also greatly increase the
quality of their stay here by having understanding teachers and ones they
can relate to. If a teacher is getting terrible feedback upon the class then
obviously something is going wrong and we have every right to know about
it before we go setting ourselves up for failure because of a teacher who is
on a power trip or does not know how to teach. Just because a teacher has12 years of college under their belt does NOT mean they understand nor
know how to teach. Understanding the concepts is one thing, but explaining
them into words that other people can understand and comprehend is
something completely different. It is very obvious that our student body
cares about providing feedback for staff, faculty and administration, so why
are our voices of change not being heard? Many would argue that the
current system is already in place and is used almost universally by all 4-
year universities. This is true yes, but we could lead the way to bring course
evaluation online and help go green and protect the environment. We can
satisfy the needs and wants of staff and students, and create a system that
is more efficient, better quality, and easily accessible to anyone and
everyone who is paying the amounts of money that they are paying to try
and obtain a teacher and class of the highest quality possible.
We Want Change
From the random surveys we obtained from students around campus it
couldnt be more obvious that we as students are in desperate need of
change.
8/9/2019 Group One Writeup
6/31
Fig. 3. Data collected by Group One. This chart reflects the opinions of OSU students
surveyed spring term, 2010.
From the two charts above (Figures 2 and 3), we can see that 60% of
OSU students ALWAYS fill out the Scantron sheet while 30% sometimes do.
This means only 1 out of 10 students do NOT fill out the Scantron course
evaluation sheets. This means that 90% of the students are regularly filling
out the Scantrons for class and 75% of the students feel that the current
evaluation system is flawed and needs changed. This means we as students
are still filling out the evaluation sheets even though we dont agree with the
material in it, or just do not like it at all. We are the ones that pay the money
to make this school run. We are the ones that are taking the classes. We are
the ones that are here to decide our future. I figure since we are the ones
doing all these things, then we are the ones who get to decide what kind of
feedback on teachers and classes we require. We should be able to obtain
any statistics of any class at anytime.
Other University Course Evaluation Systems Matthew
Nollar
Taking a look at our system of teacher evaluations, we can come to
only one conclusion. Our current system serves the teachers and
departments that they are affiliated with. Currently our system is the
standard Scantron form with a questionnaire form that the students fill out.
In the near future, Oregon State University's system of teacher evaluations
will move to an online base. We as a group have decided to take a look at
8/9/2019 Group One Writeup
7/31
other universities' system of teacher evaluations to see how their system
differed and find any similarities, if any.
We took a look at a large handful of universities. Our scope was
focused on universities located in the Northwest but not limited to it. The
reason behind taking a look at other universities in the surrounding area was
to see what system they were currently implementing for teacher
evaluations. If they had a different system then ours then we could pull ideas
from that system. The universities we looked at concerning the teacher
evaluations were the University of Oregon, University of Washington,
Western Oregon University, Portland State University, Arizona State
University, Boise State University, Linn-Benton Community College, ClarkCounty Community College, Portland Community College, Marylhurst
University, and Stanford University. These universities are highlighted in the
map below (Figure 4).
Figure 4. A map showing the location of Universities thatwe looked at. The map was constructed by Google Maps.
8/9/2019 Group One Writeup
8/31
To our surprise, we found that the vast majority of the other
universities were using the same Scantron teacher evaluation that Oregon
State University was. The universities that were implementing the Scantron
system of teacher evaluations include University of Oregon, University of
Washington, Western Oregon University, Portland State University, Arizona
State University, Boise State University, Linn-Benton Community College,
Clark County Community College, Portland Community College, Marylhurst
University, and Stanford University. Only Western Oregon University uses a
different system. Their system of teacher evaluations was the same as the
Scantron system. However, the school had the students fill it out online. They
claimed that they were trying to move to a paperless system, much like whatOregon State University is going to be doing in the near future. Looking at
these systems, we found that the Scantron system was prevalent amongst
these universities. This is due to two reasons: one being that this is an
effective way for universities to conduct teaching evaluations, the second is
that it is a standardized system.
Seeing how all of these systems are very similar to Oregon States
University's system, we can safely say that their system serves the teachers.
All of the systems require that students fill out the evaluations. All of the
systems also disclose all of the information that is gathered by the student
filled out evaluations. To our groups surprise, we did not expect that all of
the universities that we looked at in the surrounding area had the same
system as Oregon State University. This led us to conclude that either the
current system is the best that it can be or the current best system that is in
existence out there is what all of the universities have implemented into
their teacher evaluations. On the other hand, having a standard
questionnaire and a comment form benefits the teachers because they can
adjust their teaching according to the results. This is why this Scantron
system is so prevalent.
8/9/2019 Group One Writeup
9/31
Another reason why the system is the way it is in the area is due to the
fact that the staff of the departments organize it. The students take no part
in designing the survey. It is not even required of them to fill it out. It's all
voluntary. With that said, one can see why the staff of the departments
choose not to release the information. They are doing all of the work
organizing the teacher evaluations. The other reason why the universities
that we looked into choose not to release the information that is collected
from the evaluations is that the departments want to protect the teachers
reputation and/or pride.
After talking with some of the students who worked in the offices of the
other universities, some of the students showed interest in a teacherevaluation form where students had access to the results. They thought that
it would be a better system that served the students. Some also said that
they would prefer a system where the results are posted. They continued to
tell me that they would also register for their classes based on those results.
Instructor Interviews Michael Kiso
We interviewed a large number of people at the Oregon State Campus.
We tried to incorporate all the different groups in order to collect a
representative sample of our student body. Along with interviews with
students, interviews with instructors were also conducted. In total, we
interviewed two instructors and one graduate teaching assistant. All of them
teach/specialize in different subjects at Oregon State.
Heather Bremicker is a Graduate Teaching Assistant Instructor (GTA).
She specializes in writing, and has been instructing the WR 121 course at
OSU. We asked her a series of questions via email on May 17, 2010. We
decided to interview her based on the fact that she teaches in small-
8/9/2019 Group One Writeup
10/31
classroom settings, a different instruction characteristic when compared to
our other interviewees.
We asked, What types of experiences have you had with the
Instructor Evaluation System? Were They Positive? Negative? To this, she
replied,
I have experienced the current evaluation forms on many different
levels- from administering them as office staff, to filling them out as
a student, as well as receiving them from students of my own. The one
thing that I can say about these forms is that they seem completely void
and disengaged from what the students true concerns really are. I can
say this much that the scoring system is concise- but I also do not think thatit entirely reflects the concerns that students may voice if given a
chance to do so.
Next we asked, If there was to be a new Instructor Evaluation System,
what types of information would be most helpful for you? She replied with,
Again, I think that the students who are given this as the only voice
they have, should be the ones to answer this question. In other words, the
kind of information needed, in my opinion, for a new Evaluation system,
should be one created by students- even though it is a tool that can be
used with my CV when applying for jobs- I would like the Evaluation to
reflect the genuine experience that my students had in my classroom.
Example: Was instructor able to make their subject available and
understandable to you? Did they try to reach you on a variety of different
levels with different mediums? etc....
Heather Bremicker does not see the current Instructor Evaluation
System as a benefit to the students, particularly in the sense of the students
using the Instructor Evaluations as a means of exercising their voice in the
school system. For her, the current system is a great means to build her
Cover Letter, which demonstrates how the Evaluations favor the Instructors,
not the students. She also believes that if a new Evaluation system were to
8/9/2019 Group One Writeup
11/31
arise, that it should include information which students would benefit and
want. She made a comment that her system seemed a bit idealistic, but
achievable none-the-less.
Our second interview was conducted with Instructor Paul Paulson on
May 24, 2010 via face-to-face interactive interview. Paul Paulson is a
Computer Science instructor and has been teaching at Oregon State
University for a total of eight years. Not only is he an instructor of the
Computer Science courses, but is also a MECOP advisor for the engineering
department.
Instructor Paul Paulson believes that the current system in place atOregon State is as good as it gets, and sees it as effective. He believes
that the in-class student evaluations kind of force the students to participate
in the evaluations. He made note that if there was an online system put into
place, it would have a significantly lower number of participants when
compared to the in-class surveying method.
Instructor Paul Paulson believes it would be a positive change for the
students to post the results of the course evaluations, but it would create
resentment amongst the instructors. He believes that instructors teaching
the same course would give better grades to students and make the class
easier in order to get a higher course evaluation, and promote students to
take their course over another instructor. This creates a competitive attitude
amongst instructors and the faculty of Oregon State.
There is a section of the current Course sheets which gives the student
a chance to make comments about the instructor and/or course. Paul
Paulson says that most, if not all instructors take a look at these portions of
the evaluations, and use them to make changes to their course for the
following term. Most of the changes made by the instructor are not seen by
the students who previously took their class simply because they will not be
taking the course again. However, there is some content of courses that the
8/9/2019 Group One Writeup
12/31
instructors are not able to change about a course. An example being the
types of information a course is required to cover.
Paul Paulson noted that if instructors receive bad scores for three
consecutive terms, that they are fired from the university faculty. This
applies to all instructors, except for those who have earned tenure.
Over-all, Instructor Paul Paulson sees the current system as a positive,
and emphasized that if the current system was changed, there would be
more cons than pros that would come from the situation.
Our next interviewee was Instructor Steve Cook. Steve Cook teaches
Geo Science courses, and strongly believes that there must be a change tothe current evaluation system at Oregon State University. We interviewed
Steve Cook on May 21st, 2010 via email.
First, we asked Steve Cook, Do you find the current system of course
evaluation useful/valuable? Why or why not? He replied with,
Not so much. I think that students don't believe that faculty pays any
attention to these forms. I've suggested on several occasions to faculty
that the results of student evaluations should be posted, but most faculty
disagree. I also know that when U of O did post evaluations, few
students perused the results. I find the responses to my qualitative
"Student Survey" to be of more value, as I can ask questions that are more
specific to the course and my presentation of it.
Next, we asked, What prompted you to publish the results of your
evaluations online? He replied with,
Perhaps because I get good student evaluations. I have no way of
knowing how many new students actually look at the results from the
previous term. I do try to put it on the screen at the first of the term and
make students aware that the info is there.
8/9/2019 Group One Writeup
13/31
Last, we asked, What information about a course would you like
students to have before they register? He replied with, Basic outline,
grading scheme, format, and topics emphasized.
Instructor Steve Cook wanted to emphasize that OSU is indeed moving
towards an online based course evaluation system, and that contacts have
been made with a Portland company to set up the system. He also wanted to
note that he instigated the move towards a different course evaluation
system.
Methods Caroline Hilburn
Since the intention of this project was to generate a course evaluation
system that would serve the students at OSU, student input was the focal
point of the development project. Because of the diversity of student
experience and opinions we needed many responses. To this end, we
designed a survey intended to provide quantitative data. In order to increase
the response rate, all of the questions are closed ended with the option of
writing in other answers. The survey is one page long with six questions. Thesurvey is included at the end of the report as Supplemental Figure 1.
Survey Questions:
What resources do you use to decide what classes to take?
(circle all that apply)
This question was intended to determine where students
currently look to decide how to schedule their courses. Also, this
question was supposed to determine whether RateMyProfessors.com
should be considered a model for a new system of course evaluation or
as competition.
Do you feel that your feedback about instructors and classes is
valued at OSU?
8/9/2019 Group One Writeup
14/31
The aim of this question was to determine how badly OSU needs
a new course evaluation system. The secondary aim of this question
was to motivate the university administration to accept a new system
of course evaluations.
Do you fill out the course evaluation Scantrons at the end of
each class?
One problem with the current course evaluation system is the
low response rate. This question was intended to determine what the
response rate is and what it is about the current system that convinces
people to fill out the course evaluation surveys.
What information would you like to have about a course beforeregistering? (circle all that apply)
Since the new system being designed is intended to give the
students all of the information they want, we asked students what
information would be useful. Ten individual pieces of information were
provided for students to choose from. While a write-in answer was
provided, no one chose that option indicating that the list is fairly
comprehensive.
In order to get this info, would you be willing at the end of
every course to (circle all that apply)
The purpose of this question was to determine how to administer
the new course evaluation system to maximize the response rate.
How would you rate the current course evaluation system used
at OSU?
The final question was written to echo the questions asked on
the current course evaluation Scantrons. The results were designed to
demonstrate the need for a redesigned evaluation system in a form
that the university administration would identify with.
A total of 63 responses were collected from current OSU students. Each
member of Group One collected surveys from students in different areas of
8/9/2019 Group One Writeup
15/31
campus over the course of two weeks. By spreading out the process of
collecting surveys, we ensured that the responses were random, providing a
representative sample of the students here at OSU.
The Logic behind the Madness: explaining why we made the
new system the way we did. -- Richard Bradley
Based upon the data from our survey on the current student
evaluations system, it is safe to conclude that the current system is
insufficient to the needs of the students at Oregon State University and that
the focus of the new student evaluation survey should be on giving students
the information that is relevant to them.
We were able to assess that the quality of the current system was not
working well for students from how the respondents rated the quality of the
current system. According to our results, 71% of the students who responded
thought that our current system was of either fair or poor quality. Given the
data, it is clear that the current system is leaving a majority of students
unsatisfied and we are in need of modifying it or creating a new student
evaluation system that focuses on the needs of students.
We also found that although most students are unsatisfied with the
current system, 60% of them always fill out the evaluations, while 29% of the
respondents claimed that they filled out the current evaluations
sometimes, and the most common explanation for why they filled out the
current student evaluations was, because I was told to fill it out. This
allowed our group to reason that if students were filling out the form because
they were told to, then it is our responsibility to tell students to fill this
form out.
8/9/2019 Group One Writeup
16/31
It was also implied that our audience was going to be very egocentric,
lazy, and that the most effective way to create a good system for students is
to tailor it for this type of audience. This altered our own aims because it
made us realize that we needed to make this new system as accessible as
possible and that we had to figure out ways of reminding people to fill out
this survey. It also suggested that we should make the new form as efficient
as possible, in that, we needed to put as much relevant information into the
new survey without making it long. We also found that no one filled in the
blank spaces on the survey where they were meant to write out an open
ended response. Rather than looking at this as a lack of data, we concluded
that this just meant that the new system would be mostly all bubbled in.
Fig. 5. Data collected by Group One. This chart reflects the opinions of OSU students
surveyed spring term, 2010
The survey also helped us conclude that the best way to disperse the
information collected from the new evaluations would be to use the
resources that are already most commonly used by students. According to
our results, the three most popular resources used to help students choose
classes are the coarse catalog, advisers, and friends. What we found most
interesting about our data was how the students found their friends just as
credible as advisers. In our findings, 56% of students claimed that they used
advisers and 53% claimed that they use their friends as credible sources, but
this difference is insignificant from a statistical standpoint because of oursmall sample size (Figure 5).
As a result, the data suggests that a student run system would be a
credible source for the students at Oregon State University. It was through
this data that we concluded that the course evaluations should be a quick
8/9/2019 Group One Writeup
17/31
link on both Blackboard and the course catalog would be the most
convenient and efficient means to put it online and installing the new system
online would make it easy to remind students to do the evaluations.
Fig. 6. Data collected by Group One. This chart reflects the opinions of OSU students
surveyed spring term, 2010.
From the survey we also found what information was relevant for
students and concluded that the most relevant information for the students
was: how difficult the course is, the effectiveness of the instructor, the
amount of time necessary for studying, what type of resources were most
helpful, and how expensive the books were (Figure 6). For our new survey we
asked them to rate the effectiveness of the instructor on a scale of 1-10; with
10 meaning that the instructor made the class clear and that they were
engaged with the class. We also wanted to rate the difficulty of the coarse on
a scale of 1-10 and 1 means that it was an easy class and 10 means that the
class was exceedingly difficult. For the new survey we also wanted to know
how many hours per week are spent doing homework or studying for the
course and figured that the most effective way to measure something like
this was to rate it on a scale of 0 10. We did these questions on a scale of
1-10 because we thought that it would be the clearest most effective way to
rate something as opinionated and varying as this. We also had people mark
what grade they thought they would receive in the class so that we are able
to gauge what type of student is mad or angry with the instructor.
In the new survey we also have a question about what resources they
found most valuable in this class. We decided that the easiest most effective
way to get these responses would be to list common resources used by
students to help them learn and then rate them on a scale of necessary,
8/9/2019 Group One Writeup
18/31
useful, and not helpful. The resources that we decided to include were office
hours, teachers aid, MLC, Google, and nothing. We wanted students to
know if they would be able to get help when they needed it and this is also
an effective tool at gauging the best resource and learning method for the
class.
When we asked students what they used the textbook for the goal was
to phrase the answers they would give in a way that describes whether you
actually need it or not. The answers that we have listed are: what
textbook?, cramming, reading before the test, help with homework, and
reading for each lecture. Within these answers you can tell how often you
need to use the textbook and thus whether you really need to buy it. We also
have a question revolved around how much the price of the textbook is,
because it is useful to know if the book is necessary for a course and
whether or not it is in your price range.
In conclusion, we made the new student evaluation like we did so that
it would be as accessible, convenient, and efficient as possible. We were
able construct our student surveys at the convenience of students at Oregon
State University and we did it in a way that supports our two main goals of:
providing as much relevant information about classes while still having the
respondents complete the survey while using as little energy as possible.
The new course evaluation prototype is available through GoogleDocs
at this link: https://spreadsheets.google.com/viewform?
hl=en&formkey=dE0tVW8xN2xadzF3WmNtUWoxdUlHcGc6MQ#gid=0
8/9/2019 Group One Writeup
19/31
Fig. 7. A Course Evaluation Prototype by WR214 Group One.
A Proposed New System Aaron Bicknell
Our proposed system for teacher evaluations will be a new feature for
the students and staff to both get aquatinted with, however; the benefits will
be remarkable. The proposed system will work similar to 3rd party websites
such as RateMyProfessors.com in the sense that students will be able to
look up their classes through feedback from everyone that has previously
taken that class (Figure 7). This system will incorporate the fact that both
students and staff will benefit from taking the time to actually complete the
8/9/2019 Group One Writeup
20/31
evaluation system. One of the key problems with the current system in place
is that it doesnt benefit the students.
The evaluation forms are strictly for the professors/instructors criticism
and praise. Students showed through surveys that our team provided that
they should be entitled to at least a portion of this information, since they
are the people who take the course. Staff will no longer be the only people to
see the feedback that is given through the system,
which leaves students completely in the dark. The
information about the course, other than what is
provided in the student handbook or through other
sources such as advisors/peers/etc. will now be
available to the students as well. All of the information
gathered from the students will be available online
through a website we developed and tested. Students
may access this webpage and see highly valuable
information about courses that may possibly interest
them. The site will incorporate several key factors that
students look for in courses, such as homework
workload, cost, homework study time etc. All of this
information will be at the click of a button, taking the
pain and uncertainty of selecting a course.
The proposed system will take place online
through the blackboard system that all students and
staff at the university are already familiar with. Filling
out the evaluation will be similar to checking a course
grade or reading the syllabus. There will be a new tab
on the blackboard page labeled Evaluation (Figure
8). Students will click this link and within minutes they
will be filling out the evaluation form, and logging precious information into
. #. Mock-up of
e Blackboard
. 8. A Mock-up of
e Blackboard link.
8/9/2019 Group One Writeup
21/31
the database for that course. This makes the process a very smooth
transition for everyone. The benefit of moving the evaluation system online
is that students and staff will save a considerable amount of time taking the
evaluations online because it can be done virtually anywhere that has
Internet access. Students will no longer have to fill out the Scantron system
in the classroom, which wastes precious time and is often considered a
waste of time, especially due to the fact that the current system of teacher
evaluation is given during "deadweek", where most students already feel the
pressure of upcoming finals. The last thing students want is the annoyance
of filling out a Scantron for the staffs benefit rather than prepare for the
upcoming final in that class by asking questions, or even possibly going overa final exam review. Eliminating downtime in classrooms is a great way to
keep both the staff and students on the task at hand, whether its an
approaching final, or other class related subject. The staff will no longer have
to leave the room while students fill out forms, and students will no longer be
responsible for the delivery of those forms to an office where they are to be
counted through an automated machine that really provides no feedback to
the people who need it: the students.
Not only will the evaluation be online through blackboard, but students
will also be able to view all of the evaluations online as well. Every evaluation
that has been filled out correctly through the blackboard link will be recorded
to an outside website that will take that information, and store it into a large
database system that students can use to navigate to find desired
information about any course offered at the university. Every course will
have a special lookup code that you can type into the database, and quicklylook up any and all information that has been previously recorded by other
students previously enrolled in that course. This system can also possibly
eliminate the waste of printing off thousands of course documents that the
university currently distributes to students and staff. Oregon State University
8/9/2019 Group One Writeup
22/31
could add this astonishing contribution to an already outstanding green
aware campus that the university already loves to show off.
Making the evaluation from online through blackboard not only saves
time, but also money as well. The current system features generic Scantron
forms that obviously cost money to produce. If the average student takes 14-
16 credits for the fall, winter, and spring terms, at 3 credits per class, that
student is filling out anywhere between 3-5 evaluation forms per term. OSU
is proud to say that they enroll 21,996 students, making the math come out
to be anywhere between 65,998 to 109,980 Scantrons for the university to
record in just one term alone. Since the proposed system will be online,
literally tens of thousands of Scantrons will not be needed, and that money
will save the university a chunk of money every term.
While saving both time and money are huge benefits for the university,
another attractive feature of the proposed system is that it is paperless. In
todays society everyone is concerned with the state of the environment.
Going green is the new trend; saving as much paper, recycling as much as
possible etc. The proposed system will remove literally thousands of pieces
of paper a term. The university is already very proud of their efforts in going
green and the addition of anything that is beneficial to the environment
should be very welcomed here at the university. Many courses here already
have made the transition to paperless classes which is a great idea for
most classes that require any form of essay, or writing assignment. Changing
the system can be a great benefit to the environment, time management,
and even saving the university, the staff, and the students a little extra
money from printing UN-necessary documents and wasting ink.
The facts and benefits are clear that this system could be a great
addition to the campus. The time saving factors benefit not only the staff,
but the students as well. The University has already become a leader for the
8/9/2019 Group One Writeup
23/31
green environmental movement, and quite frankly saving money is
something everyone will appreciate. Even though the system is still a
prototype, with a little time and hard work, the proposed system can be a
major advantage that Oregon State can hold over other Universities.
Student Views of the Course Evaluation Prototype Alexa
Tucker
Introduction:
In order to get further opinions and feedback on the new course
evaluation system that our group designed, we had several students take a
moment to test it out for themselves. I first showed them the display of
blackboard with the Course Evaluation tab. I told them that we chose
Blackboard since all students know how to use it and that it would provide
easy and convenient access to the Course Evaluation link. Each student then
filled out the online survey we created as if evaluating a course they are
currently taking. After completing it, I showed them the results page ofstatistics and percentages of previously completed surveys. Though these
results are hypothetical and dont accurately represent an actual course, I
explained to everyone that this is a preview of how our new proposed system
would work. I explained to them that this would be the kind of information
they would have access to when choosing classes and that their input would
contribute to these results. In total, five students were interviewed about
their experiences with the course evaluation system.
1. Ashley Milliron -- The response to this new Course Evaluation
system was very positive. She said that she would be more
willing to fill out the evaluation considering the fact that it would
be easy to access through blackboard and due to the simplicity
8/9/2019 Group One Writeup
24/31
of the survey. It wouldnt be very time consuming and I would
rather be able to fill it out on my terms rather than in the middle
or at the end of class. Im more likely to skip filling out the in-
class evaluations if they are at the end of class because I just
figure everyone else is doing it and one less opinion from me
wouldnt matter much. Ashley stated. Like Ashley, this could be
the mindset of many other students who are asked to fill out in-
class evaluations. They may think that it doesnt matter if they
fill them out because there are plenty of other people already
doing it and itd be a waste of time since they dont even get to
see the evaluation results.
2. Kaylie Sundburg -- She said she liked the layout of the online
evaluation survey and that it was very easy to use. She also
told me that she would be more likely to fill out the evaluation if
it was posted on Blackboard and she could do it when she
wanted to. Kaylie said, I like that this system will also benefit
the students. Not only does it give results of what students are
saying about the instructor, but other aspects of the class like its
difficulty and how much the textbooks are used. She told me
that one of her favorite things about the information given from
this system was how much textbooks were used. Books are
expensive, and it sucks when you pay all of that money for it
because the syllabus says its required, but in the end, you rarely
even used it throughout the term. Most information needed
these days are posted on Blackboard through outlines and other
sources. This system could help students decide if they really
want to spend the money for a textbook based on how much
previous students actually used them throughout the course. It
can also help students decide if theyd rather share a book with
8/9/2019 Group One Writeup
25/31
someone else and split the cost if the results say there is some
reading required.
3. Jake Goodman -- Jake told me that he always fills out the in-
class evaluations because he doesnt mind doing so, but he said
that he likes the idea of an online evaluation much better. Im a
big supporter of going green, so if we can save paper this way
and have it still be effective, Im all for it! Obviously having the
course evaluations online would cut costs on paper and printing
and would be more convenient since the internet is used for so
many things these days. I think results would get in faster andstudents would really like being able to actually see the results.
Jake said. He also stated that he feels that many other green
supporters would highly encourage this system and would be
more willing to fill it out if they knew they were going to be
cutting down on resources and money. Not to mention, the U.S.
is all about fast, easy, and convenient. Thats exactly what this
system seems to be. Though I think people are getting too lazy
these days with technology and the internet, thats just how it is
and people will be drawn to the simplicity of this system. Jake
stated.
4. Lauren Butterfield -- She agreed with other three students
that the system was simple and easy to use and that she would
be more likely to fill out an online evaluation. Lauren said, I
definitely like the idea but I think the results page needs some
work The bar graphs are a little confusing and could be
inaccurate because these are just showing what a certain
amount of people said. What happens when two hundred
8/9/2019 Group One Writeup
26/31
students fill out this application? She said that she believes
simple numerical data would be better, something more straight
to the point than graphs. I dont like clutter, I just like things to
be straightforward. What you have now is along the right idea,
just needs some alterations. Lauren did however, like the
simplicity of the actual survey. Of course, our representation is
just an example of what our proposed system would be, but we
need this kind of feedback to decide what changes need to be
made or what needs to be added/taken away. If this system were
to be put in place, it should be tested several times by students
to make sure most everyone likes the end result.
5. Hallie Borgen -- One thing she noted on the current online
survey was that once you clicked one of the bubbles for your
answer, you cant undo it if you want to change it. This, again,
shows that there are changes and tune ups that need to be
made to the system but could be easily made when actually
putting the system to use. Hallie liked the overall easy-to-use
system but also agreed with Lauren that the bar graphs were
slightly confusing and numerical data would be better. Im also
for going green, so if we can save some trees by using this
system I think its a great idea. I asked Hallie if she was satisfied
with the information given about the course from the results. I
like everything thats given, but if I had to add another thing I
would want to know how much students enjoyed the course. I
know not all classes are going to be super fun, but if a certain
class is a total drag, I dont want to waste my time unless I know
I myself would be interested in it she stated. Other than that
Hallie was happy with the overall idea of our new system and
said she would prefer it over in-class-evaluations since she gets
8/9/2019 Group One Writeup
27/31
to see the results and it can benefit her and other students.
Summary:
Due to the overall positive response to our system, I think that with
fine tuning and certain additions this will be a very effective system. All of
the students that tested it like the simplicity of it, supported the green
aspect of not having to use all of that paper for in-class-evaluations, and
liked that it benefited them as students. With the link to the evaluation being
on Blackboard, students would be more likely to fill it out since most
students use Blackboard regularly and they could also do it when they
wanted to. Overall, minimal negative responses were given, just suggestions
that could make the system better and more understandable. Feedback is
very important for this system because it will be the students who are using
it, therefore their opinion counts significantly in the process of completing
the overall design and function of our evaluation system. Without their
opinions, certain things may not be covered and the evaluation would
continue to have problems. Like stated before, every system proposed will
most likely have its pros and cons but our group has worked hard to create
one where the pros outweigh the cons.
Next Steps Caroline Hilburn
Before implementing any major changes to the OSU course evaluation
system, the proposed new system needs to be tested. This section outlines
the steps that still need to be taken to ensure that the new course evaluation
system is ready.
8/9/2019 Group One Writeup
28/31
Collect student reactions to the new system.
Why? Students are most likely to take advantage of and
participate in a course evaluation system if they feel that it
benefits them.
How? Allow current OSU students to test the new system
and collect feedback. Students in different academic years,
majors and backgrounds need to be polled or interviewed
to get representative feedback about the system. Data
should be collected concerning how well the survey meets
students needs and the usability of the system.
Collect professor/instructor reactions to the new system. Why? Professor cooperation would legitimize and
streamline any new course evaluation system. If
professors understand and approve of the new system
they are more likely to participate, provide previous syllabi
and use the course evaluation system to advertise their
classes.
How?Allow professors and instructors at OSU access to
the prototype system. Interview the instructors to find out
what would make them more likely to participate in a
course evaluation system for the students. Data should
also be asked about the usability of the system.
Test the prototype evaluation system on one class.
Why? Each question of the completed survey needs to be
tested for both reliability and validity. Also, testing the
survey on a class of a known size will offer an approximate
response rate.
How? Provide one large lecture class with the opportunity
to test-drive a new course evaluation system. Use the data
8/9/2019 Group One Writeup
29/31
collected to analyze response rate, reliability and validity
for each of the questions.
These three steps should be applied as part of an iterative cycle to refine the
final course evaluation system. Additionally, these considerations need to be
addressed before the new course evaluation system can be fully
implemented.
Tailoring the survey to the course.
Why? In order to ensure that all responses are relevant
to the course being evaluated, the survey answers need to
be designed specifically for that course. Specifically, theresources available for each course need to be listed.
How? Professors and Instructors should be able to select
which answers are relevant to their class from a master
list. A more generic, department specific template should
be available for courses in which the professor or instructor
does not respond.
Avoid legal and practical complications.
Why? Currently, the prototype system is powered by
GoogleDocs. Before testing the system, it would be wise to
read the fine print and determine whether or not
GoogleDocs can be used on a large-scale at a public
university. Also, whatever system is chosen, it is
imperative that it be able to handle the high traffic that
would occur on a university-wide scale.
How? Consult Google and the OSU legal office about the
legality of the proposed course evaluation system. OSU Net
and the OSU computer support staff should be consulted
about the amount of traffic the proposed system will be
able to handle.
8/9/2019 Group One Writeup
30/31
Conclusion
After six weeks of research, our group has come to the following
conclusions:
The current course evaluation system used at OSU does not
serve the needs of the student body.
Most students and professors would be willing to participate in a
new system of course evaluations that were tailored to students
needs.
There are many available models and ideas to base a new
system off of.
The course evaluation system proposed in this report is just one idea.
More research needs to be done to refine the final product before it is ready
to apply on a university wide scale. However, the student body is ready for
change.
Supplemental Figure 1.
What resources do you use to decide what classes to take? (circle all that
apply)
Course catalogue Advisors Friends RateMyProfessors.com
None
Other __________________________________
8/9/2019 Group One Writeup
31/31
Do you feel that your feedback about instructors and classes is valued at
OSU?
No Rarely Sometimes Often Yes
Do you fill out the course evaluation scantrons at the end of each class?
No, never Sometimes Yes, always NA Why?_________________________
What information would you like to have about a course before
registering? (circle all that apply)
Instructor effectiveness Class difficulty Previous term grade
average
Price of textbooks Are the textbooks used Average pages of reading per
week
Hours of homework per week TA effectiveness Ability to get extra help
Professor background Professor understandability Previous syllabus
Others ____________________________________________________
In order to get this info, would you be willing at the end of every course
to (circle all that apply)
Fill out an in-class evaluation Rate the course on blackboard
Rate the course on a student published web site Other
__________________________________
How would you rate the current course evaluation system used at OSU?
Poor Fair Good Very Good Exceptional