Date post: | 14-Jul-2015 |
Category: |
Data & Analytics |
Upload: | stewart-hutchison |
View: | 69 times |
Download: | 0 times |
Reserve 25 (R25) WebViewer
The R25 WebViewer is widely used by college campuses,
primarily used for event management & scheduling, allowing
students to:
View course information, events and activities
Access the schedule of indoor and outdoor facilities
Reserve campus space for student organizations or personal use
Business Goals
Encourage students to become more involved with campus
activities:
Initiate and join campus organizations
Fostering an active and cooperative campus environment
Target
Users A full-time undergraduate student who spends the
majority of their time on campus.
A highly motivated and active student who initiates
activities with their peers.
A student who belongs to an organization on
campus or regularly attends athletic events.
Ideal participants for
our study
Participant Screener
How many credits are you currently enrolled?
Which of the following age ranges are you in?
Are you currently living on campus?
Have you ever accessed our online Calendar of Events to view an activity or search
course/activity information?
Have you ever initiated or created an event on campus with the Center for Student
Engagement?
Do you currently belong to a student organization?
Participant Introduction
“Thank you for taking the time to meet with us today.
We’re a graduate student group interested in better understanding how people interact with the R25
WebViewer Scheduling software and have asked for your help. So we can explore the software together,
you will receive specific directions for each task scenario. You can withdraw from this study at any time, and
will still receive the extra credit points from your instructor.
If you have any questions, please feel free to ask the moderator. No personally identifying information is
required and only the graduate student team will have direct access to your responses. You are encouraged
to think aloud as you complete the scenarios, describing your thoughts and feelings to the extent you feel
comfortable.
The Missouri Western website is often busy and may take time to load new pages, so there may be some
significant “clicking-and-waiting”.
We thank you again for your time and patience.”
Task Scenarios
You are curious about today’s events on campus…
You have decided to initiate an activity on campus and would like to submit an event request…
You are a softball player and would like to participate in a tournament game on February 1, 2014. You will need to contact the event organizer by obtaining their phone number…
You have enrolled in a college writing course (ENG 104 01), which begins Jan. 14 on Tuesday and Thursdays. You still need to know the time and location of your class…
You are planning to organize a conference meeting in one of the Fulkerson rooms the last week of April, 2014. You require a room that will seat 125 people…
15 Tasks
1. Locate today's date on the calendar to view the most current event listings.
2. Select any event for more information.
3. Locate the option to view all requests that you have already made.
4. Select the option to submit a request for an event.
5. Locate the ability to view only events held at the Griffon Indoor Sports Complex.
6. Navigate to the specified date of the softball tournament.
7. Locate the event for more information and confirm the requestor’s phone number.
8. Locate the option to display both events and course information.
9. Navigate to your course using the calendar of events.
10. Select your class to determine the time and location.
11. Locate the option to display building locations only.
12. Navigate to the desired week to schedule your conference.
13. Find and select one of the Fulkerson rooms for more information.
14. Select the room again above the calendar to determine the seating capacity.
(Repeat this step until you have found a suitable room for your audience).
15. Determine an available time and select the time slot to request a conference.
Testing Procedure
Two minutes anticipated per task; participants given 30 minutes total.
One assist was given for every minute spent on a singe task.
All testing was completed on a laptop computer (Toshiba Satellite P855).
Screen movements and audio were recorded with Camtasia Studio 8.1.
Remote testing was conducted via Google Hangouts.
#5: Locate the ability to view only events
held at the Griffon Indoor Sports Complex
#11: Locate the option to display
building locations only
Mean and Variance
#4: Select
the option to
submit a
request for
an event
#14: Select
the room
above the
calendar…
Task #4“Select the option to submit a request for an event”
Video 1/3
3:06
Mean = 4 seconds
Variance = 3 seconds
Task #4 Summary
The first two users began scrolled-down,
where the top menu was not visible.
Locating “My Requests” took considerably
longer for users #1 and #2 (72 and 103
seconds).
Once accessed, all users located “New
Request” in 6 seconds or less.
Task #5“Locate the ability to view only events held at the Griffon Indoor Sports Complex”
Video 2/3
3:14
Mean = 62.8 seconds
Variance = 279 seconds
Task #5 Summary
User #4 took considerably longer to
complete task 5 (283 seconds).
After selecting “Locations”, user #4 did not
scroll down to view “Filter” options until
prompted by moderator.
User #3 also selected “Locations” first, but
recovered quickly by navigating the “Filter”
options (16 seconds).
X
“The ability to view” = “Filters”?
Task #11“Locate the option to display building locations only”
Video 3/3
2:26
Mean = 63.6 seconds
Variance = 189 seconds
Task #11 Summary
All users began task scrolled-down, where
“Locations” was not immediately visible.
Users #2 & #3 required a prompt to look away
from the “Filters” option
Users #1, #4, and #5 completed the task under
17 seconds.
Possible learning effect at this point, where the
“Locations” option had been previously
explored.
“The option to display” = “Locations”?
X
SUS – System Usability Scale
1. I think that I would like to use this website frequently.
2. I found this website unnecessarily complex.
3. I thought this website was easy to use.
4. I think that I would need assistance to be able to use this website.
5. I found the various functions in this website were well integrated.
6. I thought there was too much inconsistency in this website.
7. I would imagine that most people would learn to use this website very quickly.
8. I found this website very cumbersome/awkward to use.
9. I felt very confident using this website.
10. I needed to learn a lot of things before I could get going with this website.
All questions are scaled 1-5 (1 = Strongly disagree, 5 = Strongly agree)
Scoring the SUS
For odd items: subtract one from the user response.
For even-numbered items: subtract the user responses from 5.
This scales all values from 0 to 4 (with four being the most positive response).
Add up the converted responses for each user and multiply that total by 2.5.
This converts the range of possible values from 0 to 100.
http://www.measuringusability.com/sus.php
SUS Scores:
User 1 = 50
User 2 = 47.50
User 3 = 67.50
User 4 = 70
User 5 = 55
A score of 68 is considered average (50% rank)
http://www.measuringusability.com/sus.php
Measure of perceived usability
Interpretation of SUS Scores
http://www.measuringusability.com/sus.php
User
#1
User
#5User
#2
User
#3
User
#4
Based on Jeff Sauro’s 500 studies
Conclusion of SUS Analysis
Participant’s average SUS Score = 58%
This is below Sauro’s average: 68%
Measure of perceived usability percentile ranking = 22%
Severity Scale – Dumas & Redish
Level 1: Prevents task completion
Level 2: Creates significant delay and frustration
Level 3: Problems have a minor effect on usability
Level 4: Subtle and possible enhancements/suggestions
Local (single page) vs. Global (entire website)
Recommendations
Include descriptive text
Event A
Event B
Event H
Event C
Event D
Event F
Event EEvent G
Event I
Areas to Improve – Disappearing Menu
Recommendation:
“Locations” and “My Requests”
A persistent view of the menu bar while
scrolling down the calendar of events
An example is given by Binghamton
University…
Most users did not initially see the top menu
bar
Areas to Improve – Aesthetics
Sorting through bolded
information is strenuous
The color of the top
navigation menu makes it
easy to overlook
Viewing information as a
grid
(Global 3)
Recommendations
Bold times instead of descriptions
Stronger color contrast to make menu more noticeable
Note about locations/additional features
Viewing information as a list (have it be default setting)
Areas to Improve
Old Information (Global 3)
Scheduling information dates from 2005
This may be the cause of significant lag and slow response times
Recommendations
Remove the old data and information from the system
Back up the data on another server/system
Priorities
Navigation bar at the top of the screen (Global 2)
Aesthetics (Global 3)
Text to describe event (Global 3)
Old Information (Global 3)