Web Page Usability Testing
Gill Bartlett (Learner Support)
Scott Castle (SSiD)
Gill Hughes (SSiD)
SSiD Web Pages: 900 pages in SSiD suite 250,000 hits per month 48,000 visits 28,000 unique sites
Format
• One working day.• Observing students navigating the ‘current
students’ web pages.• Undertake tasks based on realistic
scenarios.• One student at a time, plus a facilitator
and an observer.• First year undergraduates.• 50 minutes per test.
Resources
• Student Volunteers• Computer, audio & video equipment• £10 book tokens
The Script
• informed students why we were conducting the tests.
• reassured them that we were testing the web site and not them.
• asked them to be critical and honest.
• asked them to do all their thinking ‘out loud’.
Scenarios
• Examination Timetables
• Examination Clash forms
• Change of Address
• Summary of Results
• Access to Learning Funds (ALF)
• Short Term Emergency Assistance
• Things Not Going Right
Information CaptureScan Converter
Video Recorder
Microphone
Monitor
Information Capture
• Video Recording of screen
• Audio recording of comments onto video
• Observers note taking
• Creation of a .wav file for PC
• Recording the User Click Stream onto an Excel spreadsheet
Starting From: http://www.shef.ac.uk/Click Link Name Link Position To
1 Current Students Table of Contents http://www.shef.ac.uk/ssid/2 Exam Timetable main body (featured) http://www.shef.ac.uk/ssid/admin/timetbl.html3 DRAFT timetable, main body (text) http://www.shef.ac.uk/ssid/draftexamtt/all_draft.html
Starting From: http://www.shef.ac.uk/ssid/draftexamtt/all_draft.html1 Back Menu Bar http://www.shef.ac.uk/ssid/draftexamtt/all_draft.html2 Examination Clash Form main body (text) http://www.sheffield.ac.uk/ssid/forms/exam.html
Starting From: http://www.shef.ac.uk/1 Current Students Table of Contents http://www.shef.ac.uk/ssid/2 Your University Record Table of Contents http://www.sheffield.ac.uk/ssid/record3 Your University Record main body (title) http://www.sheffield.ac.uk/ssid/record/pers_inf.html4 Changing your Address main body (text) http://www.shef.ac.uk/ssid/record/pin.html
Starting From: http://www.shef.ac.uk/ssid/record/pin.html1 Your University Record Breadcrumbs http://www.shef.ac.uk/ssid/record
2Certified Copies of Degree Certificates main body (title) http://www.shef.ac.uk/ssid/record/degree_copies.html
Starting From: http://www.shef.ac.uk/ssid/record/degree_copies.html1 SSiD Breadcrumbs http://www.shef.ac.uk/ssid/
2Useful Contacts for Current Students main body (listed) http://www.shef.ac.uk/ssid/contacts.html
Starting From: http://www.shef.ac.uk/1 Current Students Table of Contents http://www.shef.ac.uk/ssid/2 Financial Information Table of Contents http://www.shef.ac.uk/ssid/finance3 Undergraduate Table of Contents http://www.shef.ac.uk/ssid/finance/financeug.html
4Financial Support administered by the University main body (title) http://www.shef.ac.uk/ssid/finance/finhlpug.html
5Financial Support for Full Time Courses main body (sub-title) http://www.shef.ac.uk/ssid/finance/ftguide.html
Scenario Progression: Phone Number
Scenario 3: Access to Learning Fund
Scenario 1: Examination Timetable
Scenario Progression: Clash Forms
Scenario 2: Change of Address
Scenario Progression: Results of Exams
Excel Spreadsheet showing the Click Stream
Findings (General)• Accessing SSiD pages• Don’t Know What They Don’t Know
– Confusion between products• Finding specific information set
– Transcript & Summary of Results• Content Volume
– Things Not Going Right• Appropriate Medium
– Web / Telephone / Email / Personal• Link Descriptions
– Student Support & Guidance• Links to Further Information
– Short Term Emergency Assistance
Changes arising from…..
• Suggestions of Students– “You should have….”
• Comments from Students– “It would be good if…..”
• Behaviour of Students– User Click Streams
• Inspired by Students
Some Comments…..
• “Lot of stuff here – might even do a find on the page”
• “I like the pictures… they tell me I can go there”
• “I tend to look at the FAQ before the A to Z because it is categorised”
• “I found the Sheffield home page a bit old fashioned when I applied compared to other universities.”
Changes made to Web Pages
• Amendments to existing pages– Less Content (Things Not Going Right)
– ‘Chunking’ (Financial Support)
– Expanded scope (Increase 16 to 120 FAQs and 24 categories)
• Links to associated pages– More on the subject (Financial Support & STL)
– Complementary Pages (Things and FAQ page )
Changes continued
• New Pages– Summary of Results (1,300 hits in February)
– Certificate of Student Status– Examples of documents
• Links where they are expected– Transcripts & Summary of Results
• Current Student ‘home’ page changes
Did it work ? – Hits/ssid/welfare/signposts.html
337
1446
1748
277
321381
375
-200
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul
2003-04
2004-05
Did it Work ? – Test Changes
• Repeated 5 scenarios from 1st Testa. Address change
b. Summary of Results/Transcripts
c. Access to Learning Fund
d. Short Term Emergency Assistance
e. Things Not Going Right?
• Faster and fewer clicks required (except e)• New Visual Identity home page
In Summary
• Simple, cheap, portable and effective exercise.
• Iterative Process– Test, analyse & evaluate, change, test…
• Test Groups– International, prospective students, students
with disabilities, postgraduates etc.
Questions?
