Web Page Usability Testing

Gill Bartlett (Learner Support)

Scott Castle (SSiD)

Gill Hughes (SSiD)

SSiD Web Pages: 900 pages in SSiD suite 250,000 hits per month 48,000 visits 28,000 unique sites

Format

• One working day.• Observing students navigating the ‘current

students’ web pages.• Undertake tasks based on realistic

scenarios.• One student at a time, plus a facilitator

and an observer.• First year undergraduates.• 50 minutes per test.

Resources

• Student Volunteers• Computer, audio & video equipment• £10 book tokens

The Script

• informed students why we were conducting the tests.

• reassured them that we were testing the web site and not them.

• asked them to be critical and honest.

• asked them to do all their thinking ‘out loud’.

Scenarios

• Examination Timetables

• Examination Clash forms

• Change of Address

• Summary of Results

• Access to Learning Funds (ALF)

• Short Term Emergency Assistance

• Things Not Going Right

Information CaptureScan Converter

Video Recorder

Microphone

Monitor

Information Capture

• Video Recording of screen

• Audio recording of comments onto video

• Observers note taking

• Creation of a .wav file for PC

• Recording the User Click Stream onto an Excel spreadsheet

Starting From: http://www.shef.ac.uk/Click Link Name Link Position To

1 Current Students Table of Contents http://www.shef.ac.uk/ssid/2 Exam Timetable main body (featured) http://www.shef.ac.uk/ssid/admin/timetbl.html3 DRAFT timetable, main body (text) http://www.shef.ac.uk/ssid/draftexamtt/all_draft.html

Starting From: http://www.shef.ac.uk/ssid/draftexamtt/all_draft.html1 Back Menu Bar http://www.shef.ac.uk/ssid/draftexamtt/all_draft.html2 Examination Clash Form main body (text) http://www.sheffield.ac.uk/ssid/forms/exam.html

Starting From: http://www.shef.ac.uk/1 Current Students Table of Contents http://www.shef.ac.uk/ssid/2 Your University Record Table of Contents http://www.sheffield.ac.uk/ssid/record3 Your University Record main body (title) http://www.sheffield.ac.uk/ssid/record/pers_inf.html4 Changing your Address main body (text) http://www.shef.ac.uk/ssid/record/pin.html

Starting From: http://www.shef.ac.uk/ssid/record/pin.html1 Your University Record Breadcrumbs http://www.shef.ac.uk/ssid/record

2Certified Copies of Degree Certificates main body (title) http://www.shef.ac.uk/ssid/record/degree_copies.html

Starting From: http://www.shef.ac.uk/ssid/record/degree_copies.html1 SSiD Breadcrumbs http://www.shef.ac.uk/ssid/

2Useful Contacts for Current Students main body (listed) http://www.shef.ac.uk/ssid/contacts.html

Starting From: http://www.shef.ac.uk/1 Current Students Table of Contents http://www.shef.ac.uk/ssid/2 Financial Information Table of Contents http://www.shef.ac.uk/ssid/finance3 Undergraduate Table of Contents http://www.shef.ac.uk/ssid/finance/financeug.html

4Financial Support administered by the University main body (title) http://www.shef.ac.uk/ssid/finance/finhlpug.html

5Financial Support for Full Time Courses main body (sub-title) http://www.shef.ac.uk/ssid/finance/ftguide.html

Scenario Progression: Phone Number

Scenario 3: Access to Learning Fund

Scenario 1: Examination Timetable

Scenario Progression: Clash Forms

Scenario 2: Change of Address

Scenario Progression: Results of Exams

Excel Spreadsheet showing the Click Stream

Findings (General)• Accessing SSiD pages• Don’t Know What They Don’t Know

– Confusion between products• Finding specific information set

– Transcript & Summary of Results• Content Volume

– Things Not Going Right• Appropriate Medium

– Web / Telephone / Email / Personal• Link Descriptions

– Student Support & Guidance• Links to Further Information

– Short Term Emergency Assistance

Changes arising from…..

• Suggestions of Students– “You should have….”

• Comments from Students– “It would be good if…..”

• Behaviour of Students– User Click Streams

• Inspired by Students

Some Comments…..

• “Lot of stuff here – might even do a find on the page”

• “I like the pictures… they tell me I can go there”

• “I tend to look at the FAQ before the A to Z because it is categorised”

• “I found the Sheffield home page a bit old fashioned when I applied compared to other universities.”

Changes made to Web Pages

• Amendments to existing pages– Less Content (Things Not Going Right)

– ‘Chunking’ (Financial Support)

– Expanded scope (Increase 16 to 120 FAQs and 24 categories)

• Links to associated pages– More on the subject (Financial Support & STL)

– Complementary Pages (Things and FAQ page )

Changes continued

• New Pages– Summary of Results (1,300 hits in February)

– Certificate of Student Status– Examples of documents

• Links where they are expected– Transcripts & Summary of Results

• Current Student ‘home’ page changes

Did it work ? – Hits/ssid/welfare/signposts.html

337

1446

1748

277

321381

375

-200

0

200

400

600

800

1000

1200

1400

1600

1800

2000

Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul

2003-04

2004-05

Did it Work ? – Test Changes

• Repeated 5 scenarios from 1st Testa. Address change

b. Summary of Results/Transcripts

c. Access to Learning Fund

d. Short Term Emergency Assistance

e. Things Not Going Right?

• Faster and fewer clicks required (except e)• New Visual Identity home page

In Summary

• Simple, cheap, portable and effective exercise.

• Iterative Process– Test, analyse & evaluate, change, test…

• Test Groups– International, prospective students, students

with disabilities, postgraduates etc.

Questions?