Usability Testing a Public ERM: Worth the Effort?

Stephanie Willen BrownElectronic Resource LibrarianUniversity of Connecticut January 8, 2008

Act Like a User

You need information on diabetes for a paper.

Two questions1. What’s the first thing you see?

2. Where can get material for your paper? Site 1 Site 2 Site 3

Community College of Philadelphiahttp://www.ccp.edu/vpacaff/library/

Salt Lake Community Collegehttp://libweb.slcc.edu/index/library-resources

York County (ME) Community Collegehttp://ww.yccc.edu/library/library.asp

What *is* Usability?

Usability … assesses how easy user

interfaces are to use. The word “usability”

also refers to methods for improving

ease-of-use during the design process.

Jakob Nielsen, Usability 101

“Making Things Findable”

UConn Libraries “Plan 2010” Goal 2: Scholar’s Portal says:“Provide immediate, unmediated, and

comprehensive access to digitized research and scholarly collections worldwide.”

Web & database usage statistics greatly outweigh individual library-user contact

Components of Usability

Goals

Learnability

Efficiency

Memorability

Error recovery

Satisfaction

Achieved by …

Thinking like a user

Consistency

Tweaking text

Thinking Like a User

“… let’s … acknowledg[e] … the vital importance of empathy for the user. Only by understanding and caring about the perspective of the individual can we design useful, usable solutions.”

- Peter Morville, Ambient Findability

ConsistencyConsistency

Color, graphicsColor, graphics

Orientation & navigationOrientation & navigation

LanguageLanguage

Reading Online is Like …

Reading Proust People like to read long sentences

online because it’s easy to keep your place, follow complex trains of thought, and flip to the next screen of dense text.

or

Skimming citationsPeople skim titles & abstracts for

keywords, take notes, and move to the next citation

Reading Online … ProustFor a long time I used to go to bed early. Sometimes, when I had put out my candle, my eyes would close so quickly that I had not even time to say "I'm going to sleep." And half an hour later the thought that it was time to go to sleep would awaken me; I would try to put away the book which, I imagined, was still in my hands, and to blow out the light; I had been thinking all the time, while I was asleep, of what I had just been reading, but my thoughts had run into a channel of their own, until I myself seemed actually to have become the subject of my book: a church, a quartet, the rivalry between Francois I and Charles V. This impression would persist for some moments after I was awake; it did not disturb my mind, but it lay like scales upon my eyes and prevented them from registering the fact that the candle was no longer burning. Then it would begin to seem unintelligible, as the thoughts of a former existence must be to a reincarnate spirit; the subject of my book would separate itself from me, leaving me free to choose whether I would form part of it or no; and at the same time my sight would return and I would be astonished to find myself in a state of darkness, pleasant and restful enough for the eyes, and even more, perhaps, for my mind, to which it appeared incomprehensible, without a cause, a matter dark indeed.

Swann’s Way / Marcel Proust

Skimming Citations Online …

ERIC search: African Americans & mathematics

TypeBooks | Journals | Reports | Videos | Web Sites

SciencesBiology | Chemistry | Microbiology | Physics | Zoology

Which is Easiest to Understand?

BiologyBooksChemistryJournalsMicrobiologyPhysicsReportsVideosWeb SitesZoology

Type• Books• Journals• Reports• Videos• Web Sites

Sciences• Biology• Chemistry• Microbiology• Physics• Zoology

Arguably …

Why? Content arranged logically, with clear headings &

bullets Alphabetical within groups Content goes down, as it does in traditional print Words are “chunked” into pieces easy for eye to scan

in one glance

Type• Books• Journals• Reports• Videos• Web Sites

Sciences• Biology• Chemistry• Microbiology• Physics• Zoology

Jargon for Librarians

On travelocity, you need the cheapest round-trip fare from Boston to London.

These are your options – which is right?

1. Flights & Prices

2. Fares

3. Three Best Itineraries

Library Jargon to Students

“ERIC, I think it’s some kind of journal … some kind of citation.”

Reference Shelf “[It’s] very general. You don’t know what to expect as it could be anything.”

Usability Testing

Define users Design questions to mimic what users

would realistically do Do usability testing early & often

3-5 users highlights 85% of errorsBetter to test several small groups than

10-15 at once Note errors, redesign and retest

Redesigning the Research Database Locator External

Literature reviewFind sites we liked

InternalUsage & query log analysisUsability testingDesign

• testing • redesign

• more testing

Literature Review

Cobus, Laura, Frances D. Valeda, and Anita Ondrusek. “How Twenty-Eight Users Helped Redesign an Academic Library Web Site.” Information Technology & Libraries 44.3 (2005): 232-46.

Krug, Steve. Don’t Make Me Think! 2d ed, Indianapolis, Ind. : Que. 2006.

Nielsen, Jakob. “Usability 101.” Alertbox: Current Issues in Web Usability.

http://gslis.simmons.edu/mw/browns/Usability

Sites We Liked

University of Toronto’s “Best research resources for your topic.” Shows article databases, research guides, and encyclopedias / dictionaries / handbooks.

NCSU’s “Browse Subjects” See Poultry Science e.g., and note tabs at the top for Summary, Books & More; Databases & Articles; Journals; Reference Tools. Full subject list at left, with Agriculture expanded on Poultry Science page.

Purdue’s “All Databases by Subject” Select main subjects from menu to see subcategories.

http://gslis.simmons.edu/mw/browns/Usability/Sites

Usage Log Analysis

March - May 2006, UConn patrons … Used the subject browse

18,000 times; Performed a keyword

search 15,800 times; and Clicked on Databases by

Title 6,600 times.

Query Log Analysis Database searches

america history and life lexus nexus infotrack

Subject searches education pharmacy medicine anthropologyu

Topic searches hamlet insane adopted children of same sex couples “why doesn’t the us have a eurpean-style welfate state?”

Are these successful?

• Subject browse: ~ 18,000; • Keyword search 15,800; and • Databases by Title: 6,600

housing market

Search “housing market”

Browse by Topic Business

Usability Testing: 3 Rounds

Who?3 undergraduates, 1 grad, 1 faculty in

each@ Storrs & regional campuses

What?First tested old systemMajor redesignTested redesignTweaked designTested again

Usability Tasks

Find articles about diabetes for your nursing class

Find a newspaper article on the day you were born

Your professor said “use a database named ERIC”

… plus 7 more …

Databases by Title – find ERIC

Design Goals

Unmediated use of database locator Focus on how most users think

*** not librarians *** Display 5 databases per subject

Users can choose to see more Shortened database descriptions

http://rdl.lib.uconn.edu/bySubject.php

Best Bets in Library Science

Side Effect: License Data

All Databases in LIS

Shortened Descriptions

From this … ABI/Inform Global Full-text articles from 1800 journals

covering business, finance, management and related functional areas. ABI/INFORM Global indexes a total of 2700 major publications. Subject coverage includes: business and management, including all functional areas.

To this … ABI/Inform Global Articles in business, finance,

management, accounting, advertising, banking, insurance, marketing, public administration, real estate, and telecommunications. 1991-current (full-text); 1971-current (index & abstracts).

Timeline … How Long?!

Began Winter 2006: “database descriptions too long, fix”All agreed. But …Rewriting database descriptions wouldn’t

solve all problems.“Maybe we should do more”

Ad hoc group started meeting spring 2006 & set up plan

From Ad Hoc to Rollout

August 2006: first usability tests Fall 2006: redesign & discussion

Possible “quick wins” tested & rejected November 2006: 2d round of usability

Redesign & more discussion January 2007 (early): 3d round of

usability tests January 2007 (mid): beta rollout March 2007: rollout (spring break)

Final Round of Testing

UConn’s Next Steps

Continue user analysisReview query logs, usage dataSolicit feedback from users, colleagues

Add / tweak …User tags“My Databases” featureBetter searching?

Note errors, redesign and retest

UConn Redesign Team

Stephanie Willen Brown, electronic resource librarian & liaison to Communication Sciences

Susanna Cowan, undergraduate education & outreach librarian

Kate Fuller, reference collection maintenance coordinator and / administrative assistant

Jill Livingston, reference librarian/liaison to the school of allied health

Tom Wood, applications developer

Useful Usability Resources

http://gslis.simmons.edu/mw/browns/Usability

“PERM” FAQ

http://gslis.simmons.edu/mw/browns/Usability/UConn-FAQ

Usability Testing: Worth the Effort!

Tony DungyCoach of the Indianapolis ColtsUpon winning 2007 Superbowl