1

Primo Usability at the University of

KansasMiloche Kottman - Scott Hanrath

ELUNA 2013

2

Discover & fix major problems before launch

New product, new features

Help support reference & instruction staff

Why?

3

Task-based user testing

Participants solicited via social media & accosting engaging students in the library

16 participants◦ 2.5 days of testing◦ ~ .5 hour per student

Method

4

Status Count (%)

Undergraduate 8 (50.0)

Graduate & Prof 6 (37.5)

Other 2 (12.5)

DemographicsHow often… Count (%)

Never 0 (0.00)

Once/twice a year 2 (12.50)

Once/twice a semester 0 (0.00)

Monthly 2 (12.50)

Weekly 7 (43.75)

Daily 3 (18.75)

No response 2 (12.50)

Confidence/ library skills

Count (%)

Not at all 0 (0.0)

A little 2 (12.5)

Somewhat 4 (25.0)

Confident 10 (62.5)

Very confident 0 (0.00)

5

Known item: physical item

Known item: ebook

Known item: journal article

Open-ended: peer-reviewed article for supplied topic

Open-ended: any resource deemed appropriate by user for user topic

5 Search Tasks

6

7

What is your overall impression?

Did you find anything unclear or confusing?

What one thing would you change?

How likely are you to use this search in the future?

Follow-up Questions

8

Overall impression and likelihood of future use

Rank Impression Use in Future

Not at all 0 0

A little 0 0

Between 2 (12.5) 0

Somewhat 1 (6.25) 3 (18.75)

Positive/Likely 10 (62.5) 7 (43.75)

Between 1 (6.25) 1 (6.25)

Very 2 (12.5) 5 (31.25)

9

Success Rate (%)1. physical item 81.252. ebook 81.253. article, known 93.754. article, supplied topic 87.5

Success Rates

10

Number of Tasks Completed

Number of Participants (%)

4 10 (62.5)3 4 (25.0)2 1 ( 6.3)1 1 ( 6.3)

# of Tasks Completed

11

“Success” involves more than just Discovery◦ Link Resolver◦ Where does the Library end?

Metadata matters◦ But also only the Title

What (you think) you know can hurt you

Takeaways

12

Link Resolver…

13

… and Beyond

14

Too smart for Primo?

15

Changes: Boolean Suggest

16

Changes: Facet Changes

17

Usability per Task Usage per Visit

Basic Search* 1.39 0.91

Advanced Search* 0.50 0.24

Result Title Click 0.98 0.67

Facet Use 0.95 1.02

Result Icon Click 0.34 0.08

View Online Tab 0.19 0.19

DYM Click 0.13 0.01

Usability vs Usage

18

Thanks!Miloche Kottmanmkottman@ku.edu

Scott Hanrathshanrath@ku.edu@rshanrath