Large-scale, web-based, user-centered assessment of library service effectiveness across multiple...

Large-scale, web-based, user-centered assessment of library service effectiveness across multiple institutions.

Co-developed by ARL and Texas A&M University, 1999

Responds to the increasing pressure for libraries to develop more outcomes-based assessment efforts, instead of relying merely on input or resource metrics.

Supported initially by a 3-year, $498,000 FIPSE grant; sustained by participant fees ($2,850/year)

What is LibQUAL+?

Grounded in the “Gap Theory” of Service Quality; addresses a set of three service dimensions:

1. Information Control—timely, convenient, and self-reliant access to information resources: local & remote, print & electronic.

2. Affect of Service—knowledge, courtesy, and responsiveness of employees; their ability to instill confidence; their willingness to help customers and provide prompt service.

3. Library as Place—a library space that is quiet, comfortable, and conducive to study and learning, for individuals as well as groups.

What is LibQUAL+?(The “Gap Theory” model and 4 dimensions of service quality)

Help libraries better understand user perceptions of service quality

Provide comparable assessment information from peer institutions

Identify best practices in library service

Continuously improve library service quality

The goals of LibQUAL+

Over 250 institutions, including:

Members of Association of Research Libraries (35)

Other college and university libraries (164)

Community college libraries (18)

A small number of governmental, public, school, and special libraries

A growing number of international participants in Australia, Canada, France, Ireland, Sweden, and the U.K.

Who participated in Spring 2007?(Groups & consortia)

University of Alberta Libraries (A) University of Arizona Library (A,G,P)University of British Columbia Library (A)University of California, Los Angeles (A)University of Chicago Library (A)University of Cincinnati Libraries (A)University of Guelph (A)University of Houston Libraries (A,G)University of Kentucky Libraries (A)University of Manitoba (A)University of Maryland Libraries (A)University of Massachusetts Amherst (A)University of Miami (A)University of Nebraska-Lincoln Libraries (A,G)University of New Mexico (A, G)University of Saskatchewan Library (A)University of South Carolina (A)University of Southern California (A,G)University of Toronto Libraries (A)University of Waterloo (A)University of Western Ontario (A)Washington University in St. Louis (A,G)York University Libraries (A)

Who participated in 2007?(Peer institutions)

Auburn University (A)Baylor University Libraries (G)Duke University Libraries (A)Iowa State University Library (A,G, P)Kansas State University Libraries (G)McMaster University Libraries (A)Ohio State University Libraries (A,P)Queen's University (A)Rice University (A,G)SUNY Buffalo (A)Texas A&M University, College Station (A,G, P)Université de Montréal (A)Université Laval (A)University of Alabama (A)

37 peer institutions from Association of Research Libraries, Greater Western Library Alliance,

and “Peer 11 Land-Grant,” including…

Gather random sample (1,600 u-grads; 800 grads; 800 faculty)

Prepare website to manage publicity, communication, etc.

Send “pre-survey” message from Dean (March 21)

Send email with imbedded URL for online survey (March 26)

Send reminder from the Dean (March 29)

Survey closes on April 6, 2007

Announce incentive prize winners (May 1)

Checklist of local activities

Who responded at ISU?(Response rates for faculty, grads, undergrads)

280 of the 3,200 users surveyed (8.8%) responded to the quantitative questions, including:

• 134 of the 800 faculty surveyed (16.8%)

• 69 of the 800 graduate students surveyed (8.6%)

• 77 of the 1,600 undergrad students surveyed (4.8%)

119 of these respondents also provided written comments (i.e., qualitative data)

Who responded at ISU?(By age & Sex)

Sex

59.4%40.6%

Age

21.2%

24.4%

19.6%

31.1%

3.6%

I use the library electronically…

I use the library on premises…

7.7%

34.7%

36.2%

19.8% 23.4%

46.1%

18.8%

9.7%

I use GoogleTM, etc. for information…

71.9%

18.4%

5.3%

Daily Weekly Monthly

Quarterly

Never

<1%

Respondents’ use of library / intranet(By age & Sex)

When it comes to… My Minimum Service level is

My Desired Service level is

My Perception of the library's service performance is

(low) (high) (low) (high) (low) (high)

1 Employees who instill confidence in users

1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 N/A

2 Easy-to-use access tools that allow me to find things on my own

1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 N/A

3 Print and/or electronic journal collections I require for my work

1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 N/A

4 Readiness to respond to users' questions

1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 N/A

5 Quiet space for individual activities

1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 N/A

Sample Survey

Dimension 1: Information Control

IC-1 Making electronic resources available from my home or office

IC-2 A library Web site enabling me to locate information on my own

IC-3 The printed library materials I need for my work

IC-4 The electronic information resources I need

IC-5 Modern equipment that lets me easily access needed information

IC-6 Easy-to-use access tools that allow me to find things on my own

IC-7 Making information easily accessible for independent use

IC-8 Print and/or electronic journal collections I require for my work

Information ControlDimension 1:

Dimension 2: Affect of Service

AS-1 Employees who instill confidence in users

AS-2 Giving users individual attention

AS-3 Employees who are consistently courteous

AS-4 Readiness to respond to users questions

AS-5 Employees who have the knowledge to answer user questions

AS-6 Employees who deal with users in a caring fashion

AS-7 Employees who understand the needs of their users

AS-8 Willingness to help users

AS-9 Dependability in handling users' service problems

Affect of ServiceDimension 2:

Dimension 3: Library as Place

LP-1 Library space that inspires study and learning

LP-2 Quiet space for individual activities

LP-3 A comfortable and inviting location

LP-4 A getaway for study, learning or research

LP-5 Community space for group learning and group study

Library as PlaceDimension 3:

Addendum:General Satisfaction

GS-1 In general, I am satisfied with the way in which I am treated at the library.

GS-2 In general, I am satisfied with library support for my learning, research, and/or teaching needs.

GS-3 How would you rate the overall quality of the serviceprovided by the library?

General Satisfaction QuestionsAddendum:

Addendum:Information Literacy Questions

IL-1 The library helps me stay abreast of developmentsin my field(s) of interest.

IL-2 The library aids my advancement in my academicdiscipline.

IL-3 The library enables me to be more efficient in myacademic pursuits.

IL-4 The library helps me distinguish betweentrustworthy and untrustworthy information.

IL-5 The library provides me with the information skills I needin my work or study.

Information Literacy QuestionsAddendum:

Question 1

Question 2

Question 3Question 4

Question 5

= Minimum= Perceived= Desired

Sample spider graph

Adequacy Gap

Superiority Gap

Text box

Perceived > Desired Perceived < Desired Perceived > Minimum Perceived < Minimum

Aggregate data (all peer universities, all users)

Print and/or electronic journal collections I require for my work

MDP

IC-8

Affect of Service

Information ControlLibrary as Place

(-0.03)

Comparison: All users (und., grad., faculty)(Graph)

Text box

Peers ISU


Information Control

Affect of Service

Information Control

Affect of Service

CoverupCoverup

CoverupCoverup Coverup

CoverupLibrary

as Place

Print and/or electronic journal collections I require for my work (-0.15)

CoverupLibrary

as PlaceIC-8

Print and/or electronic journal collections I require for my work (0.11)

Difference between perceived and minimal service:

Comparison: All users (und., grad., faculty) (1)(Table)

Peer ISU Difference

Areas in which ISU stands out from peer libraries (by >0.5)… Adequacy gap Adequacy gap [ISU-ARL]

LP-1 Library space that inspires study and learning 0.23 1.20 0.97

LP-2 Quiet space for individual activities. 0.23 1.16 0.93

LP-3 A comfortable and inviting location 0.56 1.40 0.84

LP-4 A getaway for study, learning, or research 0.49 1.29 0.80

LP-5 Community space for group learning & group study 0.82 1.54 0.72

Areas in which peer libraries and/or ISU fall below minimum…

IC-8 Print and/or e- journal collections I require for my work -0.03 0.11 0.14

Difference between perceived and desired service:

Comparison: All users (und., grad., faculty) (2) (Table)

Peer ISU Difference

Areas in which ISU stands out from peer libraries (by >0.5)… Superiority gap Superiority gap [ISU-ARL]

LP-1 Library space that inspires study and learning -1.37 -0.47 0.90

LP-2 Quiet space for individual activities. -1.12 -0.32 0.80

LP-3 A comfortable and inviting location -0.98 -0.29 0.69

LP-4 A getaway for study, learning, or research -1.00 -0.38 0.62

LP-5 Community space for group learning & group study -0.64 -0.02 0.62

Areas in which peer libraries and/or ISU exceed desired service levels

[none]

Text box

Text box

Peers ISU


Comparison: Undergraduates(Graph)

Coverup Coverup

CoverupCoverup

Coverup Coverup

Information Control

Affect of Service

Information Control

Affect of Service

Library as Place

Library as Place


Comparison: Undergraduates (1)(Table)

Peer ISU Difference

Areas in which ISU stands out from peer libraries (by >0.5)… Adequacy Gap Adequacy Gap [ISU-ARL]


LP-2 Quiet space for individual activities 0.15 0.88 0.73

IC-8 Print and/or e-juornal collections I require for my work 0.31 0.96 0.65

IC-7 Making information easily accessible for independent use 0.51 1.15 0.64


AS-9 Dependability in handling users' service problems 0.56 1.13 0.57


[none]


Comparison: Undergraduates (2)(Table)

Peer ISU Difference

Areas in which ISU stands out from peer libraries (by >0.5)… Superiority Gap Superiority Gap [ISU-ARL]


LP-2 Quiet space for individual activities -1.27 -0.53 0.74


IC-8 Print and/or e-juornal collections I require for my work -1.07 -0.57 0.50

Areas in which peers and/or ISU exceed desired service levels…

[none]

Text box

Peers ISU


Comparison: Graduate Students(Graph)

Information Control

Affect of Service

Affect of Service

Coverup Coverup

CoverupCoverup

CoverupLibrary

as Place

CoverupLibrary as Place

Information Control


IC-8


IC-8

Comparison: Graduate Students (1)(Table)


Peer ISU Difference







AS-8 Willingness to help users 0.71 1.31 0.60


IC-8 Print and/or e-journal collections I require for my work -0.18 -0.12 0.06

Comparison: Graduate Students (2)(Table)


Peer ISU Difference



LP-3 A comfortable and inviting location -1.07 -0.52 0.55


[none]

Peers ISU


Comparison: Faculty(Graph)

Information Control

Affect of Service

Information Control

Affect of Service

Coverup Coverup

CoverupCoverup

Coverup

Library as Place

Coverup

Library as Place

IC-3

IC-4

IC-6

Remote access (-0.08)

Website (-0.28)

IC-1

IC-2

Print resources (-0.09)

E-resources (-0.15)

Access tools (-0.08)

Remote access (-0.03)

IC-1

IC-4

Print and/or e- journals (-0.23)

IC-8

Community space for group learning (0.34)

LP-5

Quiet space for individual activities (0.13)

LP-2

Print and/or e- journals (-0.54)

IC-8

LP-3 Comfortable location (0.01)

E-resources (-0.1)

Comparison: Faculty (1)(Table)


Peer ISU Difference






IC-3 The printed library materials I need for my work -0.09 0.53 0.62

AS-9 Dependability in handling users' service problems 0.32 0.91 0.59

IC-5 Modern equipment that lets me easily access info 0.13 0.70 0.57

IC-7 Making info easily accessible for independent use 0.10 0.62 0.52


IC-1 Making e-resources accessible from my home or office -0.08 -0.03 0.05

IC-2 A library website enabling me to locate info on my own -0.28 0.02 0.30

IC-3 The printed library materials I need for my work -0.09 0.53 0.62

IC-4 The electronic information resources I need -0.15 -0.10 0.05

IC-6 Easy to use access tools […] to find things on my own -0.08 0.27 0.35

IC-8 Print and/or e-journal collections I require for my work -0.54 -0.23 0.31

Peer ISU Difference


LP-3 A comfortable and inviting location -0.78 0.01 0.79

LP-2 Quiet space for individual activities -0.61 0.12 0.73




LP-2 Quiet space for individual activities -0.61 0.12 0.73

LP-3 A comfortable and inviting location -0.78 0.01 0.79


Comparison: Faculty (2)(Table)


UNDERG

AdequacyGap

(P-M)

LP-5 Community space for group learning and group study 1.26

LP-1 Library space that inspires study and learning 1.19

AS-7 Employees who understand the needs of their users 1.17

IC-7 Making info easily accessible for independent use 1.15

AS-9 Dependability in handling users' service problems 1.13

LP-3 A comfortable and inviting location 1.09

AS-4 Readiness to respond to users' questions 1.06

Conclusions: Areas of challenge

Conclusions…

Areas of strength…

UNDERG GRAD FACULTY ALL

Adequacy Adequacy Adequacy AdequacyGap Gap Gap Gap

(P-M) (P-M) (P-M) (P-M)

LP-5 Community space for group learning and group study 1.26 1.68 1.63 1.54

LP-1 Library space that inspires study and learning 1.19 1.22 1.16 1.20

AS-7 Employees who understand the needs of their users 1.17 0.77 0.73 0.85

IC-7 Making info easily accessible for independent use 1.15 0.57 0.62 0.75

AS-9 Dependability in handling users' service problems 1.13 0.37 0.91 0.85

LP-3 A comfortable and inviting location 1.09 1.39 1.58 1.40

AS-4 Readiness to respond to users' questions 1.06 1.10 0.94 1.01


Conclusions…


Areas of strength lie in Library as Place (LP) and Affect of Service (AS).

UNDERG GRAD FACULTY ALL

Adequacy Adequacy Adequacy AdequacyGap Gap Gap Gap

(P-M) (P-M) (P-M) (P-M)

LP-5 Community space for group learning and group study 1.26 1.68 1.63 1.54

LP-1 Library space that inspires study and learning 1.19 1.22 1.16 1.20

AS-7 Employees who understand the needs of their users 1.17 0.77 0.73 0.85

IC-7 Making info easily accessible for independent use 1.15 0.57 0.62 0.75

AS-9 Dependability in handling users' service problems 1.13 0.37 0.91 0.85

LP-3 A comfortable and inviting location 1.09 1.39 1.58 1.40

AS-4 Readiness to respond to users' questions 1.06 1.10 0.94 1.01

IC-1 Making e- resources accessible from my home or office 0.74 0.29 0.02 0.27

IC-2 A library website enabling me to locate information… 0.60 0.26 -0.03 0.24

IC-4 The electronic information resources I need 0.86 0.21 -0.09 0.25

IC-8 Print and/or e- journal collections I require for my work 0.96 -0.12 -0.22 0.11


Conclusions…


Areas of challenge…

Areas of strength lie in Library as Place (LP) and Affect of Service (AS).

Areas of challenge lie in Information Control (IC).

Multi-year Comparison of Adequacy Gaps at ISU and Peer Institutions (2001-2007)

2001 2003 2005 2007

Affect of ServiceISUPeers


2001 2003 2005 2007


Information ControlISUPeers


Information ControlISUPeers

Library as PlaceISUPeers


2001 2003 2005 2007

General Satisfaction Questions (1)

In general, I am satisfied with the way in which I am

treated at the library.

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL




In general, I am satisfied with library support for my learning, research, and/or

teaching needs.

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL




In general, I am satisfied with library support for my learning, research, and/or

teaching needs.

How would you rate the overall quality of the

service provided by the library?

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL

Information Literacy Questions (1)

The library helps me stay

abreast of developments in my field(s) of interest.

Hidden text—Hidden text

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL




The Library aids my

advancement in my

academic discipline.


FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL




The Library aids my

advancement in my


The library enables me to be more efficient in

my academic pursuits.


FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL




The Library aids my

advancement in my




The library helps me

distinguish trustworthy /untrustworth

y information.


FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL




The Library aids my

advancement in my




The library helps me

distinguish trustworthy /untrustworth

y information.

The library provides me

with the information

skills I need in my work or

study.


FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL

FacultyFaculty

GraduateGraduate

UndergradUndergrad

ISUARL

Qualitative Data: Sample comments

Number: 61Date: 11:31 PM 3/28/2007 (CST)User Group: UndergraduateDiscipline: AgricultureLibrary Branch: Parks LibraryAge: 23-30Sex: FemaleKEYWORDS: HOURS; NOISE; CAFÉ; FOODComment:Call me a nerd, I would like the library to be open earlier on sun. Also more labeled quiet zones would be helpful. And more room to eat, drink, and socialize by bookend. I think bookends is a great addition to the library and has a great atmosphere, it would just be nice if I could take purchases out of bookend and into the rest of the library.

Qualitative Data (>50 user comments)





Qualitative Data (>5 user comments)Topic # of users

2007 (2005) (2003)

Collections-related 65 72 114

e-Library 50 64 39

Staff attitudes, behavior, & customer service 26 39 57

Building, furnishings, etc. 22 36 43

Instruction & instructional support 16 13 18

Library funding 13 15 11

ILL/DD 12 18 18

Hours 11 15 22

LibQUAL & library surveys 9 29 14

Noise & the study environment 9 22 55

Branch facilities 7 7 n/a

Services (gen'l) 7 13 n/a

Circulation services & policies 6 19 32

Reference 6 20 15

Café & food in the library 5 4 4

Media 5 4 n/a

Subject librarians 5 n/a n/a

Qualitative Data: Recurring themes…(Collections-related)

Collections-related

Buy more journals!! Avoid cancellations; try to restore some journals that have previously been cut.

Improve remote access to all e-resources, but especially journals. Improve access to books

that are in high demand (and “never available…”)

More full-text journals, including back files.

Many suggestions and requests to purchase specific titles or materials in specific subject areas.

Repeated requests to add more “seats” for SciFinder Scholar.

Qualitative Data: Recurring themes…(e-Library, Catalog, Internet, etc.)

E-Library, Catalog, Internet, etc.

Make the e-Library website more user-friendly. Improve the organization; make it less graphics-intensive.

Simplify searching for non-print media such as videos, DVDs, books-on-tape.

Make searching easier! Consider a search box on the e-Library home page.

Simplify remote access. Keep pursuing a “single sign-on.”

Qualitative Data: Recurring themes…(Staff-related)

Staff-related

Majority of respondents see library staff as courteous, patient, helpful, dedicated, efficient.

A few respondents comment on the inconsistency of staff service: the mix of professional & unprofessional behaviors. Student workers can be “hit and miss…”

Still, a few respondents describe staff as “rude,” “not friendly,” or “lacking in customer service.”

Qualitative Data: Recurring themes…(Building, equipment, furniture)

Building, furnishings, etc.

Many comment explicitly on their low use (or non-use) of the physical library. “I only need the Library’s website…” Some areas in the Parks

Library have poor ventilation, flickering fluorescent lights.

Several respondents like (and only one respondent dislikes) the new Bookends Café.

Qualitative Data: Recurring themes…(Noise)

Noise and the study environment

Importance of the Parks Library and branch facilities as places to study.

Continue to control noise levels in the Library, and to provide adequate, separate, and well-designated spaces for both “quiet” and group study.

Unlike previous years, not a single comment about cell phones!

Qualitative Data: Recurring themes…(Specific services)

Specific services

Circulation: Need continued review of Circulation and Collection Development policies (loan periods, purchase of multiple copies, etc.) to improve book availability.

Interlibrary Loan: Numerous respondents describe ILL as timely, efficient, and extremely valuable—lots of kudos to this service in 2007!

Instruction: Users need help in evaluating electronic resources. Distance learners need specialized assistance. Numerous requests for library seminars on specific topics.

Branches: A few users appeared to be unaware of the Vet Med Express delivery service.

Reproduction & reformatting: Need more and better services & equipment for copying, printing, and scanning. Try to contain costs!

Qualitative Data: Recurring themes…(Hours)

Hours; FundingSeveral requests for expanded hours, especially on weekends and during holiday and break periods.

Expanded hours are especially important to non-traditional students.

12 users comment on library’s budget constraints. Many admonish us to “keep pushing for funding…”

Next steps

• Share qualitative and quantitative data with appropriate library units, for analysis.

• Implement and document changes based on findings.

• Explore opportunities to compare findings with colleagues (GWLA, etc.)

• Repeat survey biennially (next in 2009) and watch the trajectories.

• Consider focus groups to explore areas of concern.

Recommendations

• Continue to acquire more e-journals, including backfiles, and make them accessible from both on and off campus.

• Review and act on respondents’ recommendations for specific material purchases (including SciFinder Scholar seats).

• Complete an overhaul of the e-Library website, including: more intuitive organization a homepage search box easier access to journals and journal articles (both print & electronic) easier access to non-print media (videos, DVDs, etc.)

• Improve library support for distance learning (including dedicated pages in the e-Library)

Recommendations (cont.)

• Review and act on respondents’ suggestions for specific instructional seminars and workshops in the coming year.

• Create additional quiet zones within the Parks Library.

• Use respondents’ comments/complaints regarding customer service to shape library staff development sessions in the coming year.

• Review and revise circulation and collection development policies to improve availability of “high demand” books.

• Publicize the Vet Med Express delivery service.

Date post:	28-Dec-2015
Category:	Documents
Upload:	alexander-owen
View:	215 times
Download:	0 times

Large-scale, web-based, user-centered assessment of library service effectiveness across multiple...

Documents